2. Raspberry pi as a forge for a long time: also check, the git server shim is super lightweight, its just an XRPC layer over git repositories + an sqlite3 database. there are folks running it on a riscv board with 512 megs of RAM.
3. Actions are critical and they should be runnable on my local machine: IMO this ask is slightly misplaced. it is mostly your build-systems' job to be hermetic, run anywhere, handle cross-builds etc. it would be really cool to "promote" results of such builds to the forge itself.
Surprised to see raspberry pi for hosting data that's supposed to be integral to a workflow. I've been burned too many times by SD card corruption in the past. Do you use NVME drives nowadays, just curious.
I use RPi 4 with a cheap SSD connected via USB-to-SATA connector as a tiny home server for a year already and it works without issues.
I know that not all USB-to-SATA connectors are compatible with RPi – I got lucky with the first connector I tried (Unitek). Not sure if RPi 5 has a wider compatibility.
A year isn't much. I changed I four ssds,sata,sata,nvme,nvme, about every other year, while the data store at some cheap wdc spinning rust still spins and won't stop for some years.
Raspi5 has a pci connection for ssd hardware add-ons, I use one and it runs fine with a 2 TB one, I just let it on forever. This has been on for almost a year currently without issues.
It is the job of the build system, yes. In most cases the problems people want to solve with locally runnable Actions aren't the build not working, it's the whole integration of the thing; all the YAML definition, secrets, exactly what commands it runs, how it restores tools and caches (which your build system _might_ take care of but the available primitives for that in GHA are very poor), etc.
I do think it's just an awkward problem to solve though, because it essentially devolves to needing to run the entire system somewhere else, which is why every system I've seen like this ends up being trial-and-error.
>it is mostly your build-systems' job to be hermetic, run anywhere, handle cross-builds etc.
yes, and... the idea here is that it would be neat to extend the hermetic builds idea such that this can be run locally / anywhere where there's compute easily. The root problem that's being called out here is that idea of running something until the CI says it's green when there's a change, commit, network call, in the cycle is a pain in the ass. (The best way to avoid this churncycle is to just never write bugs! TFIC ;P)
Both Radicle and Tangled miss the point; these are all for public collaborative work, but what about private repos? Many users work on side projects; they use GitHub private for this. Once you learn GitHub, then you also start public projects on GitHub.
The point I am trying to make is, until you offer a user the ability to make a private repo for side projects, it's unlikely to take off.
What people want is the ability to make a private repo, go away for a few months and come back to find their repos right there waiting for them.
Private repos provide nothing to the site by definition. The value model here is, you must pay for private repos, either by paying a subscription or hosting your own node and bearing the related costs.
A remote execution cluster and CAS for build artifacts is a good way to avoid duplicate work on local vs CI, and avoid the problem of needing to trust local builds.
Between Radicle, Tangled and Grasp, Radicle feels the coolest to me, emotionally, local first, p2p, for whatever reason it feels kind of nostalgic.
Grasp is actually pretty cool too, built on nostr, which is maybe a stronger platform in the end? I dont really know enough about it. Stronger as in, you're maybe opening up more interoperablity by putting your stuff on a "anything network" vs Radicles "p2p git data network".
To be honest they're all cool ideas, Tangled feels some how corporate though.
Graps seems neat in the way it forces you to sign your commits (which should really be standard practice) but I think Github itself has proven that interoperability is not that important.
I can understand that. It's a difficult problem design-wise and needs someone that knows crypto, so I won't hold it against software that labels itself as alpha.
i am unable to access any repository on that website. for some, it complains that ssh or https URLs are not supported by my browser? and for others its just loading indefinitely with `Failed to load file tree`. maybe its not fairly mature.
> how would you build a social graph of follows/stars and what not using user-owned git repos as a backing store?
I'm just spitballing and depending on how you want to display it, you may need more - but if I want to "follow" you I submit a signed commit to your "follow" repository, similar if I'm staring a repo; and then your system issues a signed commit back to my "followed" repo.
its linked in the original post as well, but here is an explanation of why activitypub is not a good fit for this problem, by the authors of ForgeFed themselves: https://forgefed.org/blog/actor-programming/
Reading that - I'm really not sure that AT Protocol has a much better story there either.
(as I understand it) the data has to live in a PDS, PDS are keyed by accounts, so you are similarly stymied for collaborative projects? I guess AT Proto is still a real work in progress so maybe that story has improved since the last time I checked it out.
Yeah the problems they seemed to have were over collaborative data structures with permissions. You’re right about how atproto solves that, which means you’re using CRDTs if you need to collaborate. If that’s a fit mismatch, I’d tell people to just appoint api servers which wrap a repo and provide the needed semantics.
maybe, but tangled knots actually federate. you could contribute to repos on knot.ghostty.org and knot.tangled.org with the same account. no other platform permits one identity across instances.
have a longer write up here: https://blog.tangled.org/stacking but we have "interdiffs", to view a delta from previous review. pull-requests advance in the form of immutable rounds much like the patch workflow on email.
- the first PR in the stack creates a search index.
- the second one adds a search API handler.
- the last few do the UI.
these are all related. you are right that you can do this by breaking a change into commits, and effectively that is what i do with jujutsu. when i submit my commits to the UI, they form a PR stack. the commits are individually reviewable and updatable in this stacking model.
gh's model is inherently different in that they want you to create a new branch for every new change, which can be quite a nuisance.
> - the first PR in the stack creates a search index.
> - the second one adds a search API handler.
> - the last few do the UI.
So you're saying you're going to merge (and continuously integrate, perhaps to production) a dangling, unused search index, consuming resources with no code using it, just to make your review process easier?
It's very depressing that review UX is so abysmal that you have to merge features before they're done just to un-fuck it.
Why can't the change still be a big branch that is either all merged or not... and people can review it in chunks? Why do we require that the unit of integration equals the unit of review?
The perverse logic always goes something like this:
"This PR is too big, break it up into several"
Why?
"It's easier to review small, focused changes"
Why can't we do that in one PR?
"Because... well, you see GitHub's UI makes it really hard to ..."
And that ends up being the root-cause answer. I should be able to make a 10,000 line change in a single commit if I want, and reviewers should be able to view subsets of it however they want: A thread of discussion for the diffs within the `backend` folder. A thread of discussion for the diffs within the `frontend` folder, etc etc. Or at the very least I should be able to make a single branch with multiple commits based on topic (and under no obligation for any of them to even compile, let alone be merge-able) and it should feel natural to review each commit independently. None of this should require me to contort the change into allowing integration partially-completed work, just to allow the review UX to be manageable.
This is not just about the UI, it's about the mental model and management of the changes.
Just covering the review process:
Yes, you can structure your PR into 3 commits to be reviewed separately. I occasionally structure my PRs like this - it does help in some cases. But if those separate parts are large, you really want more structure around it than just a commit.
For example, let's say you have parts A, B and C, with B depending on A, and C depending on B.
1. I may want to open a PR for A while still working on B. Someone may review A soon, in which case I can merge immediately. Or perhaps it will only be reviewed after I finished C, in which case I'll use a stacked PR.
2. The PR(s) may need follow up changes after initial review. By using stacked PRs instead of just separate commits, I can add more commits to the individual PRs. That makes it clear what parts those commits are relevant to, and makes it easy to re-review the individual parts with updated changes. Separate commits don't give you that.
Stacked PRs is not a workflow I'd use often, but there are cases where it's a valuable tool.
Then apart from the review process, there are lots of advantages to keeping changes small. Typically, the larger a change, the longer it lives in a separate branch. That gives more time for merge conflicts to build up. That gives more time for underlying assumptions to change. That makes it more difficult to keep a mental map of all the changes that will be merged.
There are also advantages to deploying small changes at a time, that I won't go into here. But the parent's process of potentially merging and deploying the search index first makes a lot of sense. The extra overhead of managing the index while it's "unused" for a couple of days is not going to hurt you. It allows early testing of the index maintenance in production, seeing the performance overhead and other effects. If there's an issue, it's easy to revert without affecting users.
The overall point is that as features become large, the entire lifecycle becomes easier to manage if you can split it into smaller parts. Sometimes the smaller parts may be user-visible, sometimes not. For features developed in a day or two, there's no need to split it further. But if it will span multiple weeks, in a project with many other developers working on, then splitting into smaller changes helps a lot.
Stacked PRs is not some magical solution here, but it is one tool that helps manage this.
> But if those separate parts are large, you really want more structure around it than just a commit.
Why? I reject the notion that large commits should be intrinsically hard to review.
GitHub already has the concept of "code owners", which are people who have ownership/review responsibility over slices of the codebase, based on globs/pattern matching. But they don't implement the other half of that, which is that a reviewer should be able to see a projection of a given PR, which matches the part of the repo they're the owner of.
There. That solves the entire problem of "this is too big, I can't look at all of it" (because your code ownership says this is the chunk of codebase you say you care about), and if that still isn't sufficient, there's a zillion UI features GitHub could add that they simply don't. Why can't I "focus" on a subset of the changes during review, in a way that helps me ignore unrelated discussions/changes? That is, even if I'm not code owner of the `frontend/` folder, why isn't there a UI affordance that says "let me focus on changes inside `frontend/` and ignore discussions/etc for the rest"?
> By using stacked PRs instead of just separate commits, I can add more commits to the individual PRs
Or you could just add commits to the PR, and if GitHub got the damned UI right, it would be natural to see the "slice" you care about, for all the new commits. Having to rearrange commits into separate PR's and slice-and-dice followup changes to file them into the right PR unit, is (to me) a workaround for how shitty GitHub's review UX is. It really shouldn't be this way.
> Then apart from the review process, there are lots of advantages to keeping changes small [...]
I agree with you on most of these points, but the decision to land smaller changes earlier should be made based on things like "let's get early feedback behind a feature flag" or "let's see how this chunk behaves in production so we can validate assumptions", or "let's merge what we have now so to cut back on conflicts", etc. That's all fine. But I'm vehemently opposed to being required to slice up my changes this way, just to work around a terrible review UI.
Personally, I review code in my development environment GitHub's UI is nonsensically terrible to read code. I could go on for hours about this[0], but when looking in my IDE I can drill into a subfolder and look at the diffs there. I can click and follow symbols. I can look at the individual diff history for any wildcarded subset of the repo, and see how the change was broken into commits. If I'm typing up some feedback to say "try doing it this way instead", I can actually try it myself first to make sure I'm not suggesting that someone do something that doesn't even compile.
And GH's discussion UX is by far the worst part of all of it. If you have a thread of discussions around a line of code, then wake up the next morning and want to see what new comments have been added? Good luck. Your best bet is to check your email inbox, because the comments are actually shown to you there. Using GitHub's "inbox" feature? All that is is a link to PR's you have to look at, with no hints at "why" (it could be a CI run finished for all you know.) Good luck figuring out "why" a PR is on your list. Did someone @-mention you? Who knows. So, find the blue dot next to the PR, click it, and then figure out for yourself what changed since the last time you looked. No, you can't just scroll and find it because GitHub hides half the discussions by default. So you have to go and expand all the collapsed sections to hopefully find that conversation you were having yesterday. But oh, you can only find it in the diff tab. So you click that, but the relevant file is collapsed by default ("Large diffs are not rendered blah blah"), so then click that. Then you may find that discussion.
Contrast this to a mailing list. The discussions are... discussion threads. You pick up where you left off. People's comments are right there in your inbox, newest one on top (or whatever your preference is.) You get notified when there's a new message, and when you tap the notification, it's the actual message, not some link to the PR that makes you click 6 more things to maybe find the message that just happened.
[0] like how the first thing you have to do when opening up the changes tab is ctrl+f search for "Large diffs are not rendered by default" to find the actually-important diffs that are not shown to you because GitHub's backend can't scale to rendering large diffs without friction. Countless times I've been burned by approving a PR because I don't see it making a change to some functionality, only to find out it actually did make said change, but GitHub just decided not to show me it. Seriously, the "large diffs" are the most important ones, and those are the ones you don't see without extra clicks. The mind boggles.)
nevermind, I see what's happening in the UI. Each `jj` change is preserved in the UI and we can see multiple versions of the same change. The stack then is not really a stack of PRs but a stack of changes (where each change has its own history, i.e., the interdiff view). Did I get it mostly right?
yes, thats right! when you submit a branch, you can choose to "stack" it, so the individual commits in the branch turn into separate PRs. these PRs evolve individually, can be merged individually, and be reviewed individually. you can also set different reviewers/labels for different PRs in the stack.
tangled.org supports native stacking with jujutsu, unlike github's implementation, you don't need to create a new branch per change: https://blog.tangled.org/stacking/
yes but tangled.org really does do most of that!
1. JJ as the VCS: tangled supports stacked PRs using jj change-ids. https://blog.tangled.org/stacking , we use it a lot to build tangled itself: https://tangled.org/tangled.org/core/pulls
2. Raspberry pi as a forge for a long time: also check, the git server shim is super lightweight, its just an XRPC layer over git repositories + an sqlite3 database. there are folks running it on a riscv board with 512 megs of RAM.
3. Actions are critical and they should be runnable on my local machine: IMO this ask is slightly misplaced. it is mostly your build-systems' job to be hermetic, run anywhere, handle cross-builds etc. it would be really cool to "promote" results of such builds to the forge itself.
reply