Think if you have that many developers you need to start doing it Linux kernel style and have maintainers responsible for certain parts of the project making sure merges go in fine.
And maybe have actual communication in other means than via comments on tickets. The whole hypothetical example could be avoided by typing "hey, I'm changing this field to be that, anything against?" on team's chat
> Since my_branch was merged before it was rebased onto someone_elses_branch, this error goes unnoticed and BAM - main is broken! This is known as a semantic merge conflict — a merge is technically possible, but results in a regression.
Do people not merge master into their branch before pushing? All that seems like mostly problem caused by having too long living branches.
I'm also confused how that even fixes anything, someone still have to go back and fix that code.
> Do people not merge master into their branch before pushing? All that seems like mostly problem caused by having too long living branches.
On projects with a high commit velocity and a large team, simply merging main and running the tests is not enough. The likelihood that a new commit lands on main before your tests finish is high, and restarting the process every time there's a new commit is time consuming. Merge queues are the only way to ensure that main does not break from conflicts that arise "in between" commits.
> Do people not merge master into their branch before pushing?
No, they rebase! (Sorry, couldn't resist, yeah the other half does this (!attention, opinion!) ugly cross merges that make the commit graph look like my cables under the desk...
> Merge queues are the only way to ensure that main does not break from conflicts that arise "in between" commits.
Not the only one (though you may argue this is a degenerated queue): use merge locks! Yeah, we really use git svn style, the locking is done via a chat channel. No, please don't laugh. Acquire the lock, merge develop or rebase, check everything again, release lock :D :D
(No again: I am not suggesting this to anyone!! :D )
The point the GP was making is that if merging to main is frequent (bigger team) and build times are long (bigger projects), catching your branch up to master before merging can be hard, as master is moving ahead faster than you rebase+build.
Fully understood.. we also have a big team, a too slow CI, and much too many LOCs. The "merge lock" will ensure you catch up, you keep the lock as long as it takes (or until others get too annoyed and urge you to step back) - as long as you have it nothing comes between you and main at the point you claimed it.
Right but biggest projects out there like Linux kernel manage it just fine
> The likelihood that a new commit lands on main before your tests finish is high, and restarting the process every time there's a new commit is time consuming. Merge queues are the only way to ensure that main does not break from conflicts that arise "in between" commits.
But the code is still broken and still someone has to fix it ? That doesn't help in that.
As for "broken main", how is just not allowing code that doesn't pass tests to merge to main not enough ?
It all seems like a lot of complexity added because devs don't want to change shitty work practices...
> Do people not merge master into their branch before pushing? All that seems like mostly problem caused by having too long living branches.
Well, yes, but you have to run your tests.
The article outlines this scenario, but to give a more concrete definition, imagine your tests take 10 minutes, and this timeline happens:
10:00 Co-worker rebases their branch with master and begins the unit testing.
10:05 You rebase your branch with master and begin unit testing.
10:10 Co-worker's tests finish and everything looks green, pushes to master.
10:15 Your tests are done and everything is green. You try to push, but your branch is now out of date because master contains your co-workers changes! You rebase on master, restart your tests...
10:25 Your tests finish, it's green, and you finally push to master.
Ten minutes got wasted. It doesn't seem like a lot, but with enough engineers (Or long enough testing time), this scenario can happen a lot, especially with very short-lived branches.
10:00 Co-worker pushes their code to the merge queue. CI/CD pipeline begins running unit tests.
10:05 You push your code to the queue.
10:10 CI/CD finishes the tests. It looks green. Their code gets merged to master. CI/CD starts running your unit tests with the new master branch and your code applied.
10:20 CI/CD finishes the tests with your code, tests are green, and merges to master.
> The whole hypothetical example could be avoided by typing "hey, I'm changing this field to be that, anything against?" on team's chat
This doesn't scale. With 15 developers, you could be working on ten different features at the same time; getting alignment on any field change (not even APIs) between that many people and projects does not scale.
And 15 is still considered a small team.
> Do people not merge master into their branch before pushing? All that seems like mostly problem caused by having too long living branches.
Again, this does not scale . Every time you merge master into your branch, you have to wait for the CI to pass before you can merge. If in the meantime someone else merges their change before you - and this happens asynchronously, whenever it's ready / whenever someone gets to it - you have to merge master again and wait for CI again. And that takes your focus off of whatever else you want to work on next, if you're waiting for CI and hoping nobody else merges before you, you're epitomizing the "Compiling!" comic.
Your idea works, but only for smaller teams, faster CI pipelines, and lower frequency of merge attempts.
Merge queues on the other hand is fire & forget; it enforces a linear process - all features will be "master + this one feature" so you don't get two merges interfering with each other - and it allows the owner to move on to the next. Unless CI breaks, which is actionable. It allows for an asynchronous process with less gatekeeping, coordination, waiting for things to happen, etc. It scales better.
When i starter my career working for large corp we had dedicated people for this called Release Engineers. The concept of specialization seems to have fallen out of favor within engineering orgs…
I'm convinced that all repositories with more than a few hundred PRs definitely need a merge queue. Glad to see discussion in this space, I've been frustrated by the lack of merge queues at previous companies.
The first description I read of a merge queue came from Graydon Hoare, the creator of the Rust programming language, describing a CI system his team cobbled together in 2001[0]. He called this system "The Not Rocket Science Rule Of Software Engineering"
They're super annoying if you have flaky tests. If the tests are such that you can automate retrying and get to a reasonably low false-positive rate, it can be usable.
A good MQ impl allows you to make the MQ checks different than in PR. And you choose for MQ builds only a subset of tests which are known to be non-flaky.
Merge queue builds should be fast and rock stable, following the 80/20 rule.
Not useless, you get sprurious rejections and have to resubmit, like running CI. and you can retry—on-failure to increase the odds. Some test frameworks support rerunning flaky tests.
From what I've observed semantic merge issues always get picked up at compile time - hence only the compile phase need pass to allow a queued merge through, but I guess if you're working a lot with weakly-typed or interpreted languages it'd be different.
It's hard not to think there should be some way of automatically flagging likely issues where, say, commits in one PR refer to particular identifiers that were modified in parallel PRs, but it definitely goes well beyond what diff-merge tools typically do today.
> From what I've observed semantic merge issues always get picked up at compile time - hence only the compile phase need pass to allow a queued merge through, but I guess if you're working a lot with weakly-typed or interpreted languages it'd be different.
I agree that 90% of the time, with strongly typed languages, a compile is enough.
But even in those languages, you can still get semantic merge issues which aren’t compilation-related.
Suppose your app has a config file. The first PR renames one of the configuration keys in the config file, the second PR adds some new code which reads that configuration key. The combination compiles fine, but the second PR fails in integration tests, because the config key it was expecting wasn’t there any more.
Of course this is a sign of a design flaw - the name of the config key should occur only once in the program (say as a constant), so both PRs would use the constant, and the second PR would get the change in constant value from the first, or else the first renames the constant so the second doesn’t compile. Or even a higher-level API which deals with the semantics of what the config represents instead of just key-value strings.
But a lot of large legacy code bases contain design flaws like this, and fixing them can be a lot of work, so they don’t get fixed, at least not any time soon.
Sure, there are always going to be occasions where a combination of two bits of work done separately cause unexpected behaviour changes - but generally if there are integration/automated E2E tests that are capable of picking that sort of thing up I'd expect they typically get triggered after the mainline merge anyway.
OTOH, I can understand at least wanting to ensure that it's not possible for two consecutive merges to the mainline branch to cause it stop building.
Oh, `--update-refs` is cool. I think this is what I want sometimes. I'll have a commit I want to test results of against branchA and branchB (e.g. I have `master` and `branchB` with two different implementations of some thing and I want to test some code in ref Y against it. Previously, I had to create a separate branch that was master+Y and branchB+Y and I might alter the branchB and have to rebase branchB+Y on it.
Actually I think it doesn't solve the problem now that I wrote it out, but it is interesting nonetheless.
Sure, but we use a stacked branch to solve the same problem that a merge queue solves - preventing semantic merge conflicts by predetermining a merge order.
Just looked at graphite.dev and it looks like it only works with Github.
Wild that an entire product can exist that depends on another commercial SaaS offering to have any value at all. That feels absurdly risky to me. Like, I am getting anxiety from the thought of working on that product.
Then again, I guess AWS exists and we all make our peace with it.
> Like, github PRs are pretty shit, which is an opportunity for reviewable.
I've used a bunch of VCS's and managed solutions, and PR's are probably the _best_ I've had the pleasure of using. Working with perforce, everyone is one (or a small number) of long lived branch(es), chucking around WIP code as shelves. PlasticSCM/Unity DevOps Version Control (seriously, that's what they renamed it to) provides absolutely no support for a merge-based workflow other than some half-baked tools that barely work.
It's like a rebase queue, but with more ref-log, and less tree inversion. Pull queues counter-intuitively lean on force-push-with-lease sub-commands, whereas monorepo merge queues have remote build caches for... sorry I blacked out.
I bought that hook line and sinker :) I was even feeling a real blackout start at the word "remote". Comedic timing is hard to pull off in text. Is any of that even real?
And maybe have actual communication in other means than via comments on tickets. The whole hypothetical example could be avoided by typing "hey, I'm changing this field to be that, anything against?" on team's chat
> Since my_branch was merged before it was rebased onto someone_elses_branch, this error goes unnoticed and BAM - main is broken! This is known as a semantic merge conflict — a merge is technically possible, but results in a regression.
Do people not merge master into their branch before pushing? All that seems like mostly problem caused by having too long living branches.
I'm also confused how that even fixes anything, someone still have to go back and fix that code.