It's annoying UX, but unfortunately I've come to the conclusion that the alternative is worse. When channels are opt-in, it makes discoverability effectively infeasible in practice. This is what the Element clients that I've seen do (following the IRC convention), and it just means that everyone clusters in the default channel and the others all wither on the vine.
That said, maybe there's a middle ground. If a server could mark, say up to 20 channels as default/opt-out, and the rest as backrooms/opt-in, that might suffice for 80% of servers while avoiding the long-tail worst-case UX of manually muting 100 channels in a server because there's only one you care about.
Another alternative would be to use threads more. But they are terrible in Discord, even worse than in Slack. When two Discord users are chit-chatting about their dogs all through the night, a mod should just drag-select all of these messages and put them in a "dog chat" thread.
I still want to try Zulip to see how well it works.
That said, maybe there's a middle ground. If a server could mark, say up to 20 channels as default/opt-out, and the rest as backrooms/opt-in, that might suffice for 80% of servers while avoiding the long-tail worst-case UX of manually muting 100 channels in a server because there's only one you care about.