I got used to platforms like TeamSpeak where everybody had individual volume sliders. It was excellent. Whatever Zoom does to automatically adjust everyone's volume is a poor replacement and it leads to exactly the issue you describe. Sure it's cheaper than sending out a bunch of individual audio streams or doing multiple transcodes server-side, but we've lost a lot of flexibility. I'd quite like P2P systems like Jitsi to bridge the gap.