This process happened in Madrid a few years ago. It's an incredible quality-of-life improvement for everyone in the city. Diesel buses were phased out a few years ago. The newer hybrid CNG buses are uncannily quiet, and some lines in the city core, where distances are smaller and speed is slower, are fully electric.
Yup, wasn’t being entirely serious. They’ve gotten progressively quieter over the last few decades; when I was a kid, they really did sound like they were within seconds of complete failure when going up a hill. The electric ones are a shock the first time you get on one, though - very futuristic.
Busses are not going to be quiet. Around here we have pretty cool "Van Hool ExquiCity" trolley busses and they can glide gracefully at slow speeds, but when they are hurtling down patched up streets they are not much quieter than the traditional busses.
Most city buses spend most of their time going fairly slowly. The electric buses are absolutely a lot quieter, at least around here. Crucially, also, they don’t vibrate; I’ve never been on a diesel bus where vibration from the engine wasn’t at least somewhat noticeable.
On the older busses in the city I live in, the level of vibrations actually make me sooo motion sick, I have gotten off the bus and immediately evacuated my insides.
I hope you're using "should" as in "that's what I'm accustomed to" rather than "that's how it ought to be"... right? :-D
Personally I feel like quieting buses would be a huge step toward making day-to-day city life more pleasant.