The problem is not the number of ears we have, but the amount of air moved by the instruments themselves and how they interact with each other.
A symphony orchestra is miced per group normally (2 for violins, 2 for trumpets, etc.), but if you're around 60 people, you can mic every instrument individually.
To reproduce the sound 1 to 1, you need to mic every instrument individually, and playback them with speakers matching the frequency response and air pressure . So you need speakers equal to the number and characteristics of instruments themselves. On top of that you need to record them ideal microphones and store them loslessly in the process.
Otherwise, you can't create the sound by recording 100 people with 20 microphones, and downmixing them to two channels. It's not possible. I played in double bass in an orchestra, listened countless orchestras, listened the recordings of our own concerts. The gap is enormous.
Thank god we have more than 2 ears.