There was a really bad Dark Ages of remote exploits which peaked around the early Windows XP era. The OS itself and Internet Explorer were a horror show.
Other than that, yes, it's been fairly rare to have malware infections that don't start with tricking the user into executing a binary, though it certainly can happen.
In 30+ years of having computers I’ve had issues twice.
Once was on in the 3.11 days. Turned out the copy of KidPix we bought had a virus on the installation disks.
The other was in the late 2000s at work on XP. I got got some piece of malware (showed ads IIRC) from a drive by using a Java exploit in an applet showed by an ad network.
If you’re not doing high-risk stuff it’s not hard to avoid. But I’m glad MacOS has this built in just in case (like MSE on Windows).
Other than that, yes, it's been fairly rare to have malware infections that don't start with tricking the user into executing a binary, though it certainly can happen.