Absolutely 100% agree. Like Bluetooth and USB, it's built to work with a whole pile of stuff out in the real world that the devs didn't have access to for testing / didn't exist yet when it was standardized, so it's got some grabasstical corner cases.
... problem is, it's also got like 36 million users, so it's incumbent upon anything that claims to be a "streaming service" to account for those quirks to work in 36 million living rooms (in the same sense that it's not Bluetooth's "fault" when the new car or headphones on the market don't pair with already-existing popular brands of mobile phone).
If we had one streaming app to rule them all, it could consolidate the quirks fixes and we'd be all good. The fact that market competition means everyone has to solve for those quirks individually and is disincentivized to share solutions is a PITA for end users.
I'm not privy to the details of how AirPlay is configured, but knowing Apple, they probably built the streaming part of the protocol atop their own proprietary software tech stack (because that's how they usually solve these issues) and anyone who doesn't pay-to-play (with the corresponding cost burden of confirming interoperability with the infrastructure Apple built) is left out in the cold. Evidence I have for this is that you can control an AirPlay device from anything running Apple software (including a Windows machine running iTunes)... But not Android. Quick Googling confirms that the on-the-wire AirPlay protocol is undocumented.
That's Apple's style and it probably works better for an application like this (there are a lot of problems for which dictatorship is the solution in the technical space... I wish we had a streaming service dictator to solve the Chromecast problem from the other direction).
But ChromeCast overall I’ve found to be crappy technology.