The broader consumer base will install anything a bad actor wants them to and then blame the manufacturer for not stopping them with some draconian rule.
This expectation exists because Apple/Google/monopolists sell it. But it's not realistic, the app stores are cesspools. We need a culture shift to being more selective about installing software. Clearer permissions, better architecture, and trusted repositories with reputations to maintain. Installing software without any validation of safety should be possible but scary (show a terminal log reporting useful technical information or something).
Permissioning is a mess – among other things of course. I feel that permissions to access any resource(image, location, files etc) could be given without necessarily giving access to the PII value that resource holds, e.g. running a no side effects function on it on-device or on a trusted service that is readonly to me and write-only to 3rd parties.
Please, side-loading is usually an annoying involved process targeted at developers which the average consumer cannot accidentally do.
A bad actor would have better luck telling a victim "just ship me your phone and login password for some emergency maintenance" than instructing a user through sideloading an app onto their smart fridge.
I, personally, would be happy if iOS had android-style side-loading where you have to enable developer mode, promise you're not an idiot, and go from there.