I mostly do stick to baseline widely available, but once in awhile something can only be done with a niche API unless perhaps I include a 10 MiB, slow, clunky polyfill. And for a hobby site without paying users, I basically just don't care.
Can you name some examples? Are those APIs required for core functionality of your websites or are they limited to features that could be made optional?
