My only concern with DRM in browser is that, as far as I understand the technology, any website can put a small piece of arbitrary DRM content and your browser will send information to that website uniq to your device, and your device will be tracked by DRM providers and probably also by that website.

Firefox gives you the option to disable DRM in the settings. Chrome also has it but it is buried deeply in advanced settings. Safari does not seem to have it at all.