I agree, there is a small subset of such files where small pieces (or even the whole) can be compressed. If we just say "images" or "videos" there are also "uncompressed" variants of such, but such images or videos are always those which aren't even prepared to be on the web. E.g. "uncompressed AVI" or "Windows BMP." Those prepared for the web (e.g. MP4 or PNG) certainly aren't losslessly compressible.
Those that want to deliver the videos or pictures should use the proper format and not depend on Brotli or zlib.
All these can't be compressed more than they already are, forcing any lossless (like zlib and brotli) compressor on them is just the loss of CPU time.
Try and compare.
Compression 101.