Changing Firefox's UserAgent to Chrome's, results in a speed-up. Because when the useragent isn't Chrome, YouTube checks for feature availability, whilst on Chrome, they just assume it is all there.

Thanks for giving me a specific example. At this point in the evolution of web standards and their implementations, that kind of UA check is stupid enough that I can believe there is at least some level of intent to not also making assumptions about available features on other browsers.