The displays always seem to be the challenge.

With my Mac Pro, as soon as I wake it, I can see the LED on my monitors are 'alive', but it still takes that same 10s or so to be displaying data.

Also on macOS, if you have an always-on VPN, that can absolutely cause wake time challenges.