Modern CPUs have a "return address stack" which basically mirrors the real stack and allows them to perfectly predict returns (for normal code anyway).

First explanation I found on Google. Haven't read it:

https://one2bla.me/cs6290/lesson4/return-address-stack.html