Yes, you can't push cross-origin.[1] However, there's still a lot of use-cases where this is useful, such as if your entire site is static content, or if your app servers are behind the CDN as well.
[1]: Yet. I believe the web packaging standard (intended, among other things, to replace AMP) allows pushing bundles signed by other origins.
[1]: Yet. I believe the web packaging standard (intended, among other things, to replace AMP) allows pushing bundles signed by other origins.