couple live web demos with similar approaches you can check (!need a webcam for this to work) https://cables.gl/p/nrv_-a https://cables.gl/p/2Zf0Hk

I believe you need immense precision of tracking for it to look decent, cause brain notice all the imperfections and gives you motion sickness