VPS (even with a super basic CMS ready set up) and managed web hosting with integrated editor are completely different products. If you want to make comparisons,
look at WordPress.com, 1mb.site, Wix, etc.
How so? I can have a preinstalled WordPress with a quality theme such as Astra with Elementor and I am done in less than 5 minutes and $4. It is pure laziness not doing it this way and waste of money/time. Not to mention unreliable as you trust another company not to jack up their price. If a price change happens to a VPS you simply move to another cheap hosting for a $3-5 per month with your own WordPress.
OVH also has SoYouStart which is around $2 per month. I also remember getting yearly commitments and under $3 per month deals. You have to subscribe to their email blasts to get these deals but $3-4 per month is average for the industry.
There is also Digital Ocean which is around $4 per month. I've been using them for over a decade. Not even one time down. Sometimes I see deals under $3 per month. https://www.digitalocean.com/pricing
What does it have to do with S3? Average webpage with JS will weight at about 200kb per initial load, without images. Do the math how many page views and indexing bots will consume before the real user can even find the page.