They could also cost the company ludicrous amounts if they built a Frankenstein at the software layer which made a mess of operations, security or performance.
It’s unlikely that a “good” sysadmin / SRE, or even a team of them, could efficiently replicate the versatility and reliability offered by cloud services.
Thats true. But you don't need that versatility or reliability a lot of the time. For a bog-standard webapp that talks to a database, gets decent traffic, and is growing at a reasonable pace (say doubling every year), dedicated hosting or colo will save you a LOT of money.
It’s unlikely that a “good” sysadmin / SRE, or even a team of them, could efficiently replicate the versatility and reliability offered by cloud services.