Well, there are huge downsides to using cocaine, whether it is undesired health impacts, or addiction, or threat of arrest, or mere cost, or even just social stigma.

I'm not sure there are downsides to living out your life in a simulation while robots take care of your physical form.