Would you ban people from saying "just eat healthy to beat cancer"? People have already died from that sort of thing, notably Steve Jobs. It's a free country, and you're allowed to be a dumbass about your personal medical decisions.

Also, ChatGPT has allowed people to get their rare conditions diagnosed, quite possibly saving lives. Is it an unmitigated good because it did that?