Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

If the LLM tells the untrained passer-by to do a tracheotomy, it's not going to go well.


Emergencies can freak people out but not once in my eight years in rescue have I ever encountered a scenario where a random bystander might do as drastic of an intervention as as a tracheotomy.

I have shown up at scenes where people have googled what to do though and, you know what, it was super helpful.

If someone is dumb enough to perform a tracheotomy because an llm, google, or a passerby told them to. The issue isn’t any of those factors. That person is just so incredibly dumb as to be a danger to everyone around them.


I've been a firefighter for 22 years. I'm sure neither of us will ever cease to be amazed at what otherwise intelligent people will do when they're in a panic.

People also do amazingly dumb things because a piece of software with a tone of authority told them to do it, even when they're not under duress. Look at the number of people who find themselves stranded or dead because they uncritically followed the directions of a navigation app, and who weren't in a panic state when they did it.




Consider applying for YC's Fall 2025 batch! Applications are open till Aug 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: