Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

https://localghost.dev/robots.txt

User-Agent: * Allow: /



I contacted the author, she said because no-one respects it, she hasn't even tried.


Not like anyone respects that anyways.


Also, I wanted tldrbot to summarize this page. /s


That's a good point. It's not a black and white issue.

I personally see a bot working on behalf of an end user differently than OpenAI hoovering up every bit of text they can find to build something they can sell. I'd guess the owner of localghost.dev doesn't have a problem with somebody using a screen reader because although it's a machine pulling the content, it's for a specific person and is being pulled because they requested it.

If the people making LLM's were more ethical, they would respect a Creative Commons-type license that could specify these nuances.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: