Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Earlier this month I was downloading top 1 million websites' robots.txt and some guy just served 100mb of whitespace :/


Once I was trying to scrape from some internal API, and made a request that used parameters the real client wouldn't make. I didn't expect curl to scream "FUCK YOU <some name>" at me. Presumably the name belonged to someone who scraped it before.


LOL, that is great.


I love it!




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: