Hacker News new | past | comments | ask | show | jobs | submit login

There are thousands of pieces of software that hit a URL and attempt to do something useful with the contents. Just try linking to a page on your own site on Twitter and tail your logfiles to see all the interesting bots that show up.

A couple of examples off the top of my head:

If you link to a web page on Reddit, they run a simple crawler to try and extract an image they can use as a thumbnail representation: https://github.com/reddit/reddit/blob/af09fa8dee69bef4f65a16...

If you link to a URL in Slack, they do something similar: https://api.slack.com/docs/unfurling




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: