Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Crawlers impact the operations of the Wikimedia projects (wikimedia.org)
13 points by manuelfcreis 3 months ago | hide | past | favorite | 3 comments



Seems like a serious issue many sites have nowadays. I really don't want this solved by Captchas everywhere.


If it comes to it, just block all known IP blocks of cloud providers.


There are likely a number of beneficial bots[0] running from cloud providers. And residential proxies are also widely available for very determined scrapers.

[0]: <https://stuartgeiger.com/talks/2013-08-03-wikisym-levee-brea...>




Consider applying for YC's Fall 2025 batch! Applications are open till Aug 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: