Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
theamk
on April 27, 2024
|
parent
|
context
|
favorite
| on:
So you want to scrape like the big boys (2021)
Google benefits from legal scraping - ban them from robots.txt and they'll stop.
Please don't mix consensual and non-consensual scraping, the difference is huge.
Consider applying for YC's Fall 2025 batch! Applications are open till Aug 4
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search:
Please don't mix consensual and non-consensual scraping, the difference is huge.