Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

"Google's average ratio of pages crawled to visitors referred was 2:1. Six months ago, that ratio had increased to 6:1. Today, according to the Prince, it's 18:1", that's interesting but are we sure it has something to do with AI?


Compare to "According to Prince, OpenAI's ratio of pages crawled to visitors referred has gone from 250:1 to 1,500:1 while Anthropic's ratio has gone from 6,000:1 to 60,000:1."


I’d be stunned if my sites have as much as 1% non-bot traffic, even outside of the ones that are vaguely straddling the border between bot and Ddos.




Consider applying for YC's Winter 2026 batch! Applications are open till Nov 10

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: