Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

The rules are that you can't send google different page content than regular browsers, but there's no reason they have to run all the background processes on googlebot requests -- can't they just send it the most recent cached version?


Not a bad idea, but seems like it would be tricky to get right. You do kinda want Google to have the most recent version of a page, all other things being equal.


Agreed, there's a sweet spot somewhere though, and it might not be the same for googlebot as a regular viewer).


Why not put the cached content in the page by default, then do an update via AJAX only in the case where the cached version is old? That way it's not triggered for crawlers. It's probably secondary content anyway.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: