Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

There is an API via which you can describe your URLs (it's called a sitemap) and you can ping to Google your sitemap when its content changes. You can have multiple sitemaps and ping only your changes. More on www.sitemaps.org .

But Google reserves the right to crawl non-sitemaps URLs, for obvious reasons. It would be quite a bad decision for them to restrict their crawls only to API-provided URLs.



We try to give search engines hints with the update frequency in our sitemaps.

We re-build our sitemaps nightly and make sure that new or recently-updated content is listed with an update frequency of "daily" or "weekly" and all other content pages are listed as being updated "monthly."

To be honest, I've never measured if it works, but it can't hurt.


How do you ping only changes? Don't you have to resubmit the whole sitemap?


You can have multiple sitemaps. You can ping just 1 sitemap containing only the links that you want to notify. You can use the optional <lastmod> tag to indicate a URL's last change date.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: