Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

There are definitely at least two different, shall we say classes of crawlers: The Googlebot and the Google Web Preview crawler. I don't know the extent of Googlebot's javascript parsing but the web preview crawler appears to parse javascript like plain webkit.

Here's a link where they show the UA for the Google Web Preview crawler: http://www.webmasterworld.com/search_engine_spiders/4353651....

I believe this crawler renders your pages for the preview snippets you get in search results when you hover over the arrow that appears on the right side of a result.

The previews I've seen would only look that way if javascript was being rendered and allowed to run for ~10-20s by my estimation--based on the progress of an animation that was previewed.



Consider applying for YC's Winter 2026 batch! Applications are open till Nov 10

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: