I use htmlunit for scraping, it's a headless browser although using WebKit will ... | Hacker News

Hacker News new | past | comments | ask | show | jobs | submit

login

wslh on March 22, 2011 | parent | context | favorite | on: PhantomJS - minimalistic headless WebKit

I use htmlunit for scraping, it's a headless browser although using WebKit will be far better.

It's sad that WebKit lacks some easier integration (good COM/.NET object in Windows).

vitovito on March 22, 2011 | [–]

Crowbar is a Gecko-based scraper, if you're interested in using a real browser: http://simile.mit.edu/wiki/Crowbar

wslh on March 23, 2011 | | [–]

The issue is using Crowbar in latest Gecko versions.

ojbyrne on March 23, 2011 | [–]

I've used htmlunit as well, basically because it seems to be the most mature. I find it chokes on some google maps js.

Consider applying for YC's Spring batch! Applications are open till Feb 11.
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact