Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

This seems perfectly logical to some people:

Google ignores robots.txt: "Google is evil! They're trespassing on my webserver!"

Google follows robots.txt: "Google search results suck! They're not indexing GitHub!"



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: