I am just logically arguing here, not in support for anyone. But hasn't Facebook... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		xtacy on June 17, 2010 \| parent \| context \| favorite \| on: Facebook employee responds on robots.txt controver... I am just logically arguing here, not in support for anyone. But hasn't Facebook worked "hard" to setup the infrastructure to "collect" users' data? If you want users' data, go ask the users; not via scraping Facebook.

curio on June 17, 2010 [–]

but their robots.txt says that you're welcome to scrape it:

  User-agent: *
  Disallow: /ac.php
  Disallow: /ae.php
  Disallow: /album.php
  Disallow: /ap.php
  Disallow: /feeds/
  Disallow: /o.php
  Disallow: /p.php
  Disallow: /photo_comments.php
  Disallow: /photo_search.php
  Disallow: /photos.php

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact