Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
wsh
on April 1, 2019
|
parent
|
context
|
favorite
| on:
“Google just started mass banning/limiting Archive...
Except archive.org
doesn’t
obey robots.txt files any more [1], and they also ignore requests to remove content.
[1]
https://blog.archive.org/2017/04/17/robots-txt-meant-for-sea...
0815test
on April 1, 2019
|
next
[–]
They don't obey robots.txt files
posted after-the-fact by domain hoarders that have zilch to do with the original content
. This is entirely proper on their part.
superkuh
on April 1, 2019
|
prev
|
next
[–]
Archive.org is not the archive team.
trevyn
on April 1, 2019
|
prev
[–]
I doubt they ignore DMCA requests.
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search:
[1] https://blog.archive.org/2017/04/17/robots-txt-meant-for-sea...