*> User-Agent: bender > Disallow: /my_shiny_metal_ass* This was there when I che... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		nitrogen on March 4, 2021 \| parent \| context \| favorite \| on: HTTPWTF > User-Agent: bender > Disallow: /my_shiny_metal_ass This was there when I checked just now; was it removed and re-added?

grishka on March 4, 2021 [–]

Yes I saw that, but the parent comment said about it being a header ;)

hunter2_ on March 5, 2021 | [–]

The way I see it, the parent said bender is in robots.txt, and you said it's not, and then you also said x-moose was in robots.txt :)

You thought you were replying a level higher.

Consider applying for YC's Winter 2026 batch! Applications are open till Nov 10
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact