Question: Is there are software which can be easily installed on apache or insid... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		tlogan on Nov 4, 2011 \| parent \| context \| favorite \| on: Ask PG: Are there IP blocks? Question: Is there are software which can be easily installed on apache or inside app to detect crawlers? We use ipban but that is not what we want: we want a system which can easily detected "bad" crawler or "abusing" user and ban them for some time. As of now, we have a simple script going thru apache logs and sending list us list of IP and their activity.

cft on Nov 4, 2011 [–]

We use fail2ban (available as ubuntu package) very successfully for this. You can point it to apache log and finesse the rules by browser string, URL or whatever you want.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact