More like only letting humans into it, and only one type of robot, the googlebot. Then this company is ignoring the posted rules and sending armies of robots into the store to photograph every square inch of the business.

Well I'm sorry, but if I opt to wrap a cURL call in a bash for loop, I'm still a human that tries to access the same resources, only with a different user agent.

There is legitimate individual interest in both scraping and non-browser HTTP sessions.