These are basically a community book exchange for rich people (when they have any selection at all), agreed. But there's a good reason they only exist in places where the book cost is immaterial -- if you put these in many poor neighborhoods people would just take all the books and sell them.
This has been my observation as well: I see these often in slightly more affluent neighborhoods, not at all in less well-off neighborhoods. It doesn't help that LFL's are much harder to get going (approval from management) in apartment complexes. For this reason I take all my books to local laundry mats: most have a book repository area with a sign to encourage swaps. Since laundry mats are often used by apartment dwellers without built-in laundry facilities, it at least covers some of that factor.
Regardless though, even if LFL tends towards more well-off neighborhoods, that doesn't take away from the merit of the concept. It just means that book access for other areas of the socioeconomic ladder need different solutions.
You realise anyone with a street facing yard can set one up right? Including any resident of any neighbourhood, regardless of wealth or ethnicity. You literally just need a box and some books. Both of which are widely available for free or a bit of dumpster diving. Why the victim mentality?