I've never heard this phrase used out of genuine concern for those in poverty. It's just another way of saying "don't make me feel guilty by reminding me that it exists".

The condescending presumption behind the phrase seems to be that those in poverty would actually prefer it if you hid behind your gated community.