It's wrong, full stop. No less wrong than it would be for someone to set up a homeless encampment on property you own without your consent.

Sure, it's (probably) a good cause. It's still wrong.