No, you are not breaking into my home and stealing stuff. Nevertheless, an analogy can be made between breaking into my home and stealing stuff, and taking my data without consent. "Analogy is a comparison or correspondence between two things (or two groups of things) because of a third element that they are considered to share." - try again to think what the third element could be in this case - I'm sure you can do it!
It's theirs, not yours. Fundamentally, it's not about harm - it's about you getting stuff you have no (moral, cultural, and in many places legal) right to.
As for harm: there is possibility of it, a lot of software does collect data for it to be used against users' interests, and I have no reason to believe yours isn't one of them.
