Discord is for gamers and quite a lot of people will be playing a game and tell someone "add me on discord my tag is xyz". Not allowing invites would seriously cut into the usability.
I'm... not actually clear on what those reasons are? For the adder, the experience is exactly the same - the only difference is that there's no longer an adder and an addee - instead there are two adders.