If you orphan the object by not having anything point to it, it goes away when "git gc" is run. That happens automatically after about two weeks by default.

Even things like "git add" will create objects stored in the .git folder.