With Skype I can play games with a group and put the mic in always on mode. No feedback, no audio from speakers, and no keyboard clicks. It's pretty damned amazing to be honest.
Ventrilo and Mumble do have far superior support for massive number of people in a chat room. If you're participating in large MMO raids or Eve Online type festivities the Skype doesn't have the necessary features. For my ~5 player parties Skype blows everything else out of the water and isn't even close.
I don't think it's about compression. I think Skype's invested most of their resource into, eg. noise and echo cancellation, reducing environment noise, working around glitchy / clicky sound capture devices, latency, ...
I have about 5 guys that play, and the "chat room" design of Mumble makes a lot more sense for our use case (which could be different from yours). We hop in the server whenever we want, just to hang out, then use it for games when we are all in.
It's also secure, FOSS, cross-platform, and implements a consistent UI. Skype has none of these.