Same. It feels like a passive-aggressive elitism, and not simply an anti-abuse feature.
Given the site we're on, I don't have a problem as much with the elitism (I'm not the target demographic, after all -I'm older and not working in tech) as much as with the "passive-aggressive" part of the rhetoric behind withholding downvote capability.
It IS an anti-abuse / anti-sockpuppet feature. I can see why you think it is elitism since, in a way, it is. But I don't understand why you think it's "passive aggressive".
You should be able to downvote comments, since you have more than 501 karma. The analog for downvoting submissions is the "super downvote" of flagging (and in turn vouching), which requires 31 karma.