My previous comment was downvoted to at least -3 and it rose up again to 1. I think that proves that with enough eyes a good comment will at least get back to 1. Most comments occur on highly trafficked posts, which means most comments should be seen by enough eyes.
Again, in the grand scheme of things one up vote or down vote is relatively unimportant.
it's not about the final score the comment gets, it's about the amount of influence early in the story.
edit: Also, I kicked off the downvoting when you were on 1 point, as an experiment. I wonder if the other 3 people who downvoted to get you to -3 would have done so if I hadn't initially put you at 0. Also, clearly three other people upvoted you. If I hadn't put you at 0 and then 3 others followed my lead, would you now be at +4?
It makes a big difference in the perception of opinion. Simply by initially putting you at 0, I was able to make it seem as though your opinion was worth less than mine. The difference between a +8 comment and a +1 (or -3) comment is much larger than the difference between a +8 and a +4.
Again, in the grand scheme of things one up vote or down vote is relatively unimportant.