A high rate of comments without upvotes degrades a story's ranking, you seem to have done the right thing. I don't know exactly how much flaming there has to be before that kicks in, though.