Seems like a middle ground solution might be to restrict the display of downvotes to videos that meet a certain viewership threshold. That way organic voting would make brigading more difficult.
Steam has a useful chart which shows the votes/reviews over time with the ability to filter by language, purchase type, play time etc.
Youtube could implement something similar to help shine a light on brigading
The steam review graph is brilliant and an example of getting it right. YT seems to have succumbed to it's incumbent position in the market and has stopped innovating - it now seeks only to maintain the status quo and apparently thinks dislikes are a threat to that.