I reckon that the format of the site caps how large of a community it could build, and its (well-earned) reputation for being the dregs of the internet has continuously selected and pushed new people meeting that description in (forcing others out). The result is distillation. As the internet gets bigger and bigger, 4chan gets worse and worse.
Combine with that the fact that anonymity combined with a relatively small community (relative to, say, Reddit) creates the perfect grounds for false consensus building, and a real echo chamber forms.
I'd love to understand the sociology behind the change in vibe that happened there.