In other words, it's the bureaucracy of the environment that's caused this? This is not uncommon amongst other file formats and implementations too, where those which were created by large organisations or multiple groups also tend to be the most complex ("design by committee").
Or alternatively, someone could just decide "let's use this bit, no one else seems to be", and everyone else who didn't think of the idea first can follow along.
But then you have a problem: because anybody could make use of some currently unused header bit, anybody else could at the same time decide to make use of the same bit for a totally different purpose, and chaos would ensue - therefore everybody has to be prudent and avoid creating such potential conflicts.
Or alternatively, someone could just decide "let's use this bit, no one else seems to be", and everyone else who didn't think of the idea first can follow along.