The incentives aren't there to perform better. Even after all these failures you mentioned, they haven't really been punished in any significant way whatsoever, so why improve?
Correct. The incentives are only truly there, generally, as an organization, for intelligence agencies to do what retains them capabilities, and their budget.
And when every department is granularly information insulated, it's easy for an individual to get caught in a task that serves the above while not even realizing.