The article mentions infection of blood vessels so it seems likely that the fungi make their way into the brain from other sites.

Perhaps the fungi avoid the immune system by being very slow and relatively non-disruptive, and the reduced effectiveness of the immune system in old age allows it to take hold and spread.