Interesting, I have a similar ability but for code. I write some of the best code when I'm not at a computer because it doesn't take any time to refactor and I can play freely. A lot of the time when I write new code it's just transcribing the systems I created from memory. It's similar for reading code, being able to keep a lot of system complexity and behavior in my head at once. I wonder how common this is for mastery in other circumstances, like sports or art.
I know what you mean, I also "see" code in a similar way as the OP author explains numbers.
It's though mostly "blocks" that interact with other "blocks" and a large application is comprised of probably hundreds of blocks organised in specific shapes with interaction lines between them.
This helps me spot "poor" application design when blocks that should be separate are actually intertwined (tightly-coupled or concerns not separated).
It's sometimes hard to describe these in architecture documents or PR's as it seems not everyone is seeing the program on this level.