How much original thought is going into the floor plan of the next McMansion to be built? Or the next 3-storey steel-framed office block? Are these truly developed from scratch by deeply original thought every time? I admit I have no experience of the building industry, but I seriously doubt it. I believe they are created by following standard recipes and checklists (square feet per desk, maximum distance to restrooms, maximum distance to window). I work on a business park full of 3 storey office blocks. They are all minor variations on the same theme. I live in a housing estate where all the houses are also minor variations on the same theme. Thats what I'm talking about. Maybe architects don't design these buildings. Maybe I'm talking about a more junior part of the construction industry. Either way, it completely lacks any originality.
I think there are software analogs of your office park.
You hit on a good insight in the essay: we don't know how we program. But I think you incorrectly assumed that this is unique to software development. Instead, I think it's an inherent part of any job that requires problem solving.
I work in an office that has a doorway that opens onto a brick wall. Fortunately they knocked a new doorway into another wall before walling up the old one, but it's a bit of a hack.
Even architects find it hard to plan for growth it seems :)
Instead of comparing that to software written, compare it to software used. My copy of MS Office looks a lot like my neighbor's, my uncle's, my father's brother's nephew's cousin's former roommate's. While there are some unique combinations, the pieces are largely the same.
The McMansions and office blocks aren't that different from 95% of people using Windows or Mac. It's just more painful in architecture because it's so visible.
I think the ratio of meaningful architecture to buildings is about 1:10,000, maybe even more. To make architecture, almost exclusively original thought is needed. I spent most of my University days unlearning preconceptions about solutions to building problems for that very reason.