When writing software, the goal is often to take some task which a human once did manually, and to partially or completely automate it. I automate as much of my daily work-flow as possible. If I could, I would automate myself right out of writing software all together.
I think this attitude explains why I loath the employee-employer fixed pay relationship so much. Selling your time, day-in, day-out, to write software, is antithetical to the very act of writing software, because, as the developer, you haven’t automated yourself out of any future work.
I think that the reason for what you describe is essentially a paradigm mismatch. Automation aims to reduce the amount of work while businesses have the goal of making as much money as possible and, by extension, since money can only be acquired by working, essentially attempt to automate as little as possible. Personally, I think that this is an old and outdated way of living. It's the same thing that simple biological organisms do - go towards pleasure and away from pain. It explains why, for example, so many software houses only look at the short term and don't take time to design decent systems which are adaptable to change, even though they know that requirements are always shifting. I call this approach 'dumb-lazy', whereas the automation approach is 'smart-lazy'.
"businesses have the goal of making as much money as possible and, by extension, since money can only be acquired by working, essentially attempt to automate as little as possible."
What? If a business gets paid $10 for something that costs them $9 to do, they have lots of profit incentive to automate the work. If they can make it cost $0.01 to do, they will be rolling in profit. Nearly every business understands this.
Maybe they don't want their clients to get this automation, but that's a different question.
I completely agree with your last point and half-agree with your first one. If we were talking about Vulcans then, yes, the optimizations would always be implemented. My personal experience is mainly with technology companies and in that field, at least, the optimizations are more often not implemented, even if they will indeed save money in the long run. I think that these problems are reflected in the general population where only a fraction of people make investments and the vast majority tend to chase after quick profit rather than lasting wealth. Business has a higher percentage of the latter but there is still plenty of the former.
Edit: Also, we might be talking about slightly different situations, I'm mainly referring to optimizations which require a time investment up front, not 'free' ones.
That doesn't explain software house behavior at all. Development organizations take shortcuts and intentionally incur technical debt in order to hit the market window. If you don't have short-term sales success then there's no revenue to fund long-term improvements.
And how can you design "decent systems" that are adaptable to change? It's impossible to reliably predict what changes will be needed. Most software products evolve in entirely unpredictable ways. There are some general software engineering best practices for modularity and layering but beyond that you're getting into YAGNI territory.
I think that a mature organization has enough of a reputation to not need to do that. This approach to running an agency is necessary for the bootstrapping phase but beyond that it will become a burden very quickly.
Best practices are pretty much what I'm talking about. It may be impossible to predict what changes are needed but it's easy to predict that there will be changes.
I think this attitude explains why I loath the employee-employer fixed pay relationship so much. Selling your time, day-in, day-out, to write software, is antithetical to the very act of writing software, because, as the developer, you haven’t automated yourself out of any future work.