If your "cost per transistor" calculation includes amortization of the fixed costs of taping out a chip, over the expected production volume, then you can sometimes genuinely end up with newer process nodes being more expensive. Design for more advanced nodes keeps getting more expensive, and mask sets keep getting more expensive. Even more so if you're pricing out a mature process node compared to early in the production ramp up of a leading edge node.
There's significant demand for older process nodes and we constantly see new chips designed for older nodes, and those companies are usually saving money by doing so (it's rare for a new chip to require such high production volumes that it couldn't be made with the production capacity of leading-edge fabs).
Intel and AMD have both been selling for years chiplet-based processors that mix old and newer fab processes, using older cheaper nodes to make the parts of the processor that see little benefit from the latest and greatest nodes (eg. IO controllers) while using the newer nodes only for the performance-critical CPU cores. (Numerous small chiplets vs one large chip also helps with yields, but we don't see as many designs doing lots of chiplets on the same node.)
There's significant demand for older process nodes and we constantly see new chips designed for older nodes, and those companies are usually saving money by doing so (it's rare for a new chip to require such high production volumes that it couldn't be made with the production capacity of leading-edge fabs).
Intel and AMD have both been selling for years chiplet-based processors that mix old and newer fab processes, using older cheaper nodes to make the parts of the processor that see little benefit from the latest and greatest nodes (eg. IO controllers) while using the newer nodes only for the performance-critical CPU cores. (Numerous small chiplets vs one large chip also helps with yields, but we don't see as many designs doing lots of chiplets on the same node.)