But anyone with enough money can make infra. Maybe not at the scale of Google, but maybe that's not necessary (unless you have a continuous stream of fresh high-quality training data).
If making infra means designing their own silicon to target only inference instead of more general GPUs I can agree with you, otherwise the long-term success is based on how cheap they can run the infra compared to competitors.
Depending on Nvidia for your inference means you'll be price gouged for it, Nvidia has a golden goose for now and will milk it as much as possible.
I don't see how a company without optimised hardware can win in the long run.
The silicon can be very generic. I don't see why prices of "tensor" computation units can't go down if the world sees the value in them, just like how it happened with CPUs.
