I agree. I think there are potentially multiple solutions to this since there are multiple bottlenecks. The most obvious is probably network overhead when talking to a database. Another might be storage overhead if storage is being used.
Frankly another one is language. I suspect type-safe, compiled, functional languages are going to see some big advantages here over dynamic interpreted languages. I think this is the sweet spot that grants you a ton of performance over dynamic languages, gives you more confidence in the models changes, and requires less testing.
Faster turn-around, even when you're leaning heavily on AI, is a competitive advantage IMO.
It could go either way. Depends very much on what kind of errors LLMs make.
Type safe languages in theory should do well, because you get feedback on hallucinated APIs very fast. But if the LLM generally writes code that compiles, unless the compiler is very fast you might get out-run by an LLM just spitting out JavaScript at high speed, because it's faster to run the tests than wait for the compile.
The sweet spot is probably JIT compiled type safe languages. Java, Kotlin, TypeScript. The type systems can find enough bugs to be worth it, but you don't have to wait too long to get test results either.
I agree. I think there are potentially multiple solutions to this since there are multiple bottlenecks. The most obvious is probably network overhead when talking to a database. Another might be storage overhead if storage is being used.
Frankly another one is language. I suspect type-safe, compiled, functional languages are going to see some big advantages here over dynamic interpreted languages. I think this is the sweet spot that grants you a ton of performance over dynamic languages, gives you more confidence in the models changes, and requires less testing.
Faster turn-around, even when you're leaning heavily on AI, is a competitive advantage IMO.