I think you're getting too distracted with the analogies. LLMs have been shown to spit out easily code for patterns that are very common such as todo apps. That isn't very impressive in itself.
LLMs are cool and show some interesting emergent properties, but knowing why they work getting them to make a very common type of app doesn't show a great deal of emergent ability beyond searching a compressed space and reproducing a common pattern found in it.
It's easy to underestimate the amount and variability of the training data used. There are possibly tens of thousands of variations on todo apps available on github alone.
So be creative and ask it to make something that isn't a Todo app with thousands examples on github. Knowing that it's really really really advanced autocomplete based on matrix math that does have some funny edge cases because of a numbering subreddit is an interesting degenerate case, but the sheer vastness of the training data it has picked up makes it able to dig into a simultaneously deep and shallow Lego box. it does fall over and go in circles, and having learned to program before the Internet was mainstream, I'm able to go into the code a fix it manually, doesn't impeach it's ability to get that far in the first place.
if it's only able to do the first 90% of the work, and I have to do the last 90% of the work, it's still saved me from doing that first 90% of the work.
I may be wrong but I coming to the conclusion that the promise that a piece of software can generate any piece of software you can possibly desire and describe is basically saying that P=NP and the halting problem can be solved by a Turing machine that approaches infinite speed.
I'm not claiming that it can generate any piece of software you can possibly desire, but that there are enough examples of quite a lot of pieces of them, and that it can compose them into something that it hasn't directly seen before. Like an astronaut on a horse that was popular for Stablediffusion. It didn't have that directly in the training data but it had enough of the pieces that it's able to create a reasonable looking version of one. That it produces too many fingers on hands and has no concept of writing is a glaring obvious shortcoming, just like LLMs generating buggy inefficient code is another one. Catching it's fuckups is missing the point.
My point is that it's such a game changer that you ignore it at your own peril. Just go into a side project, half-cocked and get shit built. Yes the code will be ugly but it got built. Maybe. It has its limits, as does the operator's patience, so it's entirely possible you'll run into a bug it can't fix. But a smart operator knows when to stop it and dig into the problem and fix it manually.
Funnily enough though, if you give it some toy code that doesn't ever complete, like a Fibonacci number generator and ask it if it will halt, it's able to point out that it won't. That, of course, is because those are in the training data, but it's cute nonetheless.
But I think if you know how to code well you can use code generators, whereas if you know how to use code generators you don't necessarily learn how to code. And being able to review generated code stems from mastering code, not from fiddling with prompts.
More generally, I think the premise that using code generators is a skill is empty. They don't require special skill beyond understanding programming well so it is probably a better use of my time to keep on coding and if someone wants to give me money to use code generators I am in a better position having spent lots of time writing code.
And if code generators become so good that it requires no skill to create software, then all this is moot. But of course I think that is nonsense because it seems to me that depends on the foundations of computer science being wrong.
Hm, that's fair. I suppose I should be more transparent as to my bias. I learned to program as the Internet was coming online and before Google, Stack Overflow, and now, ChatGPT. Not trying to brag, merely trying to show that I know how to code.
I can't say how I'd actually feel if I was just starting out, but your position isn't unreasonable. I will disagree and say there is a certain skill to prompting, even though calling prompt engineering is maybe ridiculous. Given the way the industry is trending, getting code generators into a corner; having it run around in circles, and then having in skill to fix it by hand, quickly, is the skill to hone. Whether that comes from practicing writing code from scratch, or from using code generation and then fixing the bugs it creates though, is up for debate. I obviously think the latter but that's just a random person who's not in your shoes' opinion.
Best of luck to you in your career! Hopefully it goes well, no matter which direction it takes.
LLMs are cool and show some interesting emergent properties, but knowing why they work getting them to make a very common type of app doesn't show a great deal of emergent ability beyond searching a compressed space and reproducing a common pattern found in it.
It's easy to underestimate the amount and variability of the training data used. There are possibly tens of thousands of variations on todo apps available on github alone.