Precisely! And neither can generating a handful of unit tests. As EWD would say, they only prove the existence of one error. Not that there are no errors.
If we want more programs that are correct with respect to their specifications we need to write better, precise specifications… not wave our hands around.
However for a lot of line-of-business tasks we’re generally fine with ambiguous, informal specifications. We’re not certain our programs are correct with respect to the specifications, if we had written them out formally, but it’s good enough.
I think most businesses that are writing software that needs to be reliable and precise are not going to benefit from these kinds of tools for some time.
This is true in aerospace software. Lots of process, lots of specification, lots of verification. I wouldn't want to say that GPT-seque tools would be useless here, but I really don't see them offering the same kind of magic leverage that they might offer on some other projects.
And vice-versa! Most software projects do not benefit from the rigor used in aerospace, because it's just not needed, and would be a waste of time.
I am definitely seeing ways that GPT tools could speed up some aerospace work, but we need to be really really sure that things are being done correctly... not just mostly correct, or seemingly correct.
If we want more programs that are correct with respect to their specifications we need to write better, precise specifications… not wave our hands around.
However for a lot of line-of-business tasks we’re generally fine with ambiguous, informal specifications. We’re not certain our programs are correct with respect to the specifications, if we had written them out formally, but it’s good enough.
I think most businesses that are writing software that needs to be reliable and precise are not going to benefit from these kinds of tools for some time.