While C compilers are technically not "black boxes", if you looked at lawyerly discussions about UB I don't think you can in full conscience say that `gcc -O2` is much better than a LLM.
I already said they are technically not black boxes. We are not in disagreement here.
The fact that code is all there and deterministic isn't sufficient. People use newer versions of compilers and you can't predict what the people who write compilers will do (people are black boxes). The compilers may do something you don't expect but still conform to the spec.
The point is that there is no one implementation of C. And the same is true for any other language too (to a lesser extent).
Unless you're coding against very specific versions of OS and compiler and runtime environment, you don't have code to inspect. You are not inspecting all the underlying dependencies when you write your code. You just assume they work.
It doesn't matter whether they are black boxes if you don't routinely inspect the boxes. Perhaps you do, but for most people they don't. So it doesn't matter as much.
> I already said they are technically not black boxes. We are not in disagreement here.
I’m saying they are not black boxes at all. Not just “technically”
> The point is that there is no one implementation of C. And the same is true for any other language too (to a lesser extent).
That doesn’t matter for my argument.
> It doesn't matter whether they are black boxes if you don't routinely inspect the boxes. Perhaps you do, but for most people they don't. So it doesn't matter as much.
It does matter. That’s my whole point.
For two main reasons:
First, you have the option to understand how a compiler translates C to assembly. You do NOT have that option with LLMs
With a compiler, you are abstracting the underlying machine code, but you’re not hiding it entirely.
Secondly, with an LLM there’s no tractable connection between your prompt and your code. That’s a huge issue when it comes to understanding the resulting program.
Imagine a world where LLMs turn raw language into assembly (or, more realistically, LLVM IR)
How would you reason about your resulting program?
You could give the same prompt to the same LLM and get wildly different resulting code, since LLMs are nondeterministic.
How would you debug? How would you ensure that small changes to your LLM prompt doesn’t change parts of your resulting code that worked well?
How would you even understand what your program is doing?
You couldn’t do any of those things, because there’s no deterministic relationship between the prompt and the resulting program.
LLMs can be deterministic if you set the temp to zero.
It's not feasibly debuggable though, I'll give you that.
On this point it's not really different in principle from having a non-OSS compiler. Using OSS compilers are great because you can look at the source and trace problems and fix them, but people also use closed source compilers in their work too.
