My entire point here is that one can, in fact, reasonably claim to "understand" a system without being able to model its high level behaviors. It's not a mistake; it's disagreeing with you about what the word "understand" means.
For the sake of this conversation "understanding" implicitly means "understand enough about it to be unimpressed".
This is what's being challenged: That you can discount LLMs as uninteresting because they are "just" probalistic inference machines. This completely underestimates just how far you can push the concept.
Your pedantic definition of understand might be technically correct. But that's not what's being discussed.
That is, unless you assign metaphysical properties to the notion of intelligence. But the current consensus is that intelligence can be simulated, at least in principle.
Saying we understand the training process of LLMs does not mean that LLMs are not super impressive. They are shining testiments to the power of statistical modelling / machine learning. Arbitrarily reclassifying them as something else is not useful. It is simply untrue.
There is nothing wrong with being impressed by statistics... You seem to be saying that statistics is interesting and there for to say that LLMs are statistics dismissed them. I think perhaps you are just implicitly biased against statistics! :p
Is understanding a system not implicitly saying you know how, on a high level, it works?
You'd have to know a lot about transformer architecture and some reasonable LLM specific stuff to do this beyond just those basics listed earlier.
When it's not just a black box but you can say something meaningful to approximate its high level behavior is where I'd put understand. Transistors won't get you to CPU archiecture and transformers don't get you to LLMs.
