But I tend to instinctually (as a mere human) think of a "hallucination" as something more akin to a statement that feels like it could be true, and can't be verified by using only the surrounding context -- like when a human mis-remembers a fact on something they recently read, or extrapolates reasonably, but incorrectly. Example: GPT-5 just told me a few moments ago that webpack's "enhanced-resolve has an internal helper called getPackage.json". Webpack likely does contain logic that finds the package root, but it does not contain a file with this name, and never has. A reasonable person couldn't say with absolutely certainty that enhanced-resolve doesn't contain a file with that name.
I think a "mistake" is classified as more of an error in computation, where all of the facts required to come up with a solution are present in the context of the conversation (simple arithmetic problems, "how many 'r's in strawberry", etc.), but it just does it wrong. I think of mistakes as something with one and only one valid answer. A person with the ability to make the computation themselves can recognize the mistake without further research.
So hallucinations are more about conversational errors, and mistakes are more about computational errors, I guess?
But again, I agree, it gets very difficult to distinguish these things when you dig into them.
The reason it gets very difficult to distinguish between the two is that there is nothing to distinguish between the two other than subjective human judgement.
When you try to be objective about it, it's some input, going through the same model, producing an invalid statement. They are not different in no way, shape or form, from a technical level. They can't be tackled separately because they are the same thing.
So the problem of distinguishing between these two "classes of errors" reduces to the problem of "convincing everyone else to agree with me". Which, as we all know, is next to impossible.
But I tend to instinctually (as a mere human) think of a "hallucination" as something more akin to a statement that feels like it could be true, and can't be verified by using only the surrounding context -- like when a human mis-remembers a fact on something they recently read, or extrapolates reasonably, but incorrectly. Example: GPT-5 just told me a few moments ago that webpack's "enhanced-resolve has an internal helper called getPackage.json". Webpack likely does contain logic that finds the package root, but it does not contain a file with this name, and never has. A reasonable person couldn't say with absolutely certainty that enhanced-resolve doesn't contain a file with that name.
I think a "mistake" is classified as more of an error in computation, where all of the facts required to come up with a solution are present in the context of the conversation (simple arithmetic problems, "how many 'r's in strawberry", etc.), but it just does it wrong. I think of mistakes as something with one and only one valid answer. A person with the ability to make the computation themselves can recognize the mistake without further research.
So hallucinations are more about conversational errors, and mistakes are more about computational errors, I guess?
But again, I agree, it gets very difficult to distinguish these things when you dig into them.