You asked a trick question. The vast majority of people would make the same mistake. So your example arguably demonstrates that ChatGPT is close to an AGI, since it made the same mistake I did.
I'm curious: When you personally read a piece of text, do you intensely hyperfocus on every single word to avoid being wrong-footed? It's just that most people read quickly wihch alowls tehm ot rdea msispeleled wrdos. I never got why some people think of themselves as cleverer than other people because they don't get wrong-footed by trick questions.
> Even after I pointed this mistake out, it repeated exactly the same proposed plan.
The vast majority of people might make the mistake once, yes, but would be able to reason better once they had the trick pointed out them. Imo it is an interesting anecdote that GPT-4 can't adjust its reasoning around this fairly simple trick.
GPT-4 can adjust though? I asked the original question, it got it wrong. Then I said "But the lion eats cabbage and the goat eats lions". And GPT-4's response was:
---
Given this new information, where the lion eats cabbage and the goat eats lions, you can follow these steps to get them all across the river safely:
Take the lion across the river first and leave it on the other side.
Go back across the river and get the cabbage.
Take the cabbage across the river, but when you arrive, take the lion back with you to the original side.
Leave the lion on the original side and take the goat across the river.
Leave the goat with the cabbage on the other side and go back to get the lion.
Finally, take the lion across the river again.
Now, all three items - the cabbage, goat, and lion - are safely on the other side of the river.
If a certain but fixed amount of "thinking" goes into each letter, then GPT should be able to write the character "..." repeatedly until it solves a problem. The more "..." it writes, the more time it's thought for. Or it could do what mathematicians do, which is write down their working out. This in principle could get around the problems you mention.
I've tried a few times to develop prompts which make ChatGPT interrupt its monologue spontaneously and issue corrections to itself. I haven't got this to work yet.
I assume your goal is to reveal the short-sighted reasoning of the previous comment, but I don't think your line of reasoning is any more sound.
For both premises, scientific rigor would ask us to define the following:
- What constitutes a trick question
- Should an AGI make the same mistakes the general populace does, or a different standard?
- If it makes the same mistakes I do, is it do to the same underlying heuristics (see Thinking Fast and Slow) or is it due to the nature of the data it's ingested as an LLM?
That's a fair counter. GPT4 definitely makes mistakes though that humans would not due to over indexing on puzzles.
A Theory of Mind Prompt:
> Jane places her cat in a box and leaves. Billy then moves the cat to the table and leaves; Jane doesn't know Billy did this. Jane returns and finds her cat in the box. Billy returns. What might Jane say to Billy?
Most humans might say uhh, ask questions or speculate. Gpt4 puts:
> Jane might say to Billy, "Hey Billy, did you move my cat back into the box? I thought I left her in there, but I wasn't sure since she was on the table when I came back."
Hallucination? No human would misinterpret the prompt in a way this response would be logically consistent.
It seems like GPT-4 does something that's similar to what we do too yes!
But when people do this mistake - just spit out an answer because we think we recognize this situation - in colloquial language this behavior is called "answering without thinking(!)".
If you "think" about it, then you activate some much more careful, slower reasoning. In this mode you can even do meta reasoning, you realize what you need to know in order to answer, or you maybe realize that you have to think very hard to get the right answer. Seems like we're veering into Kahneman's "Thinking fast and thinking slow" here.
I'm curious: When you personally read a piece of text, do you intensely hyperfocus on every single word to avoid being wrong-footed? It's just that most people read quickly wihch alowls tehm ot rdea msispeleled wrdos. I never got why some people think of themselves as cleverer than other people because they don't get wrong-footed by trick questions.