My brain knows where it's going to go by the end of the sentence as well, it's conceptualized the whole sentence and my hands are on the road to completing the sentence, I'm not only aware of what I've written so far.
> My brain knows where it's going to go by the end of the sentence as well
How would you know that?
Sincere question, because to me it feels like my brain is improvising word by word when typing out this sentence. I often delete and retype it until it feels right, but in the process of typing a single sentence, I'm just chaining words one after another the way they feel right.
In other words, my brain doesn't exactly know the sentence beforehand - it improvises by chaining the words, while applying a fitness function F(sentence) -> feeling that tells it whether it corresponds to what I wanted to say or not.
In my case it's because my internal dialogue is saying the sentence before I get to the end of it. I usually have the entire sentence in my inner dialogue before I even start typing. Will I edit during typing? Sure, but I have a first version in my head before I start.
> I usually have the entire sentence in my inner dialogue before I even start typing
Interesting. Perhaps the question then becomes, does your inner dialogue simply chain the words one after another, or does it come up with sentences as whole?
I have a friend who listens carefully and starts to mouth and even mumble his brain’s predictions for the next words I am going to say. So, I fuck with him by unexpectedly (even to myself) changing my words (or even the sentence) half way through when I catch him doing it. Predictably, I don’t like being predictable.
The fact that ChatGPT does so well is perhaps a sign we do somewhat generate sentences on the fly. Obviously we mostly listen and read sequentially.
One could test it by using an external random bell to signal you should try and make a significant sentence change e.g. from English to Spanish. How much practice would it take?
From 1984: In the middle of Hate Week, the speaker is halfway through a sentence about hating Eurasia, he is given a note, and he continues the sentence except now Oceania is not, and has never been, at war with Eurasia, and it is Eastasia that is now hated.
It's impossible to form meaningful sentences without having a model of the sequence of the words, this is as true for a human as for a markov chain, but that doesn't mean it then follows that human sentences are just sophisticated markov chains, the nature of sentences means that they must be a sequence of words regardless of the implementation details behind the hood of the sentence creator.
Why would it? It would make more sense to start with concepts ("dog", "brown"), and then pull it into words ("That dog is brown") - given that we're trying to convey a thought that seems to be far more likely than us chaining things - we start with the intent to communicate something, the markov chain does not.
Your eyes are telling you what color that shade of red is on the wall. But if you put it next to a different color, you’ll suddenly get a different answer from your eyes.
I think if our eyes can deceive us at a fundamental level, it’s arrogant to think we aren’t deceived by our thoughts.
Well then those who think they are coming up the sentence one word at a time are equally as easy to fool, no? They could have the whole sentence figured out and their attention is only coming to a single word at a time, possibly revising what's in their field of mental view if they come up with a better word. That's the fun with these sorts of doubts, if we assume nothing, then we can form an argument for anything.
> Sincere question, because to me it feels like my brain is improvising word by word when typing out this sentence. I often delete and retype it until it feels right, but in the process of typing a single sentence, I'm just chaining words one after another the way they feel right.
That’s your conscious experience, but it doesn’t necessarily match what your subconscious mind has actually been doing. I’d hazard a guess that it’s thinking several moves ahead, like a good chess player. What you end up being consciously aware of is probably just the end product of multiple competing models - including ones that want to stop writing altogether and go do something else.
I don’t generally think in individual words when writing. When I want to write a sentence, the first thing I think about is the idea that I want to convey. Then clusters of words start to come into my mind, as ways of expressing that idea, and I select and organize those fragments into a sentence, sometimes fine tuning words to avoid undesirable connotations or emphasize desired ones. Finally I glue the tuned fragments together with syntax and connecting words. Throughout the process I’m constantly iterating.
I don’t think that what ChatGPT does is anything remotely like what I do to communicate. But maybe I’m weird.
I definitely just start saying things or hammering out words and then figuring out midway that I need to reword the paragraph at some point or at least choose to sacrifice legibility for my own laziness. It reminds me of improvised music or comedy. You just gotta roll with it and build a structure out of what you’ve already said. Like 60s amphetamine beat poetry epics.
> I'm just chaining words one after another the way they feel right.
I think we know where we are going in a conceptual sense, the words start feeling right because they are taking us to that destination, or not.
If I leave a sentence in the middle for some reason, when I return I often have zero idea how to finish the sentence or even what the sentence fragment means.
Here's a theory. Thoughts exist on certain time scales. Everything about perception exists on certain time scales. The brain wave frequencies we use to represent things is directly related to the time scale of the thing it represents. The end of the sentence which you are trying to reach departed on its internal journey though the brain around the same time as whatever triggered the thought. It just so happened to take a more circuitous route, delaying it in preference of other words until it was ready. The longer the distance of logical relations between parts of the sentence, the larger the relevant time scale, the lower the relevant brain waves used to represent it.
Its like a juggling act. The ball with the conclusion is thrown up highest, a bunch of other balls are thrown up in between, and they should all start arriving back in your hands, in the correct order, one at a time, without having known the exact sequence to expect when they were first thrown.
Sometime the juggler misjudges and the train of thought is scrambled and lost.
> Its like a juggling act. The ball with the conclusion is thrown up highest, a bunch of other balls are thrown up in between, and they should all start arriving back in your hands, in the correct order, one at a time, without having known the exact sequence to expect when they were first thrown.
"experience tranquility"
--zenyatta overwatch
Yeah, that metaphor works. ;) As an extremely ADHD person, every thought comes with extra bonus thoughts (and parentheticals!), and the trick is knowing when to introduce each supporting point without re-introducing concepts needlessly but also try to have my bizarre brain make sense. Internet arguing and trying to preemptively address counterarguments with supporting points has seriously broken my brain and it leads to very longwinded posts. Keeping it short and coherent is specifically something I really have to work at because I love to write and people don't want to read a novel every comment. It's a matter of effective communication though.
Personally the description of the transformer as "writes words and then edits the output as a unit once the words are complete" really describes my writing at both a sentence and paragraph level. I'll go back and edit a comment a ton to try and tune it and clarify exact meaning/nuance with the most precise language I can.
A ton of people read my comments and are like "did an AI write this!?" and yeah only the finest biological neural net.
Another friend described it as "needing to slow his brain down" and perhaps a similar metaphor would be a database pivot - taking sparse facts and grouping them into a clustered dense representation as an argument. It's an expensive operation especially if there's more there than you thought.
I was just trying to do some metacognition and observe how I write, but apparently it's really just word by word. I neither form full sentences, nor full words or even just abstract imagination in my head. The words just appear from the "darkness", with some kind of sophistication postprocessor that tries to make some output more verbose or use more appropriate adjectives. Is this how people with aphantasia live? I don't like it. I expected something more sophisticated. Maybe that's why my writing often appears like a barely connected verbal diarrhoea that looks like an "inner monologue" writing task back in school.
There is definitely something else going on ... if I stop writing and come back to it often the sentence fragment I was writing makes no sense at all. If it was word by word, I'd just start writing again.
Also, I usually "hear" the next segment fragment in my head before I'm typing it.
That's how it looks like of your mind was below your consciousness level, i.e. if your focus was on higher ideas, and you just commanded the mind to produce specific thoughts. That's similar to how we control our body. However you might be at the ideas level already, but unable to see those ideas, so it looks like darkness to you. I believe you shpuld be able to see thoughts again simply by persistently focusing your attention on them, just like you could (but shouldn't) control low-level body functions by switching attention to them. If your consciousness was focused on thoughts, you'd see something like a very active 3d whiteboard filled with fine shapes, that you can create, change and destroy at will.
I have an idea or concept first, then I translate that into analogies/stories, then I put those down as sentences/paragraphs, then I wordsmith to make it flow better.
At no point am I in a mode where I say a word and think "What's most likely to come next?". The concept/idea comes first. Likely I will try different angles until I find what lands with the audience.
ChatGPT works more like a stereotypical extrovert: It doesn't think then output, it uses output to think. Which can be a fine mode for humans too. Sometimes, when you don't know what you're trying to say yet or when you need to verbalize what your gut is thinking.
There's a lot of answers here, but as Feynman said "that which I can't build I don't understand." If you can't make something that writes for you, you don't really understand how you write. That feels impossible, to be able to do something without understanding how you do it. Brain be like that sometimes.
I'd be careful about interpreting that. Can you "build" numbers? Yes, in one sense. No in another. And he doesn't specify whether being able to build something is sufficient for understanding it or merely necessary.
And even when you've built it, what about other ways that it could be built? If you implement binary search iteratively, then perhaps you understand binary search. But do you understand its recursive implementation?
Notice I didn't say anything about chatGPT. chatGPT is at least something that's been built. Disagree with the theory that we are overgrown text prediction all you want, but you can't deny that it is an "understanding" in the sense that knowing it lets you build and test it. Let's call it a "potential understanding". This is different from an "illusory understanding", which feel like explanations but when you go to implement it you realize it was all a word game and your lack of understanding has just been swept under different words or under equally hard to understand concepts. An illusory understanding doesn't even have the potential to be an understanding. You can't use it to build your own version of GPTchat.
All these glib answers people here are giving aren't even potential understandings. They are illusory understandings. They sound nice so long as you don't try to use it as a spec to code up your own language model. If you can't build anything out of an "understanding", even a wrong thing, then you don't truly have an understanding. You have a feeling of understanding.
> Disagree with the theory that we are overgrown text prediction all you want, but you can't deny that it is an "understanding" in the sense that knowing it lets you build and test it.
Yeah, and it helps us understand how "overgrown text prediction" works, but not how humans work. In the same way that building a robotic arm won't help you understand how muscles work.
> If you can't build anything out of an "understanding", even a wrong thing, then you don't truly have an understanding.
Not entirely true. We can't build a star, but we do have a pretty good theory of how stars work.
But I do agree that we don't understand the human brain.
Building a robot arm does help understand how muscles work. By having a robot arm to play with, you can pinpoint exactly the abilities which aren't accounted for by your current understanding. Boston Dynamics built that robot dog thing. It took decades to make robotic kinematics that good. The dog demonstrates an understanding of a system of muscles that solves the same problems as its biological counterpart.
You say that overgrown text prediction isn't how humans work. And full disclosure you're probably right. But me put on my contrarian hat and say that actually you're wrong and that really is all there is to the brain. At what point does my theory break? What types of things can't be done with just overgrown text prediction, and what features are relevant to a system that could do those things? Don't just appeal to intuition and tell me humans obviously don't work that way. Find the actual flaw where the theory breaks down. That is the value of this experiment.
If you can find the words / experiments to demonstrate why overgrown text prediction isn't an accurate understanding of human thought, in the process you will have in fact distilled a better understanding of human thought. Information on how the brain doesn't work is also information about how the brain works.
> Building a robot arm does help understand how muscles work.
I think we're talking about different levels. A robotic arm helps understand the mechanics of an arm, but not cell metabolism, myosin motors, etc. Any understanding of muscles you might get from a robotic arm is superficial.
> You say that overgrown text prediction isn't how humans work.
To be fair, I did say that, but what I meant was that humans don't work the way ChatGPT does. Maybe we do use "overgrown text prediction" but we don't use word vectors, tensor calculus, and transformers.
We know that humans have some pure text prediction ability. People who've seen Mary Poppins can complete supercalifragili... even though it has no meaning. But how? We don't know, even after building LMs.
> What types of things can't be done with just overgrown text prediction
That's a different claim and not one I'm making. What types of things do humans not do with text prediction? Anything that doesn't involve the language processing parts of the brain, at least.
>What types of things do humans not do with text prediction? Anything that doesn't involve the language processing parts of the brain, at least.
Let's separate out the robotics problem from the consciousness problem. Sure the brain solves both, but the things that a computer can't do yet because it has no body aren't fundamental limitations. We can just hook the computer up to a robot body eventually.
So to rephrase, what types of things can the brain of a blind paralyzed person do that text prediction cannot?
>To be fair, I did say that, but what I meant was that humans don't work the way ChatGPT does. Maybe we do use "overgrown text prediction" but we don't use word vectors, tensor calculus, and transformers.
Well, at least not consciously.
Really, the question about the question comes down to which one you care about: figuring out the phenomena of consciousness in general (studying humans as our only accessible reference implementation), or figuring out how human consciousness works in particular. Its easy to conflate the two.
Like the Aliens in Arrival? But seriously I have a general idea of what I’m going to say and then decide the words as I write it, with some edits after the initial draft.
I'm curious how do you write?