I have played with GPT3 and similar lesser models for a while, and in my opinion it doesn't understand anything. It's good at correlation (it might figure out a pikachu is a pokemon or that The Wall is a music album) which is kinda amazing but ultimately seems to be really "useless"* and it's not deterministic.
I think the current "AI" models will never understand a thing unless they become able to update their own models and have some guiding logic to them. Sure it has the context tokens so it might give a little illusion of following along, but the randomness of it all also means it will get wrong stuff that it got right before (and vice versa) and will clearly break the illusion when prodded ("pikachu is a purple dragon", "The Wall was composed by Metallica"). It's unfortunate.
GPT3 and other text models have no sense of state. You can have the same instance work with 2 (or more) different persons and text provided by user A will have no bearing on anything user B does, since it's just trying to complete the provided text in form of tokens. As computationally intensive as it is, the whole thing is a lot simpler than it seems.
It's like we managed to get the "learning" part of intelligence more or less right, but nothing else yet. I always wanted to have an AI "trusted assistant" kinda thing, but even the biggest GPT3 models don't give me much in terms of trust. Wouldn't trust it to turn a lamp on and off as it is. (I know GPT3 is just a text model but you know what I mean). And if it happens, it won't be run locally because the hardware requisites for this stuff are obscene on a good day, which opens a few cans of worms (some of which have been opened already).
*useless as in getting anywhere close to intelligence. It can become decent entertainment and might have some interesting uses to correlate datum A to datum B/C/D, recognition and so, but so far it's just fancy ELIZA. An "educated guess engine" at best.
I think the current "AI" models will never understand a thing unless they become able to update their own models and have some guiding logic to them. Sure it has the context tokens so it might give a little illusion of following along, but the randomness of it all also means it will get wrong stuff that it got right before (and vice versa) and will clearly break the illusion when prodded ("pikachu is a purple dragon", "The Wall was composed by Metallica"). It's unfortunate.
GPT3 and other text models have no sense of state. You can have the same instance work with 2 (or more) different persons and text provided by user A will have no bearing on anything user B does, since it's just trying to complete the provided text in form of tokens. As computationally intensive as it is, the whole thing is a lot simpler than it seems.
It's like we managed to get the "learning" part of intelligence more or less right, but nothing else yet. I always wanted to have an AI "trusted assistant" kinda thing, but even the biggest GPT3 models don't give me much in terms of trust. Wouldn't trust it to turn a lamp on and off as it is. (I know GPT3 is just a text model but you know what I mean). And if it happens, it won't be run locally because the hardware requisites for this stuff are obscene on a good day, which opens a few cans of worms (some of which have been opened already).
*useless as in getting anywhere close to intelligence. It can become decent entertainment and might have some interesting uses to correlate datum A to datum B/C/D, recognition and so, but so far it's just fancy ELIZA. An "educated guess engine" at best.