You're not wrong, but until you actually play with ChatGPT yourself, you just don't understand how _dumb_ it is.

All people see is the cheating, and possibly this scary new AI that's going to get smarter than humans in a short period of time.

I suspect the best way to educate people on both the powers and limits of the technology is to get them to sit down for 15 minutes with it.

When I read this I feel people must be using it in the wrong way. I use it all the time to quickly solve tech problems I mostly know something about, however it’s so smart it regularly takes 1-2 hour problems for me and turns them into 10 mins ones. That is definitely not dumb from my perspective, but obviously it’s also not smart in it will give me profound understanding of something, but ok whatever, it’s still a massive productivity booster for many problem.

When you call it dumb, what do you mean? Can you give some examples?

Please don’t give computational examples we all already understand it does inference and doesn’t have floating point computational capabilities or reasoning, and so many give such examples for some silly reason.

It's dumb in the sense that it doesn't actually have a symbolic understand of what it's actually saying.

I use it quite frequently too, mostly for solving coding problems, but at the end of the day it's just regurgitating information that it read online.

If we took an adversarial approach and deliberately tried to feed it false information, it would have no way of knowing what's bullshit and what's legit, in the way that a human could figure out.

A lot of people who've never used ChatGPT make the mistake of thinking it has symbolic reasoning like a human does, because its language output is human too.

How many book mistakes have you found, as a human, so far? How about deliberate mistakes hidden in plain sight? I once re-validated for 3 times the same test set and was still finding mistakes.

>How many book mistakes have you found, as a human, so far?

All the time. Probably every single day I read something and say "that's clearly bullshit".

Agreed on getting excellent hints from it, shortening the time to figure out stuff. But eg. just now it gave me an example algorithm that, only at 2nd look, turned out to be complete nonsense. You know That colleague, who shines in the eyes of his managers, but peers know that half what he does is garbage.

Its ability to correct itself is very impressive when given the feedback. I imagine with the proper feedback loop it can advance very fast. E.g., when asked to write a piece of html markup, if it could "see" how the rendered layout is different from what was asked for, it could adjust its solution without human involvement. If it could run the deployment script and see where it fails, it could apply all the fixes itself until it works. If it could run the unit tests and see where its solution breaks the other parts of the system, it would need much less handholding.

And this is the AI effect in practice. We are past the point where the original idea of the Turing test has been met by machine intelligence. We are at that point.

The problem with people is we keep pushing it to "Only AGI/machine superintelligence is good enough". We are getting models that behave closer to human level. The 'doesn't know everything, is good at somethings, and bullshits pretty well'. Yea, that's the average person. Instead we raise the bar and go 'well it needs to be a domain expert in all expert system' and that absolutely terrifies me that it will get to that stage before humanity is ready for it. This is not going to work well trying to deal with it after the fact.

Sure, but in our present reality those tests and interviews are how we currently gatekeep upper middle class jobs, so it is at least of some practical interest.

Also, I think this is a bit overstated. Programmers (and smart people in general) like to think that their real job is high level system design or something, and that "mere regurgitation" is somehow the work of lesser craftsmen. When in reality what GPT shows is that high-dimensional regurgitation actually gets you a good fraction of the way down the road of understanding (or at least prediction). If there is a "buried lede" here it's that human intelligence is less impressive than we think.

While I agree with this sentiment, I think we should be careful assuming that our jobs as knowledge workers are much more than “retaining and regurgitating information”. Even the emotional intelligence and organizational strategy portions of what we do may boil down to this as well.

> all they're proving is that tests, and interviews, are bullshit constructs that merely attempt to evaluate someone's ability to retain and regurgitate information

No, all they are proving is that either tests are bullshit constructs, or ChatGPT is human-level.

The ability to retrieve and then synthesize the retrieved information into an answer tailored to the question is completely new. The applications of this go far beyond passing an interview, its a fundamental capability of humans that gets used everyday at work