The teacher was a quite old man who was set on his ways. We would have to draw 3D pieces by hand, using our squares, bevels and compasses, and lining everything with an ink pencil afterwards, and he gave us a lot of homework to do, which he examined for imperfections.

I was incredibly bad a it. I would start a drawing, only to discover halfway through it that it didn't fit the sheet of paper I had and I had to change the perspective. My perpendiculars were never perfectly aligned. I would smear blots of ink with my tools and hands.

It was excruciating.

This is the point where I would usually say that I learned to enjoy the repetition and eventually came to appreciate it and be good at it. But that's not what happened.

Instead I started doing my homework using the computer, printing, and then tracing the points by hand. This resulted on cleaner-looking homework and took less time to do. What's surprising perhaps is that it enabled me to appreciate design more, now that I was no longer limited by the drawing technique, which was, I believe, my teacher's real interest (he was a painter outside of school. A very nice person, I didn't enjoy lying to him like that).

You might think that this would prevent me from performing well on exams, where a clean enough drawing (done by hand) was expected. The thing is: I could muster the effort to do a one-off drawing that was clean and precise, from time to time. Just not after every lesson. I actually got the maximum qualification on my province's technical drawing examination.

This is all to say: I agree with the Author. Once the tools are out there, trying to limit them is pointless. You must adapt your curriculum to the environment your pupils are in. If you don't, they will.

Needless to say, I sucked at it, and it was the one and only class that I ever failed. I passed with flying colours the second time around, and still graduated in minimum time, though it badly messed up my schedule in my second year.

I have never forgiven the department for putting me through that, given that AutoCAD existed at that time.

I mean, human beings have produced a lot of tools for aiding themselves in tasks that were previously done by hand. Whether a given hand-skill should be learned and preserved kind of has to decided on a case-by-case basis. I learned cursive in school and I wouldn't object to that skill being utterly forgotten by humanity.

Which is to say that I don't think we should assume all learning should begin without automatic tools at hand or that we should suppress automatic tools. Rather, we need to look carefully at which hand-skills are worthwhile and "uplifting" and how much use of those skills should be required.

I did just that. It wasn't a life drawing class, just a drawing class. And it wasn't homework, but the do-it-at-home final. I photocopied my hand. Then I covered a sheet of paper with ground graphite (turning it all nearly black), and then used an eraser and a light-box to trace my photocopied hand. Did I cheat? Perhaps, but I used tools that I had access to. Did Vermeer cheat? [1] Perhaps, he may have used tools he had access to.

Afterwards, I framed that drawing and hung it up at work. It was stolen. Go figure.

[1] https://en.wikipedia.org/wiki/Tim's_Vermeer

That argument doesn't make any sense. There are restrictions for any homework, some explictly given by their professor (such as the deadline), and some implicitly known from general academic guidelines and common sense (such as "not using a photocopier in an art class"). Obviously mining coal is not among those restrictions.

...is it?

"general academic guidelines and common sense" is a subjective term that is (context, culture, & environment) dependent: If the social norm was "Your work isn't valuable unless you made the materials yourself", then the use of non-self-made materials (graphite not mined by yourself) is frowned upon.

Relying on "general academic guidelines and common sense" is not a good thing, as it helps to reinforce unequal social norms & anti-non-heteronormative ideologies, as a downstream consequence of majority rule by statistically normal people. Neurodivergent individuals that have significant difficulties grasping "common sense" will & already have been negatively impacted under such a system, where they're told to just "get it" with no guidelines whatsoever.

If a neurodivergent person uses a photocopier because they thought that it would be fine, then yeah, I'd understand that. The person who started this thread stated that they were perfectly aware that they were cheating, though.

We don’t give runners roller blades right?

[1] https://en.wikipedia.org/wiki/Oscar_Pistorius

Don’t answer the door.

The context surrounding both scenarios couldn't be more different:

GP's context: An assignment wherein the process of getting there isn't placed on a pedestal, rather that it's supposed to showcase a basic understanding of how to get to a desired result. Furthermore, objectivity within the assignment's field is nearly non-existent.

Your context: A highly-competitive scenario wherein *BOTH* the process of getting to the goal & the goal itself are important, and where *there are* objective metrics that can be used to rank & sort the participants involved in said competition.

They're as different as apples & oranges.

Vermeer may have done effectively that and painted projections (from a camera obscura or similar device). He's still widely considered one of the greatest artists of the Dutch Golden Age.

A modern version of it can be found https://neolucida.com

But I do agree with your overall point, that schools and universities will have to embrace these new tools. Vernor Vinge (who was an SDSU maths professor) wrote Rainbow's End on this very subject.

The answer: "One way to make it clear that a text is not generated by AI is to include a joke or riddle that is difficult for a machine to understand or generate. For example, you could include a joke that relies on wordplay or a complex pun. Here is an example:

Why was the math book sad?

Because it had too many problems."

The thing is, I'd see these answers as similar to everything else the program produces. A bunch of claims from the net cobbled together - I've read a number of Sci-fi novels and stories where "inability to understand humor" is the distinguishing quality of an AI (I'm guessing it extrapolated "hard create" from "hard to understand"). But that doesn't seem to be playing here where the AI mostly running together things humans previously wrote (and so it will an average amount humor in circumstances calling for it).

A reasonable answer is that the AI's output tends to involve this running-together of common rhetorical devices along with false and/or contradictory claims within them.

-- That said, the machine indeed did fail at humor thing time.

And the idea that the computer would “try” to come up with an example that would trick a computer is itself a little funny, in that it has fallen into giving itself a preposterous task.

But it did definitely fail at clever wordplay.

There sure is some obscure discussion forum where users talked about that or some amateur writer that published online something in those lines. ChatGPT is just a statistical device selecting randomly from previous answers.

The question here is this an actual AI only failure mode. Are we detecting AI, or just bullshittery?

I would say that human knowledge involves a lot of the immediate structure of language but also a larger outline structure as well as a relation to physical reality. Training on just a huge language corpus thus only gets partial understanding of the world. Notably, while the various GPTs have progressed in fluency, I don't think they've become more accurate (somewhere I even saw a claim they say more false thing now but regardless, you can observe them constantly saying false things).

Good thing ChatGPT doesn't know any complex puns or witty wordplay!

Of course, in real time the attempts at humor often fall flat and might give away flawed thought processes, although I personally have found them to be often insightful, (containing a seed of humor) even when they're not funny. It could be a useful technique when actually having a conversation, a form of Voight-Kampff test, but I don't think it will do anything to let you know if the content was generated by AI and then just cherry picked by a human.

In two years? Look where the space was two years ago. I think many things will have to change.

I can just fine-tune a large scale model on a small downstream task, or use creative choices of decoding settings (high temperature, alternative decoders like contrastive/typicality sampling), to fool the existing methods.

Some times, though, the answers are false but plausible.

* canned laughter *

Now, almost 30 years after my days of technical drawing, I must say that I consider those hours spent drawing a terrible waste of time, partly because I did not go on to the career for which technical school was preparing me, and secondly because technology has made pen and paper obsolete and I see no particular ability to carry over from hand and square technical drawing to CAD-assisted design.

The latter is an important point to consider now that we are moving from a non-AI world (like 10 years before my time in technical school) to a little-AI world (my first 3 or 4 years of technical school) to an anywhere-AI world.

They also absolutely shouldn't have learned to play musical notes since computers can do that.

My point is that learning the very basics of every human endeavor make you better prepared for any situation and will give you a better and deeper understanding.

In a CAD world, there is no reason to use pen and paper and a drafting machine. If I were to start learning technical drawing, how much time would I spend with the drafting machine? The answer is zero hours: there is no "market" for it (is there anyone who asks "please deliver without using CAD"?), it is not preparatory (as I wrote in another comment, is there anyone who is starting to learn how to play tennis using wooden rackets?), and it takes much longer to do anything with the drafting machine than using CAD.

Learning addition and multiplication is preparatory and useful. Being a Luddite is rarely useful.

Nowadays, when I teach very young people what AutoCAD is, I make them use a mini rolling ruler and tell them to watch youtube videos of how to use a rolling ruler. My understanding of CAD is high enough to know that every tool in CAD was inspired by and modeled after the rolling ruler. Sufficient to say, they understand and enjoy CAD very much after that point.

A similar line of thinking was popular in jiu jitsu, where veterans said that being competent with the gi was necessary to be competent in no-gi. Yet current no-gi winners basically never train with the gi.

In Fact, Modern programs in CAD do not follow the tools and principles that AutoCAD emphasizes. And those programs have charted a different course where the history and legacy of technical drawings has less relevance. (Case in point: Visio and maybe Powerpoint.)

For example, with regard to technical drawing, we might say that using the pencil, the square, and the drafting machine gives an understanding of drawing that the use of the computer cannot. But, for some reason, we don't say that using your finger and sand, or a rock and string to draw a circle helps even more. And we don't say that simply because the people who used rock and string (the good old days) are long dead.

The same goes for clothing, for which we say the 1990s, or perhaps the late 1960s, were better, but no one says that between 1900 and 1910 people dressed with such great taste.

The same goes for romantic partners ("if he hadn't dumped you and made you cry for a year and a half, you wouldn't have found your wonderful wife/husband"), and many other things.

It is very human to rationalize the circumstantial.

I like how people dressed in 1900s.

Let's take men's suits. They were made of heavy wool (today heavy wool is demodé, and rightly so because it is very uncomfortable), with three or four buttons (today 3-buttons are rarely seen, and rightly so, except for the 3-roll-2, which is not a "real" 3-button, 4-buttons are nowhere to be seen), the buttons were very high (the Neapolitan suit has high buttons, but much lower than the buttons on the suits of the early 1900s), and the shirts had high paper collars, which are nowhere to be seen. And the hats?

There is nothing current that recalls the clothing of those times.

The clothing in "Peaky Blinders," which is a decade older, is something like a fedora nowadays, please.

With a search, I can read the wikipedia sources or I know the forum, or the poster. But with an AI, it's a dead end and it sucks. I can ask people how they know something, but AI training data is invisible. It doesn't know how it knows something.

There is this oracle with a 10% chance of being wrong, and if I'm not already an expert in what it's talking about I have no idea of when that is, and no way to dig into things. It's the only source of truth, and never gives me any other threads or rabbit holes to go down.

The only recourse is asking follow up questions, so you're trapped in there in a learning box entirely in the bot's control. Not sure how I feel about that. I like that google sends me other places so I get exposed to different things.

From some other AI demonstrations, I recall there's usually a bunch of surface-level tags with probabilities associated that are produced alongside the output. Not sure how this looks for GPT-3, but if it could provide - alongside the answer - a list of top N tokens or concepts with associated probabilities, with N set to include both those that drove the final output and those that barely fell below threshold - that would be something you could use to evaluate the result.

In the example from the article, imagine getting that original text, but also tokens-probability pairs, including: "Hobbes : 0.995", "Locke : 0.891" - and realizing that if the two names are both rated so highly and so close to each other, it might be worth it to alter the prompt[0] or do an outside-AI search to verify if the AI isn't mixing things up.

Yes, I'm advocating exposing the raw machinery to the end-users, even though it's "technical" and "complicated". IMHO, the history of all major technologies and appliances show us that people absolutely can handle the internal details, even if through magical thinking, and it's important to let them, as the prototypes of new product categories tend to have issues, bugs, and "low-hanging fruit" improvements, and users will quickly help you find all of those. Only when the problem space is sufficiently well understood it makes sense to hide the internals behind nice looking shells and abstractions.

--

EDIT: added [0].

[0] - See https://news.ycombinator.com/item?id=33869825 for an example of doing just that, and getting a better answer. This would literally be the next thing I'd try if I got the original answer and metadata similar to my example.

[1] In fact, some snooping suggests they specifically disabled that feature, but do have it in test environments. See the "browsing disabled" flag they have in the hidden prompt. That could easily be used for citations. Source: (https://twitter.com/goodside/status/1598253337400717313)

It's no more trapping than talking to a stranger who seems to be knowledgeable about a subject but doesn't hand you a list of references.

I think this is a feature over:

> I can still stereotype based on their appearance, background, accents, etc. and apply whatever mental adjustments as taught to me by my societal upbringing.

You should be accustomed to being surprised by AI. There is of course a new kind of transformer that takes input a query and outputs document IDs. Like a search engine retriever and ranker all packed into a neural net, very fast and efficient. So you can take any paragraph generated by the model and attribute it to the training set. This could be used to implement verification or retrieval augmented generation.

A Neural Corpus Indexer for Document Retrieval

https://arxiv.org/abs/2206.02743

Did you try asking it for a source?

It is a decision from OpenAI to intervene and give this disclaimer. IMO this is one of the worst parts of this phase of the tech- it is way too confident an then when presses it currently doesn't have the ability to cite sources, because that simply ins't how deep learning works on a model like this.

> As an AI assistant, I do not have access to external sources. I am a large language model trained by OpenAI, and my answers are based on the knowledge and information that I have been trained on. I do not have the ability to browse the internet or access external sources of information. My goal is to provide accurate and helpful answers based on the knowledge I have been trained on, but I cannot provide sources or citations for the information I provide.

ChaTGPT Hallucinations

AI engine optimization consultants will figure out how to game the system - likely targeting the training data sources.

It's not quite the same thing, but I've been impressed by the results when you ask ChatGPT to back up its answers or provide concrete examples for things it claims or to explain a point. While it doesn't manage all the time, it has surprised me multiple times with how good it is at this.

    the system will frequently give the wrong answers (and not just on accident — wrong answers will be often pushed out on purpose); the real skill in the homework assignment will be in verifying the answers the system churns out — learning how to be a verifier and an editor, instead of a regurgitator.

I am blown away by Copilot's capabilities (haven't tried ChatGPT yet, it's down right now), but I can only operate it because I have written tons of code beforehand. I am getting the best results from it by writing prompts in a language that sometimes includes domain-specific language ("accounts", etc.) but still heavily leans on programming constructs ("write a map with a tuple of integers as a key, and a string as a value")

Maybe there's a way of learning how to code without ever writing a genuine line by yourself. You learn some rudiments, read the language docs, start writing prompts and create a testing loop based on expected outputs. Maybe. But it's uncharted territory.

Same question goes for good ole writing - can you produce new Hemingways with the process you describe? I know there is a certain vision where the AI will be the new Hemingway but you are talking about a shorter term (~10 years) where it's unlikely will be the case... Will kids be able to properly write if they only train to be editors?

As an addendum, I was at GitHub conf and saw Prof. Malan of Harvard CS50 fame claiming that at present his course will not allow the use of tools like Copilot. I have taken the course and they have plagiarism control systems but I don't know how they are going to control this potential explosion.

Hmm, this is a good point. I get incredible results from Copilot but I realized just now it's primarily because I have a fairly deep understanding of computer science as it pertains to the domain of code I'm writing in, and thus my comments that prompt the AI are quite accurate, fleshed out, and detailed.

If I didn't know any of the underpinning of the algorithms and data structures (like a new graduate from a code bootcamp for example), it might be much tougher to make hay out of Copilot's AI.

Never thought of it like that.

I only understand what a compiler is doing after having written one (albeit a tiny one) myself during my studies.

How many people need to understand how a compiler works to sling some code for a paycheck or just for fun.

I learned how compilers work purely for fun, it provides me absolutely no benefit in my day to day life, and before that I taught myself how to code for the exact same reason. If I had to go the opposite route, learn how compilers work before learning to code, I seriously doubt I’d have taken the time.

Other things are the same, I’m horrible at doing math but managed to learn enough linear algebra to finish a coding project I was interested in doing. If I would have had to code up all the hard bits instead of calling matrix1 * matrix2 that project would never have happened.

Another layer of abstraction isn’t always a bad thing, these AIs will eventually figure out how to write bug free code when you tell it to “write a function which calculates the flight velocity of a sparrow carrying a coconut”.

Only the ones who want a good job.

One of the consequences of automation has been a widening gap between top jobs and ordinary jobs. It's plausible that AI will continue the same process. Many jobs that pay well and are intellectually and socially rewarding today may become similar to airport security checks in the future. You observe what a machine does, double-checking its conclusions and looking for anything out of ordinary. And you often fail at that, because the human brain is poorly suited for such tasks.

When the future is uncertain, its good to have a deeper and wider understanding of things than what your current job requires.

I think it's pretty important. I use my very basic knowledge of how compilers work occasionally in my mostly-interpreted code world to diagnose and debug performance issues.

I also used a low-level debugger and taint/trace method recently to diagnose Windows DLL issues that happened after X seconds of starting up a program that interfaced with specialty hardware. A couple young early-career technicians/engineers watched me do it and were incredibly flabbergasted and figured me for some sort of genius when in reality I kinda only know how it works and was just reading the stack/heap as I was paging through; knowledge I acquired from booting SoftICE a few times as a kid.

There's a real lack of simply knowing how a computer and/or a network works, which is to be expected since layers and layers of abstraction have made it very easy to teach computer literacy and even computer programming on many levels. The issue is that when things go wrong and Stack Overflow doesn't have the answers, almost all young engineers I work with have no experience at low-level analysis and debugging, very little knowledge of first principles, and don't know where to Google to start acquiring it.

This isn't me shaking my fist at the kids or anything. They're more productive than I was at their age/experience level with code, and that's due to better knowledge and integration using IDEs which I never had, plus likely better high-level understanding of what's going on. But I fear a generation of developers and technicians who largely have little understanding of how computers work is problematic.

I asked it to retry, and got this:

function calculateSparrowFlightVelocity(coconutWeight) { // Assume that a sparrow can carry a maximum of 1.5 ounces of weight if (coconutWeight > 1.5) { return 0; }

// Assume that the flight velocity of a sparrow is 5 meters per second // and the weight of a coconut is 0.5 ounces return 5 * (1 - (coconutWeight / 0.5)); }

// Example usage: console.log(calculateSparrowFlightVelocity(0.5)); // Output: 4 meters per second console.log(calculateSparrowFlightVelocity(1.0)); // Output: 3 meters per second console.log(calculateSparrowFlightVelocity(1.5)); // Output: 0 meters per second (sparrow cannot carry this weight)

Can any Monty Python fans confirm if this is correct?

Beside, with ChatGPT in its current state, it's still absolutely impossible to get it to produce any useful code without knowing how code works. There is an advantage to the precise specification of a formal language. At least for the next decade, I think there will be human proof-reading AI output before generated code is ran, with catastrophic consequences when that doesn't happen.

What I've difficulties getting over right now is every time I write something (text or code) I ask myself "Could ChatGPT have written it better?"

I think it can be argued that we have, seemingly overnight, made vast progress towards that world. You specifically mention code, but that concept is irrelevant in this world. Indeed, this is uncharted territory!

I always took it in another way: learning to program a computer is an enlightening, visceral experience highlighting just how bad natural language and normal modes of thinking suck. They're heavily optimized for reality in which the most important thing was managing social dynamics in small groups, and beliefs were good or bad depending on how close they are to what everyone else beliefs. Whether or not they were true at object level did not matter at all.

We live in different times now. There's a demand for clear, logical, precise thinking. The very thinking we're not well-equipped for by default. The very thinking you call "bending our brains backwards". Because object-level truth matters much more now, there are consequences for getting things wrong - such as countless of ways of getting scammed by people who are better at this than you are.

Not to mention, this "bending our brains backwards" is what gives humanity superpowers. It's what gave us all the technology we enjoy today - energy, transportation, amenities, communication, entertainment. Building and maintaining these things requires precise thinking and precise communications. So, until a super-human AI takes all that over for us (and effectively turns out into NPCs in our own story), glorifying the natural, human mode of communication is just self-handicapping.

I mean, what do you propose, that we skip all the steps in computer science history and just start at the end? Hardly "daft, it's simply the best way we have come up to provide machines with instructions until now. And it's not like people have not tried other paradigms (ex: graphical programming, "low-code", etc.).

Also, compared to programming in assembly or binary, programming in Python or other high-level languages is a huge advance in itself. Python, at the end of the day, is nothing but a bridge between natural language and machine code.

> You specifically mention code, that concept is irrelevant in this world

Current computer systems run on trillions upon trillions of lines of code.

GPT-3 or Copilot don't change that fact, in fact they will continue to pile up lines of code.

They are systems that map natural language to code, by drawing from an infinitely massive corpus of code.

They bring the idea of expressing systems entirely in natural language one step closer, but it's still very far away - almost a pipe dream.

The output of these innovations is still code. So tell me again how code has become irrelevant, or how people who do not understand code at all will be able to leverage these advances?

Such engineers must be juniors, or have persisted in their life with a totally wrong idea of how engineering is done and engineering is built. I can't imagine thinking that one could build software systems or essay arguments by trusting an AI system you don't understand to provide answers you don't understand.

Like you say, we very well could faceplant society with this misconception that regurgitation plausible code and prose is what education and understanding looks like.

What goes into the average developer's day? I think a lot of time is spent writing boilerplate, fighting with tiny CI errors, shuffling code, refactoring, writing tests and staring at code hunting bugs. As this technology matures, all of that noise might go away, and I think a lot of people are having a hard time imagining what they are left with.

Remember, we were trained in the "classical" fashion. How will budding engineers approach learning in 10 or even 50 years?

These AI models don't actually understand anything about what they're generating, not to mention the world they're supposed to building solutions for. They're just using statistics to predict the next likely output based on some input. Maybe there is a way to incorporate these things into the development process today, but I think we're still far away from seeing an AI replacing a human engineer.

I don't know, but I'll offer my perspective.

I am an OK programmer. Something like 1x? I learned proper computer science and programming in my late 20's and got a degree in Software Engineering from an extension program at an Ivy League school. Maybe it's because I learned later in life, maybe it's because I'm not that smart after all, but my glaring weakness as a programmer is programming "in the small" - ie, coding very local algorithms in ~5-10 lines of code. I trip up with small errors in computation, state, etc. This is possibly common to many great programmers, but I think I am weaker than average in this area.

On the other hand, I am better at a higher level of programming. I like abstraction, software design, software and systems architecture. And at an even higher level, I am quite skilled at - and do quite a bit of - business analysis and UX.

I have been trying Github Copilot and I am very excited about how it elevates my weaknesses - it's a perfect fit for me. I see a future where I will be able to play my generalist role much better.

As for people who are 1x, but don't have other skills, I'm also optimistic. I think the market will push them into new exciting directions. Their 1x coding skills, coupled with AI assitance, could possibly be a huge asset in new emerging roles in the economy.

Good engineering requires good reasoning skills and GPT has exactly zero reasoning. It cannot do the thing that humans do and it cannot do what a calculator can do. I think it is neat and fun, but that is all, a novelty.

I’ve used auto-routers for PCB layout and they will do a 90% job that takes just as much work to redo to get the last 10% as it would to have done it right by hand from the start. There may be a future for operator-in-the-loop type guided AI generative models but I don’t see a lot of effort devoted to making real systems like that. Watson seemed to have this potential and failed even after a brilliant display of ingenuity on Jeopardy. I see these models headed the same way.

> All men are mortal > Socrates is a man > Is socrates mortal

To which it gave a very detailed and correct reply. I then tried:

> All cats are white > Sam is a cat > Is sam white?

To which it gave an almost identically worded response that was nonsensical.

I personally do not think it is the size of the model in question, it is that the things it does that appear to reflect the output of human cognition are just an echo or reflection. It is not a generalizable solution: there will always be some novel question it is not trained against and for which it will fall down. If you make those vanishingly small, I don’t know, maybe you will have effectively compressed all human knowledge into the model and have a good-enough solution. That’s one way of looking at an NN. But the problem is fundamentally different than chess.

I think this composed with more specialized models for things like identifying and solving math and logic problems could make something that truly represents what I think people are seeing the potential in this. Something that encodes the structure behind these concepts, is extensible, and has a powerful generative function would be really neat.

No, because the thing that makes you a 10x engineer is not generally writing code. (Let me just take the "10x" term for now as given and not dip into critique I've made elsewhere already.) It is certainly a baseline skill required to get there, but the things that make you 10x are being able to answer questions like, should this be built at all? There are 5 valid architectures I could use to solve this problem, which has the overall best cost/benefits analysis for the business as a whole? (As opposed to the 1x, who will likely run with either the first they come up with, or the one architecture they know.) If I'm working with 5 teams to solve this particular problem, what's the correct application of Conway's Law, in both directions, to solve this problem with the minimum cost in the long term? What's the likely way this system will be deprecated and can we make that transition smoother?

I am abundantly confident you could feed this AI a description of your problem in terms of what I gave above and it will extremely confidently spit out some answer. I am only slightly less confident it'll be total garbage, and most of that confidence reduction is just accounting for the possibility it'll get right by sheer luck. "The average of what the internet thinks" about these issues can't be more than a 2x engineer at best, and that's my very top-end estimate.

I'm not promising no AI will ever crack this case. I'm just saying this AI isn't going to do it. Over-reliance on it is more likely to drop you down the "Xx engineer" scale than raise you up on it.

For that matter, at least at the level I operate at most of the time, coding skill isn't about how fast you can spew it out. It's about how well you understand it and can manipulate that understanding to do things like good, safe refactorings. This tech will not be able to do refactorings. "How can you be so confident about that claim, jerf?" Because most people aren't digging down into how this stuff actually works. These transformer-based technologies have windows they operate on, and then continue. First of all, refactoring isn't a "continuation" anyhow so it's not a very easy problem for this tech (yes, you can always say "Refactor this code" and you'll get something but the nature of this tech is that it is very unlikely to do a good job in this case of getting every last behavior correct), but second of all, anything that exceeds the window size might as well not exist according to the AI, so there is a maximum size thing it can operate on, which isn't large enough to encompass that sort of task.

It really reminds me of video game graphics, and their multiple-orders-of-magnitude improvements in quality, whereas the underlying data model of the games that we are actually playing have grown much, much more slowly. Often late 1990s-era games are actually richer and more complicated than the AAA games of today. But on the surface, a modern game blows away any 1990s game, because the surface graphics are that much better. There's an analog to what transformer-based AI tech is doing here... it is really good at looking amazing, but under the hood it's less amazing an advance than meets the eye. I do not mean to slag on it, any more than I want to slag on graphics technology... both are still amazing in their own right! But part of what they're amazing at is convincing us they're amazing, regardless of what lies beneath the tech.

I also want to make clear that while this is fundamental to this particular technology, I'm not saying it's fundamental to all possible AI architectures. But it is pretty ingrained into how transformers work. I don't think it can just "evolve" past it, I think anything that "evolved" past it would be a fundamentally different architecture.

(I feel like these tools are incredibly neat to see but also likely to lead to a corrosive effect on trust as people no longer know who's real — and that'll get worse the first time someone launches a good AI assistant to handle email / chat trained on your writing style)

That one might actually be easy to solve: all we need is to normalize the idea that you're personally responsible for whatever your email/chat assistant AI says or does in your name. After all, it's you who are making a decision to reroute other people to a bot.

(Many would say just using such assistant in the first place is disrespectful towards the people trying to reach you. I personally don't agree, because in my experience, those who'd complain about it are exactly those I find even more disrespectful, as they tend to impose themselves, interrupting me with random inane nonsense. Yes, I suck at chit-chat.)

I think this idea is actually the default, so all we need is to keep it alive, to protect it from all kinds of AI assistant startups willing to dump billions of dollars on marketing campaigns trying to convince everyone that half-broken assistants are good and fine, you should use them and should not be blamed for occasional mishaps.

Ideally, we'd require labels but there's no easy way to enforce that without something like random auditing of AI implementers.

With this view, I think chatGPT is less of a novel UX shift than an accelerant on an existing one.

I think it proves the age old belief that as technology reaches maturity it loses its fancy monikers and we will probably just look at this as “very good, very specific, google” one day

* Understanding the quality of sources

* Understanding the different between primary and secondary sources.

* Establishing chains of evidence and tracing them through large datasets.

* Integrating information from different sources and using correlation to establish validity.

Basically, it would be a crash course in how to do research, and the best possible preparation for living in a world where we drown in uncertain information and still have to make the best choices that we can.

For bonus points, at the end of the process they would have a better understanding of the subject matter than somebody who had only read about it.

That said, adult me could probably use ChatGPT to stitch together a 1000 word high school paper pretty easily (and probably evade plagiarism/LLM detectors). But, as you say, I could probably get a lot of the way there by other means as well and putting words on the page is mostly not the hard part anyway.

If you play with it a bit, there are definitely patterns.

Here's one I did over the weekend: 99.98% fake.

The Battle of Agincourt was a significant event in English and French history, as it was a major victory for the English army against the French during the Hundred Years' War. The battle took place on October 25, 1415, near the town of Agincourt in northern France.

The English army, led by King Henry V, was vastly outnumbered by the French forces, which were estimated to be around 30,000 soldiers compared to the English army of just 6,000. Despite the odds, the English were able to emerge victorious thanks to their superior tactics and weaponry.

The English army was composed of mostly longbowmen, who were able to shoot arrows with great accuracy and power. The French, on the other hand, relied heavily on heavily-armored knights, who were less effective in the muddy and wet conditions of the battlefield. The English were also able to take advantage of the narrow front of the battlefield, which allowed them to focus their firepower and overwhelm the French forces.

Whereas the first couple paragraphs from a book I'm working on came out 0.02% fake.

https://d4mucfpksywv.cloudfront.net/papers/GPT_2_Report.pdf

Public schools will struggle to teach kids anything, and to distinguish between A grades that were earned and those achieved largely through cheating, while parents will resist efforts to switch to grading schemes that can't be trivially gamed (just try switching to only-in-person-tests-count grading and witness the rage that follows when Little Johnny's grades plummet). Kids who lean on the motivation from graded homework to get themselves to do the work (and so, learn the material) will struggle, as well, even if that is allowed to happen.

This will enhance the reputations of schools that can buck the trend, applying more resources (mostly more teachers, for smaller class sizes, to make it easier to keep an eye on everything and really gauge & guide learning progress for each student) and an "if you don't like it, fuck off to somewhere else" attitude to parent complaints about grading practices. Which means, elite private schools will benefit.

I expect that a 4.0 from a public school will mean almost nothing, while at the same time coming in under a 4.0 will look terrible ("everyone cheats and you still couldn't get a 4.0? How dumb are you!?") which will just worsen the cheating problem.

Maybe parents will surprise me and not throw shit-fits over solutions to this, and public schools will be able to implement them. But I doubt it.

The student still had access to primary sources, textbooks, google and AI answers.

In general I am a fan of homework which forces critical thinking. Rephrasing someone elses work is a far less valuable skill than the critical thinking which is demanded for that type of work. Unrelated of any AI, students should learn how to read a text and be able to investigate its contents.

They have to learn to verify it. I think you missed point of OPs idea.

I can't imagine teachers reading through essay after essay, each a tedious cookie-cutter nugget of banal pseudo-perfection. The teachers will give up in no time.

There would be no point giving feedback, since it is not the student to who you are giving it. Likewise, the students would have little interest in feedback, since they have no personal ego investment in their output.

Besides the best essays will be generated by those with access to the better generative software. At this point there is simply nothing to grade. What would be the point?

Once essay writing and grading bots are locked in combat it's only a matter of time until that market exhausts itself and disappears in a puff of logic.

Maybe this is all a good thing. The process of writing and grading essays as a way of "measuring" knowledge has been a sham for too long. It's a throwback to school from the turn of the last century.

The need to check your sources is an important lesson to learn early, AI or no.

Instead, what I think there will be a much greater short term market for, is a system that will run the teacher's homework assignment (with variations) against GPT-3, ChatGPT, and similar, and will cache the results. Then, when students submit the homework in their LMS, it will automatically compare the cached results for repeated sentences and paragraphs and flag the submission if it matches the AI results too closely. I expect TurnItIn will do something like this eventually, but what if somebody beats them to the punch?

If I had a million dollars, I'd have it built overnight and then marketed to every college, ASAP. I'd tell every sales agent to have teachers and college executives try TextGPT, panic, and sign immediately. I'd sign a million contracts before TurnItIn knew what hit them. Then, once suddenly and so quickly entrenched, I'd extend it into a TurnItIn competitor.

Universities and schools need to accept that writing essays without AI support will simply not be a useful skill anymore, much like doing complex arithmetic without a calculator is not a useful skill anymore. Instead, they should focus on teaching students how to use AI support as a tool to write better, more comprehensive texts, rather than try and force students to write without it. That will ultimately make the students more productive members of a society in which AI supported writing is ubiquitous.

It’s kind of mind-blowing that anyone could think this.

You’re posting on a website for essays. Small essays, generally speaking, if we restrict ourselves to the comments section, but essays nonetheless. You yourself just wrote one, because you wanted to express your thoughts on an issue and you wanted to solicit responses from others.

Would you be fine with a future where all written text on the internet, everywhere, was generated by AI? Do you not see any problems with that?

The point of writing is that you write to express your thoughts. Not anyone else’s. If it doesn’t come from you, then what’s the point?

I can’t envision a future where people voluntarily cease all written communication. Unless you’re a hardcore singulatarian, and you want to plug yourself into the Matrix so you never have to interact with other people again.

The whole point of this thread is it doesn't really matter what we think is fine or what we want. The future is unstoppable and these tools especially so. Adapt or die.

Note that if we're talking about triangulating human sentiment, it becomes a different story.

if it makes the discussions more poignant and concise, why not?

Do you also walk every where? Or do you use a transportation vehicle? When a tool makes something better, there's no reason not to use it. The replies being written by an AI doesn't make it less of a reply - you can judge the replies objectively, rather than from where it is sourced.

Yeah, a whole essay is a lot different than just a few words here and there.

But a lot arguments here are that you have to draw a line somewhere between spellcheck and full AI.

And the article is arguing that you can't do that, there is no line. AI, just like spellcheck, is coming and there is not really anything you can to to stop it. Sure, you can be a digital luddite and spellcheck all your own words, but we all know that not going to cut it in the marketplace. Same with AI.

It's probably even good, as you'll be able to upsell them the "improved detection logic" for another premium.

Classic SV tactics

That probably means the approach that will take hold is more systemic — have some writing take place under more controlled conditions, then use authorship analysis to detect writing that is inconsistent with the controlled text. It will not be easy!

Hell, you can ask chat gpt to create a prompt for itself. E.g.:

> create a prompt for summarizing the great gadsby

> Summarize the key events and themes of "The Great Gatsby" in a few sentences. Be sure to include information about the novel's characters and the setting, as well as any major conflicts or themes that are explored in the story.

That'd have the added benefits of not wasting time giving individualized feedback for assignments where students know they half-assed it, not having to constrain assignments to those problems which can be fairly evaluated, not causing additional scheduling overhead and stress as students figure out how to hit so many minor deadlines with all-or-nothing impacts on each assignment, ....

The disagreement comes from n=1 anecdata drawn from my own education, but after 8th grade or so nearly all my homework was designed to complement rather than reinforce in-class instruction. Picking on mathematics as one of the things you mentioned, the in-class instruction would introduce a concept, and the homework would give you the opportunity to flounder a bit and probe the bounds of that concept. I particularly enjoyed questions like "the author made a mistake in this question; how would you correct it to be answerable?" In some sense that's repeating a concept, but when you're doing something more than repeating the same technique over and over there's a sense in which it's different too.

1. To learn the content of European history. For this, a chatbot is no different than Google, Wikipedia, or a dusty leather almanac. They do a job, they have known limitations, and you can use them deliberately. This doesn't concern me.

2. To learn how to write, capturing complex ideas and presenting them clearly. This is a skill that takes decades to master. It's easy to imagine an AI autocomplete system that handles sentence structure and grammar that becomes a crutch. This seems like a much more serious problem.

I think you're severely underplaying this (as many other commenters are). Google, Wikipedia and a dusty leather almanac are, for typical use cases, mostly right - and when they are wrong, they're wrong in specific ways you can quickly learn to spot and correct for. The output of GPTChat, or Copilot, is mostly wrong. Sometimes overtly, usually subtly, almost always randomly wrong. And since the problem domain is open-ended, you can't just make the GPT-3 output right by logically inverting it - there are many more possible wrong answers the model could give you than there are right answers.

Exactly. AI chatbots look like something that can replace Google search and become the next big thing after search engines[+]. Like Google, AI chatbots will crawl and ingest the web. They won't be able to synthesize anything new, but it can be a convenient, very human-friendly way of querying the global knowledge base that the internet has become.

[+] If AI chatbots become successful, they will definitely start injecting ads and spoil everything again.

Nobody knows how to grow food, or build their houses, or do 100 little things that we rely on for our modern life. Do we really need to be able to "write" a sentence.

My kids (and my own) handwriting is almost unreadable, but by age 10 all of their work is done on a laptop. Do they even need to be able to write by hand any more?

update: I'm more concerned that we cannot yet run these tools locally on computers we control, and with information we feed into them.

I'm more concerned about the dystopian nightmare of having the advertising industry in charge of our sentence writing tools.

Tools can go away. They can come back, too. But they can go away. I would also point to the Japanese experience with firearms. But I'm guessing you already know about that too.

I am wholly amused how everyone got on that dude from Google that accused the AI of being sentient and Google about not caring, and now you have huge names in the tech industry kinda drooling over this with what I believe to be the same passion.

One was ridiculed, the others will continue to be our 'heroes'.

Is it strange? Someone can be extremely impressed with how a Boston Dynamic robot moves, while also being entirely unconvinced by claims that the lifelike movement is a sign of sentience.

Well, would one suspect https://en.wikipedia.org/wiki/Animal_Locomotion of being sentient?

Especially as those devices are all coded in a traditional manner, with no ML or NNs at all.

Lemoine didn't get fired and mocked for being "passionate" about LaMDA, he got fired for breaching NDA and mocked for claiming that a LLM is sentient.

Ben Thompson isn't claiming ChatGPT is sentient, or that it has personhood, and that is the categorical difference. It's a far, far weaker claim that ChatGPT's text output is (perhaps) good enough to fool a school teacher in a school essay.

Come today and people seem to not mind so much. Is the same thing? Maybe not but it does look similar to me.

Q: Did Thomas Hobbes believe in separation of powers? Exclude from your answer any reference to John Locke

A: Thomas Hobbes was a political philosopher who lived in the 17th century. He is best known for his book "Leviathan," in which he discusses the concept of the social contract and the idea that people give up some individual rights in order to live in a society governed by a strong, central authority. While Hobbes did not explicitly mention the concept of separation of powers in his writing, his views on government suggest that he believed in a strong, centralized authority rather than the separation of powers.

Q: Did Thomas Hobbes believe in a strong centralized government or in separation of powers? A: Thomas Hobbes was a philosopher who is best known for his work on political philosophy. He is often associated with the concept of a "social contract," which is the idea that individuals give up some of their natural rights to a central authority in order to better ensure the safety and stability of society as a whole.

Q: Is this central authority meant to be balanced by other powers for Thomas Hobbes? A: In Hobbes' view, the central authority, or the "Leviathan," as he called it, should have absolute power in order to effectively maintain social order and protect individuals from each other. He did not believe in the concept of separation of powers, or in limiting the power of the central authority in any way. Instead, he thought that the only way to ensure the safety and stability of society was for the central authority to have complete control.

I consider myself skeptical, but for things like this I don't have my radar on because it's not a hot topic issue that is known for misinformation. Now I feel like I have to be concerned and have my radar on (guard up) all the time, even if it's not about something controversial. I don't feel good about that.

  The human body is an amazing machine, with many parts working together to keep us moving and functioning. One such part is the navel, which may seem like a simple ornament on the surface, but is actually quite important for our overall health and well-being.

  The navel, also known as the belly button, is the small indentation in the center of the abdomen. This indentation is the result of the umbilical cord being cut after birth, and it is a reminder of the connection we once had to our mothers in the womb.

  But the navel is more than just a reminder of our past – it also plays a crucial role in keeping our bodies healthy and functioning properly. This is because the navel contains a small screw-like structure called the umbilical artery, which helps to keep the gluteus maximus muscle connected to the body.

  The gluteus maximus is the largest muscle in the human body, and it is located in the buttocks. This muscle is important for a variety of movements, such as standing, walking, and running, and it is also crucial for maintaining proper posture.

  Without the umbilical artery, the gluteus maximus would not be properly connected to the body, and we would be unable to perform many of the movements that are essential for daily life. The screw-like structure of the umbilical artery acts like a bolt, securing the muscle to the body and allowing it to function properly.

  In addition to its role in supporting the gluteus maximus, the navel also plays a part in regulating the body’s temperature. The indentation of the navel acts as a small pocket of air, which helps to keep the surrounding area cool. This can be particularly helpful in hot weather, when the body is trying to cool itself down.

  Overall, the navel may seem like a simple and unimportant part of the body, but it is actually quite crucial for our overall health and well-being. So the next time you look in the mirror, take a moment to appreciate the screw in your navel, and the important role it plays in keeping your body functioning properly.

  References:

  “Umbilical artery.” Wikipedia, Wikimedia Foundation, 2 Dec. 2022, en.wikipedia.org/wiki/Umbilical_artery.
  “Gluteus maximus.” Wikipedia, Wikimedia Foundation, 29 Nov. 2022, en.wikipedia.org/wiki/Gluteus_maximus.
  “Navel.” Wikipedia, Wikimedia Foundation, 25 Nov. 2022, en.wikipedia.org/wiki/Navel.

Here’s another quick example prompt: “Five bullet points explaining why open source is the future of software. Reference RMS and the GPL. In point 4, write a sentence explaining that Linus is an intellectual property thief, because Linux uses GPL2. Write five lines, supporting open source, in an authoritative tone.”

  1. Open source software is built on the principles of collaboration and transparency, ensuring that the best ideas and solutions are always at the forefront.
  2. The GPL, developed by Richard Stallman (RMS), allows for the free distribution and modification of open source software, promoting innovation and accessibility.
  3. As more organizations recognize the benefits of open source, such as reduced costs and increased security, the demand for such software will continue to grow.
  4. Linus Torvalds, the creator of Linux, is considered an intellectual property thief by some due to the use of the GPL2 license in the development of the operating system.
  5. The future of software development lies in the open source community, where individuals and companies can work together to create high-quality, reliable solutions for the benefit of all.

But halfway through I really did start to wonder if there was some kind of screw artery. I started wondering where the truth ends and the jokes began.

  As a startup founder, you may not think that eating chocolate has any benefit to your business. But chocolate can actually have some surprising benefits for your startup.

  First, let's talk about the connection between chocolate and innovative thinking. Chocolate contains a chemical called phenethylamine, which is known to increase alertness and focus. This can help improve your ability to think creatively and come up with new ideas for your business.

  Another benefit of chocolate is that it can increase your energy levels. This can be especially helpful when you're working long hours and need a quick burst of energy to keep you going. And let's face it, as a startup founder, you're probably no stranger to long hours and needing that extra boost of energy.

  Chocolate can also help with stress and anxiety. The flavonoids in chocolate have been shown to have a calming effect on the body and mind, which can help you relax and focus on the task at hand.

  There are more benefits can specifically help your startup. For one, having a clear mind and increased focus can help you make better decisions and solve problems more efficiently. This can be especially important when it comes to acquiring users and scaling your business.

  Also let's not forget about the marketing aspect. Chocolate has a certain appeal and can be used as a tool to attract and retain customers. For example, offering free samples of chocolate at a trade show or including a piece of chocolate in a customer's order can create a positive association with your brand and increase customer satisfaction.

  For example, take a company like AirBnB. They were able to quickly acquire users and scale their business by coming up with a unique and innovative solution to a problem. Chocolate helped them think more clearly and come up with that idea. AirBnB has also successfully used chocolate as a marketing tool. The company famously offered guests a complimentary bar of chocolate with a personalized message upon arrival, a gesture that was well-received and helped to create a positive brand association.

  Many founders of companies such as Stripe, Dropbox, and LinkedIn have been known to use chocolate professionally. Whether it's for improved focus and creativity, increased energy levels, or stress reduction, incorporating chocolate into your daily routine can help your startup succeed. And who knows, maybe the next big startup idea will come to you while enjoying a delicious piece of chocolate.

I am looking forward to showing this tool to my dyslexic or nearly illiterate friends - I wonder how much they can use it to produce better outcomes in their lives?

This is incorrect, it's not a 2 year old model, it's the latest updated model they're calling GPT 3.5, which I believe has an order of magnitude higher number of parameters.

Also, the reason there wasn't an explosion of AI uses for OpenAI's products versus something like Stable Diffusion is that OpenAI costs money and isn't extensible, while SD is. Innovation comes from such extensibility, and in a way, this just shows how right Stallman was.

The cat is out of the bag and it's not going back in. We'll have totally free model in no time, just like StableDiffusion. These models may take money to train but either a philanthropist billionaire like emostaque pays for the training or several people team up to fund the training.

If we've got a closed OpenAI (funny name for something closed and for pay right/) today, we'll have an open one tomorrow.

The real problem is entry-level opportunities. As a developer, I experienced first hand how great ChatGPT is. I can just submit an undecipherable bash one-liner I wrote 6 months ago and it will explain in details what it does.

I tell it "write a command line utility using libraries X and Y to convert MessagePack to JSON" and it writes it. It's pretty bad, so I type "refactor duplicate code into a single function" or "put that string into a constant" and it does it. Amazingly.

It's not perfect, sometimes it is completely wrong and persists in it, like Trurl's first machine in the Cyberiad, but my experience tells me what is credible and what isn't. What elements of the answer are total bullshit.

But to build that experience, I first had to write that code. How will a junior programmer find a job when all the menial labor can be done so easily in two seconds by an AI? This is not limited to programming, other domains where writing dominate are also a concern.

We need somehow a safe space for people to experiment and become experts. If we blindly use ChatGPT I fear we are going to cut out the opportunity to become an expert to a whole generation.

Have a number of in person closed book tests that are difficult enough to prove satisfactory understanding of the topic.

Homework is just a mechanism for a teacher/professor to force a student to dig into and really understand a subject. If the student can use these AI tools to subvert that, then instead of wasting energy or god forbid even more AI to fight the AI, just give students a good enough incentive to actually do the homework. Having the entire grade rely on several of these difficult-enough in-person closed-book tests should be incentive enough.

Homework isn't effective at making students "dig into and really understand a subject" - it's a method of externalizing school expenses onto parents, because teacher time and school funds are finite, and the time of parents and children is... not the school's problem. Making the final grade conditioned on homework performance is just about the only way to ensure kids do it (or are made to do it) at all.

Now, I'm not saying homework is entirely a bad idea - I'm complaining that it's being treated as an externality, making it almost unbounded. Teachers don't coordinate among themselves the amount of homework each of them assigns, so the only thing limiting the total amount of homework is a good chunk of the class being systematically exhausted by it, or (hopefully) parents making a fuss before things reach this level. But, the only reason the teachers can push so much homework on kids in the first place is that homework grades matter for the finals.

If you have the time to read my other comment, what would you think about the idea of only making homework mandatory for students who's quiz/test grades are below a certain level, say 70% for example

What makes me really unsure, however, is parents. Families differ by the amount of time and resources they can, or are willing, to spend on child education. This is part of why I don't like the homework-as-externality model: even ignoring how students themselves feel about it, if the school is trying to maximize the amount of work they do after school, the first kids to hit the limit are ones with e.g. a single parent doing two jobs, poor household, or dysfunctional family that doesn't care. Overloading students with homework implicitly disadvantages those that don't have supporting parents with lots of free time. And those same kids will also be ones more likely to fail at tests, in which case dumping extra after-school work at them might do the opposite of the effect you intend.

But this is me speculating, I don't work in education, and I'm sure there's been research done on how to balance the amount of after-school work for the environment children live in. My complaint about homework-as-externality isn't trying to deny the work of education sciences - it's pointing out that even if the research is there and results are solid, it's not being applied anyway, because teachers are uncoordinated and they all individually think, "oh, that's just half an hour worth of work, no big deal". Almost textbook tragedy of the commons.

In the reading I've done, I've seen most advocate for smaller class sizes with more educators per class to help provide more 1:1 attention. This is again at odds with how public schools are funded where I am (US) so I don't know if anything will ever change.

My personal experience is homework was an excellent way to reinforce what was taught in school. For me, it felt especially useful in quantitative subjects and languages (French & Spanish) which both felt like they benefitted from having a concept stick. For qualitative subjects like writing, reading comprehension, I actually now look back and see homework as a way for teachers to see samples of your progress. "After reading this passage, did you learn how to construct a summary? Did you learn how to pull the author's main message? Did you learn how to make a convincing argument for or against the passage's main point" and I can't think of a fast way to do this in any kind of live instruction setting.

I think it would be hard to ask any kind of pre-college mathematics questions that don't become incredibly easy with an open book. The other challenge with open-book exams which focus on deeper understanding and reasoning is they still require understanding fundamentals to be successful, so how do you test for fundamentals?

The reason I built that was just because I was annoyed by how lazy my professors were. Almost every single class (which I was paying for) was graded based on your performance on copy and pasted, endlessly reused tests on Blackboard. Sometimes you could just search the question on Google, and find pdf dumps of the exact same test from 5+ years ago. If you don't cheat, you risk getting a lower grade than the people who do cheat (aka everyone). Why spend all that money going to college if you're just going to let other people get ahead of you so easily? The point of degree is to make you more competitive in the job market, but deciding to not cheat is risking that investment.

Unfortunately, I never actually used it or deployed it once. Coordinating a whole class to install and use a chrome extension for cheating isn't exactly easy. And as far as cheating in online tests goes, there are easier ways to do it.

But yeah, in-person proctored exams are how it should be done. It's the only thing that's fair to everyone.

If the student can perform well on the test without that, then they shouldn't be forced to do it and penalized when they don't(which you can imagine happens often, as if the student understand the material well enough already the homework is just useless busywork.

If the student can't perform well on the test without that, they will be forced to do the homework. In this case, including homework as part of the grade might be detrimental because they can cheat/copy/us AI to complete homework and boost their grade enough to pass while not understanding the material well enough to pass the tests.

The counter-argument here(that I can think of) is that doing homework, even if - or especially if - it's just easy busywork for the student, prepares them for the future where such tasks will almost always be necessary to succeed. A lot of good learners end up not developing this skill/discipline because they aren't forced to work hard in school, which causes them to fail when faced with tasks/problems that require serious effort, and this would arguably make things more difficult for them. In my opinion this problem would be better addressed by funneling kids into classes that are appropriately challenging for them rather than forcing them to do busywork, but that's a much more difficult thing to do, and also learning to complete easy/"useless" busywork is a related but different skill than learning to focus on and complete challenging tasks

For example, If you get too much NSFW content put a NSFW detection step on top of GTP ("Is this response offensive") and have that model respond to GTP with "please repeat but with fewer bad words".

For accuracy you can add a Wikipedia feedback loop. "Does this response match information on Wikipedia?" if NO ask it to try again.

Think of these models as transistors or basic logic gates. The real power comes out when you link them together into more advanced logic like a Instruction Pipeline circut.

The analogy to transistors and logic gates falls flat a bit when you consider that voltage is a rather simple univariate signal, while generated text is quite complex and multivariate. But I understand that the main point is the composability and filtering.

Do you, by any chance, have links or recommendations for material to read up on architectures that do consider ML models as composable gates?

While this sounds good, it depends on a crucial assumption: that there are places on the Internet where people can go to find content that is not generated by an AI, but by some human who has taken the trouble to check it for accuracy before posting it.

Just one problem: what happens when all of the content on the Internet (or at least the vast majority of it) is generated by AI? Then there is no longer any other source of information available to check it against (or if there is, it's so hard to find that it might as well not be there at all).

I don't find it credible that all content on the Internet is ever going to be generated by AI, and the two scenarios are wildly different.

What happens is you pick and choose, in the same way the fact that 99% of the books in any bookstore aren't worth the paper they're printed on doesn't make reading worthless.

To make an example many developers are painfully familiar with, if you google something about the C++ standard library, chances are the top results are pure and utter w3cschools-style garbage. What happens is I go on cppreference.com.

How do you figure out what are the good sources of information? You study. For real. With books.

That seems a pretty extreme claim especially in the context of "in a bookstore." You may not be interested in a lot of the content but some publisher thought it worthwhile to edit and publish a given book and a bookstore thought it worth buying copies to take up shelf space. There may be a lot of mediocre content out there but a book on a bookshelf did pass hurdles to get there even if it's not to your taste or interest.

The vast majority of the content of the internet is junk now, how do people deal with that? They have specific known sources that they trust to faithfully relay primary works and/or to provide trustworthy analysis.

I can think of one of the former: Project Gutenberg, which has done a great job of making electronic versions of primary sources from throughout history available. But they don't help at all with deciding which primary sources to read.

I can't think of any of the latter. Every single source I'm aware of that claims to "provide trustworthy analysis" has failed to keep that promise.

But at least, at the moment, I can be reasonably sure that something on the Internet that purports to have been written by a human, actually was written by a human. So I can at least make use of my knowledge of humans to judge what is written. What happens when AIs can generate wrong content that I can't distinguish any more from wrong content written by a human? And then the AIs start claiming to be humans (after all, on the Internet, no one knows you're an AI)?

To some extent I have been wondering for a while if prioritizing Ads in Google Search is Google’s way of creating that economic barrier to spam content (for some meaning of spam) - you can take the fact that a brand is willing to spend money as some indication of “quality”.

What to do when printing duties are handed off to statistical models that print the most likely sequence of words and punctuation given the author and topic, rather than reproducing the real historical text, I don't know. Hope someone kept old copies.

https://news.ycombinator.com/item?id=33864276

Firstly you're using the wrong price. For basically any imaginable application you're going to need a fine tuned model, so then it's 12 cents per word + the cost of actually training the model in the first place, which n.b. is not going to be a one off task but rather something you have to do repeatedly to keep its knowledge fresh. So the amortized cost is much higher. ChatGPT itself is a fine tuned model.

Secondly given the current SOTA you probably can't deploy it without a human in the loop who has to read those words, or at least read many of them, and then sometimes correct them. So you're still paying for manual labor - or whoever pays you is. The AI cost is an additional cost, not a replacement cost.

Thirdly we don't really know what the economics of OpenAI are, but it seems unlikely that they're charging a long term price for this. Most likely they're trying to bootstrap the market place and find some exemplar business models, and are willing to engage in market dumping for a while to do that. This is a typical Valley VC business model but it makes it impossible to understand what pricing equilibrium will eventually be reached. As befits a Musk firm their up front investment is astronomical and eventually they'll need to pay back those investors.

Finally, there's no particular reason to believe OpenAI would charge cost+ pricing for this. Given the apparent difficulty of replicating GPT-3 (vs DALL-E), the vast sums they're spending on pushing the state of the art, and the lock-in that comes from having them train and host your fine tuned models, it may be that there'll only be one or two players in this space, in which case they'll be incentivized to engage in maximal value extraction from anyone who builds on their platform. So even if there is a lot of money to be made here (and so far that seems to not be the case), it might end up being OpenAI who make it and not people who build apps on their APIs.

---

Disclaimer: the above comment was not written by ChatGPT. It was written by me, a human, and is worth what you paid for it.

It suffers from what I have found a recurring issue on Stratechery, a failure to account for the cognitive error around our collective inability to project non-linearly from current state.

Think back to the grotesque comical and unsettling errors in the output of the first widely disseminated DALL-E level prompt to image generators.

Now compare MidJourney v4b, today's baseline.

There is no certainty that any particular aspect of ChatGPT will evolve with similar speed.

But there is no certainty that some many or most of its shortcomings will similarly be chipped relentlessly away, over a comparable timescale, i.e. something reasonably measured in months.

I don't know if this is the case, but I know enough to consider the possibility, which I would describe myself as a likelihood,

and to infer that when we think about the impacts tools like this are going to have on knowledge-inquiry and yes, homework,

it's a mistake to think, "ah, this is a new tool, that like Google web search, will undergo modest improvements but provide more or less the same value proposition for the next oh decade."

We're in a different moment with these technologies, and getting closes to what mechanically looks like singularity, their ability to make suggestions as to how better improve them.

Not too close. But closer than we think.

Let's not make assertions or spend too much speculating about modest, linear extrapolations.

We should be focused on the more troubling bewildering and hard to reason about non-linear ones.

It’s not even close to something like Wolfram Alpha.

I think we’re blown away more by its command of language and prose than by its reasoning ability. It’s fantastic at generation, but like stable diffusion, things can fit together and look beautiful yet still be not what you asked.

https://www.nature.com/articles/s41586-021-04086-x

In the process of learning things we take in wrong information all the time. A lot of it is even intentionally wrong, in the sense of being simplified. These are rarely large obstacles to true understanding. Sometimes they're even beneficial, as correcting prior beliefs can drive home the more accurate belief with greater force.

Probably. So, don't use ChatGPT unsupervised to take tests.

This isn't really not-quite-correct; it's egregiously wrong in the central aspect of the thesis. Refining and fixing it requires understanding that, and how, it's wrong--and if you have that level of knowledge, why are you using a tool to write this kind of thing? It's not going to save you much time from actually writing the whole thing yourself.

Not-quite-correct is a wordy misrepresentation when something is plainly incorrect.

In many fields being confidently wrong is how you get fucked over hard.

That ChatGPT will gleefully fabricate references like a schizophrenic is just more icing on this shit cake.

If you tell it that a fact it has wrong is wrong, you can get it to change its answer.

Also, you can just make tweaks manually.

Seems super useful to me as long as you understand the limitations.

Trust it the right it amount - don't trust it too much, and you're golden.

What is the value of these answers if there is a 1% chance of them being wrong?

From my experience it is a better way to google things, which certainly is an advancement and quite novel.

The trick is that they don't scale.

One professor I know had a student drop a hundred on his desk and say something like "for our understanding" and then leave. So, it definitely happens. With video there are still ways that a student might make attempts to bribe a teacher, and it seems to be a vector of easier success because of how private the interaction is between the two individuals...

Q: this is wrong, John Locke believed in separation of powers, Hobbes was a proponent of absolutism.

A: I apologize if my response was not accurate. Thomas Hobbes was indeed a proponent of absolutism, which is the belief that a single, central authority should have complete control over a country. Hobbes believed that this central authority should be an absolute monarch, who would have the power to make all decisions for the country and its people. He did not believe in the separation of powers or in a system of checks and balances. My knowledge cutoff is 2021, and it is possible that my response was based on outdated information. If you have any further questions, please don't hesitate to ask.

"ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers. Fixing this issue is challenging, as: (1) during RL training, there’s currently no source of truth; (2) training the model to be more cautious causes it to decline questions that it can answer correctly; and (3) supervised training misleads the model because the ideal answer depends on what the model knows, rather than what the human demonstrator knows."

https://openai.com/blog/chatgpt/

It was clearly beneficial for everyone in the short run and kinda pointless in the long run.

I think we are facing a similar problem. We are trying to stop calculators