Plato's _Phaedrus_ features Socrates arguing against writing; "They will cease to exercise memory because they rely on that which is written, calling things to remembrance no longer from within themselves, but by means of external marks."
I have heard people argue that the use of calculators (and later, specifically graphing calculators) would make people worse at math; quick searching found papers like https://files.eric.ed.gov/fulltext/ED525547.pdf discussing the topic.
I can't see how the "LLMs make us dumber" argument is different than those. I think calculators are a great tool, and people trained in a calculator-having environment certainly seem to be able to do math. I can't see that writing has done anything but improve our ability to reason over time. What makes LLMs different?
Because they do it all for us and they frequently do it wrong. We're not offloading the calculation or the typing to the thing we're using it to solve the whole problem for us.
Calculators don't solve problems, they solve equations. Writing didn't kill our memories because there's still so much to remember that we almost have to write things down to be able to retain it.
If you don't do your own research and present the LLM with your solution and let it point out errors and instead just type "How do I make ____?" it's solving the entire thought process for you right there. And it may be leading you wrong.
That's my view on how it's different at least. They're not calculators or writing. They're text robots that present solutions confidently and offer to do more work immediately afterwards, usually ending a response in "Want me to write you a quick python script to handle that?"
A thought experiment, if you're someone who has used a calculator to calculate 20% tips your whole life, try to calculate one without it. Maybe you specifically don't struggle because you're good at math or have a lot of math experience elsewhere but if you have approached it the way this article is calling bad, you'd simply have no clue where to start.
I guess my point is that the argument being made is "if you lift dumbbells with a forklift, you aren't getting strong by exercising". And that's correct. But that doesn't mean that the existence of forklifts makes us weaker.
So, I guess I'm just saying that LLMs are a tool like any other. Their existence doesn't make you worse at what they do unless you forgo thinking when you use them. You can use a calculator to efficiently solve a wrong equation - you have to think about what it is going to solve for you. You can use an LLM to make a bad argument for you - you have to think about the inputs you're going to have it output for you.
I was just feeling anti-alarmist-headline - there's no intrinsic reason we'd get dumber because LLMs exist. We could, but I think history has shown that this kind of alarmism doesn't come to fruition.
Fair! I'd definitely agree with that! I don't really know the author's intentions here but my read of this article is that it's for the people that ARE skipping thinking entirely using them. I agree completely, to me LLMs are effectively a slightly more useful (sometimes vastly more useful) search engine. They help me find out about features or mechanisms I didn't know existed and help demonstrate their value for me. I am still the one doing the thinking.
The bit of the dialogue you quote is Plato telling a story.
On another bit he says directly: anyone can plainly see there is nothing detrimental about writing itself.
And further down the line he warns against relying on text solely to learn and not other learned people because you might lose context and internalize the wrong interpretation.
Writing or calculators likely do reduce our ability memorize vast amounts of text or do arithmetic in our heads; but to write or do math with writing and calculation, we still must fully load those intermediate facts into our brain and fully understand what was previously written down or calculated to wield and wrangle it into a new piece of work.
In contrast, LLMs (unless used with great care as only one research input) can produce a fully written answer without ever really requiring the 'author' to fully load the details of the work into their brain. LLMs basically reduc ethe task to editing not writing. As editing is not the same as writing, so it is no surprise this study shows an serious inability to remember quotes from the "written" piece.
Perhaps it is similar to learning a new language wherein we tend to be much sooner able to read the new language at a higher complexity than write or speak it?
I have a kid in high school who uses LLMs to get feedback on essays he has written. It will come back with responses like "you failed to give good evidence to support your point that [X]", or "most readers prefer you to include more elaboration on how you changed subject from [Y] to [Z]".
You (and another respondent) both cite the case where someone unthinkingly generates a large swath of text using the LLM, but that's not the only modality for incorporating LLMs into writing. I'm with you both on your examples, fwiw, I just think that only thinking about that way of using LLMs for writing is putting on blinders to the productive ways that they can be used.
It feels to me like people are reacting to the idea that we haven't figured out how to work it into our pedagogy, and that their existence hurts certain ways we've become accustomed to measuring people having learned what we intended them to learn. There's certainly a lot of societal adaptation that should put guardrails around their utility to us, but when I see "They will make us dumb!" it just sets of a contrarian reaction in me.
YES, fully agree, and your kid is definitely doing it right!
I've also found LLMs to be very helpful in proofreading to find inconsistencies, missing items, stray edits, etc..
Here's one variant of prompt I've used with ChatGPT-4o that worked well:
"Focus particularly on inconsistencies and editing errors (stray words or characters, etc.). Be exact and do not include compliments to the author. Please ignore apparent duplicate listings of part numbers and dimensions which are under the illustrations, and single stray or inconsistent spaces."
I'm quite sure that LLMs used in the right way can be amazing for teaching, and I've used them to learn quite a few things. In fact, it seems this is one of the strengths of LLMs — they are not so good at 'reasoning' about unusual content at the edge of a field of knowledge, but are fantastic for compiling info that is commonly used by humans but not yet familiar to a particular human.
And yes, the "They will make us dumb!" response kind of depends if you are starting out dumb — just as a hammer will make you smarter if you use it to pound nails to build a house and a school, but dumber if you hit yourself in the head with it...
The analogy falls apart because calculating isn't math. Calculating is more like spelling, while math is more akin to writing. Writing and math are creative, spelling and calculating are not.
I worked at a supercomputing facility for a few years. The codes are typically decades old, maintained by hundreds of people over the years. By and large, they understand their performance profiles, and are working to squeeze as much out of the code as they can.
In addition, the performance engineers tend to be employed by the facilities, not the computational scientists. They're the ones who do a bunch of legwork of profiling the existing code on their new platform, and figuring out how to squeeze any machine-specific performance out of the code.
A lot of these codes are time-marching PDE solvers that do a bunch of matrix math to advance the simulation, so the kernel of the code is responsible for a vast majority of the time spent during a job. So it's not necessarily a huge chunk of code that needs to be tuned to wring better performance out of the machine.
The parallel communication they do is also to an API, not an ABI - the supercomputing vendors drop in the optimizations in the build of the library for their machine, to take advantage of network-specific optimizations for various communications patterns. If you express your code in the most-specific function (doing a collective all-to-all explicitly, say, rather than building your own all-to-all out of the point-to-point primitive) the MPI build can insert optimized code for those cases.
There's some misalignment because the facility will be in the top 500 for a few years, while the code lives on and on and on. If your supercomputer architecture is really out of left field (https://en.wikipedia.org/wiki/Roadrunner_(supercomputer)) it's not going to be super worth it for people to try to run on it without porting support from the facility.
Then as things shifted back to linear depreciation, it made building/running malls much less attractive, and we're seeing that play out over the 20-30 year capital lifecycle you mention.
You might enjoy http://www.iquilezles.org/live/ where he live codes some ray-marching using some kind of opengl editor. It's not quite what you're talking about, since it's just running the opengl code, but you could imagine it going through some kind of compiler/visualizer pipeline like you're thinking about.
Yes, absolutely. They did something like this at the Sun Microsystems field office outside of Chicago while I worked there. You would log into a sunray with your smartcard and pick up whatever you had left behind in your session, with no permanent desk assignment.
It was unpopular, to say the least. Your personal belongings went into a pedestal on wheels that you could take to whichever workspace you wound up at that day. This was in 2000/2001 or so.
And they can be great. My work issue laptop is a Pixel 2 ChromeBook, and I love it. But that's mine, which I alone use (except when I lend it to someone), and the grease on the keyboard comes from my fingers.
My favorite interviewing question as an IC was "Tell me about someone on your team you admire". It let me learn about what people valued based on why people were admired, and gave some depth-of-bench sense whether there were lots of distinct names, or if everyone was in awe of the one good person on the team.
If you're looking for cross-team health, maybe you could adapt it to "Tell me about someone on the other team that you admire?"
This is an awesome question. What sort of responses have you seen from this? Do most folks have a quick answer or do you get some thought? As an interviewer, I'm not sure I would ever expect a question like this.
Truth be told, I don't think I'm calibrated on the question yet; I've only used it twice. In one org, there was a shining star who attracted all the answers. In the other org, someone laughed because of the number of good answers, and started rattling off names and reasons.
In hindsight, I wish I'd had enough experience with the question and possible scenarios to ask for a second answer from people in the first org; I suspect there were more good answers available, but one obvious answer that everyone snapped to first.
Regarding "theoretical best", I think that is "in the absence of mitigations". I think you can build a service with a higher SLA than one of its dependencies, but only if you recognize that impedance mismatch and build in defenses.
As a contrived example, if you've got a microservice that provides data FOO about a request that isn't actually end-user critical, you can mitigate your dependency on it by allowing your top-level request to succeed even if the FOO data is missing. Or maybe you can paper over blips of unavailability with cached data.
But, yes, know what you depend on and how reliable they are, then see if you need to take more action than that if your target is higher than the computed target.
(Tedious disclaimer: my opinion only, not speaking for anybody else. I'm an SRE at Google)
Building reliable services out of unreliable dependencies is a part of what we do. At the lowest level, we're building services out of individual machines that have a relatively high rate of failure, and the same basic principles can be applied at every layer of the stack: make a bunch of copies, and make sure their failure modes are uncorrelated.
The queue are everywhere - your messaging queue, the threadpool, the hardware threads, and other layers of the stack and APIs you use. The video adds the interesting detail that as you add more tellers (workers) you learn of impending disaster only in the outlier p99 (or higher) latencies; by the time your p85 latency rises, you're already about to stall out.
I have heard people argue that the use of calculators (and later, specifically graphing calculators) would make people worse at math; quick searching found papers like https://files.eric.ed.gov/fulltext/ED525547.pdf discussing the topic.
I can't see how the "LLMs make us dumber" argument is different than those. I think calculators are a great tool, and people trained in a calculator-having environment certainly seem to be able to do math. I can't see that writing has done anything but improve our ability to reason over time. What makes LLMs different?