Please don't anthropomorphise. These are statistical text prediction models, not people. An LLM cannot be "deceptive" because it has no intent. They're not intelligent or "smart", and we're not "teaching". We're inputting data and the model is outputting statistically likely text. That is all that is happening.
If this is useful in it's current form is an entirely different topic. But don't mistake a tool for an intelligence with motivations or morals.
> It seems like the author hasn't really used the latest models
The author addresses this point.
> While I’m sure the technology and its costs will continue to improve, it’s hard to see how that would mitigate most of these harms. Many would just as likely be intensified by greater speed, efficiency, and affordability.
> This sort of technology distributes instability to the many at the bottom, while consolidating benefit at the top—and there has arguably never been a more efficient mechanism for this than AI.
Personally, I'm really tired of every criticism of AI being met with "you haven't tried the latest models". The model isn't the point. It doesn't matter how good it is, it cannot possibly outweigh the harms.
> I wrote my last line of code about a month ago after 20+ years coding
You are exactly the kind of person the author talks about
> To be an AI optimist, I’m guessing you must not be worried about where your next job might come from, or whether you can even find one. The current dire state of the job market, I have to assume, doesn’t scare you. You must feel secure. Maybe it’s because you’ve already made a name for yourself. Maybe you’re known at conferences, or on podcasts. Maybe you’re just senior enough that your résumé opens doors for you.
I fear you've entirely missed the point of the article. Just because you believe you can get value from it, does not make up for the downsides to everyone else, and it's quite literally privilege to ignore that.
The world changes, you either adapt or not. People who saw this coming could have positioned themselves with plan a b/c. No different from when other societal changing technologies arrived in the past. What does crying about it in blog posts achieve.
The adoption of AI in society at large is not foregone conclusion. Acting as if it's unstoppable and washing your hands of the consequences is wilful ignorance. But, it doesn't have to be this way. You do have a choice to not use or encourage this technology.
Think of all of the other public health changes over the years: CFCs, leaded gasolene, asbestos, etc. Apparent miracles of technology that looked unassailable in their ubiquity, and through the blood and tears of many they were all but eliminated.
That's the crux of the article. There are harms, and if you ignore them it's because you don't think it'll affect you.
I'm not trying to be rude, we all make our choices and nobody is a saint. I still eat meat even knowing the damage of the meat industry. But don't pretend you don't have a choice here, or wash your hands of the harms because you feel you won't make a difference.
It's more comparable to the internet, mobile phones and social media. It is likely to cause some harms as well as provide great utility. I disagree it's not inevitable though and I do think the harms can be mitigated, that where the effort should be focused.
You'd be out of your mind to trust an OpenAI built Slack competitor. Slack, for all of it's many faults, is two things:
- Reliable, both in terms of "service uptime" and in terms of "Slack isn't going to rugpull your features"
- Secure. Slack don't have a history of major breaches or data exposure.
Both of these means that people feel comfortable relying on it. Who would possibly trust OpenAI with data security, or that their app will still be around in 3 years.
> Layers upon layers of abstractions that abstract nothing meaningful, that solve problems we shouldn’t have had in the first place, that create ten new problems for every one they claim to fix.
LLM generated code is the ultimate abstraction. A mess of code with no trusted origin that nobody has ever understood. It's worse than even the worst maintained libraries and frameworks in every way.
They've tried to play it both ways. Not AI enough for the AI fanboys, but want to keep a toe in there for everyone else. They'd be better placed rejecting AI entirely, and then when the bubble pops they'll be well positioned to sweep in and eat VSCode's lunch with a product that actually works.
> They cite cherry picked announcements showing that LLM usage makes development slower or worse. They opened ChatGPT a couple times a few months ago, asked some questions, and then went “Aha! I knew it was bad!” when they encountered their first bad output instead of trying to work with the LLM to iterate like everyone who gets value out of them.
"Ah-hah you stopped when this tool blew your whole leg off. If you'd stuck with it like the rest of us you could learn to only take off a few toes every now and again, but I'm confident that in time it will hardly ever do that."
To be fair, that does seem be a very common usage pattern for them, to the point where they're even becoming a nuisance to open source projects; e.g. https://news.ycombinator.com/item?id=45330378
I'm impressed that such a short post can be so categorically incorrect.
> For years, despite functional evidence and scientific hints accumulating, certain AI researchers continued to claim LLMs were stochastic parrots
> In 2025 finally almost everybody stopped saying so.
There is still no evidence that LLMs are anything beyond "stochastic parrots". There is no proof of any "understanding". This is seeing faces in clouds.
> I believe improvements to RL applied to LLMs will be the next big thing in AI.
With what proof or evidence? Gut feeling?
> Programmers resistance to AI assisted programming has lowered considerably.
> It is likely that AGI can be reached independently with many radically different architectures.
There continues to be no evidence beyond "hope" that AGI is even possible, yet alone that Transformer models are the path there.
> The fundamental challenge in AI for the next 20 years is avoiding extinction.
Again, nothing more than a gut feeling. Much like all the other AI hype posts this is nothing more than "well LLMs sure are impressive, people say they're not, but I think they're wrong and we will make a machine god any day now".
Strongly agree with this comment. Decoder-only LLMs (the ones we use) are literally Markov Chains, the only (and major) difference is a radically more sophisticated state representation. Maybe "stochastic parrot" is overly dismissive sounding, but it's not a fundamentally wrong understanding of LLMs.
The RL claims are also odd because, for starters, RLHF is not "reinforcement learning" based on any classical definition of RL (which almost always involve an online component). And further, you can chat with anyone who has kept up with the RL field, and quickly realize that this is also a technology that still hasn't quite delivered on the promises it's been making (despite being an incredibly interesting area of research). There's no reason to speculate that RL techniques will work with "agents" where they have failed to achieve wide spread success in similar domains.
I continue to be confused why smart, very technical people can't just talk about LLMs honestly. I personally think we'd have much more progress if we could have conversations like "Wow! The performance of a Markov Chain with proper state representation is incredible, let's understand this better..." rather than "AI is reasoning intelligently!"
I get why non-technical people get caught up in AI hype discussions, but for technical people that understand LLMs it seems counter productive. Even more surprising to me is that this hype has completely destroyed any serious discussions of the technology and how to use it. There's so much oppurtunity lost around practical uses of incorporating LLMs into software while people wait for agents to create mountains of slop.
> why smart, very technical people can't just talk about LLMs honestly
Because those smart people are usually low-rung employees while their bosses are often AI fanatics. Were they to express anti-AI views, they would be fired. Then this mentality slips into their thinking outside of work.
No, it can still be modeled as a finite state machine. Each state just encodes the configuration of your memory. I.e. if you have 8 bits of memory, your state space just encodes 2^8 states for each memory configuration.
Any real-world deterministic thing can be encoded as a FSM if you make your state space big enough, since it by definition there has only a finite number of states.
You could model a specific instance of using your computer this way, but you could not capture the fact that you can execute arbitrary programs with your PC represented as an FSM.
Your computer is strictly more computationally powerful than an FSM or PDA, even though you could represent particular states of your computer this way.
The fact that you can model an arbitrary CFG as an regular language with limited recursion depth does not mean there’s no meaningful distinction between regular languages and CFG.
> you can execute arbitrary programs with your PC represented as an FSM
You cannot execute arbitrary programs with your PC, your PC is limited in how much memory and storage it has access to.
>Your computer is strictly more computationally powerful
The abstract computer is, but _your_ computer is not.
>model an arbitrary CFG as an regular language with limited recursion depth does not mean there’s no meaningful distinction between regular languages and CFG
Yes this I agree. But going back to your argument, claiming that LLMs with a fixed context-window are basically markov chains so they can't do anything useful is reductio ad absurdum in the exact same way as claiming that real-world computers are finite state machines.
A more useful argument on the upper-bound of computational power would be along the lines of circuit complexity I think. But even this does not really matter. An LLM does not need to be turing complete even conceptually. When paired with tool-use, it suffices that the LLM can merely generate programs that are then fed into an interpreter. (And the grammar of turing-complete programming languages can be made simple enough, you can encode Brainfuck in a CFG). So even if an LLM could only ever produce programs with a CFG grammar, the combination of LLM + brainfuck executor would give turing completeness.
I never claimed that. They demonstrate just how powerful Markov chains can be with sophisticated state representations. Obviously LLMs are useful, I have never claimed otherwise.
Additionally, it doesn’t require any logical leaps to understand decoder only LLMs as Markov Chains, they preserve the Markov Property and otherwise be have exactly like them. It’s worth noting that encoder-decoder LLMs do not preserve the Markov property and can not be considered Markov chains.
Edit: I saw that post and at the time was disappointed by how confused the author was about those topics and how they apply to the subject.
> A2UI lets agents send declarative component descriptions that clients render using their own native widgets. It's like having agents speak a universal UI language.
Why the hell would anyone want this? Why on earth would you trust an LLM to output a UI? You're just asking for security bugs, UI impersonation attacks, terrible usability, and more. This is a nightmare.
If done in chat, it's just an alternative to talking to you freeform. Consider Claude Code's multiple-choice questions, which you can trigger by asking it to invoke the right tool, for example.
None of the issues go away just because it's in chat?
Freeform looks and acts like text, except for a set of things that someone vetted and made work.
If the interactive diagram or UI you click on now owns you, it doesn't matter if it was inside the chat window or outside the chat window.
Now, in this case, it's not arbitrary UI, but if you believe that the parsing/validation/rendering/two way data binding/incremental composition (the spec requires that you be able to build up UI incrementally) of these components: https://a2ui.org/specification/v0.9-a2ui/#standard-component...
as transported/renderered/etc by NxM combinations of implementations (there are 4 renderers and a bunch of transports right now), is not going to have security issues, i've got a bridge to sell you.
Here, i'll sell it to you in gemini, just click a few times on the "totally safe text box" for me before you sign your name.
My friend once called something a babydoggle - something you know will be a boondoggle, but is still in its small formative stages.
> None of the issues go away just because it's in chat?
There is a wast difference in risk between me clicking a button provided by Claude in my Claude chat, on the basis of conversations I have had with Claude, and clicking a random button on a random website. Both can contain a malicious. One is substantially higher risk. Separately, linking a UI constructed this way up to an agent and let third parties interact with it, is much riskier to you than to them.
> If the interactive diagram or UI you click on now owns you, it doesn't matter if it was inside the chat window or outside the chat window.
In that scenario, the UI elements are irrelevant barring a buggy implementation (yes, I've read the rest, see below), as you can achieve the same things as you can do that way with just presenting the user with a basic link and telling them to press it.
> as transported/renderered/etc by NxM combinations of implementations (there are 4 renderers and a bunch of transports right now), is not going to have security issues, i've got a bridge to sell you.
I very much doubt we'll see many implementations that won't just use a web view for this, and I very much doubt these issues will even fall in the top 10 security issues people will run into with AI tooling. Sure, there will be bugs. You can use this argument against anything that requires changes to client software.
But if you're concerned about the security of clients, mcp and hooks is a far bigger rats nest of things that are inherently risky due to the way they are designed.
This article seems to suggest that businesses are going to swap domain-specific SaaS tools, written and tested by people knowledgable in the domain with specific SLAs for vibe coding everything. But your AI subscription is still a SaaS?
All you've done is swapped a SaaS built for your problem domain with another, more expensive SaaS that has no support at all for your actual problem. Why would anyone want that? People buy SaaS products because they don't want to solve the problem, they just want it fixed. AI changes nothing about that.
> Vibe coding actually works. It creates robust, complex systems that work. You can tell yourself (as I did) that it can’t possibly do that, but you are wrong.
This is such a bad take. I'm convinced that engineers simply don't understand what the job is. The point was never "does it output code that works", the point is "can it build the right thing in a way that is maintainable and understandable". If you need an LLM to understand the output then you have failed to engineer software.
If all you're doing is spitting out PoCs and pure greenfield development then I'm sure it looks very impressive, as the early language models did when it looked like they were capable of holding a conversation. But 99% of software engineering is not that kind of work.
If this is useful in it's current form is an entirely different topic. But don't mistake a tool for an intelligence with motivations or morals.
reply