Impressive numbers, but have you ever tried to use one of these things to get actual work done? They're terrible at it! Until these companies start demonstrating some actual value this is merely a hype bubble. Nvidia is slightly different categorically because they're selling the picks and shovels in this hype gold rush--but they were also in the last one (web3/crypto/nfts). At some point if these folks don't actually start delivering the value they're hyping it's all coming crashing down.
The thing is, it's mostly managers and the like getting wow'd by how cool it seems initially, and they're the ones driving the hype. I can ask it in plain English and it answers with flowery language that reads like a professional wrote it, wow!
Of course, once you actually try to use it for any real work that isn't a toy project with a trillion examples online, you very quickly run into the myriad of flaws. But they're not using it deeply like that, they're just interested in how they can cut costs or prop up their own product by wow-ing other managers with AI features even if they don't make a lick of sense from an end-user perspective.
This isn't even to say it's completely useless, just that the hype and marketing is so insanely divergent from the reality, it's shocking. For me, it's at best a 5% productivity boost for very specific tasks like formatting structured text or something like that, but I wouldn't say that justifies a 500 billion dollar investment...
Your opinion, like many others, is wildly misinformed.
It’s not “just managers”. People are legit using copilot and it is saving time and helping develop better code. There are 100+ engineers in my company using it. And it’s not just copiloting, it is better than Wikipedia for explaining RFCs or academic papers. That is very important for ramping people when they can interactively ask hundreds of questions to learn about something without bothering top engineers.
You really need to actually get some real world experience as not just pop off.
Anyone who refers to LLMs as “stochastic parrots” has made their choice and is not interested in understand what areas can benefit and what areas cannot. They’re just scared and ignorant.
Copilot? I personally found Copilot pretty useless. And very distracting trying to constantly incorrectly guess what I was doing.
I do use chatgpt/claude for quickly creating first draftw of frontend pages as I find it quicker, but I have to do extensive edits to get it right. But anything non-boiler plate I am faster writing it myself.
I have also quickly found it is a stochastic parrot but that's ok for boilerplate stuff. It is useless at anything complicated and quickly starts hallucinating methods, parameters and functions when you point out problems with the code.
I am beginning to believe that anything niche, where it can't steal from its training data, and it will always be utterly useless.
I am curious how you feel it's so important to 100+ engineers, how do you use it that makes it effective for you?
You put this better than I could have. I am really impressed by LLMs' ability to respond to a prompt with code which compiles and runs. That's a hugely impressive result. But it doesn't mean the technorapture is just around the corner. There's a vast gulf between what LLMs can do currently and what skilled people do, and it's not "obvious" that there's some path from here to a world in which LLMs become some kind of superintelligent engineer-gods. It could be physically impossible, for all we know. That's why the pseudoreligious twaddle gets under my skin so much--it's the naive assumption that progress is linear, instead of a jumpy "fits and starts" process of accidental discoveries building on one another (or encountering dead ends). There's no way to know whether these things can be improved enough to be truly useful for real work and pretending it's "right around the corner" won't necessarily make it happen.
Yeah yeah, I've heard all of this a million times. Yet, I open up Claude and it keeps forgetting to not give me React code after I remind it multiple times to not give me React code. It still randomly drops all context of the conversation at some point and enters a loop where it regurgitates the same answers it already gave me. It still gets basic facts wrong, even when the reference/factsheet is provided to it a sentence ago. The code the juniors and mediors are shitting out with the help of LLMs is all dogwater that doesn't pass the sniff test, and the worst part is they themselves can't explain half the lines they're shoveling.
I suspect the next refrain is gonna be along the lines of me using it wrong, or using the wrong model, or the wrong prompt, or this or that. At the end of the day, if even after arduously trying to use it pretty much daily I still find all the models so far useless for real work, then no amount of fanboyism is going to convince me to disregard the reality of it as I experience it.
I can write the most insanely in-depth prompt on the planet (wasting 10 minutes of my time in the process), and 9 times out of 10 the reply I get is barely any more coherent than a single sentence along the lines of "Give me X" would've given me back.
> it is better than Wikipedia for explaining RFCs or academic papers.
I dread the future we're creating where people are relying on LLMs to parse RFCs or papers.
> Anyone who refers to LLMs as “stochastic parrots” has made their choice and is not interested in understand what areas can benefit and what areas cannot.
You're arguing with a strawman as I haven't used the word stochastic or parrot anywhere in the comment you're replying to or have implied anything of the sort about LLMs, and I'm even saying that I don't think it's completely useless, just not nearly as useful as the hype would lead one to imagine. Did you perhaps put my comment through an LLM and thus were arguing against imaginary points that I never made?
> I dread the future we're creating where people are relying on LLMs to parse RFCs or papers.
Amen & Hallelujah (plz allow western reference)
Arseh0lez in the mix? go ahead, Hit yer FLAGGIT
We're ALL Jock Tamshun's Bairns ... Meanin' "ALL G__ Children"
Myself are Scot's Arysh. Readers might be in China, Iraq, N. Korea. It still stands ~ ALL meanin' ALL Jock Tamshun's bairn. All means ALL, ya see?
That said, this story is far from over.
'AI' don't know peanut butter from cat poop
How'd AI / LLM do for IRL (in real life) Jeopardy - NO play on words.
Let's try THIRST (no water) for $100, Alex.
Has AI / LLM ever slaked THIRST ~ no drinking water? Living beings?? Did ever 'AI/LLM' put a water tank on a pickup truck? SHOW US (or What profit will ye' boogaloo's gain there?) We're open minded, simply SHOW US
How about Starvation for $500, Alex?
WHOOPEE The Daily Double!
Has AI / LLM EVER fed the STARVING Or rain Manna (overlook Western ref) upon starving refugees? Wilderness? Please, (please?) show factual examples? PLEASe show us what AI/LLM has done to remedy this NEED
Let's try immaculate conception for $700
- Invoke our absent SArCASM flag -
Explain how's this Whiskey-Tango-Foxtrot BOOGALOO AI might approximate companionship much less intimacy never mind concieve life. Merely curious
This is a Genuine need of all life forms (cockroaches included) even P. Dorov (telegram)
~ end snarkasm ~
Does there exist ANY verifiable evidivence supporting how this "AI / LLM" provides any HELP for the needs of ALL life, including even primates?
MAKE US UNDERTAND
Exactly (and be specific)
What does 'AI' (et alia) bring to mitigate fundamental NEED.
Water,
Food.
Company.
LIFE?
HOW DOES tHIS HELP
Genuinely & with Deep Sincerety we ask?
Sincere Respects,
Virginwidow
AKA An0n.1984
AKA Ånne Äůghnimax
And... While we're at it kids (acronyms baffle old folks) try GFOD. Try Urban Dictionary, Wikipedia. Giggle
> Impressive numbers, but have you ever tried to use one of these things to get actual work done? They're terrible at it!
If you mean "have you tried using an LLM like ChatGPT or Claude for doing development work", then you're wrong.
I'm very confident in saying this, both because I personally use AI for programming and get a lot of value from it, and also because almost everyone else I know in real life also uses AI for programming.
But also because there are dozens/hundreds of accounts of very good developers getting a lot of value from AI in their work.
So at this point, insisting AI products are terrible doesn't say much about the AI products, so much as it says something about your ability to get value out of them.
you should probably disclose what you work on as this makes you not exactly impartial, it would be quite bad for you to criticize value of what you sell:
> I'm the CEO of Hipposys Ltd, a boutique Data & AI Engineering shop (www.hipposys.com). We specialize in building RAG systems, Data Warehouses, and anything else related to the intersection between Data & AI. If you have any questions, please feel free to reach out to me at edan@hipposys.com, and we can see if we are a good fit!
First of all, I'm not sure what you mean by "disclose" - you took that from my HN profile which is a click away. Do you think I should mention this in every comment I make about LLMs?
Secondly, you're confusing cause and effect. I've been in the industry for more than twenty years working on many different things, and my company has done mostly data engineering work for most of its existence, the AI engineering is a relatively-new (past year or so) shift.
I don't think highly of AI because I work in AI - I work in AI because I think highly of it!
there are MANY of us who have not only tried it but using it daily to do amazeballs shit. you can read numerous other posts here of people saying how indispensible they have become in their day-to-day work. just like with any other tool, some will learn its strengths and limitations and take them to the limit while others will say “this is no good…”
if I was in the first cohort I would think long and hard how to move to the second…
