Two things are holding back current LLM-style AI of being of value here:
* Latency. LLM responses are measured in order of 1000s of milliseconds, where this project targets 10s of milliseconds, that's off by almost two orders of magnitute.
* Determinism. LLMs are inherently non-deterministic. Even with temperature=0, slight variations of the input lead to major changes in output. You really don't want your DB to be non-deterministic, ever.
From what I understand, in practice it often is true[1]:
Matrix multiplication should be “independent” along every element in the batch — neither the other elements in the batch nor how large the batch is should affect the computation results of a specific element in the batch. However, as we can observe empirically, this isn’t true.
In other words, the primary reason nearly all LLM inference endpoints are nondeterministic is that the load (and thus batch-size) nondeterministically varies! This nondeterminism is not unique to GPUs — LLM inference endpoints served from CPUs or TPUs will also have this source of nondeterminism.
"But why aren’t LLM inference engines deterministic? One common hypothesis is that some combination of floating-point non-associativity and concurrent execution leads to nondeterminism based on which concurrent core finishes first."
How do you propose we measure signal? Lines of code is renowned for being a very bad measure of anything, and I really can't come up with anything better.
The OP said that they kept what they liked and discarded the rest. I think that's a reasonable definition for signal; so, the signal-to-token ratio would be a simple ratio of (tokens committed)/(tokens purchased). You could argue that any tokens spent exploring options or refining things could be signal and I would agree, but that's harder to measure after the fact. We could give them a flat 10x multiplier to capture this part if you want.
I personally discard code for the tiniest of reasons. If something feels off moments after I open the PR, it gets deleted. The reason we still have 1.2K open PRs is because we can't review all of them in time.
The most likely solution is to delete all of them after a month or two. By that time the open PRs on this project alone will be at least 10-20 more.
Doesn't seem like too efficient process, no? Seems to me like investment in better quality of the output is exactly what is needed here, wouldn't you agree?
I feel they sit of on the opposite end of the OP here. One wants to write out specs to control the agent implementation to achieve a one shot execution. Other side says: let’s won’t waste time of humans writing anything.
I’m personally torn. A lot of the spec talk and now here in combination with TDD etc feels like the pipe dreams of the mid 2000. There was this idea of the Architect role who writes UML and specs. And a normal engineer just fills in the gaps. Then there was TDD. Nothing against it personally. But trying to write code in test first approach when you don’t really have a clue how a specific platform/system/library works had tons of overhead. Also the side effect of code written in the most convenient way to be tested and not to be executed. All in all to throw this ideas together for AI now…
But throwing tokens out of the window and hoping for the token lottery to generate the best PR is also not the right direction in my book. But somebody needs to investigate in both extremes I say.
Actually, nobody said the spec needs to be written by humans.
My personal opinion: with today's LLMs, the spec should be steered by a human because its quality is proportional to result quality. Human interaction is much cheaper at that stage — it's all natural language that makes sense. Later, reasoning about the code itself will be harder.
In general, any non-trivial, valuable output must be based on some verification loop. A spec is just one way to express verification (natural language — a bit fuzzy, but still counts). Others are typecheckers, tests, and linters (especially when linter rules relate to correctness, not just cosmetics).
Personally, on non-trivial tasks, I see very good results with iterative, interactive, verifiable loops:
- Start with a task
- Write spec in e.g. SPEC.md → "ask question" until answer is "ok"/proceed
- Write implementation PLAN.md — topologically sorted list of steps, possibly with substeps → ask question
- For each step: implement, write tests, verify (step isn't done until tests pass, typecheck passes, etc.); update SPEC/PLAN as needed → ask question
- When done, convert SPEC.md and PLAN.md into PR description (summary) and discard
("Ask question" means an interactive prompt that appears for the user. Each step is gated by this prompt — it holds off further progress, giving you a chance to review and modify the result in small bits you can actually reason about.)
The workflow: you accept all changes before confirming the next step. This way you get code deltas that make sense. You can review and understand them, and if something's wrong you can modify by hand (especially renames, which editors like VS Code handle nicely) or prompt for a change. The LLM is instructed to proceed only when the re-asked answer is "ok".
This works with systems like VSCode Copilot, not so much with CC cli.
I'm looking forward to an automated setup where the "human" is replaced by an "LLM judge" — I think you could already design a fairly efficient system like this, but for my work LLMs aren't quite there yet.
That said, there's an aspect that shouldn't be forgotten: this interactive approach keeps you in the driving seat and you know what's happening with the codebase, especially if you're running many of these loops per day. Fully automated solutions leave you outside the picture. You'll quickly get disconnected from what's going on — it'll feel more like a project run by another team where you kind of know what it does on the surface but have no idea how. IMO this is dangerous for long-term, sustainable development.
From my experience, LLMs understand prompt just fine, even if there are substantial typos or severe grammatical errors.
I feel that prompting them with poor language will make them respond more casually. That might be confirmation bias on my end, but research does show that prompt language affects LLM behavior, even if the prompt message doesn't change/
Infinitief scrolling is only mentioned in the title. The actual legislation focuses on addictive patterns of which infinite scroll is just one. The exact formulation will of course matter a lot, but it will not simply be banning infinite scroll, as that would be trivial to circumvent.
The "lethal trifecta" is a limited view on security, as it's mostly concerned with leaking data. This solution focuses on a different aspect: the ability of rogue actions (instead of rogue communications per #3).
Not updating your DL after changing your address is a crime* in all US states. I'm not as familiar with law elsewhere, but would be surprised if that's not true most other places.
*There are exceptions for active duty military personal and other limited exceptions.
It is a law but rarely enforced, also some places like Washington are primarily digital meaning you update your DL address online but they don’t print a new ID unless you request it or your DL is expired
Unless you’re wild camping, campsites have addresses. So do marinas where a ship would need to be docked more or less regularly to establish residency.
As for being a nomad, you don’t need a driver’s license or any kind of ID to wander if you’re willing to sleep rough. If you want to drive on public roadways though, you better have a primary address where the courts can send someone if you kill someone in a traffic accident and bail.
Docking is expensive, so no. It's also only needed once per 5 years or so for maintenance.
Government fining you a ticket doesn't mean your address has to be on the drivers license. They could register the number plate to an SSN for instance.
Did you skip my last sentence? A traffic ticket is not the worst thing you can do in an automobile. And not everyone eligible for a drivers license will have an SSN.
Laws of the government can't override laws of physics. If you don't have a place where you can receive mail, do they just arrest you or what? Do they assign a PO box to you?
This is especially true if the marketing team claims that humans were validating every step, but the actual humans did not exist or did no such thing.
If a marketer claims something, it is safe to assume the claim is at best 'technically true'. Only if an actual engineer backs the claim it can start to mean something.
There's a daily token limit. While I've never run into that limit while operating Claude as a human, I have received warnings that I'm getting close. I imagine that an unattended setup will blow through the token limit in not too much time.
> How close are we to smart dust I wonder? How small can we make wireless communications?
There's two limiting factors for 'smart dust': power (batteries are the majority weight and volume of this vape), and antennae (minimum size determined by wavelength of carrier wave).
I believe you can fit an NFC module in a 5x5mm package, but that does externalize the power supply.
We are going to have to rethink power for smart dust. Like consider that no creature out there is powered by batteries. From the biggest land animal to the smallest microbe it’s all chemistry.
Maybe the smart dust will have to eat microbes and stuff to stay active.
As for communication, we can’t go shoving antennas in them as then they’d be larger than dust. And you can’t use the optical part of the spectrum because of interference with basically everything. You can’t use wavelengths smaller either as you get into UV and high radiation. There is the terahertz radio spectrum [0] between 3mm and 30um that is pretty open and not utilized at all because we haven’t figured out how to make good transmitters. Plus the spectrum isn’t very useful as it isn’t very penetrating and water vapor absorbs it… and it requires lots of power.
Smart dust might have to be more of a distributed computer or something. Or a micro machine that uses chemistry and mechanical magic to do its operations.
> From the biggest land animal to the smallest microbe it’s all chemistry.
Batteries are chemistry. ATP is a chemical battery.
The difference between living things and our machines is primarily in manufacturing methods: we do things in bulk, because we reach from the top with crude, meter-scale tools; nature glues things up from lots of tiny biomolecular nanomachines, and each of those tiny machines has to carry its own power source!
Still, it's highly likely that any form of "smart dust" will resemble living cells as much as, or even more so, it will resemble miniature devices we build today, simply because that's the kind of chemistry that's efficient at smaller scales.
RFID tags are powered wirelessly, one could imagine powering smaller particles when operating on higher frequencies (RFID is on 13.something MHz requiring relatively large coils). A directional antenna could send a pulsed beam to power a subset of the particles in the area and afterwards receive their signals.
It needs to be in the infrared spectrum at least to be useful for smart dust, otherwise the package size is still dominated by the size of the antenna. Even mm-wave radar is marginal here.
* Latency. LLM responses are measured in order of 1000s of milliseconds, where this project targets 10s of milliseconds, that's off by almost two orders of magnitute.
* Determinism. LLMs are inherently non-deterministic. Even with temperature=0, slight variations of the input lead to major changes in output. You really don't want your DB to be non-deterministic, ever.
reply