You can run some models pretty decently using CPU inference only, things like Gemma 3 that are built for exactly that use case or some tiny speech to text models via llama.cpp that I have tested out (not so good). Although not the best for "heavy" tasks, if you just need a decent text generator that can produce more or less sensible, generic output you are good to go.
It's more about demonstrating what's possible on a Pi than expecting GPT-4 level performance. It's designed for LLMs that specialize in tiny, incredibly specific tasks. Like, "What's the weather in my ant farm?" ;)
The vision processing boost is notable, but not enough to justify the price over existing HATs. The lack of reliable mixed-mode functionality and sparse software support are significant red flags.
(This reply generated by an LLM smaller than 8GB, for ants, using the article and comment as context).
This is such a stupid argument. A very significant amount of code never makes it into the public sphere. None of the code I've written professionally in the last 26 years is publicly accessible, and if someone uses a product I've written they likely don't care if it was written with the aid of an LLM or not.
Not to mention agent capabilities at the end of last year were vastly different to those at the start of the year.
Even if a portion of software is not released to the general public, you'd still expect an increase in the amount of software released to the general public.
Even if LLMs became better during the year, you'd still expect an increase in releases.
I found it very opaquely worded the whole way through. I think the work being presented is simply an implementation of the technique in eduos, but short of going and reading the paper I don’t know.
What a load of ableist bullshit. Guess what, you can have autism, ADHD and anxiety and still be really fucking clever. The article starts with the premise that they are incompatible.
They both offer virtualized guests under a hypervisor host. EC2 does have more offload specialization hardware but for the most part they are functionally equivalent, unless I'm missing something...
Yes, the negativity is infuriating. This is the mindset that is going to get left behind. I'm no LLM maximalist but they clearly have their uses in the right context and the right hands.
" Run Any App: If it runs on Windows, it can run on WinBoat. Enjoy the full range of Windows applications as native OS-level windows in your Linux environment"
Bold claim. Lets see it run Fortnite with anti-cheat.
