Yeah who wouldn't love a dip in the sulphur pool. But back to the question, why can't such a model recognize letters as such? It cannot be trained to pay special attention to characters? How come it can print an anatomically correct eye but not differentiate between P and Z?
At what context sizes? I've just run the same prompt and query on my RTX3080 with openwebui as frontend.
When I set the context size to 2048 (openwebui's default), the inference is almost twice as fast as when I set it to 4096. I can't set the conext size any higher because my GPU only has 12GB of RAM and ollama crashes for larger context sizes.
Still, I find that thoroughly odd. Using the larger conetext size (4096), the GPU usage is only 50% as seen in nvtop. I have no idea why.
Not sure why this hasn't got any comments yet, so I'll add one.
While my first thought was - admittedly - ugh, yet another LLM/AI code assistant, I'm so glad I tried it. It's just... amazing. The "act" mode is a game changer for me. I've heard Cursor does it similarly, but I haven't tested that yet.
I have however used Continue.dev for quite a while, and even though continue is great, Cline is even better. I didn't know what I was missing.
So, I'd encourage everyone to at least try it.
---
Just a question to Nick: what's the deal with the Roo fork?
I think it's supposed to quite the opposite, at least it was like that for me. When gpt3 first launched, it seemed almost like magic for me, and I have some limited ml background too. Much later I saw the 3blue1brown Video about transformers, and I was almost disappointed to see the math itself is rather simple.
My main take away was that even simple basics can produce astonishing results, I this case if they're just scaled large enough. That incredibly complex and useful emergent behavior can result from what seems like conways-game-of-life like principles.
It's the large training data which contains the knowledge for that complex and useful emergent behavior. It's like if you could import vast information about the world into Conway's game of life to enable increasingly complex levels of emergence.
Oh idea of "vast patterns" in Conway's game of life seems like a great metaphor for LLM transformers. Explaining hidden markov chains with some extras or whatnot to a layman doesn't give a good mental image but is what I guess it's like.
Probability chains alone don't seem to give a good mental visualization of how such a system comes to certain "decisions" or "thought patterns". But watching the Game of Life you can see fascinating patterns which emerge and lead to interesting patterns. That's easy to extrapolate.
Maybe in the future NNs will be understable sorta like Game of Life, "oh that NN section is running pattern 27 on XY input data. That's gonna be an unstable behavioral element combined with pattern 38c over here." Not sure if that's a fascinating or dreadfully boring prospect though.
I don't know, I've installed openwrt on each device I've owned especially because their original firmware wasn't supported anymore (or crap to begin with).
Often because the cheap devices were either all I could afford or because I've even gotten them for free or basically free, like on flea markets.
I think they actually did manage to fuck up even the small unmanaged switches. I have three unmanaged switches at home, one on the ground floor and two in the first floor. Ground floor is an 8 port netgear, first floor are one to link and one d link.
Every couple of weeks, the entire wired network goes down. Not even pinging adresses works. The d links ports leds are all flashing (perfectly in sync!) until I power cycle it. Then everything goes back to normal.
I have no idea what happens, and I should probably replace the d link soon.
Are you aware about broadcast storms? Perhaps you somehow accidentally introduced a loop in the network? The symptoms fit that exactly. https://en.wikipedia.org/wiki/Broadcast_storm
It could be missing IGMP Snooping Protocol support in a network with IPTV or custom VLAN setups. There are 3 versions (IGMP snooping (v1, v2, and v3)), managed switches have them all, unmanaged usually don't have them. To avoid problems, only pass a single VLAN to the unmanaged switch (it must be behind the managed switch for that), otherwise the unmanaged switch can and usually will bring a network down after some time. Or just use a switch with IGMP snooping support.
If the D-link has a wall wart which you could easily replace, try that. (And maybe a real surge strip, if you've got one handy.) Iffy power can cause all sorts of bizarre behavior.
I haven't enabled jumbo frames knowingly on my system, but even if I had, why would the issue occur only every few weeks? Also, it seems to be rather independent of the actual network load.
A friend had networked speakers that would freeze until a manual reboot time to time. It turned out to be the Linux running within the speakers that crashed on the occasional jumbo frame.
