I realized, being a manager overseeing people, chasing after career progression was a wrong path. I should have stuck to what I like more, engineering, not bullshitting.
Haha not exactly like predicting actual questions. Just trying to find patterns or what topics show up often. I made this to help my study, didn’t think people would care this much.
It is shown running on 2 or 4 raspberry pis; the point is that you can add more (ordinary, non GPU) hardware for faster inference. It's a distributed system. The sky is the limit.
It doesn't even scale linearly to 4 nodes. It's slower than a five year old gaming computer. There is definitely a hard limit on performance to be had from this approach.
ah, Thanks! but what can a distributed system like this do? is this a fun to do, for the sake of doing it project or does it have practical applications? just curious about applicability thats all.
I'm going to get downvoted for saying the B-word, but I imagine this growing up into some kind of blockchain thing where the AI has some goal and once there's consensus that some bit of data would further that goal it goes in a block on the chain (which is then referenced by humans who also have that goal and also is used to fine tune the AI for the next round of inference). Events in the real world are slow enough that gradually converging on the next move over the course of a few days would probably be fine.
The advantage over centralizing the compute is that you can just connect your node and start contributing to the cause (both by providing compute and by being its eyes and hands out there in the real world), there's no confusion over things like who is paying the cloud compute bill and nobody has invested overmuch in hardware.
Eh, something that's trapped in a blockchain and can only move forward when people vote to approve its next thought/block is a lot less scary to me than some centralized AI running behind closed doors and taking direction from some rich asshole who answers only to himself.
In a distributed system, the overall performance and scalability are often constrained by the slowest component.
This Distributed Llama is over Ethernet..
The raspberry pis aren't really the point, since the raspberry pi os is basically debian, this means you could do the same thing on four much more powerful but still very cheap ($250-300 a piece) x86-64 systems running debian (with 32, 64 or 128GB RAM each if you needed). Also opening up the possibility of relatively cheap pci-express 3.0 based 10 Gbps NICs and switch between them, which isn't possible with raspberry pi.
About the diverging roles, i agree its likely go down that path, based on my own career experience as well. The fact that I enjoy the freedom to be as innovative with quick turnaround, i could be in terms of directly able to solve business issues, rather than process entanglements set forth for managing developers.
Exactly! And as the boundaries between specializations like backend, frontend, embedded, and data science are going away, developers can express their creativity on an entirely new level. Generalists will thrive in the future!
I do see the trend in corporates to push this agenda, not sure if it will survive though...becoming a developer again may sound like downsizing your career progression in many cultures.
