I've been wondering recently if there's some practical path forward for some sort of co-op based LLM training. Something which puts the power in the hands of the users somehow.

You may check Prime Intellect's prime-diloco and Nous Research's DisTrO

while we're talking pronunciation I'm on an (entirely pointless) one man mission to have "lemon" stick as a pronunciation of "LLM".

The only two models I ever hear non technical people mention are ChatGPT and occasionally Gemini

I hope you're right, but is there any guarantee that there will continue to be institutions willing to spend the money to produce open models?

I almost wonder if we need some sort of co-op for training and another for hosted inference

There doesn't seem to be any sign of Chinese companies stopping to produce open models to destroy the American moat.

Given that a lot of the R&D in China is state sponsored that also seems to be a good pawn in US-China relations.

Eventually there'll be some kind of standard for licensing that's required of LLM runtimes, like software and digital media. Of course people will figure out workarounds, but just like pirated software, half of it will be infested with malware so most people will just pay for the license.

I've been taking a similar approach with my own exploration.

I worry that if the reality lives up to investors dreams it will be massively disruptive for society which will lead us down dark paths. On the other hand if it _doesn't_ live up to their dreams, then there is so much invested in that dream financially that it will lead to massive societal disruption when the public is left holding the bag, which will also lead us down dark paths.

It's already made it impossible to trust half of the content i read online.

Whenever i use search terms to ask a specific question these days theres usually a page of slop dedicated to the answer which appears top for relevancy.

Once i realize it is slop i realize the relevant information could be hallicinated so i cant trust it.

At the same time im seeing a huge upswing in probable human created content being accused of being slop.

We're seeing a tragedy of the information commons play out on an enormous scale at hyperspeed.

I agree.

While I'm generally sympathetic to the idea that humans and LLM creativity is broadly similar (combining ideas absorbed elsewhere in new ways), when we ask for something that already exists it's basically just laundering open source code

Same for me. I also make extensive use of adding links to anything relevant. Spent a bunch of time discussing something in a slack thread: link it. Read some documentation: link it. Had a chat with an llm in a chat window: link it. Writing notes about how a bunch of code works : link to the functions. For this last one I've registered a custom vim:// URL scheme on my system which lets me link to a symbol within a given file, and when clicked focuses the relevant tmux window and navigates the relevant vim instance (using named pipes) to the symbol, or opens a fresh one if not already open.

I generally try to avoid adding external links as I found that those resources tend to get lost very fast. Of course, this is not always feasible, but whenever I can try to copy over the contents into my notes.

After at least a century of labour saving devices being produced and widely adopted in all areas of our lives, how much less time do we spend labouring now?

I'm currently reading "The disappearance of rituals" by Byun-Chul Han which is highlights the roles of ritual to bring people together into those shared experiences which bind people into a shared narrative and world view.

There are parts of the book I agree with and some I disagree with, but an earlier version of me would have dismissed the whole topic as fluff and any notion of ritual or narrative as superstitious nonsense that needed to be swept away by the light of reason and science. Ultimately though the things that really matter in people's lives tend to be those things which are not coldly rational - love and a place in a wider narrative