If you don't mind a stupid question, is this essentially dynamic quantization? I'm trying to understand how this is different from using a regular quantized model to squeeze more parameters into less RAM.
To summarize, they rejected Nvidia's offer because they didn't want one outsized investor who could sway decisions. And "the company was also able to turn down Nvidia due to its stable finances. Hugging Face operates a 'freemium' business model. Three per cent of customers, usually large corporations, pay for additional features such as more storage space and the ability to set up private repositories."
Their business model is essentially the same as GitHub. Host lots of stuff for free and build a community around it, sell the upscaled/private version to businesses. They are already profitable.
This is what Sourceforge did too, and they still had the DevShare adware thing didn't they?
GitHub is great -- huge fan. To some degree they "sold out" to Microsoft and things could have gone more south, but thankfully Microsoft has ruled them with a very kind hand, and overall I'm extremely happy with the way they've handled it.
I guess I always retain a bit of skepticism with such things, and the long-term viability and goodness of such things never feels totally sure.
These are all big hardware firms, which makes a lot of sense as a classic 'commoditize the complement' play. Not exactly pro-consumer, but not quite anti-consumer either!
All of them are public companies, which means that their default state is anti-consumer and pro-shareholder. By law they are required to do whatever they can to maximize profits. History teaches that shareholders can demand whatever they want, with the respective companies following orders, since nobody ever really has to suffer consequences and any and all potential fines are already priced in, in advance, anyway.
Conversely, this is why Valve is such a great company. Valve is probably one of the only few actual pro-consumer companies out there.
Fun Fact! Rarely is it ever mentioned anywhere, but Valve is not a public company! Valve is a private company! That's why they can operate the way they do! If Valve was a public company, then greedy, crooked billionaire shareholders would have managed to get rid of Gabe a long time ago.
> By law they are required to do whatever they can to maximize profits.
I know it's a nit-pick, but I hate that this always gets brought up when it's not actually true. Public corporations face pressure from investors to maximize returns, sure, but there is no law stating that they have to maximize profits at all costs. Public companies can (and often do) act against the interest of immediate profits for some other gain. The only real leverage that investors have is the board's ability to fire executives, but that assumes that they have the necessary votes to do so. As a counter-example, Mark Zuckerberg still controls the majority of voting power at Meta, so he can effectively do whatever he wants with the company without major consequence (assuming you don't consider stock price fluctuations "major").
But I say this not to take away from your broader point, which I agree with: the short-term profit-maximizing culture is indeed the default when it comes to publicly traded corporations. It just isn't something inherent in being publicly traded, and in the inverse, private companies often have the same kind of culture, so that's not a silver bullet either.
It's a worthwhile point to make because if people believe that misconception then it lets companies wash their hands of flagrantly bad behavior. "Gosh, we should really get around to changing the law that makes them act that way."
I once tried hugging face because I wanted I worked through some tutorial. They wanted my credit card details during the registration as far as I remember. After a month they invoiced me some amount of money and I had no idea what it was. To be honest, I don't understand what exactly they do and what services I was paying for, but I cancelled my account and never touched it again. For me that was a totally intransparent process.
Hey, I made the same decision (except I went with the 24gb model, not the 16gb). The other thing I like about having it on a separate Mac Mini is that it it's completely sandboxed, and I don't log into anything with it on my personal machine. It's VERY nice to have this as an isolated environment, and the extra VRAM means that I can run my own local models, and it's got enough beef to do long-running tasks (right now I have it chugging through several gigs of images and building embeddings for them with DINOv2) -- that's the sort of local workload that would crush a Raspberry Pi, but the Macbook is hitting 17 images per second -- all managed by OpenClaw.
All that to say, don't let the naysayers get you down. I bought my Mac Mini last week and have been really happy with it as an isolated environment. Way better than futzing around with VMs. The always-on nature of OpenClaw means that it's nice to be able to restart my personal laptop or do gaming or whatever else I want and I'm not fighting for GPU resources in the background.
I've wondered about such things, and it feels like the 17 Lands dataset might be a good place to scrape play-by-play game data between human players. Feels like it could be adapted to a format usable by this structure, and used as a fine-tuning dataset.
Oh, fascinating - I didn't realize they released actual replay data publicly. It doesn't look like it's quite as rich as I'd like, though - it only captures one row per turn, so I don't think you can deduce things like targeting, the order in which spells are cast, etc.
(I also thought about pointing it at my personal game logs, but unfortunately there aren't that many, because I'm too busy writing analysis tools to actually play the game.)
Another thing that I've thought about doing is to use some sort of computer vision to watch streamers of online games and use STT to capture not just play datasets, but also datasets of their narrated reasoning about why they play what they play.
Would be a lot of work to go through and use computer vision and some measure of reasoning to create these datasets, but some players do an excellent job of narrating their reasoning for their players (thinking of players like Cheon or LSV), so would be fascinating.
Caleb Gannon [0] is one such streamer who does a good job of narrating his plays, and he's also a computer scientist who is very interested in machine-learning projects (he's done several of his own). If you contacted him, I could definitely see him being willing to consent to his videos being used as a fine-tuning dataset for such purposes.
I would be willing to help with creating this dataset if you helped me understand what you would like to see in the final output format.
Down the road I can definitely imagine being interested in that (basically split out the "web-based replay viewer" part from the "LLM harness that I want to debug with a replay viewer" part, and then ingest non-LLM games into the viewer), but for now they're super entangled and I'm not prioritizing separating them cleanly. I'll definitely keep this offer in mind for the future, thanks!
I believe it's even possible to match up game IDs so that (hypothetically) if both players are using 17 Lands, then you can match up a game from both sides and get full information re: the hands of each player as well.
It obviously wouldn't be the full set of games (because not everyone uses 17 lands), but it would certainly be a nonzero dataset.
reply