Hacker Newsnew | past | comments | ask | show | jobs | submit | chid's commentslogin

Given the high bar of entry 160VRAM GPU - is there anything practical one can use this for?

The model being 32B could run in <20GB VRAM with Q4 quantization (minimal loss of quality), or 80GB unquantized at full fidelity. The quoted 160GB is for their recommended evaluation settings.

There's a few pre-quantized options[0] or you can quantize it yourself with llama.cpp[1]. You can run the resulting gguf with llama.cpp `llama-cli` or `llama-server`, with LM Studio or with Ollama.

0: https://huggingface.co/models?search=cwm%20q4%20gguf

1: https://huggingface.co/spaces/ggml-org/gguf-my-repo


I see, still a fair more VRAM than I have access to. Thanks for sharing that information.

Interesting though one would think this is also an obvious finding.

Quantifying this would be interesting though.


it definitely feels like it.


Took quite a while for it to show up oddly.


I can't think of any other than potentially oil and gas (though they probably use a lot of it in head office type environment).


Has this been implemented anywhere else in the world other than China?


Do they even have advertising normally?


yes i believe so


Anna doesn't have any advertising. Their income is purely driven off donations, most of which are part of subscription packages that offer faster downloads.


yeah meant more libgen, which i believe some instances have ads.

anna's has "donations for speed" and dark pattern hide the links to the fast external websites


I just had look and seems most have a warranty of five years, it doesn't seem there are many outliers to that. Funnily though it seems the AC unit is most likely going to operate many times longer, as the OP points out just the controller isn't working properly, and how long would a consumer be willing to fight this in the summer in Perth.


I’ve used the Australian Consumer Law several times at this point and most times you say to the customer service rep “The Australian Consumer Law gives guarantees of acceptable durability that I do not think this product has met. How are you planning on remedying the situation?”

They’ll then generally go to speak to a supervisor.

The’ll then come back and say they’ve been instructed to help you escalate to ‘senior management’.

A day later they’ll contact you with how to get your product fixed for free.

I’ve not had a significant delay, but you’d be spewing if it was a heat wave in summer.


I agree, practically this would've been much more in terms of combined efforts (to collect, parse, and validate) this analysis. Absolutely bargain if McKinsey was able to create from scratch ;)


according to this page the mini (current model) will have it activated I suspect when the Homepod 2 comes out.


Ahh thanks, I checked the actual mini product page which doesnt seem to be updated yet.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: