Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

> For people without a 24GB RAM video card, I've got an 8GB RAM one running

What're you using for this? llama.cpp? Have a 12GB card (rtx 4070) i'd like to try it on.



https://ollama.com/library/devstral

https://ollama.com/

I believe its just a HTTP wrapper and terminal wrapper around llama.cpp with some modifications/fork.


Does ollama have support for cpu offloading?


> Does ollama have support for cpu offloading?

https://www.reddit.com/r/ollama/comments/1df757o/high_cost_o...

https://github.com/ollama/ollama/issues/8291

Yes.


A perfect blend of LMGTFY and helpfulness. :)


lol. I try not to be a total asshole, it sometime even works! :)

Good luck to you mate with your life :)




Consider applying for YC's Winter 2026 batch! Applications are open till Nov 10

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: