No, the ollama default quantisation is 4 bit | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		wkat4242 on June 16, 2024 \| parent \| context \| favorite \| on: Cost of self hosting Llama-3 8B-Instruct No, the ollama default quantisation is 4 bit

brrrrrm on June 16, 2024 [–]

I meant 8b -> 8billion rather than 70b

wkat4242 on June 16, 2024 | [–]

Ah sorry!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact