qwen 2.5 coder 1.5b @ q4_k_m: 1.21 GB memory
qwen 2.5 coder 1.5b @ q8: 1.83 GB memory
I always assumed this to be the case (also because of the smaller download sizes) but never really thought about it.
qwen 2.5 coder 1.5b @ q4_k_m: 1.21 GB memory
qwen 2.5 coder 1.5b @ q8: 1.83 GB memory
I always assumed this to be the case (also because of the smaller download sizes) but never really thought about it.