How long before it runs on a 4 gig card? | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		anshumankmr on May 14, 2023 \| parent \| context \| favorite \| on: Run Llama 13B with a 6GB graphics card How long before it runs on a 4 gig card?

rain1 on May 14, 2023 [–]

You can offload only 10 layers or so if you want to run on a 4GB card

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact