If you're working with local LLMs, why do you care about cost? | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		kbelder 4 months ago \| parent \| context \| favorite \| on: Context Rot: How increasing input tokens impacts L... If you're working with local LLMs, why do you care about cost?

jsemrau 4 months ago [–]

You can use a lower-end GPU (like the RTX 3060), which also uses less energy. But you are right, you won't be encountering model API costs when running it locally.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact