Yep, it can decohere really badly with bigger context. It's not only context rel...

		tvshtr 4 months ago \| parent \| context \| favorite \| on: Context Rot: How increasing input tokens impacts L... Yep, it can decohere really badly with bigger context. It's not only context related though. Sometimes it can lose focus early on in a way that is impossible to get it back on track.