It also lets a language model answer questions while citing a source, something it fundamentally cannot do on its own.
Everyone talks about "reducing hallucinations" but from a system perspective, everything a LLM emits is equally hallucinated.
Putting the relevant data in context gets around this and provides actual provenance of information, something that is absolutely required for real "knowledge" and which we often take for granted in practice.
Of course, the ability to do so is entirely reliant on the retrieval's search quality. Tradeoffs abound. But with enough clever tricks it does seem possible to take advantage of both the LLMs broad but unsubstantiated content, and specific fact claims.
Everyone talks about "reducing hallucinations" but from a system perspective, everything a LLM emits is equally hallucinated.
Putting the relevant data in context gets around this and provides actual provenance of information, something that is absolutely required for real "knowledge" and which we often take for granted in practice.
Of course, the ability to do so is entirely reliant on the retrieval's search quality. Tradeoffs abound. But with enough clever tricks it does seem possible to take advantage of both the LLMs broad but unsubstantiated content, and specific fact claims.