No, the basic model already exists. You could throw together something like this yourself: encode your documents with a Transformers model into a series of vectors. Then you're merely a nearest neighbor search away from finding the most semantically relevant documents. Feed those documents to GPT-3 or some such LLM as contextual state, along with the query, et voila! you have Q&A on documentation.
(Spoiler: I worked on this feature @ Slite) But in practice, the effectiveness of your pipeline will depend greatly upon exactly how you implement each piece.
Here are some of the things we've had to consider and develop, in order for the Q&A to be production worthy:
- at which level is it best to encode documents (sentence, paragraph, whole doc) and how can those tiered embeddings be weighed effectively?
- how to use the natural document hierarchy in the workspace?
- where to add NLI, so we can actually compare in-model "does this statement imply that statement" rather than just comparing vectors
- how to prioritize passages using additional fine-grained ranking against the question (cross-encoding, etc)?
- how to compute a "confidence" score so that we actually take the generative (GPT) part out of the equation and bail out early, if the documents do no contain anything which strongly implies an answer to your specific query
These are just a few of the pieces. But what we learned quickly is that solving the problem of building a great Q&A means first solving many problems that are deeply intertwined with search algorithms, natural language understanding, and other text problems in general.
Thanks for all this, most of which flew right over my head!!
I wonder, can you recommend a resource where one could get "quickly" up to speed on how this stuff conceptually works, something one could ingest in a weekend or so to get a decent handle on it (but not necessarily be capable of doing anything)?
Like, I think I have a decent understanding of the probabilistic "next word" aspect of LLM text generation, but I'm assuming there's an initial "vector query" that happens prior to this part of it?
Also, could you offer any advice on the notion of how AI/ML is able to extract various meanings from text? For example, let's say I wanted to be able to scrape this thread and then for each comment extract various ideas/topics mentioned in each comment (ideally with influence from a custom ontology), detect emotions, claims of fact, etc etc etc - is that sort of thing possible?
