I don't know anything about the T0pp, so can't comment on that.
I agree with you on tuning of LLMs. We did some work at my last job before I retired (as CTO of a major medical clinical and research organization) using GPT-3 up-trained on medical vocabulary to generate physician's notes as a summary of a transcribed visit. The results were impressive. Still not usable though. Most of what it generated was correct (and essentially all of it was well composed and readable), but false statements and non sequiturs still crept in at an unacceptable rate.
I think the technology is amazing, and very valuable. But I do also think that tying it to "hard" knowledge models - akin to the way deep physics is done, but coupling to the language model, rather than to generalized neural networks, is going to prove will eventually make it a complete success in specific domains.
I agree with you on tuning of LLMs. We did some work at my last job before I retired (as CTO of a major medical clinical and research organization) using GPT-3 up-trained on medical vocabulary to generate physician's notes as a summary of a transcribed visit. The results were impressive. Still not usable though. Most of what it generated was correct (and essentially all of it was well composed and readable), but false statements and non sequiturs still crept in at an unacceptable rate.
I think the technology is amazing, and very valuable. But I do also think that tying it to "hard" knowledge models - akin to the way deep physics is done, but coupling to the language model, rather than to generalized neural networks, is going to prove will eventually make it a complete success in specific domains.