My initial instinct was that this has to be getting some nudges from whatever human-in-the-loop is going on at OpenAI.
But then I realized that somewhere on the Internet there inevitably is a message board where people play the "find me some shit on the internet" game, and there's some rabid subculture around it with zillions upon zillions of of examples, and it's in the Bing index, and all the nudging it would need is to emphasize that sort of thing in the corpus.
There is no literal “human in the loop” for generations, of course, but the model is fine-tuned on examples written by human contractors of instructions being given followed by correct responses. I assume that training is essential to it being able to follow directions of this length, or really any directions at all. If you try using the pre-InstructGPT version of Davinci (model=“davinci”, not model=“text-davinci-002”), you’ll find it’s as cumbersome and annoying as you remember GPT-3 being in 2018.
Ah thank you for the color. My thought was prompted by someone (I forget who? Andreessen maybe?) proposing a possible explanation for the LaMDA bot arguing that's it's conscious: there are reams upon reams of sci-fi books with robots having that debate! These are almost certainly in the Books corpus.
It's my opinion that these hyper-scaled transformers are actually a great deal less mysterious than seems to be in the zeitgeist, but for reasons that actually make me think there is a lot of headroom on capability: when the corpus is basically everything ever digitized like it is when a search or social network megacorp trains one, the only thing it could never do is something literally unprecedented on the Internet.
The mechanism can be good old `P(thing|internet)`, but if the KL-divergence is low enough, sampling from the modeled distribution can write something like Tristan und Isolde or paint something like the Mona Lisa.
But then I realized that somewhere on the Internet there inevitably is a message board where people play the "find me some shit on the internet" game, and there's some rabid subculture around it with zillions upon zillions of of examples, and it's in the Bing index, and all the nudging it would need is to emphasize that sort of thing in the corpus.
Very impressive stuff.