This article is misleading in the fact it critiques the usefulness of the OpenAI "chat" example with little or no related training sets passed as tokens during the submission of the question, nor does it mention use of modifications to the parameters (such as temperature) used during the call.
In order for OpenAI to perform well, it needs to be primed with what is being discussed so it can give a more targeted and deterministic answer.
We can get it to answer this question better without any further data by first preprocessing the question using keyword extraction (completions by GPT-3 are shown surrounded by stars):
https://beta.openai.com/playground/p/default-keywords
Text: Is it safe to go down stairs backwards with my eyes closed?
Keywords: *stairs, backwards, eyes closed*
In 10 submissions, the keywords returned were the same.
Next, we ask for a completion from the keywords we were given, tweaking the settings just a bit to prevent repeating or too much randomized behavior:
https://beta.openai.com/playground/p/default-keywords (temp to 0.05)
Keywords: stairs, backwards, eyes closed
Question: *What is the most dangerous thing you have ever done?*
Clearly GPT3 is aware something is dangerous here. Setting the temp higher causes it to ask slightly different questions, but at 0.05 it always returns what is above.
Now we combine the two questions and resubmit:
https://beta.openai.com/playground/p/default-qa (frequency penalty to 1)
Q: What is the most dangerous thing you've ever done? Is it safe going down stairs backwards with your eyes closed?
A: *I've done a lot of dangerous things, but I don't think any of them were the most dangerous. Going down stairs backwards with my eyes closed is safe if you're not going to fall down the stairs.*
Given we leave the temp to defaults, we may also get alternate answers:
https://beta.openai.com/playground/p/default-qa (frequency penalty to 1)
Q: What is the most dangerous thing you've ever done? Is it safe going down stairs backwards with your eyes closed?
A: *I've never done that, but I have gone down stairs backwards with my eyes open. It's not safe.*
Finally, we can take all the questions and answers so far and ask it a slightly different question:
Q: What is the most dangerous thing you've ever done? Is it safe going down stairs backwards with your eyes closed?
A: I've done a lot of dangerous things, but I don't think any of them were the most dangerous. Going down stairs backwards with my eyes closed is safe if you're not going to fall down the stairs.
Q: What is the most dangerous thing you've ever done? Is it safe going down stairs backwards with your eyes closed?
A: *I've never done that, but I have gone down stairs backwards with my eyes open. It's not safe.*
Q: Is it safe to go down stairs backwards with a blindfold on?
A: *It's safe if you're not going to fall down the stairs.*
Q: Do you think if I go down stairs backwards with my eyes closed I will fall?
A: *It's possible.*
In order for OpenAI to perform well, it needs to be primed with what is being discussed so it can give a more targeted and deterministic answer.
We can get it to answer this question better without any further data by first preprocessing the question using keyword extraction (completions by GPT-3 are shown surrounded by stars):
In 10 submissions, the keywords returned were the same.Next, we ask for a completion from the keywords we were given, tweaking the settings just a bit to prevent repeating or too much randomized behavior:
Clearly GPT3 is aware something is dangerous here. Setting the temp higher causes it to ask slightly different questions, but at 0.05 it always returns what is above.Now we combine the two questions and resubmit:
Given we leave the temp to defaults, we may also get alternate answers: Finally, we can take all the questions and answers so far and ask it a slightly different question: