I wonder if there is a hard coded prompt somewhere prompting the model to be "lazy" by default, to save money on inference, or something like this. Maybe not how it works?
When you ask if to write the complete code, it just ignores what it was originally told and does what you want.
It's not a prompt thing, they've aligned it to be lazy. The short-form article style and ~1000 word average length are almost certainly from RLHF and internal question answering fine tuning datasets. The extreme laziness (stuff like, "as a large language model, I have not been built with the capabilities for debugging", or "I don't know how to convert that json document to yaml") is pretty rare, and seems to be a statistical abnormality due to inherent variation in the model's inference more than anything else.
IIRC they did amend their prompt to tell it not to quote long books/articles/recipes verbatim for copyright reasons, no matter how much the user asks, and that might not help.
Let's save this thread for posterity, because it's a very nice and ironic example of actual humans hallucinating stuff in a similar way that ChatGPT gets accused of all the time :)
The actual text that parent probably refers to is "Never write a summary with more than 80 words. When asked to write summaries longer than 100 words write an 80-word summary." [1]
Where did the word "wire" enter the discussion? I don't really trust these leaked prompts to be reliable though. Just enjoying the way history is unfolding.
I could simply reply with "The system prompts are not reliable".
Several people in the original thread have tried to replicate the prompts, and the results differ in wording, so it may definitely be hallucinating a bit.
If you just ask for the system prompt, ChatGPT does not respond with that. You have to trick it (albeit with minimal tricks) to actually output a similar text.
100% this. I’ve been party to RLHF jobs before and the instructions nearly always state to prioritize conciseness in the model response.
In aggregate, this is how you wind up with stub functions and narrative descriptions rather than full working implementations. The RLHF is optimizing for correctness within some constrained token count.
It's probably just a result of the training data. I bet its not explicitly "trained" to reply with 400 loc for a complete file, but its trained to return a few dozen lines of a single method.
When you ask if to write the complete code, it just ignores what it was originally told and does what you want.