A language model being able to do something that rarely has to be done in the first place within milliseconds or less that would take a human a minute to do just does not seem that impressive. (I tried it without reading any of these comments below, and I got everything except "clinching" in about 20 seconds.)
I don't know. I am having a hard time overcoming the likelihood that scrambled and cipher-encoded words/solutions are part of the training corpus, thus fully explaining the phenomenon.
If someone can get it to decipher something like the zodiac killer's cipher, then I might be more impressed.
That's ok. You get to be impressed or not impressed by whatever impresses or doesn't impress you. Maybe nothing is impressive! I'm extremely fine with a statement like that.
I question whether your bounding the human time to a minute is valuable here though. If the jumbled content were multiple pages long instead of only 23 words, would it be somehow more impressive despite the process being exactly the same?
> I am having a hard time overcoming the likelihood that scrambled and cipher-encoded words/solutions are part of the training corpus, thus fully explaining the phenomenon.
Scrambled words are part of my training corpus too, but it still takes me a lot longer than the machine, and I don't even need to give the machine a hint about what's going on. I just say "tell me what this says" and a moment later it does.
I don't know. I am having a hard time overcoming the likelihood that scrambled and cipher-encoded words/solutions are part of the training corpus, thus fully explaining the phenomenon.
If someone can get it to decipher something like the zodiac killer's cipher, then I might be more impressed.