Then where is the publication that uses GPT-2 for a long-term NLP task, like, for example, reconstructing the rules of the great vowel shift from a corpus of all pre 20th century English?
Synchronously parsing the meaning a single text has at one moment in time involves no time series at all.
The parent comment isn't thinking time-series like you and I are.
Being able to follow multiple agents and correctly deduce their relationships at a given time t is very hard.
NLP "time-series" does a fine job at making back references within a text, but wouldn't be able to have multiple representations of a word or character through the years.
It's very hard to get the computer to say "ah, the context is 16th century, so here are the relationships" without fudging it / tailoring models via tailored corpuses.
Depends. In the articles Uber example, the failure wasn't with detection, it as with context switching.
The detection kept changing, and so the model kept going "oh, new object, restart decision process."
Lacking the ability to generate and maintain it's own context is an area where a human would do better. We might not know what the object was, but our "slow down" response wouldn't keep resetting depending on what we classified the object as.
Same as words switching meanings within a piece or sentence. It's hard, but most humans can pickup when the usage changes
Synchronously parsing the meaning a single text has at one moment in time involves no time series at all.