The need for new data seems like it has outpaced the rate at which real data is being generated. And most of the new data is llm slop.

So you might improve algorithms (by doing matrix multiplications in a different order.... it's always matrix multiplications) but you'll be feeding them junk.

So they need ever increasing amounts of data but they are also the cause of the ever increasing shortage of good data. They have dug their own grave.