Here's a provocative idea: Maybe it's because deep learning and all the other popular AI algorithms are complete and utter rubbish.
Maybe using them has nothing to do with standing on the shoulders of giants but much more with standing on the shoulders of the local maximum that is achievable by throwing insane amounts of data at dumb algorithms.
If you could choose between access to algorithms and data structures that exactly mimic the human brain and a data set that contains everything all humans taken together know, what would you choose?
Trained networks of artificial neurons are function approximators, so they are basically algorithms , we just don't care enough for their analytical expressions. The analytical expression is appealing but some problems may prove irreducible or very-little-reducible. I don't see a problem with either approach since they both achieve the same effect.
They're not rubbish, they have their uses. You kind of outlined where deep learning tends to shine - when you have large amounts of data and massive CPU infrastructure. Pattern recognition and machine learning work, but typically require more manual overhead than throwing tons of data at "dumb" algorithms to achieve the same output.
In end, you need both the algorithms and the data to do the work, and choosing between the two leaves you still wanting the other.
Of course, and I don't seriously claim that they are rubbish. They are useful and I admire some of the people who have developed them.
But I think we need to question why data seems to have this outsized value compared to algorithms. I don't think it is some sort of information theoretical invariant. It's a relationship between the specific algorithms and the specific sort of data we have.
Maybe using them has nothing to do with standing on the shoulders of giants but much more with standing on the shoulders of the local maximum that is achievable by throwing insane amounts of data at dumb algorithms.
If you could choose between access to algorithms and data structures that exactly mimic the human brain and a data set that contains everything all humans taken together know, what would you choose?