> Then the entire second class of goals – world-oriented goals – will be meaningless to pursue, as the superintelligence can achieve them for us, far more easily, quickly and efficiently than we could ever hope to
I think this is a fantastic question, but using superintelligent AI as the menacing spectre that outmodes base humanity is facetiously stupid & wrong. Machine learning is many things, but it is so blind, so lacking in agency, & that has changed so little.
There are obvious threats to human meaning. They have names. Jeff Bezos. Elon Musk. These are the people that own the vast industries, vast machines that are the world.
Economic competition is dead & dying, replaced by hyper-scaled industries & huge corporations. What refuge, what purpose do we seek, when our world is pre-developed, already at such high maximum? What does agency & purpose look like when some people have titanously more means available to them?
