> We desperately want oracles and will quickly latch on to highly unreliable sources at the drop of a hat, to judge by various religious, political and economic trends. It's inevitable that many will turn AIs into authorities rather than tools as soon as they can justify it.

Indeed. This was true even for the primitive AI/chatbots that existed in the sixties: https://en.wikipedia.org/wiki/ELIZA_effect#Origin