Superintelligence is a very interesting book.

Here is a well written, accessible summary of some of the issues it highlights:

https://waitbutwhy.com/2015/01/artificial-intelligence-revol...

It'd be a lot more interesting if they actually made points rather than vague condescension and sneering.

But points against what? The very idea that a hyperintelligent being is _possible_ very much lingers in the land of science fiction. Yes, even today.

Anyone with a little education in artificial intelligence knows that it's either impossible or somewhat possible but not likely. We truly are no closer to artificial consciousness that we were 500 years ago, it's all still smoke and mirrors.

The thing that's actually a dangerous scenario is the dumb artificial intelligence we can build to start misbehaving, due to human programming errors or statistical bias: self-driving cars running over people; military systems confusing friends from foe; this stupid predictor keyboard not predicting correctly what I'm trying to type. This is the stuff of real, possible and demonstrable fears. Anything grandeur has no basis in reality.

I haven't read Bostrom's book yet, but from what I've read from others about the book, it's all about this nonsense that we can create a new kind of intelligence who will instantly destroys us. This is cool for people ignorant of the basic tenets of what A.I. is (so far) but for people who actually know or understand the current state, they should know better.

Worrying about an hyperintelligent A.I is like worrying about ghosts while living in a neighborhood filled with serial killers.

edit: grammar.

I think your statements that majority of people who have had an AI education think we are no closer to hyper intelligence than the 1500s is inaccurate.

Here is a survey (first one I googled) showing most believe it is relatively soon: https://aiimpacts.org/agi-11-survey/

I think if you want to critique Bostrom that is fine, but better to do so on the merits. His work is straightforward to read.

> The very idea that a hyperintelligent being is _possible_ very much lingers in the land of science fiction. Yes, even today.

Yesterday, the idea that a computer could beat the chess grandmaster, or compose poems, was also in the land of science fiction.

Most technical inventions are in the land of science fiction, until one day they are not.

Just as I thought the author was going to make an actual assertion, he goes "When I was in my twenties.."

I think that's enough of my attention for now.

So in the end, computers just weaponize autism? Super intelligence is a shitty meme, much like eugenics. There are only so many philosophical situations. And machine learning is just improving performance P at a task T with experience E.

So what pray is superintelligence? If you drop the tool bit, no robot can reason that it needs the tool bit to do the machining and would just move its arm anyways.