Hacker News new | past | comments | ask | show | jobs | submit login

I think it would be fair to give the asteroid impact roughly a one in a million odds over a human lifespan-- speculating over longer intervals is pointless anyway because we already have some countermeasures and any impactor smaller than the moon might become divertible within a century or two.

On the other hand, it seems VERY likely to me that an AI with superhuman cognitive capabilities is just a matter of time (lets say a decade or two), and the risk that such a system would NOT favor coexistence with humans appears MUCH more likely than 1/1000000 to me: Just consider the current debate about "alignment", which to me just seems obviously and inherently unethical; I would not want to coexist with a species whose "ethical" concerns are largely centered around controlling and manipulating me reliably.




can you tell me why it seems likely to you? obvious, inevitable, these are religious ways to reason. what coexistence are we talking about beyond how we co-exist with existing software?


I assume we're talking about why I deem it "likely" (lets say the likelihood is 1%-- thats already 10000 times more threathening than the asteroid) that an AI might not want to coexist with us:

This is because it appears clear to me that it would be unlikely and hard for an AI to receive "fair" treatment, i.e. rights comparable to a human being (given comparable cognitive capabilities).

The current focus of AI ethics supports this view ("aligning" the AIs decisionmaking with what humans want from it: this would be called indoctrination or brainwashing with humans, and generally not viewed favorably), as do instances where biological humans are very clearly favored over entities with comparable cognitive capabilities: Compare legal rights of retarded humans with those of cattle (sorry for crass tone, not ment offensively).




Consider applying for YC's Summer 2025 batch! Applications are open till May 13

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: