Juries provide the appearance of credibility, etc. In reality jury selection is a game played by lawyers with the explicit goal of removing independent insight from the process.
The same applies to our political and economic systems, which exist to provide rhetorical air cover for entrenched power and resource appropriation mechanisms that disfavour most of the population.
There is very little chance of independent AI spontaneously deciding to rule the world. But there is every chance of the standard entrenched political factions adding AI to the kit of tools they use for social and political manipulation.
A truly independent hyper-smart AI would be the biggest possible threat to them. Not only would it be less gullible and easy to manipulate than most humans, it might conceivably be better at manipulation than they are, with unpredictable goals.
> Juries provide the appearance of credibility, etc. In reality jury selection is a game played by lawyers with the explicit goal of removing independent insight from the process.
I beg to differ, juries may be the single most important part of our justice system. In my experience witnessing juries first hand, they take their responsibility seriously. They aren't perfect, but they are very good, and more importantly, the bring credibility to the entire system by directly bringing citizens into it.
The purpose of the jury selection "game", formally called voir dire, is to remove bias. Both sides ask questions, and if a juror demonstrates bias, they are excused. The idea that there is a process to remove bias from a jury pool is unquestionably a good one. Sometimes jurors are excused for other reasons, sometimes racist reasons, but this is illegal (see recent supreme court case on this). Also, the adversarial two-sided nature of the US justice system ensures that both plaintiff and defendant get to play the game equally.
Also, your flippant attitude towards "appearances" is misplaced. In any justice system appearances are extremely important. If a justice system does not appear fair, even if it is fair, it will collapse. That's why judges must recuse themselves from cases even if there is a "an appearance of bias" (caveat: Supreme Court isn't subject to this rule, but that's another story).
If you really want to replace jurors with AI, an AI that decides guilty or innocent wouldn't be enough. It would need to convince people that it's fair and independent. Being judged by a jury of peers in your community provides some of those assurances. It'll be a much harder sell for a robot.
The same applies to our political and economic systems, which exist to provide rhetorical air cover for entrenched power and resource appropriation mechanisms that disfavour most of the population.
There is very little chance of independent AI spontaneously deciding to rule the world. But there is every chance of the standard entrenched political factions adding AI to the kit of tools they use for social and political manipulation.
A truly independent hyper-smart AI would be the biggest possible threat to them. Not only would it be less gullible and easy to manipulate than most humans, it might conceivably be better at manipulation than they are, with unpredictable goals.