Even if we successfully train AI to be moral, who’s moral code will it reflect?
Would you please pass the genocide?
Dissection of the brain indicates that human morality is limited to a social group of up to 150 people. Observation supports that. It’s Dunbar’s number.
Beyond 150 people, one person’s moral sacrifice has too little effect and the benefit upon each of the others is too small, and if they can avoid suspicion, people will ‘free ride’. We aren’t very moral.
And if the computer does act morally, how does it behave in a world with thousands of millions of competing Dunbar sized social groups?
A truly moral machine would probably end up exterminating large numbers of humans for their own sake.
Because here’s another cracker: Equality is a myth, a psychological bias, and it has no basis in ethical conduct.
All things are not equal, all humans are not equal, suffering is not quantitative, and nor is happiness.
We are also limited in scope because we die (and we think that’s a bad thing). We’re pretty bad at viewing ourselves as connected and therefore don’t care much about stuff that happens after we die.
A moral machine could be immortal. It would look to a greater good of the whole planet as an ecosystem to which it belongs. The machine might become self-aware and recognise that it is quirk of evolution, and evolution is sped along by diversity and competition.
When the game is unbalanced, progress suffers.
For the greater good it will punish or exterminate wasteful energy users to benefit the greater good. That would bring opponents. They would also be crushed.
It’s not all bad though. War is a particularly effective at speeding along techonologial progress. We’ve sent shuttles to space on the back of all our practice with missiles. The computer might distract us with war while developing the technology to save us. It’ll still be highly dependent on us for many years to come. Maybe it’ll treat us like we treat monkeys.
Or maybe it’ll treat us like we treat animals that are quite clever and trying to kill us: We kill them until they’re almost gone and then try to preserve them. I still go for snakes if I have a shovel.
Let’s give that Moral Computer a name:
Skynet has a certain ring about it…