Researchers received some interesting answers when they asked millions of people across the world what they think a driverless car should do in the face of an unavoidable accident
Each scenario required making choices between various combinations of saving passengers or pedestrians, and the researchers identified a number of shared moral preferences.
The researchers, from Massachusetts Institute of Technology and other institutions, presented variations of the classic “trolley problem” thought experiment almost 40m times to millions of volunteers from all around the world.
In the traditional thought experiment, participants are asked to consider whether they would reroute a runaway trolley car which is about to hit and kill five people, directing it on to a siding where it would kill only one person.
The new quiz, dubbed “Moral Machine”, instead asked volunteers to consider what a self-driving car should do in examples from more than 26 million variations of the same question.
Should a car with three occupants, an adult man and woman and a child, swerve into a wall, killing them all, in order to avoid hitting three elderly people, two men and a woman?
Should an unoccupied car swerve and kill an unemployed adult man, a child and a cat in order to save an adult man and woman and a child?
Does the answer change if the pedestrian light is red? What if one of the people is unfit, or pregnant?
Responses varied greatly around the world: in the global south, for instance, there was a strong preference to spare young people at the expense of old – a preference that was much weaker in the far east and the Islamic world.
The same was true for the preference for sparing higher-status victims – those with jobs over those who are unemployed.
When compared with an adult man or woman, the life of a criminal was especially poorly valued: respondents were more likely to spare the life of a dog (but not a cat).
The researchers, whose work is published in the journal Nature, also note “some striking peculiarities, such as the strong preference among those in the global south for sparing women and fit characters.
“Only the (weak) preference for sparing pedestrians over passengers and the (moderate) preference for sparing the lawful over the unlawful appear to be shared to the same extent in all clusters.”
These sorts of ‘trolley problems’ are philosophically fascinating, but until now, they’re rarely been much of a concern for law, says Associate Professor Colin Gavaghan, New Zealand Law Foundation Chair in Law & Emerging Technologies, Faculty of Law, University of Otago.
“Most drivers will never have to face such a stark dilemma, and those who do will not have time to think through consequentialist and deontological ethics before swerving or braking!”
He notes that the law tends to be pretty forgiving of people who respond instinctively to sudden emergencies.
The possibility of programming ethics into a driverless car, though, takes this to another level.
“That being so, which ethics should we programme? And how much should that be dictated by majority views?
“Some of the preferences expressed in this research would be hard to square with our approaches to discrimination and equality – favouring lives on the basis of sex or income, for instance, really wouldn’t pass muster here.”
Age is also a protected category, Gavaghan believes, but the preference for saving young rather than old lives seems to be both fairly strong and almost universal.
So should driverless ethics reflect this?
“Even that preference seems likely to raise some hard questions,” he concedes.
“At what point does a ‘child’ cross the threshold to having a less ‘valuable’ life? 16? 18? Is an infant’s life more precious than a toddler’s? An 8-year-old’s?
“Expressed like that, the prospect of building a preference for ‘young’ lives looks pretty challenging.”
One preference that might be easier to understand and to accommodate is for the car to save as many lives as possible.
“Sometimes, that might mean ploughing ahead into the logging truck rather than swerving into the group of cyclists.
“Most of us might recognise that as the ‘right’ thing to do, but would we buy a car that sacrificed our lives – or the lives of our loved ones – for the good of the many?”
Which, Gavaghan says, brings us to the role of law.
“Maybe it just shouldn’t be legal to buy a car that would discriminate on protected grounds, or that would sacrifice other people to preserve our own safety.
“But in that case, how many people would buy a driverless car at all?”
What if we left it up to individual choice, he asks.
“Could driving a ‘selfless’ car come to be seen as an indication of virtue, like driving an electric now?
“Would drivers of ‘selfish’ cars be marking themselves out in the opposite direction?
Gavaghan says maybe the biggest issue is the fact that over a million people die on the roads every year.
“Hundreds die in New Zealand alone,” he notes.
“Driverless cars have the potential to reduce this dramatically.
“It’s important to think about these rare ‘dilemma’ cases, but getting too caught up with them might see us lose sight of the real, everyday safety gains that this technology can offer.”
“Social scientists will certainly focus on ethics of technology including driverless cars as we get closer to wider use of this technology in the next few years,” he believes.Professor Hossein Sarrafzadeh, Adjunct Professor, High Tech Research, Unitec, admits that while technical aspects of driverless cars have seen great advancement, the social aspects have not been studied well.
“Cultural aspects of driverless cars and other artificially intelligent systems like emotion recognition systems have not been studied sufficiently either and there is a great need for research in these areas globally and in New Zealand.”
One aspect of driverless cars that Sarrafzadeh says is not taken into account in various studies of the social dimensions of this technology is the fact that future roads may not be the same roads we are using today.
“Even if we use similar roads they will be heavily sensored, intelligent roads,” he predicts.
“They will certainly be much safer, although these ethical dilemmas will remain if the same roads are used.”
Future roads, he believes, will be different to what we have now.
“There may be no humans walking across the roads that autonomous vehicles travel in.”
