Imagine you’re driving down a road with your friend in the front seat. A child appears after chasing their football into the middle of road, and your only choices are to run them over or swerve into a tree, which could kill you and your passenger. What do you do?
It sounds like a test to determine someone’s level of psychopathy, but it’s actually a decision that the developers of autonomous vehicles have been faced with since the idea first materialised.
In an interview with Car and Driver, the manager of driver assistance systems at Mercedes-Benz Christoph von Hugo has revealed that the company’s future autonomous vehicles will always put the driver first. In other words, it would choose to run the child over every time.
Although the fact someone has to make this choice feels uncomfortable, it would be more dangerous if they didn’t, because unless a self-driving vehicle is told what to do when a child runs into the road, it won’t do anything.
Previously, manufacturers have been quiet about what would happen under these circumstances, until Mercedes-Benz’s announcement at the Paris Auto Show this month. According to von Hugo, all of the company’s future Level 4 and Level 5 self-driving cars will be programmed with the decision to save the people they carry over anything else.
“If you know you can save at least one person, at least save that one. Save the one in the car,” von Hugo said in the interview. “If all you know for sure is that one death can be prevented, then that’s your first priority.”
Claiming this is one thing, but whether it lines up with what potential buyers actually believe is another. A study published in the journal Science this year highlighted the ethical dilemmas self-driving car manufacturers are faced with, and what people believed would be the correct course of action; kill or be killed.
Just under 2000 people were polled, and the majority believed that autonomous cars should always make the decision to cause the least amount of fatalities. On the other hand, most people also said they would only buy one if it meant their safety was a priority. So when it comes to selling cars, Mercedes has probably made the right call.
“According to our results, this might help sales, because what we observe is that people largely prefer to ride in a car that will save them at all costs, even though that is not their moral preference,” co-author of the paper Jean-François Bonnefon told Business Insider. “So not only could that help sales, but it could actually make it difficult for other companies to program their cars otherwise.”
At the show, von Hugo justified the decision by saying that if the lives of the people inside the car are uncertain, the subsequent effects are also unknown. For example, if you think back to the earlier situation, if you’d swerved and hit the tree, the child might still have been killed by a falling branch as a result. Or maybe you cause a secondary collision with a school bus behind you that’s holding 30 children, who all die. Maybe someone else runs the child over anyway.
“We believe this ethical question won’t be as relevant as people believe today. It will occur much less often,” von Hugo said. “There are situations that today’s driver can’t handle that — from the physical standpoint — we can’t prevent today and automated vehicles can’t prevent, either. [The self-driving car] will just be far better than the average driver.”
Bonnefon said the decision from Mercedes is brave because it brings the discussion out into the open. It’s also a clever move from the company. Last month the department of transportation in the US issued 15 criteria that have to be met before autonomous vehicles can use the roads. One of the criteria is ensuring that the programming can deal with ethical choices.
“It’s fantastic that Mercedes-Benz has chosen to speak openly about this issue, because other companies have been silent,” Bonnefon said. “At this stage we’re sort of in the Wild West of self driving cars ethics, and it seems like in this case Mercedes drew first.”
If a fatal accident does occur as a result of this decision, though, Bonnefon warns that it’s important to remember that the backlash would be huge. It’s up to the individual manufacturers whether they want to take on that responsibility.
“So is that going to set the tone? That’s very hard to say. Is it going to provoke government regulations? We’ll see,” he said.
Business Insider Emails & Alerts
Site highlights each day to your inbox.