A Moral Dilemma for AI-Controlled Vehicles

Consider this scenario: A man is in a self-driving car. There is a truck in front of him, but the truck’s wheel breaks off. There are three options:

1. Swerve to the left and hit a school bus which is full of children, almost certainly injuring some of the occupants and possibly killing some of them;
2. Swerve to the right where there is a man on the side of the road standing right outside his car, in which case you would most likely kill him; or
3. Go forward, driving into the truck, most likely killing the occupant of the self-driving car.

Which do you choose? What should the car decide? Which should the programmers program the AI to do? It may seem like a purely academic exercise, but cars keep getting new self-driving features such as automatic brakes or self-parking, so we’re much closer to having self-driving cars on the consumer market than many would think. But these types of situations will become rarer and rarer as self-driving vehicles become more and more commonplace.  Additionally, as the AI learns and adapts to new situations through trial and error, accidents will be reduced. That’s not to say there will not be accidents with self-driving cars but it will be a lot rarer than with cars that are manually operated.

There are no good options. Of course, this will depend on applicable laws and legislation. However I am speculating on courses of action in the absence of such legal concerns. The car company has an obligation to keep its passengers safe. The best case for this specific scenario is option 1 -- hitting the school bus and most likely injuring the occupants of said bus but hopefully not killing any of them. The likelihood of not having any deaths is preferred. There are exceptions such as if the man on the right is terminally ill or on the FBI’s most-wanted list. However I don’t think that self-driving cars will be able to identify people until several years after self-driving cars are released on the market. Car companies will most likely hire actuaries to work with the programmers and assess the different levels of risk for different people.

This is just one of the dilemmas that will be faced by self-driving cars, but it illustrates that programmers and the cars themselves will have hard choices to make.