By Samiran Ghosh1
1Independent Consultant
The modern world increasingly runs on intelligent algorithms created by humans. The data-hungry, self-improving computer programs that underly the AI revolution already determine Google search results, Facebook newsfeeds and online shopping recommendations. Increasingly, they also decide how easily we get a mortgage or a job interview, the chances we will get stopped and searched by the police on our way home, and what penalties we face if we commit a crime, too. So, these systems would have to be beyond reproach in their decision-making, correct? Wrong. Bad input data, skewed logic or simply the prejudices of their creators mean AI systems all too easily reproduce and even amplify human biases – as the following examples show.
COMPAS is an algorithm widely used in the US to guide sentencing by predicting the likelihood of a criminal reoffending. In perhaps the most notorious case of AI prejudice, the US news organization ProPublica reported in May 2016 that COMPAS is racially biased. According to the analysis, the system predicted that black defendants pose a higher risk than they do, and the reverse for white defendants.
Another algorithm, PredPol, has been designed to predict when and where crimes will take place (a real world Minority Report), with the aim of helping to reduce human bias in policing. But in 2016, the Human Rights Data Analysis Group found that the software could lead police to unfairly target certain neighbourhoods. When researchers applied a simulation of PredPol’s algorithm to drug offences in Oakland, California, it repeatedly sent officers to neighbourhoods with a high proportion of people from racial minorities, regardless of the true crime rate in those areas.
In February 2018, a researcher at MIT found that three of the latest gender-recognition AIs from leading global tech companies could correctly identify a person’s gender from a photograph 99% of the time – but only for white men. For dark-skinned women, accuracy dropped to just 35%. A 2015 study showed that in a Google images search for “CEO”, just 11% of the people it displayed were women, even though 27% of the CEOs in the US were female. A few months later, a separate study led by Anupam Datta at Carnegie Mellon University in Pittsburgh found that Google’s online advertising system showed high-income jobs to men much more often than to women. The same is also true in the financial services industry where algorithms are making decisions on who can access loans and on what terms.
While we are still trying to find solutions to these biases, futurists are already predicting the rise of self-aware artificial intelligence – meaning morality is the next stage of the evolutionary journey of AI. Teaching morality to machines is hard because humans are unable to objectively convey morality in metrics that make it easy for machines to process. In fact, it is questionable whether we, as humans, even have a sound understanding of morality that we all agree on. In moral dilemmas, humans tend to rely on gut feeling instead of elaborate cost-benefit calculations. Machines, on the other hand, need explicit and objective metrics that can be clearly measured and optimized. But how can we teach a machine what is fair unless the engineers designing the system have a precise conception of what fairness is?
At first glance, the goal seems simple enough: make an AI system that behaves in a way that is ethically responsible. However, the task is actually far more complicated than it initially seems, as there are innumerable factors that come into play. Moral judgements are affected by rights (such as privacy), roles (such as in families), past actions (such as promises), motives and intentions, and other morally relevant features. These diverse factors have not yet been built into AI systems. This new form of AI will probably be nothing like us. Would human morality apply to it? And if so, which one?
For AI systems to be used in the service of society, they will need to make recommendations and decisions that align with ethical norms and values. However, it is a huge challenge to specify what exactly we mean by human values, let alone take the technical steps needed to incorporate them into an AI system. Any discussion of morality must also consider the different values held by different people and groups, and the risk of endorsing values held by a majority which may lead to discrimination against minorities.
Stephen Hawking argued that “once humans develop full AI, it will take off on its own and redesign itself at an ever-increasing rate”. Elon Musk warns that AI may constitute a “fundamental risk to the existence of human civilization”. Does this mean that we need a more ethical implementation of AI systems, that we should imbue them with a sense of ethics?
The concerns over morality often arise while talking about AI in areas like self-driving cars. Who dies in the car crash? Should the autonomous vehicle protect the passengers or passers-by or itself? The Moral Machine, an initiative by MIT, has attempted to gather a perspective on moral decisions made by AI and machine learning using a crowdsourced approach. As a part of this initiative, thousands of participants were asked to give their opinions on what AI in cars should do when confronted with a moral dilemma.
Some of the indicative questions asked in this initiative to “crowdsource” morality were:
Although it sounds like an interesting concept, how reliable can crowdsourced morality be? It could not be trusted to make complex decisions especially around the question of preserving human life. Experts believe that it is not acceptable to decide on hundreds of millions of variations based on the views expressed by a few million people. While anecdotally interesting, crowdsourced morality doesn’t make AI ethical.
The journey of morality in AI, while a noble quest, is far from complete and the question of what constitutes morality is far from being resolved, either in humans or in machines.