By: Justine Brooks
12 Sep, 2022
Believe the Impossible: The Future of… Fairness in AI
September 12, 2022
Artificial intelligence (AI) technology is extremely prevalent, but is it fair? Dhanya Sridhar, Canada CIFAR AI Chair, discusses algorithmic fairness and AI’s future
From credit scores, to health-care services, to the criminal justice system, AI is helping humans make decisions. But are they decisions that benefit everyone?
Dhanya Sridhar, a Canada CIFAR AI Chair at Mila and assistant professor at the Université de Montréal, explains how AI can perpetuate existing inequalities. “Historically, humans have perpetuated injustices and inequities and now, algorithms sometimes learn from this data. While AI, ideally, doesn’t have its own cognitive biases, it does inherit harmful patterns from historical data.”
By focusing on causality over correlations, Sridhar hopes to develop an AI system capable of generalization, a process by which AI can work with data that is completely different from what it was trained on. Teaching AI the difference between causality and correlation could be one way of promoting algorithmic fairness.
In a Q&A with CIFAR, Dhanya Sridhar discusses how far AI has come — and the exciting potential of where it could go.
CIFAR: What is fairness in AI?
Dhanya Sridhar: I’ll call this broad field algorithmic fairness. Historical data is used to train predictive models, mostly in service of risk assessment or scores that we then use to make decisions about people. One example is in the criminal justice system. Algorithms use past data on recidivism to generate scores used in deciding who to release pre-trial. We’re at the point where I think machines and humans can be co-collaborators in decision-making; it’s not that we’re purely entrusting decision making to AI, but it’s certainly becoming a crucial component in aiding human decision making. It’s an interesting interface, where humans have cognitive biases and AI doesn’t have biases, per se, but it’s learning from data that’s created by human beings. That data includes all sorts of patterns that arise because of human bias, and the algorithmic assessments produced by machine learning and AI could potentially exacerbate the kinds of biases we already see in modern society. This includes direct discrimination against various protected groups, but it could also perpetuate stereotypes or inequalities that already exist.
CIFAR: What are some ways that AI has altered our day-to-day lives that we don’t even think about? Why is it important for those to be fair?
Dhanya Sridhar: There are definitely lots of examples: assistive technologies with driving, human-AI decision-making in the health-care system, and AI systems, like DALL-E, that take human language and generate images. And, like I said, the criminal justice system is a great example. It’s been studied a lot, especially as of late, because there are these recidivism risk assessments which have faced a lot of scrutiny around how fair they are. The essence of this is basically, from historical data, you might end up learning that certain groups are more likely to recidivate and so these risk assessment scores will potentially favour releasing white defendants at a higher rate than most. There are many reasons for this correlation to exist, but they don’t reflect any causal mechanism; it could be that in the past there was a racist judge that made bail decisions that disadvantaged non-white defendants or because of historical events that worsened disadvantage for non-white folks. So then, when we use these forces to make decisions in the future, it will just perpetuate the inequality that has already happened. Humans have cognitive biases and there’s perhaps a hope that algorithmic agents could help us overcome these, but we have to be cautious about how they’re trained. We want to make sure that in the decision-making process, we look towards the future and think, what are the kinds of worlds and systems that we want to create? How can we adjust our AI algorithms to not just do what’s been done historically?
CIFAR: What are you working on right now? How do you apply this concept of ethical AI to your work?
Dhanya Sridhar: What I’m interested in right now is developing methods and criteria that help machine learning to be more robust, avoiding just correlations and rather focusing on information that’s more stable, potentially capturing more causal influences and sources of information. In terms of fairness, the goals here are similar in that we have a historical context in which our algorithms are trained but we want to deploy them in a different context and we want them to do better. We want them to make inferences that don’t just mimic patterns from historical data, but generate new, fair conclusions. At the heart of all of this is, how do we get machine learning to not just memorize what it has seen, but to instead draw inferences and conclusions, to generalize, in the way that we want it to?
CIFAR: What do you hope to achieve in your work over the next decade?
Dhanya Sridhar: Looking ahead to the next decade, what I’m very broadly interested in is integrating AI into decision-making. AI has made huge strides, but we haven’t gotten to the point where algorithmic systems are totally integrated into human decision support, and there are so many missing components to get there. For example, a big one is we have to start studying not just the implications of AI in a stationary way, but instead consider a sequential process that integrates AI decisions and how that changes the next time point. There’s a lot to consider in terms of how AI systems are going to explain their decisions to human beings, so that humans could understand why the prediction has been made, how it’s been made, and then integrate it into their own model of the world. The vision that I have is to really start taking seriously the idea of AI reasoning agents that work together with humans to make decisions about the world.
CIFAR: What are some things in your research or field that once seemed impossible but are now real?
Dhanya Sridhar: I think one of the most spectacular examples is what we’ve been able to do around human language. At the time that I was starting to get into this field, around 2013, the idea of statistical language models was to count the frequencies of word sequences and try to understand patterns about what makes language linked. At the time these were not particularly scalable; they were simplified to the point that they were not used to generate text or human-like communication. And then with the amazing advances that we’ve had in natural language processing, around deep learning and large pre-trained language models, now we have amazing systems like DALL-E. You give it a phrase and it will automatically generate an image, and this is not like retrieving an image from Google that looks like the thing that you wrote. It is literally creating a new image from scratch. Other impressive examples are image captioning, automatic image generation, code completion, natural language generation, where you give an AI a prompt and it completes the text for you. A decade ago, we really did not think that our language models would get to the point that they were actually going to be passable for human standards. And, of course, we still have a very long way to go, but I think that the progress so far has been astounding.
CIFAR: What excites you about the future of your field?
Dhanya Sridhar: What really excites me is that with our advancements in machine learning we have these systems that can learn amazing things from many sources of data: from text, from images, from networks, and so forth. We have this amazing opportunity to make progress in precision medicine, autonomous driving, and decision-making. I think we have this tool in our hands to be able to potentially make the world a much better place for lots of people. But before we get there, there are lots of really interesting technical challenges around explainability, and around fairness and robustness.
As part of CIFAR’s 40th anniversary celebration, “Believe the Impossible: The Future of…” highlights researchers whose big ideas could transform their field over the next 40 years.
Related Articles
-
Looking ahead: the future of AI in Canada
March 05, 2025
-
Three 2024 Nobel Laureates among CIFAR’s acclaimed community of researchers
October 15, 2024
-
Canada CIFAR AI Chairs gather in Banff for annual AICan meeting
June 20, 2024
-
How does the brain give rise to the mind?
June 13, 2024