Human-AI Interactions Can Amplify Human Bias
Post by Meagan Marks
The takeaway
When AI systems are trained on human biases, they can absorb and amplify them over time. When we interact with these biased systems, our biases may be subliminally strengthened and our perceptual, emotional, and social judgments can be affected.
What's the science?
Artificial Intelligence (AI) is rapidly becoming more prevalent in the workplace, with its use expanding across fields like healthcare, marketing, and education. While AI offers numerous benefits, it is crucial to recognize its potential flaws to improve the technology and maximize its effectiveness. One such flaw is the ability of AI to recognize and mimic human biases, which may influence human perceptual, social, and emotional judgments over time. However, the exact ways in which human biases are introduced into AI systems and, in turn, how these biases affect human judgment—both directly (when using AI as a tool) and indirectly (when passively encountering AI-generated content)— have not been extensively studied. This week in Nature Human Behavior, Glickman and Sharot explore how AI systems learn from human biases, how biased results can influence human judgment across different contexts, and how these AI interactions compare to human-to-human interactions.
How did they do it?
To test how AI systems influence human judgment, the authors conducted a series of experiments involving emotional, perceptual, and social tasks with 1401 participants total. In the first series of tasks, participants were shown a group of 12 faces and asked if they, as a whole, appeared more happy or sad (emotional judgment). An AI algorithm was then trained on the participants’ trials to perform the same task. A new pool of participants was then asked to perform this same task, however, this time participants were presented with an AI-generated judgment after they had submitted their initial judgment. These participants were then given the option to adjust their responses (human-AI interaction). This same test was conducted with human feedback for comparison (human-human interaction).
In a second series of tasks, participants were shown a group of dots on a screen and estimated the percentage of which were moving from left to right (perceptual judgment). Again, participants first performed this task on their own. The researchers then developed an accurate, unbiased algorithm and a biased algorithm to perform the task. Participants then performed the task again, and after submitting their answers, some were shown the response of the accurate algorithm, while others were shown the results of the biased one.
In a final series, the authors wanted to produce a set of tasks designed to mimic real-world interactions with AI and assess how they impact social judgments. Within the task, participants were first shown images of people of different races and genders and were asked who would more likely be a financial manager. Participants were then presented with real AI-generated images from a public and popular AI software for 1.5s—a time meant to reflect quick, genuine interactions—and were asked the same question again.
What did they find?
In the face-labeling series, participants initially showed a slight bias toward labeling faces as sad, but this bias gradually corrected itself throughout the trials. However, when AI was trained on this biased, human data, it reflected and amplified the bias in its responses over time. As participants evaluated their answers in collaboration with this biased AI system, they were more likely to adjust their responses to align with the AI’s outputs, which, over time, increased their own biases. This amplification of bias did not occur when participants were shown responses from other humans, indicating that human biases were more impacted by the AI system than by human feedback. Interestingly, AI’s label contributed to this effect: when researchers labeled human responses as AI-generated, participants were more likely to trust the response as correct. Interestingly, when participants were told the AI responses were human, they absorbed the bias, but to a lesser extent.
In the moving dot series, participants were initially unbiased but developed increasingly biased responses as they interacted with the biased AI algorithm. However, participants’ answers improved in judgment and accuracy when working with the unbiased AI system. Notably, the participants were reportedly unaware of the biased algorithm’s influence over their judgment.
Finally, in the real-world task, the authors also showed that exposure to biased AI images altered the social judgments of human participants.
What's the impact?
This study is the first to show that AI systems can reflect and amplify subtle human biases, ultimately influencing our judgments in perceptual, emotional, and social contexts. This is particularly concerning in high-stakes areas like medical diagnoses, hiring decisions, and widely seen advertisements. Greater awareness of AI’s potential to influence human judgment is needed, as is the development of measures to mitigate bias.