AI algorithms at work: How to use AI to help overcome historical biases

Considering its advantages, there is no slowing down the pace of the AI revolution. However, we must ensure that AI algorithms are generating fair outcomes and be transparent about how they are achieving such outcomes. There are two ways that we can leverage the advantages of AI algorithms in selection processes while promoting fairness:

• The anonymity and efficiency afforded by AI algorithms can reduce the influence of stereotyping prevalent in human decision-making. 
• Our knowledge about stereotyping can improve AI algorithms, from how they leverage data to preventing the perpetuation of historical outcomes.

It’s no wonder the general public feels precarious about AI algorithms dictating their fate. Headlines about AI-enabled selective algorithms range from wildly optimistic to dismal: 

• The Bad: A variety of newspapers featured exposées about an AI algorithm used by Florida’s criminal justice system that falsely identified African-American defendants as “high risk” at nearly twice the rate as the mislabeled white defendants.³
• The Good: Experts have predicted that AI algorithms could potentially eradicate the cognitive biases that disadvantaged historically marginalized jobseekers.⁴

These contradicting portrayals send mixed signals about the reliability of  AI algorithms, leading to unease about their widespread use. 

Even when AI algorithms are performing well, we’re more likely to lose confidence in algorithmic forecasters than human forecasters—even if they’re making errors at the same rate.⁵ Increasing public trust in AI algorithms allows us to make use of their many advantages (e.g., efficiency, cost-efficacy). Nevertheless, establishing trust remains a complex challenge —especially considering organizations’ haphazard use of AI algorithms in the past. 

When speaking of algorithms, a "bias" refers to a disproportionate weighting towards a particular outcome based on certain variables. Algorithms exhibit unwanted bias when they are trained with past data that is influenced by historical prejudice.

For example: In 2018, Amazon discovered that their AI-enabled recruitment tool unintentionally downgraded resumes including the word "women" and consequently, overlooked all candidates from all-women's colleges.⁶  The algorithm was tailored to select applicants based on ones that were historically successful. As a result, the algorithm was attuned to identify male candidates as preferable—even if their work experience and education level was identical to their female competitors. 

Ultimately, AI algorithms are both a means of perpetuating historical bias and a helpful tool for overcoming them. To prevent the former outcome, it is imperative that organizations evaluate the historical prejudice baked into their data sets from decades prior. In the words of Dr. Kirsten Martin from University of Notre Dame’s Mendoza School of Business, “data is not clean of the sins of the past.”⁷ 

AI algorithms can be responsible for unjust outcomes, but humans aren’t any better. We’re prone to stereotyping because our cognitive mechanisms have adapted to rely on heuristics to synthesize information quickly.

Stereotyping is detrimental to selection processes because it lends itself to favoring historically privileged demographics. Human evaluators favor male applicants over female applicants, despite having the same work experience.⁸

Algorithmic bias does not create unheard-of, harmful outcomes—it merely amplifies the trends that we see in human decision-makers.

During the stages of selection processes when stereotyping is the most prevalent—such as preliminary screening stages—AI algorithms can be strategically implemented. AI algorithms can anonymize applicant’s demographic information and efficiently process data better than a human can.

Multivariable human decision-making is inefficient, and we are prone to stereotyping under time constraints. We’re also vulnerable to becoming overwhelmed by the magnitude of information that requires processing. Under such pressure, recruiters will often limit their review of the applicant pool to the top 10-20% of applicants they expect to show promise, i.e. those coming from Ivy league campuses or employee-referral programs.⁹

In the past, filtering applicants based on these stereotypical attributes has tended to favor less diverse applicants. Alternatively, AI selection algorithms enable applicants to be evaluated based on relevant variables (i.e. the quality of their work, relevant credentials) at every stage in the pipeline. Here, AI algorithms provide a fairer alternative to time constrained human evaluators who are pressured to shrink the pipeline based on stereotypical features. 

Research in behavioral science can help identify the superficial features that we tend to prioritize and ensure that algorithms are selecting applicants using relevant variables. Although AI algorithms are powerful and effective, they cannot tell us what constitutes a “fair” metric or outcome. That’s where interdisciplinary discussion comes into play. 

Besides optimizing efficiency and providing anonymity, AI algorithms can shed light onto unwanted bias that is systematically perpetuated within organizations. Cognitive biases are challenging to identify because the way a single person arrives at a particular outcome cannot be systematically analyzed and quantitatively measured.

Alternatively, the outcomes produced by AI algorithms can be interrogated with a high degree of precision through data analysis. For example, if a candidate’s outcome is affected by their race—while all other matters are considered equal—then there is evidence of algorithmic bias.¹⁰ 

There are already accessible tools available that can help organizations analyze their training data before AI algorithms are built: Google’s Know Your Data allows researchers and product developers to identify unwanted bias latent within their data sets through basic criteria (e.g., “diversity of images”). Knowledge about behavioral science can help organizations analyze the type of biased outcomes that they’re likely to see, since the biases of human decision-making have already been codified in historical data. 

The capacity for algorithmic outcomes to be rigorously analyzed provides a significant advantage over human decision-making.

We are often reluctant to admit our decisions are influenced by cognitive biases. According to a survey, 97% of admissions committees at prestigious universities agree on the importance of fair admissions, but less than half believed that biases could be a factor in their own admissions process.¹¹

AI algorithms are advantageous because they can reveal the systemic bias through quantitative evidence—so we don’t have to rely on people to realize the consequences of their biased decision-making decades later. AI algorithms can achieve this by providing feedback to developers in real-time.

When implementing AI algorithms in selection processes, it’s imperative that systematic bias—whether it be “cognitive” or “algorithmic”— is mitigated at all costs. We can increase the public’s trust in AI algorithms by anonymizing applicant’s biographic information to prevent the influence of stereotyping, and addressing algorithmic bias by quantitatively measuring how historical prejudice affects training data. 

The Decision Lab is a behavioral consultancy that examines how research in behavioral science can promote social good. If you are interested in learning about how to promote fairness through your adoption of AI algorithms, please contact us to hear more about our services.