Data Science 1: Introduction to Data Science (CS 109A) – Fall 2021

Module Topic: Algorithmic (Un)fairness
Module Author: Sophie Gibert

Course Level: Upper-level undergraduate
AY: 2021-2022

Course Description: “Data Science 1 is the first half of a one-year introduction to data science. The course will focus on the analysis of messy, real life data to perform predictions using statistical and machine learning methods. Material covered will integrate the five key facets of an investigation using data: (1) data collection – data wrangling, cleaning, and sampling to get a suitable data set; (2) data management – accessing data quickly and reliably; (3) exploratory data analysis – generating hypotheses and building intuition; (4) prediction or statistical learning; and (5) communication – summarizing results through visualization, stories, and interpretable summaries. Part one of a two-part series. The curriculum for this course builds throughout the academic year. Students are strongly encouraged to enroll in both the fall and spring course within the same academic year.”

Semesters Taught: Fall 2019, Fall 2020, Fall 2021, Fall 2022

Tags

• algorithms (CS)
• machine learning (CS)
• algorithmic design (CS)
• prediction (CS)
• bias (phil)
• discrimination (phil)
• fairness (phil)
• transparency (phil)
• label (CS)
• training data (CS)
• proxy variable (CS)
• feedback loop (phil)
• calibration (CS)
• luck-egalitarianism (phil)
• model cards (CS)

Module Overview

This module focuses on the questions of when machine-learning (ML) algorithms are unfair and how we can prevent them from being so. The module begins with an overview of the ways in which unfair bias can enter ML algorithms. The module then focuses on a case study involving a racially biased healthcare risk-prediction algorithm. In small groups, students attempt to articulate the sense in which the healthcare algorithm is unfair. They are then introduced to the promises and pitfalls of a luck-egalitarian answer. To close, the module provides a set of ethical questions that students should ask during the design and examination of ML algorithms and introduces the practical tool of a “Model Card for Model Reporting.” In the post-class assignment, students practice generating key ethical questions about an algorithm’s design.

    Connection to Course Technical Material

Students in this course learn to analyze real data and perform predictions using statistical and machine learning methods. The module poses ethical questions about how to avoid reproducing unfair biases when employing such methods.

Goals

    Module Goals

By the end of the module, students will be able to:

1. Identify multiple stages of development at which bias can enter ML algorithms.
2. Describe a major healthcare risk-prediction algorithm and explain how it discriminates against Black patients.
3. Articulate why the outcome of this algorithm is unfair by appealing to luck-egalitarianism; problematize this answer and offer an alternative.
4. Generate ethical questions to ask during the development and evaluation of a given ML algorithm that help to address unfairness

    Key Philosophical Questions

1. How can unfair bias arise in ML algorithms?
2. Why is it unfair that Black patients are sicker, on average, than White patients with the same risk score?
   • How might a luck-egalitarian explain this unfairness?
   • How might a critic of luck-egalitarianism explain this unfairness?
3. What ethical questions should we ask at each stage of algorithm development to address the problem of unfair bias?

Materials

    Key Philosophical Concepts

• Bias
• Fairness/unfairness
• Luck-egalitarianism

    Assigned Readings

• Barocas, Hardt, and Narayanan. “Introduction” to Fairness and Machine Learning.
• Ledford. “Millions of Black People Affected by Racial Bias in Health-Care Algorithms.” Nature NEWS.

Implementation

    Class Agenda

1. Introduction: Ubiquity of ML algorithms, why we use them, the problem of unfair bias, terminology.
2. How bias arises in ML algorithms: problem formulation, dataset construction, deployment, feedback loops.
3. Case study: How and why a major healthcare risk-prediction algorithm is racially biased.
4. Class activity/discussion.
5. Luck-egalitarianism and key objections.
6. “Relabeling” as a solution to the problem of racial bias seen in this case.
7. Ethical questions to ask at each stage of algorithm development; Brief introduction to model cards.

    Sample Class Activity

Students are asked to discuss the following questions in groups of 2-3 and share their responses with the class during a subsequent large-group discussion:

1. Why might someone think it is unfair for Black patients to receive lower risk scores than their White counterparts in this context?
2. Why might someone think it is not unfair?

.

    Module Assignment

Before class, students are asked to answer the following reading response questions:

1. What 1-3 things did you learn or take away from today’s reading?
2. What was unclear, or do you have more questions about today’s reading?

After class, students complete a post-class quiz. They are asked to answer three questions:

1. Why might a committed luck-egalitarian think that it is unfair for Black patients to receive lower risk scores than their White counterparts when these risk scores are used to recommend enrollment in a high-risk care program?
2. Do you think the luck-egalitarian gives a compelling explanation of why this is unfair? Why or why not?
3. Imagine that the healthcare risk-prediction algorithm discussed in class is used to determine insurance premiums, rather than inclusion in a high-risk care program, and that patients who are considered to be at high risk are charged higher insurance premiums. Articulate two ethical questions that you would ask during the process of deciding whether this practice is fair.

    Lessons Learned

Student responses to this module were overwhelmingly positive. Students found the concept of luck-egalitarianism particularly useful for thinking about the ethical issues discussed. They also appreciated the overview of how bias can arise at different stages of algorithm development and enjoyed discussing a single example in depth.

Pedagogical insights:

• Students appreciate having philosophical concepts and theories to structure their thinking about questions that seem at first to be nebulous (e.g., “Why is X unfair?”), even when those concepts and theories are simultaneously problematized.
• Real-world examples, especially from current events, go a long way in ensuring that module content is engaging and memorable.
• This module was taught to a class of approximately 250 students in a tiered, lecture-style classroom, with approximately 80 additional students participating online. The course sessions do not normally include discussion. In this setting, small-group-based (2-3 student) discussions followed by large-group discussions worked well; however, it is useful to dedicate a few minutes early in the session to having the students determine their small groups and introduce themselves to their discussion partners.