System Design Projects: Machine Learning for Social Impact (CS 96) – Spring 2023

Module Topic: How could we improve the expected moral value of machine learning projects?
Module Author: Aksel Braanen Sterri

Course Level: Advanced undergraduate
AY: 2022-2023

Course Description: “Machine learning promises a bright future through solving previously unsolvable problems. Yet, the gap between a theoretical understanding of machine learning concepts and a practical application of these tools can be fraught with technical and ethical problems. In this course, teams of 3-5 students will work with real partner organizations to use machine learning techniques on a directly impactful project. Students will learn how to effectively explore data, create and iterate on real models, communicate and work with external partners, and incorporate ethics into their technical work. In addition, the class will include guest lectures from experts in various fields of the social impact tech space.” (Course description)

Semesters Taught: Spring 2023

Tags

• Expected value [phil]
• Fairness [phil]
• Privacy [phil]
• Value trade-offs [phil]
• Multiple criteria decision-analysis [both]

Module Overview

In this module, we examine how student projects using machine learning techniques can be improved to increase their expected value. Students, who work with external partners on socially impactful projects, present their preliminary social impact assessments. In these assessments, they have explicated the values and possible disvalue of their projects. After each group presents their work, the module instructor gives immediate feedback, prompting students to explicate and clarify the values at play in their projects. The module instructor presents an applied Expected Value framework and Multiple Criteria Decision Analysis (MCDA). The students are then asked to work in their groups on identifying a value they struggle to promote in their projects and develop strategies to promote this value before assessing the merits of their newly devised strategy compared to their previous one.

    Connection to Course Material

The course is dedicated to creating a concrete technological product for partner organizations to help them achieve their broader aims. The module intends to help them think more clearly about the values the students promote in their work, the relative importance of these values, and what strategies they can employ to improve the expected value of their projects.

Goals

Module Goals

1. Clarify the value of the student’s projects
2. Clarify trade-offs in their projects
3. Get the students to identify strategies to increase the expected value of their projects

Key Philosophical Questions

1. What is ultimately valuable, and what is merely instrumentally valuable?
2. How to deal with decision-making under risk and uncertainty?
3. How to weigh values when they conflict?

Materials

    Key Philosophical Concepts

• Value
• Privacy
• Fairness
• Expected value
• Uncertainty

    Assigned Readings

• No reading

Implementation

Class Agenda

1. A brief introduction to “practical ethics” as the study of what we should do, and the two primary difficulties when creating projects for social impact: Projects that promote social impact promote values, but which values should we promote? And how do we deal with uncertainty?
2. Students present their projects and social impact assessments in class.
3. Module instructor gives immediate feedback, focusing on explication and clarification of the values their projects promote.
4. Module instructor presents expected value theory and multi-criteria decision analysis as frameworks for improving their projects.
5. Students work in groups on how to improve their projects.
6. Students present their solutions with a few clarificatory questions by the module instructor.

    Sample Class Activity

The main active learning exercise was a task where students were asked to identify a value they struggle to promote in their work or that their project risks violating. They were also asked to develop concrete strategies to promote that value and compare this new “package” with their existing strategy. See more in Module Assignment.

    Module Assignment

Identify an important value in your work that you currently struggle to achieve or that your project might violate. Identify strategies that can promote those values. Will this prevent the project from realizing other values? Will it change the project too much? Require too many resources? Will the project be better or worse? Use the expected value framework if helpful. Can you pursue a scaled-down strategy that realizes some of the value without jeopardizing the primary value?

Lessons Learned

Overall the module was successful, and students were very satisfied.

1. One particularly worthwhile part of the module was when the module instructor responded to the students’ projects and helped them clarify the values they wanted to promote and how different values relate to other more fundamental values.
2. Another worthwhile part was when they were thinking about how to improve their projects.
3. The lecture on expected value theory and multiple criteria decision analysis should be very specific.