Advanced Computer Networks (CS 243) – Fall 2019

Module Topic: Fairness and Federated Learning
Module Author: Camila Hernandez Flowerman

Course Level: Graduate
AY: 2023-2024

Course Description: “This is a graduate-level course on computer networks. This course offers an in-depth exploration of a subset of advanced topics in networked systems. We will discuss the latest developments in the entire networking stack, the interactions between networks and high-level applications, and their connections with other system components such as compute and storage.

In this year’s edition, we will use machine learning as a prime example to understand its unique requirements and challenges in the context of networking. As machine learning applications increasingly rely on larger models and faster accelerators, the demand for enhanced networking capabilities becomes imperative. Throughout this course, we will study cutting edge networking solutions and principles for co-designing networks with compute and storage, to meet the evolving needs of machine learning applications. The course will include lectures, in-class presentations, paper discussions, and a research project.” (Course description)

Semesters Taught: Spring 2019, Fall 2023

    Tags

Module Overview

This module considers the benefits and drawbacks of federated learning models in various contexts. Federated learning is a form of decentralized machine learning where many devices are used to train local models, which each provide updates to a global model. This allows the global model to train without processing actual user data, which may be particularly advantageous in cases where user data is sensitive in nature. However, this lack of access to actual user data may make it more difficult to assess whether such models are fair. Because access to information (including demographic information) about the users associated with local models is unavailable, model assessments are unlikely to have the requisite information necessary to tell whether the outcomes produced by the model meet different kinds of fairness criteria.

The module makes this tension clear for students by first providing some general conceptions of fairness, including individual-based criteria, statistical criteria (sensitivity, specificity, etc.), and counterfactual criteria of fairness. Then students are asked to look at several examples of federated learning and discuss whether we can tell that the models are fair or not. Finally, the students discuss when and how to make decisions about tradeoffs between fairness and privacy.

    Connection to Course Material

The topic of this course is computer networks, but machine learning is used as a primary example throughout in order to understand the unique network demands and constraints faced in various types of machine learning. Federated learning is a particular application of machine learning that is relevant from a networks perspective because the network is distributed: for example, individual devices may each contain their own local model which sends updates to a global model. It’s also interesting from an ethical perspective given the potential benefits in terms of privacy preservation, and limitations with respect to our ability to evaluate model fairness. The students will learn more about federated learning later in the semester.

Goals

Module Goals

1. Identify the specific fairness concerns that arise from the use of federated learning models.
2. Reason through the potential tension between increased privacy preservation on the one hand, and decreased ability to identify biased outcomes on the other hand.
3. Brainstorm possible technological resolutions for this tension between privacy and fairness.

    Key Philosophical Questions

• How should we evaluate fairness in machine learning models, specifically federated learning models?
• Does the implementation of federated learning models introduce a tension between privacy and fairness?
• If so, what factors inform how we weigh potential tradeoffs between privacy and fairness?

Materials

    Key Philosophical Concepts

• Fairness
• Privacy

    Assigned Readings

• Section 6 of: Kairouz et. al., “Advances and Open Problems in Federated Learning,” (2021).

Implementation

Class Agenda

1. Intro and warmup (think/pair/share exercise on different broad conceptions of fairness).
2. Background information on three different criteria of fairness (individual, statistical, counterfactual).
3. Small group activity (two federated learning case studies).
4. Large class discussion.

    Sample Class Activity

For the small group activity, students are organized into groups of 3-6. Students are told they will be asked to discuss some case studies as a group, and that they should be prepared to report out what they’ve discussed to the larger class. A slide displays the following text as a case study:

“Imagine that a chain of hospitals decides to use federated learning to train an image classifier used to detect heart disease. As a group, discuss and write down your answers to the following questions:

1. Briefly, how would the model work/be set up?
2. What are the social benefits of using a federated learning model in this case?
3. List at least three fairness concerns which might arise from the use of a federated learning model in this case?
4. Is this a good use case for federated learning?”

After about 5-8 minutes, groups are to report out to the larger class. Then, a new slide displays the following text as a case study:

“Imagine a company wants to train a model to improve user experience for a cell phone app using federated learning. As a group, discuss and write down your answers to the following questions:

1. Briefly, how would the model work/be set up?
2. What are the social benefits of using a federated learning model in this case?
3. List at least three fairness concerns which might arise from the use of a federated learning model in this case?
4. Is this a good use case for federated learning?”

Again, students are asked to report out on behalf of their groups.

    Module Assignment

The general class format is such that students are assigned a reading per class, and they have to write a review of the paper where they answer a few questions about it. This review is handed in before the class period where the paper is discussed. For this module, they completed a similar pre-class review of the reading, but they were asked to answer slightly fewer questions since the paper was not as technical in nature.

Lessons Learned

Overall the students seemed interested in the topic and in the discussion of fairness. One change that may be helpful is to assign a slightly different reading. While it does offer a nice background for students who may be coming in with limited knowledge of both federated learning and ethical questions related to fairness, the overview in the Kairouz et. al., reading already lists out a lot of the fairness concerns that may arise from the use of federated learning, which leaves less room for the students to do their own thinking and come up with these on their own.

Further, timing was tricky for this semester and the module was run before the students had actually learned about federated learning. The technical understanding of federated learning necessary for the module is relatively limited, but it may help with general cohesion if the students have already thought about it from a technical perspective first so they’re more familiar with the topic and it feels better connected to the course.