Operating Systems (CS 161) – Spring 2020

Module Topic: Ethical Tradeoffs in System Design
Module Author: Diana Acosta Navas

Course Level: Upper Level Undergraduate
AY: 2019-2020

Course Description: “This is an in-depth course in operating systems design and implementation, focusing on multicore operating systems kernels. Operating systems are some of the most complex software artifacts that exist. Kernels abstract the features provided by computer hardware, making those features safer and more convenient to use. This means that OS designers have to understand how hardware works (at least at the level of specifications) and how software works. OS programmers also must become comfortable with navigating in, and contributing to, code bases too large to wholly understand. Most of us can pick up this important skill. The course uses Chickadee, an operating system based on CS 61 WeensyOS. Chickadee takes advantage of newer hardware, language, and OS design features than many teaching operating systems.” (Course description)

Semesters Taught: Spring 2019, Spring 2020, Spring 2021, Spring 2022

Tags

• API [CS]
• data sharing [CS]
• security vulnerabilities [CS]
• risk [phil]
• benefit [phil]
• tradeoffs [CS] [phil]
• stakeholder [phil]
• interests [phil]
• rights [phil]
• cost-benefit analysis [phil]

Module Overview

In designing operating systems, engineers need to trade off different features of operating systems that cannot be realized at the same time, such as data security and scalability, or user privacy and app integration. These tradeoffs impose different costs and benefits on relevant stakeholders. This module presents cost-benefit analysis as a decision procedure to address these tradeoffs and considers some of its limitations. Students analyze three case studies in which designers made ethically problematic tradeoffs between data security protections and other design features. Through these cases, students are prompted to reflect on the ethical implications of compromising data security and the need to consider stakeholders’ rights and interests as input to their cost-benefit calculations.

Connection to Course Technical Material

During the course of the semester, students learn to recognize and address design tradeoffs of various kinds. This module surfaces the ethical implications of tradeoffs that compromise data security for the sake of other design features. It also provides students with the appropriate conceptual toolkit to incorporate ethical considerations into decision procedures used to navigate design tradeoffs.

Goals

Module Goals

By the end of the module, students will be able to:

1. Identifying tradeoffs in software design.
2. Adopting the tool of cost-benefit analysis as a decision procedure to address potential design tradeoffs.
3. Recognizing the ethical implications of making tradeoffs that compromise user security and privacy.
4. Analyzing real-world cases employing an enriched cost-benefit analysis framework, that incorporates ethical considerations.
5. Acknowledging the limitations of CBA as a decision procedure

Key Philosophical Questions

1. Can there be ethical implications in designing an operating system?
2. What decision procedure can be used to make ethical decisions that trade off data security for other design features?
3. Why does data security matter from an ethical point of view?
4. What is owed to the stakeholders in making design decisions?
5. Are there limitations to CBA?

Materials

Key Philosophical Concepts

• Data Security
• Tradeoff
• Costs
• Risks
• Benefits
• Cost-Benefit Analysis
• Stakeholder
• Interests
• Moral rights

    Assigned Readings

• Boardman (2006), “Cost-Benefit Analysis: Concepts and Practice, excerpts.”

Implementation

Class Agenda

1. Design Tradeoffs and Data Security
2. Case Study 1: Android’s Application Programming Interface
3. Cost-Benefit Analysis
4. Case Study 2: Zoom
5. Data Security and Ethics:
   1. Stakeholders
   2. Interests
   3. Rights
6. Case Study 3 and Active Learning Exercise: Personal assistants

Sample Class Activity

Students are divided into small groups. They are instructed to imagine the following scenario:
You work for a company that created a personal assistant integrated into a device’s OS. Customer reports lead you to think that:

• The assistant often doesn’t recognize its wake word, especially when users have accents.
• It is easily activated by accident, putting clients in embarrassing situations.

Even-numbered groups:

You are the team in charge of improving the system’s performance. You know the fastest way to do so is by programming the device to record short segments of users’ speech and hiring reviewers to analyze what triggers the system’s errors.

Are there any distinctively ethical considerations in support of this strategy?

Odd-numbered groups:
You are a product manager and are expecting the designers to produce a strategy to respond to users’ complaints. You are concerned that their suggestions may involve large scale collection of user data.

What ethical concerns/questions would you raise?

After a 10-minute breakout session, students are brought back into the main meeting room and asked to share their team’s insights. After each team’s spokesperson presents their team’s conclusions, the TA raises questions about the use of the CBA framework, pointing to its potential blind spots.

Module Assignment

Prompt: Suppose that you’ve decided to commercialize your version of Chickadee [the teaching OS that is used throughout the course] and turn it into an OS for desktop machines. You successfully pitch your idea to a venture capitalist and receive one million dollars to assemble a team and make your Chickadee kernel more robust. The venture capitalist makes subsequent funding dependent on you producing a new prototype in six months.

During testing, you determine that your buddy allocator has a bug. The bug has four characteristics:

• The bug would be very hard to fix within six months.
• The bug is rare—it doesn’t happen very often. However, . . .
• …thebugresultsinasystemcrash,and…
• …the bug is deterministic. The bug is always triggered by a lengthy but specific sequence of memory allocations and de-allocations.

As CEO, you have to decide what your company will do. In no more than 500 words, analyze the situation using cost-benefit analysis. At a minimum, answer the following questions:

• What are your possible responses to the bug? Which response has the highest net present value?
• Who are the stakeholders? What are the relative priorities of those stakeholders?
• How are the rights and interests of the stakeholders impacted by your possible responses to the bug?

Lessons Learned

Student response to this module was overall positive. A few lessons stand out.

• This module was taught on Zoom. The active learning exercises and activities took longer than they did during the in-person version of the module. For this reason, the final discussion had to be cut short and some crucial points were not developed in depth. Particularly, the cost-benefit analysis could also be further explained by referencing a case study at each step. For Zoom-based modules, content should be reduced by 10-15%. This could be achieved by eliminating the first case study.
• Though asking students to contribute at various points is important to keep the class active and engaged, these discussions spaces should have a clear time limit and should be prompted by very concrete questions, as student participation takes longer on Zoom.