Tiny Machine Learning (CS 249r) – Fall 2020

Module Topic: Privacy in Context
Module Author: Susan Kennedy

Course Level: Upper-level undergraduate
AY: 2020-2021

Course Description: “

Tiny machine learning (TinyML) is defined as a fast-growing field of machine learning technologies and applications including hardware (dedicated integrated circuits), algorithms and software capable of performing on-device sensor (vision, audio, IMU, biomedical, etc.) data analytics at extremely low power, typically in the mW range and below, and hence enabling a variety of always-on use-cases and targeting battery-operated devices. The pervasiveness of ultra-low-power embedded devices, coupled with the introduction of embedded machine learning frameworks like TensorFlow Lite for Microcontrollers, will enable the mass proliferation of AI-powered IoT devices. The explosive growth in machine learning and the ease of use of platforms like TensorFlow (TF) make it an indispensable topic
of study for modern computer science and electrical engineering students.” (Course description)

Semesters Taught: Fall 2020

Tags

• embedded ML (CS)
• IoT (CS)
• privacy (phil)
• autonomy (phil)
• fairness (phil)

    Module Overview

In this module, we first consider how the features of TinyML may pose unique challenges to privacy. More specifically, we discuss: (1) how the small size and discreet nature of TinyML poses problems for informed consent; and (2) how the possibility of deploying this technology in a wide range of contexts requires us to move beyond viewing privacy in terms of public vs. private spheres. As a framework for approaching these challenges, Nissenbaum’s argument for privacy as a right to contextual integrity is introduced. According to NIssenbaum, every context is associated with norms which govern the appropriate flow of information and, consequently, our expectations of privacy. Using this lens of analysis, students think through how new devices or practices might disrupt context- relative informational norms and generate privacy violations.

Once students are able to identify a violation of privacy, they are then asked to consider what types of ethical considerations might nevertheless justify this violation. To reinforce this lesson, students practice applying the philosophical concepts they have learned to several real-world examples of ML devices. In closing, students reflect on ways they can apply what they have learned about privacy to make more informed design choices.

    Connection to Course Technical Material

In this course, students learn to build embedded devices that utilize edge computing. A section of the course focuses on promising applications of TinyML including keyword spotting for personal voice assistants, visual wake words, and arrhythmia detection. For the Embedded EthiCS module, students evaluate relevantly similar applications of embedded devices (including fall detection systems, health wearables, and personal voice assistants) to determine whether they violate privacy while paying special attention to the context in which these devices might be deployed.

Goals

Module Goals

By the end of the module, students will be able to:

1. Understand philosophical arguments for privacy, with a special focus on privacy as a right to contextual integrity.
2. Identify violations of privacy.
3. Evaluate ethical considerations that might justify a violation of privacy.
4. Practice applying these concepts to evaluate real- world case studies of ML devices.

    Key Philosophical Questions

1. What is context?
2. What are informational norms and how do they govern the appropriate flow of information within a given context?
3. In what way might a shift in context-relative informational norms be said to constitute a violation of privacy?
4. What ethical considerations should we take into account when evaluating the desirability of a new device or practice?
5. When might a violation of privacy be justified or outweighed by other ethical considerations?

Materials

    Key Philosophical Concepts

• Nissenbaum’s argument for privacy as a right to contextual integrity
• Context
• Informational norms
• Autonomy
• Fair distributions of costs and benefits
• Power dynamics (Foucault)

    Assigned Readings

• Michael Zimmer, “How Contextual Integrity Can Help Us with Research Ethics in Pervasive Data” (July 2018) Medium.
• Helen Nissenbaum, “Contexts, Informational Norms, Actors, Attributes, and Transmission Principles,” Privacy in Context: Technology, Policy, and the Integrity of Social Life.

Implementation

Class Agenda

1. Explore how the features of TinyML pose unique challenges to privacy
2. Explain Nissenbaum’s framework for understanding privacy as a right to contextual
   integrity
3. Class activity to practice applying this framework to real-world cases

    Sample Class Activity

First, students are presented with background information about a real-world example of an ML device. Then, the Embedded EthiCS TA walks students through defining the context-relative informational norms (the context; the key actors involved including the subject, senders, and recipients of information; the attributes of information shared, and the transmission principle). Next, the Embedded EthiCS TA identifies how the device in question may disrupt the informational norms, thus violating privacy.

After the device has been flagged as a violation of privacy, students are asked to perform the second round of evaluation to determine if this violation of privacy is justified in light of other ethical considerations. More specifically, students are given several minutes to consider whether the new device or practice: 1) provides better support for contextual values; 2) promotes autonomy; 3) improves power relations; 4) creates a fair distribution of costs and benefits.

Students then complete a Zoom poll that asks them to identify which of the 4 ethical considerations listed above they found most relevant or useful in their analysis. The results of the poll are shared with the whole class and can be used by the module TA to guide the class discussion. This process is repeated for the remaining 2 case studies.

    Module Assignment

The follow-up assignment asks students to reflect on their final project and write a paragraph explanation for one of the following questions:

1. If you think your application may potentially violate privacy, please write an explanation of why it may still ultimately be desirable with reference to the following questions: Does it provide better support for contextual values? Does it promote autonomy? Does it improve power relations? Does it create a fair distribution of burdens and benefits?
2. If you think your application sufficiently protects privacy, explain how it does so by answering the following questions: What is the context? How does this new application maintain or reduce the number of key actors (subjects, senders and recipients of information), how does it maintain or reduce the attributes of the data (the type or nature of information), and how does it maintain the transmission constraints on the flow of information?

Lessons Learned

In the follow-up evaluations, quantitative feedback revealed that 100% of students found this class to be both interesting and relevant to their work. In addition, qualitative comments revealed that students found the discussion of power dynamics to be a particularly helpful concept for thinking about the ethical issues we discussed in class. After running this module, there are two lessons that are worth highlighting:

• Including an explicit discussion during class of how the ethical issues connect to the technical content in the course seemed to improve student engagement with the material. On the evaluation form, one student specifically noted their appreciation for how the module topic was connected to course’s focus on embedded ML on devices.
• Utilizing case studies for the class activity so that students have an opportunity to practice applying what they have learned is important not only for fostering engagement, but for helping students fully understand the philosophical tools and concepts as well. Several students reported feeling more comfortable with the philosophical concepts once we began discussing them in the context of real-world examples.