-
(Beyond) Reasonable Doubt: Challenges that Public Defenders Face in Scrutinizing AI in Court
Authors:
Angela Jin,
Niloufar Salehi
Abstract:
Accountable use of AI systems in high-stakes settings relies on making systems contestable. In this paper we study efforts to contest AI systems in practice by studying how public defenders scrutinize AI in court. We present findings from interviews with 17 people in the U.S. public defense community to understand their perceptions of and experiences scrutinizing computational forensic software (C…
▽ More
Accountable use of AI systems in high-stakes settings relies on making systems contestable. In this paper we study efforts to contest AI systems in practice by studying how public defenders scrutinize AI in court. We present findings from interviews with 17 people in the U.S. public defense community to understand their perceptions of and experiences scrutinizing computational forensic software (CFS) -- automated decision systems that the government uses to convict and incarcerate, such as facial recognition, gunshot detection, and probabilistic genotyping tools. We find that our participants faced challenges assessing and contesting CFS reliability due to difficulties (a) navigating how CFS is developed and used, (b) overcoming judges and jurors' non-critical perceptions of CFS, and (c) gathering CFS expertise. To conclude, we provide recommendations that center the technical, social, and institutional context to better position interventions such as performance evaluations to support contestability in practice.
△ Less
Submitted 13 March, 2024;
originally announced March 2024.
-
Physician Detection of Clinical Harm in Machine Translation: Quality Estimation Aids in Reliance and Backtranslation Identifies Critical Errors
Authors:
Nikita Mehandru,
Sweta Agrawal,
Yimin Xiao,
Elaine C Khoong,
Ge Gao,
Marine Carpuat,
Niloufar Salehi
Abstract:
A major challenge in the practical use of Machine Translation (MT) is that users lack guidance to make informed decisions about when to rely on outputs. Progress in quality estimation research provides techniques to automatically assess MT quality, but these techniques have primarily been evaluated in vitro by comparison against human judgments outside of a specific context of use. This paper eval…
▽ More
A major challenge in the practical use of Machine Translation (MT) is that users lack guidance to make informed decisions about when to rely on outputs. Progress in quality estimation research provides techniques to automatically assess MT quality, but these techniques have primarily been evaluated in vitro by comparison against human judgments outside of a specific context of use. This paper evaluates quality estimation feedback in vivo with a human study simulating decision-making in high-stakes medical settings. Using Emergency Department discharge instructions, we study how interventions based on quality estimation versus backtranslation assist physicians in deciding whether to show MT outputs to a patient. We find that quality estimation improves appropriate reliance on MT, but backtranslation helps physicians detect more clinically harmful errors that QE alone often misses.
△ Less
Submitted 25 October, 2023;
originally announced October 2023.
-
Addressing interpersonal harm in online gaming communities: the opportunities and challenges for a restorative justice approach
Authors:
Sijia Xiao,
Shagun Jhaver,
Niloufar Salehi
Abstract:
Most social media platforms implement content moderation to address interpersonal harms such as harassment. Content moderation relies on offender-centered, punitive approaches, e.g., bans and content removal. We consider an alternative justice framework, restorative justice, which aids victims in healing, supports offenders in repairing the harm, and engages community members in addressing the har…
▽ More
Most social media platforms implement content moderation to address interpersonal harms such as harassment. Content moderation relies on offender-centered, punitive approaches, e.g., bans and content removal. We consider an alternative justice framework, restorative justice, which aids victims in healing, supports offenders in repairing the harm, and engages community members in addressing the harm collectively. To assess the utility of restorative justice in addressing online harm, we interviewed 23 users from Overwatch gaming communities, including moderators, victims, and offenders; such communities are particularly susceptible to harm, with nearly three quarters of all online game players suffering from some form of online abuse. We study how the communities currently handle harm cases through the lens of restorative justice and examine their attitudes toward implementing restorative justice processes. Our analysis reveals that cultural, technical, and resource-related obstacles hinder implementation of restorative justice within the existing punitive framework despite online community needs and existing structures to support it. We discuss how current content moderation systems can embed restorative justice goals and practices and overcome these challenges.
△ Less
Submitted 7 June, 2023; v1 submitted 2 November, 2022;
originally announced November 2022.
-
Beyond General Purpose Machine Translation: The Need for Context-specific Empirical Research to Design for Appropriate User Trust
Authors:
Wesley Hanwen Deng,
Nikita Mehandru,
Samantha Robertson,
Niloufar Salehi
Abstract:
Machine Translation (MT) has the potential to help people overcome language barriers and is widely used in high-stakes scenarios, such as in hospitals. However, in order to use MT reliably and safely, users need to understand when to trust MT outputs and how to assess the quality of often imperfect translation results. In this paper, we discuss research directions to support users to calibrate tru…
▽ More
Machine Translation (MT) has the potential to help people overcome language barriers and is widely used in high-stakes scenarios, such as in hospitals. However, in order to use MT reliably and safely, users need to understand when to trust MT outputs and how to assess the quality of often imperfect translation results. In this paper, we discuss research directions to support users to calibrate trust in MT systems. We share findings from an empirical study in which we conducted semi-structured interviews with 20 clinicians to understand how they communicate with patients across language barriers, and if and how they use MT systems. Based on our findings, we advocate for empirical research on how MT systems are used in practice as an important first step to addressing the challenges in building appropriate trust between users and MT tools.
△ Less
Submitted 13 May, 2022;
originally announced May 2022.
-
The Distressing Ads That Persist: Uncovering The Harms of Targeted Weight-Loss Ads Among Users with Histories of Disordered Eating
Authors:
Liza Gak,
Seyi Olojo,
Niloufar Salehi
Abstract:
Targeted advertising can harm vulnerable groups when it targets individuals' personal and psychological vulnerabilities. We focus on how targeted weight-loss advertisements harm people with histories of disordered eating. We identify three features of targeted advertising that cause harm: the persistence of personal data that can expose vulnerabilities, over-simplifying algorithmic relevancy model…
▽ More
Targeted advertising can harm vulnerable groups when it targets individuals' personal and psychological vulnerabilities. We focus on how targeted weight-loss advertisements harm people with histories of disordered eating. We identify three features of targeted advertising that cause harm: the persistence of personal data that can expose vulnerabilities, over-simplifying algorithmic relevancy models, and design patterns encouraging engagement that can facilitate unhealthy behavior. Through a series of semi-structured interviews with individuals with histories of unhealthy body stigma, dieting, and disordered eating, we found that targeted weight-loss ads reinforced low self-esteem and deepened pre-existing anxieties around food and exercise. At the same time, we observed that targeted individuals demonstrated agency and resistance against distressing ads. Drawing on scholarship in postcolonial environmental studies, we use the concept of slow violence to articulate how online targeted advertising inflicts harms that may not be immediately identifiable. CAUTION: This paper includes media that could be triggering, particularly to people with an eating disorder. Please use caution when reading, printing, or disseminating this paper.
△ Less
Submitted 25 October, 2022; v1 submitted 7 April, 2022;
originally announced April 2022.
-
Not Another School Resource Map: Meeting Underserved Families' Information Needs Requires Trusting Relationships and Personalized Care
Authors:
Samantha Robertson,
Tonya Nguyen,
Niloufar Salehi
Abstract:
Public school districts across the United States have implemented school choice systems that have the potential to improve underserved students' access to educational opportunities. However, research has shown that learning about and applying for schools can be extremely time-consuming and expensive, making it difficult for these systems to create more equitable access to resources in practice. A…
▽ More
Public school districts across the United States have implemented school choice systems that have the potential to improve underserved students' access to educational opportunities. However, research has shown that learning about and applying for schools can be extremely time-consuming and expensive, making it difficult for these systems to create more equitable access to resources in practice. A common factor surfaced in prior work is unequal access to information about the schools and enrollment process. In response, governments and non-profits have invested in providing more information about schools to parents, for instance, through detailed online dashboards. However, we know little about what information is actually useful for historically marginalized and underserved families. We conducted interviews with 10 low-income families and families of color to learn about the challenges they faced navigating an online school choice and enrollment system. We complement this data with four interviews with people who have supported families through the enrollment process in a wide range of roles, from school principal to non-profit staff ("parent advocates"). Our findings highlight the value of personalized support and trusting relationships to delivering relevant and helpful information. We contrast this against online information resources and dashboards, which tend to be impersonal, target a broad audience, and make strong assumptions about what parents should look for in a school without sensitivity to families' varying circumstances. We advocate for an assets-based design approach to information support in public school enrollment, which would ask how we can support the local, one-on-one support that community members already provide.
△ Less
Submitted 30 March, 2022; v1 submitted 29 March, 2022;
originally announced March 2022.
-
Bridging Action Frames: Instagram Infographics in U.S.Ethnic Movements
Authors:
Darya Kaviani,
Niloufar Salehi
Abstract:
Instagram infographics are a digital activism tool that have redefined action frames for technology-facilitated social movements. From the 1960s through the 1980s, United States ethnic movements practiced collective action: ideologically unified, resource-intensive traditional activism. Today, technologically enabled movements have been categorized as practicing connective action: individualized,…
▽ More
Instagram infographics are a digital activism tool that have redefined action frames for technology-facilitated social movements. From the 1960s through the 1980s, United States ethnic movements practiced collective action: ideologically unified, resource-intensive traditional activism. Today, technologically enabled movements have been categorized as practicing connective action: individualized, low-resource online activism. Yet, we argue that Instagram infographics are both connective and collective. This paper juxtaposes the insights of past and present U.S. ethnic movement activists and analyzes Black Lives Matter Instagram data over the course of 7 years (2014-2020). We find that Instagram infographic activism bridges connective and collective action in three ways: (1) Scope for Education: Visually enticing and digestible infographics reduce the friction of information dissemination, facilitating collective movement education while preserving customizability. (2) Reconciliation for Credibility: Activists use connective features to combat infographic misinformation and resolve internal differences, creating a trusted collective movement front. (3) High-Resource Efforts for Transformative Change: Instagram infographic activism has been paired with boots on the ground and action-oriented content, curating a connective-to-collective pipeline that expends movement resources. Our work unveils the vitality of evaluating digital activism action frames at the movement integration level, exemplifies the powerful coexistence of connective and collective action, and offers meaningful design implications for activists seeking to leverage this novel tool.
△ Less
Submitted 1 November, 2021;
originally announced November 2021.
-
Modeling Assumptions Clash with the Real World: Transparency, Equity, and Community Challenges for Student Assignment Algorithms
Authors:
Samantha Robertson,
Tonya Nguyen,
Niloufar Salehi
Abstract:
Across the United States, a growing number of school districts are turning to matching algorithms to assign students to public schools. The designers of these algorithms aimed to promote values such as transparency, equity, and community in the process. However, school districts have encountered practical challenges in their deployment. In fact, San Francisco Unified School District voted to stop…
▽ More
Across the United States, a growing number of school districts are turning to matching algorithms to assign students to public schools. The designers of these algorithms aimed to promote values such as transparency, equity, and community in the process. However, school districts have encountered practical challenges in their deployment. In fact, San Francisco Unified School District voted to stop using and completely redesign their student assignment algorithm because it was not promoting educational equity in practice. We analyze this system using a Value Sensitive Design approach and find that one reason values are not met in practice is that the system relies on modeling assumptions about families' priorities, constraints, and goals that clash with the real world. These assumptions overlook the complex barriers to ideal participation that many families face, particularly because of socioeconomic inequalities. We argue that direct, ongoing engagement with stakeholders is central to aligning algorithmic values with real world conditions. In doing so we must broaden how we evaluate algorithms while recognizing the limitations of purely algorithmic solutions in addressing complex socio-political problems.
△ Less
Submitted 25 January, 2021;
originally announced January 2021.
-
Whither AutoML? Understanding the Role of Automation in Machine Learning Workflows
Authors:
Doris Xin,
Eva Yiwei Wu,
Doris Jung-Lin Lee,
Niloufar Salehi,
Aditya Parameswaran
Abstract:
Efforts to make machine learning more widely accessible have led to a rapid increase in Auto-ML tools that aim to automate the process of training and deploying machine learning. To understand how Auto-ML tools are used in practice today, we performed a qualitative study with participants ranging from novice hobbyists to industry researchers who use Auto-ML tools. We present insights into the bene…
▽ More
Efforts to make machine learning more widely accessible have led to a rapid increase in Auto-ML tools that aim to automate the process of training and deploying machine learning. To understand how Auto-ML tools are used in practice today, we performed a qualitative study with participants ranging from novice hobbyists to industry researchers who use Auto-ML tools. We present insights into the benefits and deficiencies of existing tools, as well as the respective roles of the human and automation in ML workflows. Finally, we discuss design implications for the future of Auto-ML tool development. We argue that instead of full automation being the ultimate goal of Auto-ML, designers of these tools should focus on supporting a partnership between the user and the Auto-ML tool. This means that a range of Auto-ML tools will need to be developed to support varying user goals such as simplicity, reproducibility, and reliability.
△ Less
Submitted 12 January, 2021;
originally announced January 2021.
-
What If I Don't Like Any Of The Choices? The Limits of Preference Elicitation for Participatory Algorithm Design
Authors:
Samantha Robertson,
Niloufar Salehi
Abstract:
Emerging methods for participatory algorithm design have proposed collecting and aggregating individual stakeholder preferences to create algorithmic systems that account for those stakeholders' values. Using algorithmic student assignment as a case study, we argue that optimizing for individual preference satisfaction in the distribution of limited resources may actually inhibit progress towards…
▽ More
Emerging methods for participatory algorithm design have proposed collecting and aggregating individual stakeholder preferences to create algorithmic systems that account for those stakeholders' values. Using algorithmic student assignment as a case study, we argue that optimizing for individual preference satisfaction in the distribution of limited resources may actually inhibit progress towards social and distributive justice. Individual preferences can be a useful signal but should be expanded to support more expressive and inclusive forms of democratic participation.
△ Less
Submitted 13 July, 2020;
originally announced July 2020.
-
A branch, price and remember algorithm for the U shaped assembly line balancing problem
Authors:
Abdolmajid Yolmeh,
Najmeh Salehi
Abstract:
In this paper we propose a branch, price and remember algorithm to solve the U shaped assembly line balancing problem. Our proposed algorithm uses a column generation approach to obtain tight lower bounds for this problem. It also stores generated columns in memory to enhance the speed of column generation approach. We also develop a modification of Hoffman algorithm to obtain high quality upper b…
▽ More
In this paper we propose a branch, price and remember algorithm to solve the U shaped assembly line balancing problem. Our proposed algorithm uses a column generation approach to obtain tight lower bounds for this problem. It also stores generated columns in memory to enhance the speed of column generation approach. We also develop a modification of Hoffman algorithm to obtain high quality upper bounds. Our computational results show that our proposed algorithm is able to optimally solve 255 of Scholl's well-known 269 benchmark problems. Previous best known exact algorithm, ULINO, is able to solve 233 of the 269 benchmark problems. We also examined our algorithm on a new data set and the results show that our algorithm is able to solve 96.48 percent of all available benchmark problems.
△ Less
Submitted 2 September, 2017; v1 submitted 14 August, 2017;
originally announced August 2017.
-
Huddler: Convening Stable and Familiar Crowd Teams Despite Unpredictable Availability
Authors:
Niloufar Salehi,
Andrew McCabe,
Melissa Valentine,
Michael Bernstein
Abstract:
Distributed, parallel crowd workers can accomplish simple tasks through workflows, but teams of collaborating crowd workers are necessary for complex goals. Unfortunately, a fundamental condition for effective teams - familiarity with other members - stands in contrast to crowd work's flexible, on-demand nature. We enable effective crowd teams with Huddler, a system for workers to assemble familia…
▽ More
Distributed, parallel crowd workers can accomplish simple tasks through workflows, but teams of collaborating crowd workers are necessary for complex goals. Unfortunately, a fundamental condition for effective teams - familiarity with other members - stands in contrast to crowd work's flexible, on-demand nature. We enable effective crowd teams with Huddler, a system for workers to assemble familiar teams even under unpredictable availability and strict time constraints. Huddler utilizes a dynamic programming algorithm to optimize for highly familiar teammates when individual availability is unknown. We first present a field experiment that demonstrates the value of familiarity for crowd teams: familiar crowd teams doubled the performance of ad-hoc (unfamiliar) teams on a collaborative task. We then report a two-week field deployment wherein Huddler enabled crowd workers to convene highly familiar teams in 18 minutes on average. This research advances the goal of supporting long-term, team-based collaborations without sacrificing the flexibility of crowd work.
△ Less
Submitted 26 October, 2016;
originally announced October 2016.
-
Atelier: Repurposing Expert Crowdsourcing Tasks as Micro-internships
Authors:
Ryo Suzuki,
Niloufar Salehi,
Michelle S. Lam,
Juan C. Marroquin,
Michael S. Bernstein
Abstract:
Expert crowdsourcing marketplaces have untapped potential to empower workers' career and skill development. Currently, many workers cannot afford to invest the time and sacrifice the earnings required to learn a new skill, and a lack of experience makes it difficult to get job offers even if they do. In this paper, we seek to lower the threshold to skill development by repurposing existing tasks o…
▽ More
Expert crowdsourcing marketplaces have untapped potential to empower workers' career and skill development. Currently, many workers cannot afford to invest the time and sacrifice the earnings required to learn a new skill, and a lack of experience makes it difficult to get job offers even if they do. In this paper, we seek to lower the threshold to skill development by repurposing existing tasks on the marketplace as mentored, paid, real-world work experiences, which we refer to as micro-internships. We instantiate this idea in Atelier, a micro-internship platform that connects crowd interns with crowd mentors. Atelier guides mentor-intern pairs to break down expert crowdsourcing tasks into milestones, review intermediate output, and problem-solve together. We conducted a field experiment comparing Atelier's mentorship model to a non-mentored alternative on a real-world programming crowdsourcing task, finding that Atelier helped interns maintain forward progress and absorb best practices.
△ Less
Submitted 21 February, 2016;
originally announced February 2016.
