Showing 1–14 of 14 results for author: Moss, E

Introducing v0.5 of the AI Safety Benchmark from MLCommons

Authors: Bertie Vidgen, Adarsh Agrawal, Ahmed M. Ahmed, Victor Akinwande, Namir Al-Nuaimi, Najla Alfaraj, Elie Alhajjar, Lora Aroyo, Trupti Bavalatti, Max Bartolo, Borhane Blili-Hamelin, Kurt Bollacker, Rishi Bomassani, Marisa Ferrara Boston, Siméon Campos, Kal Chakra, Canyu Chen, Cody Coleman, Zacharie Delpierre Coudert, Leon Derczynski, Debojyoti Dutta, Ian Eisenberg, James Ezick, Heather Frase, Brian Fuller , et al. (75 additional authors not shown)

Abstract: This paper introduces v0.5 of the AI Safety Benchmark, which has been created by the MLCommons AI Safety Working Group. The AI Safety Benchmark has been designed to assess the safety risks of AI systems that use chat-tuned language models. We introduce a principled approach to specifying and constructing the benchmark, which for v0.5 covers only a single use case (an adult chatting to a general-pu… ▽ More This paper introduces v0.5 of the AI Safety Benchmark, which has been created by the MLCommons AI Safety Working Group. The AI Safety Benchmark has been designed to assess the safety risks of AI systems that use chat-tuned language models. We introduce a principled approach to specifying and constructing the benchmark, which for v0.5 covers only a single use case (an adult chatting to a general-purpose assistant in English), and a limited set of personas (i.e., typical users, malicious users, and vulnerable users). We created a new taxonomy of 13 hazard categories, of which 7 have tests in the v0.5 benchmark. We plan to release version 1.0 of the AI Safety Benchmark by the end of 2024. The v1.0 benchmark will provide meaningful insights into the safety of AI systems. However, the v0.5 benchmark should not be used to assess the safety of AI systems. We have sought to fully document the limitations, flaws, and challenges of v0.5. This release of v0.5 of the AI Safety Benchmark includes (1) a principled approach to specifying and constructing the benchmark, which comprises use cases, types of systems under test (SUTs), language and context, personas, tests, and test items; (2) a taxonomy of 13 hazard categories with definitions and subcategories; (3) tests for seven of the hazard categories, each comprising a unique set of test items, i.e., prompts. There are 43,090 test items in total, which we created with templates; (4) a grading system for AI systems against the benchmark; (5) an openly available platform, and downloadable tool, called ModelBench that can be used to evaluate the safety of AI systems on the benchmark; (6) an example evaluation report which benchmarks the performance of over a dozen openly available chat-tuned language models; (7) a test specification for the benchmark. △ Less

Submitted 13 May, 2024; v1 submitted 18 April, 2024; originally announced April 2024.

Materiality and Risk in the Age of Pervasive AI Sensors

Authors: Matthew Stewart, Emanuel Moss, Pete Warden, Brian Plancher, Susan Kennedy, Mona Sloane, Vijay Janapa Reddi

Abstract: Artificial intelligence systems connected to sensor-laden devices are becoming pervasive, which has significant implications for a range of AI risks, including to privacy, the environment, autonomy, and more. There is therefore a growing need for increased accountability around the responsible development and deployment of these technologies. In this paper, we provide a comprehensive analysis of t… ▽ More Artificial intelligence systems connected to sensor-laden devices are becoming pervasive, which has significant implications for a range of AI risks, including to privacy, the environment, autonomy, and more. There is therefore a growing need for increased accountability around the responsible development and deployment of these technologies. In this paper, we provide a comprehensive analysis of the evolution of sensors, the risks they pose by virtue of their material existence in the world, and the impacts of ubiquitous sensing and on-device AI. We propose incorporating sensors into risk management frameworks and call for more responsible sensor and system design paradigms that address risks of such systems. To do so, we trace the evolution of sensors from analog devices to intelligent, networked systems capable of real-time data analysis and decision-making at the extreme edge of the network. We show that the proliferation of sensors is driven by calculative models that prioritize data collection and cost reduction and produce risks that emerge around privacy, surveillance, waste, and power dynamics. We then analyze these risks, highlighting issues of validity, safety, security, accountability, interpretability, and bias. We surface sensor-related risks not commonly captured in existing approaches to AI risk management, using a materiality lens that reveals how physical sensor properties shape data and algorithmic models. We conclude by advocating for increased attention to the materiality of algorithmic systems, and of on-device AI sensors in particular, and highlight the need for development of a responsible sensor design paradigm that empowers users and communities and leads to a future of increased fairness, accountability and transparency. △ Less

Submitted 16 February, 2024; originally announced February 2024.

The Cadaver in the Machine: The Social Practices of Measurement and Validation in Motion Capture Technology

Authors: Emma Harvey, Hauke Sandhaus, Abigail Z. Jacobs, Emanuel Moss, Mona Sloane

Abstract: Motion capture systems, used across various domains, make body representations concrete through technical processes. We argue that the measurement of bodies and the validation of measurements for motion capture systems can be understood as social practices. By analyzing the findings of a systematic literature review (N=278) through the lens of social practice theory, we show how these practices, a… ▽ More Motion capture systems, used across various domains, make body representations concrete through technical processes. We argue that the measurement of bodies and the validation of measurements for motion capture systems can be understood as social practices. By analyzing the findings of a systematic literature review (N=278) through the lens of social practice theory, we show how these practices, and their varying attention to errors, become ingrained in motion capture design and innovation over time. Moreover, we show how contemporary motion capture systems perpetuate assumptions about human bodies and their movements. We suggest that social practices of measurement and validation are ubiquitous in the development of data- and sensor-driven systems more broadly, and provide this work as a basis for investigating hidden design assumptions and their potential negative consequences in human-computer interaction. △ Less

Submitted 19 January, 2024; originally announced January 2024.

Comments: 34 pages, 9 figures. To appear in the 2024 ACM CHI Conference on Human Factors in Computing Systems (CHI '24)

Model Complexity of Program Phases

Authors: Arjun Karuvally, J. Eliot B. Moss

Abstract: In resource limited computing systems, sequence prediction models must operate under tight constraints. Various models are available that cater to prediction under these conditions that in some way focus on reducing the cost of implementation. These resource constrained sequence prediction models, in practice, exhibit a fundamental tradeoff between the cost of implementation and the quality of its… ▽ More In resource limited computing systems, sequence prediction models must operate under tight constraints. Various models are available that cater to prediction under these conditions that in some way focus on reducing the cost of implementation. These resource constrained sequence prediction models, in practice, exhibit a fundamental tradeoff between the cost of implementation and the quality of its predictions. This fundamental tradeoff seems to be largely unexplored for models for different tasks. Here we formulate the necessary theory and an associated empirical procedure to explore this tradeoff space for a particular family of machine learning models such as deep neural networks. We anticipate that the knowledge of the behavior of this tradeoff may be beneficial in understanding the theoretical and practical limits of creation and deployment of models for resource constrained tasks. △ Less

Submitted 5 October, 2023; originally announced October 2023.

Off-Diagonal Ramsey Multiplicity

Authors: Elena Moss, Jonathan A. Noel

Abstract: The Ramsey multiplicity problem asks for the minimum asymptotic density of monochromatic labelled copies of a graph $H$ in a red/blue colouring of the edges of $K_n$. We introduce an off-diagonal generalization in which the goal is to minimize a certain weighted sum of the densities of red copies of one graph and blue copies of another. We build up various properties of this new notion, including… ▽ More The Ramsey multiplicity problem asks for the minimum asymptotic density of monochromatic labelled copies of a graph $H$ in a red/blue colouring of the edges of $K_n$. We introduce an off-diagonal generalization in which the goal is to minimize a certain weighted sum of the densities of red copies of one graph and blue copies of another. We build up various properties of this new notion, including a useful "dual formulation," and use these results to solve the problem for several pairs of graphs. △ Less

Submitted 29 June, 2023; originally announced June 2023.

Comments: 28 pages

MSC Class: 05C35; 05D10

How and Why do Researchers Reference Data? A Study of Rhetorical Features and Functions of Data References in Academic Articles

Authors: Sara Lafia, Andrea Thomer, Elizabeth Moss, David Bleckley, Libby Hemphill

Abstract: Data reuse is a common practice in the social sciences. While published data play an essential role in the production of social science research, they are not consistently cited, which makes it difficult to assess their full scholarly impact and give credit to the original data producers. Furthermore, it can be challenging to understand researchers' motivations for referencing data. Like reference… ▽ More Data reuse is a common practice in the social sciences. While published data play an essential role in the production of social science research, they are not consistently cited, which makes it difficult to assess their full scholarly impact and give credit to the original data producers. Furthermore, it can be challenging to understand researchers' motivations for referencing data. Like references to academic literature, data references perform various rhetorical functions, such as paying homage, signaling disagreement, or drawing comparisons. This paper studies how and why researchers reference social science data in their academic writing. We develop a typology to model relationships between the entities that anchor data references, along with their features (access, actions, locations, styles, types) and functions (critique, describe, illustrate, interact, legitimize). We illustrate the use of the typology by coding multidisciplinary research articles (n=30) referencing social science data archived at the Inter-university Consortium for Political and Social Research (ICPSR). We show how our typology captures researchers' interactions with data and purposes for referencing data. Our typology provides a systematic way to document and analyze researchers' narratives about data use, extending our ability to give credit to data that support research. △ Less

Submitted 16 February, 2023; originally announced February 2023.

Comments: 35 pages, 2 appendices, 1 table

The Objective Function: Science and Society in the Age of Machine Intelligence

Authors: Emanuel Moss

Abstract: Machine intelligence, or the use of complex computational and statistical practices to make predictions and classifications based on data representations of phenomena, has been applied to domains as disparate as criminal justice, commerce, medicine, media and the arts, mechanical engineering, among others. How has machine intelligence become able to glide so freely across, and to make such waves f… ▽ More Machine intelligence, or the use of complex computational and statistical practices to make predictions and classifications based on data representations of phenomena, has been applied to domains as disparate as criminal justice, commerce, medicine, media and the arts, mechanical engineering, among others. How has machine intelligence become able to glide so freely across, and to make such waves for, these domains? In this dissertation, I take up that question by ethnographically engaging with how the authority of machine learning has been constructed such that it can influence so many domains, and I investigate what the consequences are of it being able to do so. By examining the workplace practices of the applied machine learning researchers who produce machine intelligence, those they work with, and the artifacts they produce. The dissertation begins by arguing that machine intelligence proceeds from a naive form of empiricism with ties to positivist intellectual traditions of the 17th and 18th centuries. This naive empiricism eschews other forms of knowledge and theory formation in order for applied machine learning researchers to enact data performances that bring objects of analysis into existence as entities capable of being subjected to machine intelligence. By data performances, I mean generative enactments which bring into existence that which machine intelligence purports to analyze or describe. The enactment of data performances is analyzed as an agential cut into a representational field that produces both stable claims about the world and the interpretive frame in which those claims can hold true. The dissertation also examines how machine intelligence depends upon a range of accommodations from other institutions and organizations, from data collection and processing to organizational commitments to support the work of applied machine learning researchers. △ Less

Submitted 21 September, 2022; originally announced September 2022.

Model-Based Reinforcement Learning with SINDy

Authors: Rushiv Arora, Bruno Castro da Silva, Eliot Moss

Abstract: We draw on the latest advancements in the physics community to propose a novel method for discovering the governing non-linear dynamics of physical systems in reinforcement learning (RL). We establish that this method is capable of discovering the underlying dynamics using significantly fewer trajectories (as little as one rollout with $\leq 30$ time steps) than state of the art model learning alg… ▽ More We draw on the latest advancements in the physics community to propose a novel method for discovering the governing non-linear dynamics of physical systems in reinforcement learning (RL). We establish that this method is capable of discovering the underlying dynamics using significantly fewer trajectories (as little as one rollout with $\leq 30$ time steps) than state of the art model learning algorithms. Further, the technique learns a model that is accurate enough to induce near-optimal policies given significantly fewer trajectories than those required by model-free algorithms. It brings the benefits of model-based RL without requiring a model to be developed in advance, for systems that have physics-based dynamics. To establish the validity and applicability of this algorithm, we conduct experiments on four classic control tasks. We found that an optimal policy trained on the discovered dynamics of the underlying system can generalize well. Further, the learned policy performs well when deployed on the actual physical system, thus bridging the model to real system gap. We further compare our method to state-of-the-art model-based and model-free approaches, and show that our method requires fewer trajectories sampled on the true physical system compared other methods. Additionally, we explored approximate dynamics models and found that they also can perform well. △ Less

Submitted 30 August, 2022; originally announced August 2022.

Comments: 8 pages, 1 figure, 1 table, 1 algorithm, presented at the Decision Awareness in Reinforcement Learning workshop held at the International Conference on Machine Learning, 22 July 2022, Baltimore MD, USA

Librarian-in-the-Loop: A Natural Language Processing Paradigm for Detecting Informal Mentions of Research Data in Academic Literature

Authors: Lizhou Fan, Sara Lafia, David Bleckley, Elizabeth Moss, Andrea Thomer, Libby Hemphill

Abstract: Data citations provide a foundation for studying research data impact. Collecting and managing data citations is a new frontier in archival science and scholarly communication. However, the discovery and curation of research data citations is labor intensive. Data citations that reference unique identifiers (i.e. DOIs) are readily findable; however, informal mentions made to research data are more… ▽ More Data citations provide a foundation for studying research data impact. Collecting and managing data citations is a new frontier in archival science and scholarly communication. However, the discovery and curation of research data citations is labor intensive. Data citations that reference unique identifiers (i.e. DOIs) are readily findable; however, informal mentions made to research data are more challenging to infer. We propose a natural language processing (NLP) paradigm to support the human task of identifying informal mentions made to research datasets. The work of discovering informal data mentions is currently performed by librarians and their staff in the Inter-university Consortium for Political and Social Research (ICPSR), a large social science data archive that maintains a large bibliography of data-related literature. The NLP model is bootstrapped from data citations actively collected by librarians at ICPSR. The model combines pattern matching with multiple iterations of human annotations to learn additional rules for detecting informal data mentions. These examples are then used to train an NLP pipeline. The librarian-in-the-loop paradigm is centered in the data work performed by ICPSR librarians, supporting broader efforts to build a more comprehensive bibliography of data-related literature that reflects the scholarly communities of research data users. △ Less

Submitted 9 March, 2022; originally announced March 2022.

A relationship and not a thing: A relational approach to algorithmic accountability and assessment documentation

Authors: Jacob Metcalf, Emanuel Moss, Ranjit Singh, Emnet Tafese, Elizabeth Anne Watkins

Abstract: Central to a number of scholarly, regulatory, and public conversations about algorithmic accountability is the question of who should have access to documentation that reveals the inner workings, intended function, and anticipated consequences of algorithmic systems, potentially establishing new routes for impacted publics to contest the operations of these systems. Currently, developers largely h… ▽ More Central to a number of scholarly, regulatory, and public conversations about algorithmic accountability is the question of who should have access to documentation that reveals the inner workings, intended function, and anticipated consequences of algorithmic systems, potentially establishing new routes for impacted publics to contest the operations of these systems. Currently, developers largely have a monopoly on information about how their systems actually work and are incentivized to maintain their own ignorance about aspects of how their systems affect the world. Increasingly, legislators, regulators and advocates have turned to assessment documentation in order to address the gap between the public's experience of algorithmic harms and the obligations of developers to document and justify their design decisions. However, issues of standing and expertise currently prevent publics from cohering around shared interests in preventing and redressing algorithmic harms; as we demonstrate with multiple cases, courts often find computational harms non-cognizable and rarely require developers to address material claims of harm. Constructed with a triadic accountability relationship, algorithmic impact assessment regimes could alter this situation by establishing procedural rights around public access to reporting and documentation. Developing a relational approach to accountability, we argue that robust accountability regimes must establish opportunities for publics to cohere around shared experiences and interests, and to contest the outcomes of algorithmic systems that affect their lives. Furthermore, algorithmic accountability policies currently under consideration in many jurisdictions must provide the public with adequate standing and opportunities to access and contest the documentation provided by the actors and the judgments passed by the forum. △ Less

Submitted 2 March, 2022; originally announced March 2022.

Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learning

Authors: A. Feder Cooper, Emanuel Moss, Benjamin Laufer, Helen Nissenbaum

Abstract: In 1996, Accountability in a Computerized Society [95] issued a clarion call concerning the erosion of accountability in society due to the ubiquitous delegation of consequential functions to computerized systems. Nissenbaum [95] described four barriers to accountability that computerization presented, which we revisit in relation to the ascendance of data-driven algorithmic systems--i.e., machine… ▽ More In 1996, Accountability in a Computerized Society [95] issued a clarion call concerning the erosion of accountability in society due to the ubiquitous delegation of consequential functions to computerized systems. Nissenbaum [95] described four barriers to accountability that computerization presented, which we revisit in relation to the ascendance of data-driven algorithmic systems--i.e., machine learning or artificial intelligence--to uncover new challenges for accountability that these systems present. Nissenbaum's original paper grounded discussion of the barriers in moral philosophy; we bring this analysis together with recent scholarship on relational accountability frameworks and discuss how the barriers present difficulties for instantiating a unified moral, relational framework in practice for data-driven algorithmic systems. We conclude by discussing ways of weakening the barriers in order to do so. △ Less

Submitted 13 May, 2022; v1 submitted 10 February, 2022; originally announced February 2022.

Journal ref: 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT '22)

A Silicon Valley Love Triangle: Hiring Algorithms, Pseudo-Science, and the Quest for Auditability

Authors: Mona Sloane, Emanuel Moss, Rumman Chowdhury

Abstract: In this paper, we suggest a systematic approach for developing socio-technical assessment for hiring ADS. We suggest using a matrix to expose underlying assumptions rooted in pseudoscientific essentialized understandings of human nature and capability, and to critically investigate emerging auditing standards and practices that fail to address these assumptions. In this paper, we suggest a systematic approach for developing socio-technical assessment for hiring ADS. We suggest using a matrix to expose underlying assumptions rooted in pseudoscientific essentialized understandings of human nature and capability, and to critically investigate emerging auditing standards and practices that fail to address these assumptions. △ Less

Submitted 12 May, 2022; v1 submitted 23 June, 2021; originally announced June 2021.

Participation is not a Design Fix for Machine Learning

Authors: Mona Sloane, Emanuel Moss, Olaitan Awomolo, Laura Forlano

Abstract: This paper critically examines existing modes of participation in design practice and machine learning. Cautioning against 'participation-washing', it suggests that the ML community must become attuned to possibly exploitative and extractive forms of community involvement and shift away from the prerogatives of context-independent scalability. This paper critically examines existing modes of participation in design practice and machine learning. Cautioning against 'participation-washing', it suggests that the ML community must become attuned to possibly exploitative and extractive forms of community involvement and shift away from the prerogatives of context-independent scalability. △ Less

Submitted 11 August, 2020; v1 submitted 5 July, 2020; originally announced July 2020.

PlanAlyzer: Assessing Threats to the Validity of Online Experiments

Authors: Emma Tosch, Eytan Bakshy, Emery D. Berger, David D. Jensen, J. Eliot B. Moss

Abstract: Online experiments are ubiquitous. As the scale of experiments has grown, so has the complexity of their design and implementation. In response, firms have developed software frameworks for designing and deploying online experiments. Ensuring that experiments in these frameworks are correctly designed and that their results are trustworthy---referred to as *internal validity*---can be difficult. C… ▽ More Online experiments are ubiquitous. As the scale of experiments has grown, so has the complexity of their design and implementation. In response, firms have developed software frameworks for designing and deploying online experiments. Ensuring that experiments in these frameworks are correctly designed and that their results are trustworthy---referred to as *internal validity*---can be difficult. Currently, verifying internal validity requires manual inspection by someone with substantial expertise in experimental design. We present the first approach for statically checking the internal validity of online experiments. Our checks are based on well-known problems that arise in experimental design and causal inference. Our analyses target PlanOut, a widely deployed, open-source experimentation framework that uses a domain-specific language to specify and run complex experiments. We have built a tool, PlanAlyzer, that checks PlanOut programs for a variety of threats to internal validity, including failures of randomization, treatment assignment, and causal sufficiency. PlanAlyzer uses its analyses to automatically generate *contrasts*, a key type of information required to perform valid statistical analyses over experimental results. We demonstrate PlanAlyzer's utility on a corpus of PlanOut scripts deployed in production at Facebook, and we evaluate its ability to identify threats to validity on a mutated subset of this corpus. PlanAlyzer has both precision and recall of 92% on the mutated corpus, and 82% of the contrasts it automatically generates match hand-specified data. △ Less

Submitted 30 September, 2019; originally announced September 2019.

Comments: 30 pages, hella long

Journal ref: OOPSLA 2019