-
Expected Utilitarianism
Authors:
Heather M. Roff
Abstract:
We want artificial intelligence (AI) to be beneficial. This is the grounding assumption of most of the attitudes towards AI research. We want AI to be "good" for humanity. We want it to help, not hinder, humans. Yet what exactly this entails in theory and in practice is not immediately apparent. Theoretically, this declarative statement subtly implies a commitment to a consequentialist ethics. P…
▽ More
We want artificial intelligence (AI) to be beneficial. This is the grounding assumption of most of the attitudes towards AI research. We want AI to be "good" for humanity. We want it to help, not hinder, humans. Yet what exactly this entails in theory and in practice is not immediately apparent. Theoretically, this declarative statement subtly implies a commitment to a consequentialist ethics. Practically, some of the more promising machine learning techniques to create a robust AI, and perhaps even an artificial general intelligence (AGI) also commit one to a form of utilitarianism. In both dimensions, the logic of the beneficial AI movement may not in fact create "beneficial AI" in either narrow applications or in the form of AGI if the ethical assumptions are not made explicit and clear.
Additionally, as it is likely that reinforcement learning (RL) will be an important technique for machine learning in this area, it is also important to interrogate how RL smuggles in a particular type of consequentialist reasoning into the AI: particularly, a brute form of hedonistic act utilitarianism. Since the mathematical logic commits one to a maximization function, the result is that an AI will inevitably be seeking more and more rewards. We have two conclusions that arise from this. First, is that if one believes that a beneficial AI is an ethical AI, then one is committed to a framework that posits 'benefit' is tantamount to the greatest good for the greatest number. Second, if the AI relies on RL, then the way it reasons about itself, the environment, and other agents, will be through an act utilitarian morality. This proposition may, or may not, in fact be actually beneficial for humanity.
△ Less
Submitted 19 July, 2020;
originally announced August 2020.
-
Inequity aversion improves cooperation in intertemporal social dilemmas
Authors:
Edward Hughes,
Joel Z. Leibo,
Matthew G. Phillips,
Karl Tuyls,
Edgar A. Duéñez-Guzmán,
Antonio García Castañeda,
Iain Dunning,
Tina Zhu,
Kevin R. McKee,
Raphael Koster,
Heather Roff,
Thore Graepel
Abstract:
Groups of humans are often able to find ways to cooperate with one another in complex, temporally extended social dilemmas. Models based on behavioral economics are only able to explain this phenomenon for unrealistic stateless matrix games. Recently, multi-agent reinforcement learning has been applied to generalize social dilemma problems to temporally and spatially extended Markov games. However…
▽ More
Groups of humans are often able to find ways to cooperate with one another in complex, temporally extended social dilemmas. Models based on behavioral economics are only able to explain this phenomenon for unrealistic stateless matrix games. Recently, multi-agent reinforcement learning has been applied to generalize social dilemma problems to temporally and spatially extended Markov games. However, this has not yet generated an agent that learns to cooperate in social dilemmas as humans do. A key insight is that many, but not all, human individuals have inequity averse social preferences. This promotes a particular resolution of the matrix game social dilemma wherein inequity-averse individuals are personally pro-social and punish defectors. Here we extend this idea to Markov games and show that it promotes cooperation in several types of sequential social dilemma, via a profitable interaction with policy learnability. In particular, we find that inequity aversion improves temporal credit assignment for the important class of intertemporal social dilemmas. These results help explain how large-scale cooperation may emerge and persist.
△ Less
Submitted 27 September, 2018; v1 submitted 23 March, 2018;
originally announced March 2018.
-
The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation
Authors:
Miles Brundage,
Shahar Avin,
Jack Clark,
Helen Toner,
Peter Eckersley,
Ben Garfinkel,
Allan Dafoe,
Paul Scharre,
Thomas Zeitzoff,
Bobby Filar,
Hyrum Anderson,
Heather Roff,
Gregory C. Allen,
Jacob Steinhardt,
Carrick Flynn,
Seán Ó hÉigeartaigh,
Simon Beard,
Haydn Belfield,
Sebastian Farquhar,
Clare Lyle,
Rebecca Crootof,
Owain Evans,
Michael Page,
Joanna Bryson,
Roman Yampolskiy
, et al. (1 additional authors not shown)
Abstract:
This report surveys the landscape of potential security threats from malicious uses of AI, and proposes ways to better forecast, prevent, and mitigate these threats. After analyzing the ways in which AI may influence the threat landscape in the digital, physical, and political domains, we make four high-level recommendations for AI researchers and other stakeholders. We also suggest several promis…
▽ More
This report surveys the landscape of potential security threats from malicious uses of AI, and proposes ways to better forecast, prevent, and mitigate these threats. After analyzing the ways in which AI may influence the threat landscape in the digital, physical, and political domains, we make four high-level recommendations for AI researchers and other stakeholders. We also suggest several promising areas for further research that could expand the portfolio of defenses, or make attacks less effective or harder to execute. Finally, we discuss, but do not conclusively resolve, the long-term equilibrium of attackers and defenders.
△ Less
Submitted 20 February, 2018;
originally announced February 2018.
