Foundational Challenges in Assuring Alignment and Safety of Large Language Models
Authors:
Usman Anwar,
Abulhair Saparov,
Javier Rando,
Daniel Paleka,
Miles Turpin,
Peter Hase,
Ekdeep Singh Lubana,
Erik Jenner,
Stephen Casper,
Oliver Sourbut,
Benjamin L. Edelman,
Zhaowei Zhang,
Mario Günther,
Anton Korinek,
Jose Hernandez-Orallo,
Lewis Hammond,
Eric Bigelow,
Alexander Pan,
Lauro Langosco,
Tomasz Korbak,
Heidi Zhang,
Ruiqi Zhong,
Seán Ó hÉigeartaigh,
Gabriel Recchia,
Giulio Corsi
, et al. (13 additional authors not shown)
Abstract:
This work identifies 18 foundational challenges in assuring the alignment and safety of large language models (LLMs). These challenges are organized into three different categories: scientific understanding of LLMs, development and deployment methods, and sociotechnical challenges. Based on the identified challenges, we pose $200+$ concrete research questions.
This work identifies 18 foundational challenges in assuring the alignment and safety of large language models (LLMs). These challenges are organized into three different categories: scientific understanding of LLMs, development and deployment methods, and sociotechnical challenges. Based on the identified challenges, we pose $200+$ concrete research questions.
△ Less
Submitted 15 April, 2024;
originally announced April 2024.
Cooperation and Control in Delegation Games
Authors:
Oliver Sourbut,
Lewis Hammond,
Harriet Wood
Abstract:
Many settings of interest involving humans and machines -- from virtual personal assistants to autonomous vehicles -- can naturally be modelled as principals (humans) delegating to agents (machines), which then interact with each other on their principals' behalf. We refer to these multi-principal, multi-agent scenarios as delegation games. In such games, there are two important failure modes: pro…
▽ More
Many settings of interest involving humans and machines -- from virtual personal assistants to autonomous vehicles -- can naturally be modelled as principals (humans) delegating to agents (machines), which then interact with each other on their principals' behalf. We refer to these multi-principal, multi-agent scenarios as delegation games. In such games, there are two important failure modes: problems of control (where an agent fails to act in line their principal's preferences) and problems of cooperation (where the agents fail to work well together). In this paper we formalise and analyse these problems, further breaking them down into issues of alignment (do the players have similar preferences?) and capabilities (how competent are the players at satisfying those preferences?). We show -- theoretically and empirically -- how these measures determine the principals' welfare, how they can be estimated using limited observations, and thus how they might be used to help us design more aligned and cooperative AI systems.
△ Less
Submitted 5 August, 2024; v1 submitted 24 February, 2024;
originally announced February 2024.
