-
Proving Theorems using Incremental Learning and Hindsight Experience Replay
Authors:
Eser Aygün,
Laurent Orseau,
Ankit Anand,
Xavier Glorot,
Vlad Firoiu,
Lei M. Zhang,
Doina Precup,
Shibl Mourad
Abstract:
Traditional automated theorem provers for first-order logic depend on speed-optimized search and many handcrafted heuristics that are designed to work best over a wide range of domains. Machine learning approaches in literature either depend on these traditional provers to bootstrap themselves or fall short on reaching comparable performance. In this paper, we propose a general incremental learnin…
▽ More
Traditional automated theorem provers for first-order logic depend on speed-optimized search and many handcrafted heuristics that are designed to work best over a wide range of domains. Machine learning approaches in literature either depend on these traditional provers to bootstrap themselves or fall short on reaching comparable performance. In this paper, we propose a general incremental learning algorithm for training domain specific provers for first-order logic without equality, based only on a basic given-clause algorithm, but using a learned clause-scoring function. Clauses are represented as graphs and presented to transformer networks with spectral features. To address the sparsity and the initial lack of training data as well as the lack of a natural curriculum, we adapt hindsight experience replay to theorem proving, so as to be able to learn even when no proof can be found. We show that provers trained this way can match and sometimes surpass state-of-the-art traditional provers on the TPTP dataset in terms of both quantity and quality of the proofs.
△ Less
Submitted 20 December, 2021;
originally announced December 2021.
-
AI Explainability 360: Impact and Design
Authors:
Vijay Arya,
Rachel K. E. Bellamy,
Pin-Yu Chen,
Amit Dhurandhar,
Michael Hind,
Samuel C. Hoffman,
Stephanie Houde,
Q. Vera Liao,
Ronny Luss,
Aleksandra Mojsilovic,
Sami Mourad,
Pablo Pedemonte,
Ramya Raghavendra,
John Richards,
Prasanna Sattigeri,
Karthikeyan Shanmugam,
Moninder Singh,
Kush R. Varshney,
Dennis Wei,
Yunfeng Zhang
Abstract:
As artificial intelligence and machine learning algorithms become increasingly prevalent in society, multiple stakeholders are calling for these algorithms to provide explanations. At the same time, these stakeholders, whether they be affected citizens, government regulators, domain experts, or system developers, have different explanation needs. To address these needs, in 2019, we created AI Expl…
▽ More
As artificial intelligence and machine learning algorithms become increasingly prevalent in society, multiple stakeholders are calling for these algorithms to provide explanations. At the same time, these stakeholders, whether they be affected citizens, government regulators, domain experts, or system developers, have different explanation needs. To address these needs, in 2019, we created AI Explainability 360 (Arya et al. 2020), an open source software toolkit featuring ten diverse and state-of-the-art explainability methods and two evaluation metrics. This paper examines the impact of the toolkit with several case studies, statistics, and community feedback. The different ways in which users have experienced AI Explainability 360 have resulted in multiple types of impact and improvements in multiple metrics, highlighted by the adoption of the toolkit by the independent LF AI & Data Foundation. The paper also describes the flexible design of the toolkit, examples of its use, and the significant educational material and documentation available to its users.
△ Less
Submitted 24 September, 2021;
originally announced September 2021.
-
The Option Keyboard: Combining Skills in Reinforcement Learning
Authors:
André Barreto,
Diana Borsa,
Shaobo Hou,
Gheorghe Comanici,
Eser Aygün,
Philippe Hamel,
Daniel Toyama,
Jonathan Hunt,
Shibl Mourad,
David Silver,
Doina Precup
Abstract:
The ability to combine known skills to create new ones may be crucial in the solution of complex reinforcement learning problems that unfold over extended periods. We argue that a robust way of combining skills is to define and manipulate them in the space of pseudo-rewards (or "cumulants"). Based on this premise, we propose a framework for combining skills using the formalism of options. We show…
▽ More
The ability to combine known skills to create new ones may be crucial in the solution of complex reinforcement learning problems that unfold over extended periods. We argue that a robust way of combining skills is to define and manipulate them in the space of pseudo-rewards (or "cumulants"). Based on this premise, we propose a framework for combining skills using the formalism of options. We show that every deterministic option can be unambiguously represented as a cumulant defined in an extended domain. Building on this insight and on previous results on transfer learning, we show how to approximate options whose cumulants are linear combinations of the cumulants of known options. This means that, once we have learned options associated with a set of cumulants, we can instantaneously synthesise options induced by any linear combination of them, without any learning involved. We describe how this framework provides a hierarchical interface to the environment whose abstract actions correspond to combinations of basic skills. We demonstrate the practical benefits of our approach in a resource management problem and a navigation task involving a quadrupedal simulated robot.
△ Less
Submitted 24 June, 2021;
originally announced June 2021.
-
AndroidEnv: A Reinforcement Learning Platform for Android
Authors:
Daniel Toyama,
Philippe Hamel,
Anita Gergely,
Gheorghe Comanici,
Amelia Glaese,
Zafarali Ahmed,
Tyler Jackson,
Shibl Mourad,
Doina Precup
Abstract:
We introduce AndroidEnv, an open-source platform for Reinforcement Learning (RL) research built on top of the Android ecosystem. AndroidEnv allows RL agents to interact with a wide variety of apps and services commonly used by humans through a universal touchscreen interface. Since agents train on a realistic simulation of an Android device, they have the potential to be deployed on real devices.…
▽ More
We introduce AndroidEnv, an open-source platform for Reinforcement Learning (RL) research built on top of the Android ecosystem. AndroidEnv allows RL agents to interact with a wide variety of apps and services commonly used by humans through a universal touchscreen interface. Since agents train on a realistic simulation of an Android device, they have the potential to be deployed on real devices. In this report, we give an overview of the environment, highlighting the significant features it provides for research, and we present an empirical evaluation of some popular reinforcement learning agents on a set of tasks built on this platform.
△ Less
Submitted 27 May, 2021;
originally announced May 2021.
-
Training a First-Order Theorem Prover from Synthetic Data
Authors:
Vlad Firoiu,
Eser Aygun,
Ankit Anand,
Zafarali Ahmed,
Xavier Glorot,
Laurent Orseau,
Lei Zhang,
Doina Precup,
Shibl Mourad
Abstract:
A major challenge in applying machine learning to automated theorem proving is the scarcity of training data, which is a key ingredient in training successful deep learning models. To tackle this problem, we propose an approach that relies on training purely with synthetically generated theorems, without any human data aside from axioms. We use these theorems to train a neurally-guided saturation-…
▽ More
A major challenge in applying machine learning to automated theorem proving is the scarcity of training data, which is a key ingredient in training successful deep learning models. To tackle this problem, we propose an approach that relies on training purely with synthetically generated theorems, without any human data aside from axioms. We use these theorems to train a neurally-guided saturation-based prover. Our neural prover outperforms the state-of-the-art E-prover on this synthetic data in both time and search steps, and shows significant transfer to the unseen human-written theorems from the TPTP library, where it solves 72\% of first-order problems without equality.
△ Less
Submitted 6 April, 2021; v1 submitted 5 March, 2021;
originally announced March 2021.
-
Learning to Prove from Synthetic Theorems
Authors:
Eser Aygün,
Zafarali Ahmed,
Ankit Anand,
Vlad Firoiu,
Xavier Glorot,
Laurent Orseau,
Doina Precup,
Shibl Mourad
Abstract:
A major challenge in applying machine learning to automated theorem proving is the scarcity of training data, which is a key ingredient in training successful deep learning models. To tackle this problem, we propose an approach that relies on training with synthetic theorems, generated from a set of axioms. We show that such theorems can be used to train an automated prover and that the learned pr…
▽ More
A major challenge in applying machine learning to automated theorem proving is the scarcity of training data, which is a key ingredient in training successful deep learning models. To tackle this problem, we propose an approach that relies on training with synthetic theorems, generated from a set of axioms. We show that such theorems can be used to train an automated prover and that the learned prover transfers successfully to human-generated theorems. We demonstrate that a prover trained exclusively on synthetic theorems can solve a substantial fraction of problems in TPTP, a benchmark dataset that is used to compare state-of-the-art heuristic provers. Our approach outperforms a model trained on human-generated problems in most axiom sets, thereby showing the promise of using synthetic data for this task.
△ Less
Submitted 19 June, 2020;
originally announced June 2020.
-
Shaping representations through communication: community size effect in artificial learning systems
Authors:
Olivier Tieleman,
Angeliki Lazaridou,
Shibl Mourad,
Charles Blundell,
Doina Precup
Abstract:
Motivated by theories of language and communication that explain why communities with large numbers of speakers have, on average, simpler languages with more regularity, we cast the representation learning problem in terms of learning to communicate. Our starting point sees the traditional autoencoder setup as a single encoder with a fixed decoder partner that must learn to communicate. Generalizi…
▽ More
Motivated by theories of language and communication that explain why communities with large numbers of speakers have, on average, simpler languages with more regularity, we cast the representation learning problem in terms of learning to communicate. Our starting point sees the traditional autoencoder setup as a single encoder with a fixed decoder partner that must learn to communicate. Generalizing from there, we introduce community-based autoencoders in which multiple encoders and decoders collectively learn representations by being randomly paired up on successive training iterations. We find that increasing community sizes reduce idiosyncrasies in the learned codes, resulting in representations that better encode concept categories and correlate with human feature norms.
△ Less
Submitted 12 December, 2019;
originally announced December 2019.
-
One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques
Authors:
Vijay Arya,
Rachel K. E. Bellamy,
Pin-Yu Chen,
Amit Dhurandhar,
Michael Hind,
Samuel C. Hoffman,
Stephanie Houde,
Q. Vera Liao,
Ronny Luss,
Aleksandra Mojsilović,
Sami Mourad,
Pablo Pedemonte,
Ramya Raghavendra,
John Richards,
Prasanna Sattigeri,
Karthikeyan Shanmugam,
Moninder Singh,
Kush R. Varshney,
Dennis Wei,
Yunfeng Zhang
Abstract:
As artificial intelligence and machine learning algorithms make further inroads into society, calls are increasing from multiple stakeholders for these algorithms to explain their outputs. At the same time, these stakeholders, whether they be affected citizens, government regulators, domain experts, or system developers, present different requirements for explanations. Toward addressing these need…
▽ More
As artificial intelligence and machine learning algorithms make further inroads into society, calls are increasing from multiple stakeholders for these algorithms to explain their outputs. At the same time, these stakeholders, whether they be affected citizens, government regulators, domain experts, or system developers, present different requirements for explanations. Toward addressing these needs, we introduce AI Explainability 360 (http://aix360.mybluemix.net/), an open-source software toolkit featuring eight diverse and state-of-the-art explainability methods and two evaluation metrics. Equally important, we provide a taxonomy to help entities requiring explanations to navigate the space of explanation methods, not only those in the toolkit but also in the broader literature on explainability. For data scientists and other users of the toolkit, we have implemented an extensible software architecture that organizes methods according to their place in the AI modeling pipeline. We also discuss enhancements to bring research innovations closer to consumers of explanations, ranging from simplified, more accessible versions of algorithms, to tutorials and an interactive web demo to introduce AI explainability to different audiences and application domains. Together, our toolkit and taxonomy can help identify gaps where more explainability methods are needed and provide a platform to incorporate them as they are developed.
△ Less
Submitted 14 September, 2019; v1 submitted 6 September, 2019;
originally announced September 2019.
-
Learning Hierarchical Teaching Policies for Cooperative Agents
Authors:
Dong-Ki Kim,
Miao Liu,
Shayegan Omidshafiei,
Sebastian Lopez-Cot,
Matthew Riemer,
Golnaz Habibi,
Gerald Tesauro,
Sami Mourad,
Murray Campbell,
Jonathan P. How
Abstract:
Collective learning can be greatly enhanced when agents effectively exchange knowledge with their peers. In particular, recent work studying agents that learn to teach other teammates has demonstrated that action advising accelerates team-wide learning. However, the prior work has simplified the learning of advising policies by using simple function approximations and only considered advising with…
▽ More
Collective learning can be greatly enhanced when agents effectively exchange knowledge with their peers. In particular, recent work studying agents that learn to teach other teammates has demonstrated that action advising accelerates team-wide learning. However, the prior work has simplified the learning of advising policies by using simple function approximations and only considered advising with primitive (low-level) actions, limiting the scalability of learning and teaching to complex domains. This paper introduces a novel learning-to-teach framework, called hierarchical multiagent teaching (HMAT), that improves scalability to complex environments by using the deep representation for student policies and by advising with more expressive extended action sequences over multiple levels of temporal abstraction. Our empirical evaluations demonstrate that HMAT improves team-wide learning progress in large, complex domains where previous approaches fail. HMAT also learns teaching policies that can effectively transfer knowledge to different teammates with knowledge of different tasks, even when the teammates have heterogeneous action spaces.
△ Less
Submitted 18 May, 2020; v1 submitted 7 March, 2019;
originally announced March 2019.
-
The Hanabi Challenge: A New Frontier for AI Research
Authors:
Nolan Bard,
Jakob N. Foerster,
Sarath Chandar,
Neil Burch,
Marc Lanctot,
H. Francis Song,
Emilio Parisotto,
Vincent Dumoulin,
Subhodeep Moitra,
Edward Hughes,
Iain Dunning,
Shibl Mourad,
Hugo Larochelle,
Marc G. Bellemare,
Michael Bowling
Abstract:
From the early days of computing, games have been important testbeds for studying how well machines can do sophisticated decision making. In recent years, machine learning has made dramatic advances with artificial agents reaching superhuman performance in challenge domains like Go, Atari, and some variants of poker. As with their predecessors of chess, checkers, and backgammon, these game domains…
▽ More
From the early days of computing, games have been important testbeds for studying how well machines can do sophisticated decision making. In recent years, machine learning has made dramatic advances with artificial agents reaching superhuman performance in challenge domains like Go, Atari, and some variants of poker. As with their predecessors of chess, checkers, and backgammon, these game domains have driven research by providing sophisticated yet well-defined challenges for artificial intelligence practitioners. We continue this tradition by proposing the game of Hanabi as a new challenge domain with novel problems that arise from its combination of purely cooperative gameplay with two to five players and imperfect information. In particular, we argue that Hanabi elevates reasoning about the beliefs and intentions of other agents to the foreground. We believe developing novel techniques for such theory of mind reasoning will not only be crucial for success in Hanabi, but also in broader collaborative efforts, especially those with human partners. To facilitate future research, we introduce the open-source Hanabi Learning Environment, propose an experimental framework for the research community to evaluate algorithmic advances, and assess the performance of current state-of-the-art techniques.
△ Less
Submitted 6 December, 2019; v1 submitted 1 February, 2019;
originally announced February 2019.
-
The Barbados 2018 List of Open Issues in Continual Learning
Authors:
Tom Schaul,
Hado van Hasselt,
Joseph Modayil,
Martha White,
Adam White,
Pierre-Luc Bacon,
Jean Harb,
Shibl Mourad,
Marc Bellemare,
Doina Precup
Abstract:
We want to make progress toward artificial general intelligence, namely general-purpose agents that autonomously learn how to competently act in complex environments. The purpose of this report is to sketch a research outline, share some of the most important open issues we are facing, and stimulate further discussion in the community. The content is based on some of our discussions during a week-…
▽ More
We want to make progress toward artificial general intelligence, namely general-purpose agents that autonomously learn how to competently act in complex environments. The purpose of this report is to sketch a research outline, share some of the most important open issues we are facing, and stimulate further discussion in the community. The content is based on some of our discussions during a week-long workshop held in Barbados in February 2018.
△ Less
Submitted 16 November, 2018;
originally announced November 2018.
