-
Towards Responsible Development of Generative AI for Education: An Evaluation-Driven Approach
Authors:
Irina Jurenka,
Markus Kunesch,
Kevin R. McKee,
Daniel Gillick,
Shaojian Zhu,
Sara Wiltberger,
Shubham Milind Phal,
Katherine Hermann,
Daniel Kasenberg,
Avishkar Bhoopchand,
Ankit Anand,
Miruna Pîslar,
Stephanie Chan,
Lisa Wang,
Jennifer She,
Parsa Mahmoudieh,
Aliya Rysbek,
Wei-Jen Ko,
Andrea Huber,
Brett Wiltshire,
Gal Elidan,
Roni Rabin,
Jasmin Rubinovitz,
Amit Pitaru,
Mac McAllister
, et al. (49 additional authors not shown)
Abstract:
A major challenge facing the world is the provision of equitable and universal access to quality education. Recent advances in generative AI (gen AI) have created excitement about the potential of new technologies to offer a personal tutor for every learner and a teaching assistant for every teacher. The full extent of this dream, however, has not yet materialised. We argue that this is primarily…
▽ More
A major challenge facing the world is the provision of equitable and universal access to quality education. Recent advances in generative AI (gen AI) have created excitement about the potential of new technologies to offer a personal tutor for every learner and a teaching assistant for every teacher. The full extent of this dream, however, has not yet materialised. We argue that this is primarily due to the difficulties with verbalising pedagogical intuitions into gen AI prompts and the lack of good evaluation practices, reinforced by the challenges in defining excellent pedagogy. Here we present our work collaborating with learners and educators to translate high level principles from learning science into a pragmatic set of seven diverse educational benchmarks, spanning quantitative, qualitative, automatic and human evaluations; and to develop a new set of fine-tuning datasets to improve the pedagogical capabilities of Gemini, introducing LearnLM-Tutor. Our evaluations show that LearnLM-Tutor is consistently preferred over a prompt tuned Gemini by educators and learners on a number of pedagogical dimensions. We hope that this work can serve as a first step towards developing a comprehensive educational evaluation framework, and that this can enable rapid progress within the AI and EdTech communities towards maximising the positive impact of gen AI in education.
△ Less
Submitted 21 May, 2024;
originally announced July 2024.
-
Using deep reinforcement learning to promote sustainable human behaviour on a common pool resource problem
Authors:
Raphael Koster,
Miruna Pîslar,
Andrea Tacchetti,
Jan Balaguer,
Leqi Liu,
Romuald Elie,
Oliver P. Hauser,
Karl Tuyls,
Matt Botvinick,
Christopher Summerfield
Abstract:
A canonical social dilemma arises when finite resources are allocated to a group of people, who can choose to either reciprocate with interest, or keep the proceeds for themselves. What resource allocation mechanisms will encourage levels of reciprocation that sustain the commons? Here, in an iterated multiplayer trust game, we use deep reinforcement learning (RL) to design an allocation mechanism…
▽ More
A canonical social dilemma arises when finite resources are allocated to a group of people, who can choose to either reciprocate with interest, or keep the proceeds for themselves. What resource allocation mechanisms will encourage levels of reciprocation that sustain the commons? Here, in an iterated multiplayer trust game, we use deep reinforcement learning (RL) to design an allocation mechanism that endogenously promotes sustainable contributions from human participants to a common pool resource. We first trained neural networks to behave like human players, creating a stimulated economy that allowed us to study how different mechanisms influenced the dynamics of receipt and reciprocation. We then used RL to train a social planner to maximise aggregate return to players. The social planner discovered a redistributive policy that led to a large surplus and an inclusive economy, in which players made roughly equal gains. The RL agent increased human surplus over baseline mechanisms based on unrestricted welfare or conditional cooperation, by conditioning its generosity on available resources and temporarily sanctioning defectors by allocating fewer resources to them. Examining the AI policy allowed us to develop an explainable mechanism that performed similarly and was more popular among players. Deep reinforcement learning can be used to discover mechanisms that promote sustainable human behaviour.
△ Less
Submitted 23 April, 2024;
originally announced April 2024.
-
BYOL-Explore: Exploration by Bootstrapped Prediction
Authors:
Zhaohan Daniel Guo,
Shantanu Thakoor,
Miruna Pîslar,
Bernardo Avila Pires,
Florent Altché,
Corentin Tallec,
Alaa Saade,
Daniele Calandriello,
Jean-Bastien Grill,
Yunhao Tang,
Michal Valko,
Rémi Munos,
Mohammad Gheshlaghi Azar,
Bilal Piot
Abstract:
We present BYOL-Explore, a conceptually simple yet general approach for curiosity-driven exploration in visually-complex environments. BYOL-Explore learns a world representation, the world dynamics, and an exploration policy all-together by optimizing a single prediction loss in the latent space with no additional auxiliary objective. We show that BYOL-Explore is effective in DM-HARD-8, a challeng…
▽ More
We present BYOL-Explore, a conceptually simple yet general approach for curiosity-driven exploration in visually-complex environments. BYOL-Explore learns a world representation, the world dynamics, and an exploration policy all-together by optimizing a single prediction loss in the latent space with no additional auxiliary objective. We show that BYOL-Explore is effective in DM-HARD-8, a challenging partially-observable continuous-action hard-exploration benchmark with visually-rich 3-D environments. On this benchmark, we solve the majority of the tasks purely through augmenting the extrinsic reward with BYOL-Explore s intrinsic reward, whereas prior work could only get off the ground with human demonstrations. As further evidence of the generality of BYOL-Explore, we show that it achieves superhuman performance on the ten hardest exploration games in Atari while having a much simpler design than other competitive agents.
△ Less
Submitted 16 June, 2022;
originally announced June 2022.
-
Learning Robust Real-Time Cultural Transmission without Human Data
Authors:
Cultural General Intelligence Team,
Avishkar Bhoopchand,
Bethanie Brownfield,
Adrian Collister,
Agustin Dal Lago,
Ashley Edwards,
Richard Everett,
Alexandre Frechette,
Yanko Gitahy Oliveira,
Edward Hughes,
Kory W. Mathewson,
Piermaria Mendolicchio,
Julia Pawar,
Miruna Pislar,
Alex Platonov,
Evan Senter,
Sukhdeep Singh,
Alexander Zacherl,
Lei M. Zhang
Abstract:
Cultural transmission is the domain-general social skill that allows agents to acquire and use information from each other in real-time with high fidelity and recall. In humans, it is the inheritance process that powers cumulative cultural evolution, expanding our skills, tools and knowledge across generations. We provide a method for generating zero-shot, high recall cultural transmission in arti…
▽ More
Cultural transmission is the domain-general social skill that allows agents to acquire and use information from each other in real-time with high fidelity and recall. In humans, it is the inheritance process that powers cumulative cultural evolution, expanding our skills, tools and knowledge across generations. We provide a method for generating zero-shot, high recall cultural transmission in artificially intelligent agents. Our agents succeed at real-time cultural transmission from humans in novel contexts without using any pre-collected human data. We identify a surprisingly simple set of ingredients sufficient for generating cultural transmission and develop an evaluation methodology for rigorously assessing it. This paves the way for cultural evolution as an algorithm for developing artificial general intelligence.
△ Less
Submitted 1 March, 2022;
originally announced March 2022.
-
When should agents explore?
Authors:
Miruna Pîslar,
David Szepesvari,
Georg Ostrovski,
Diana Borsa,
Tom Schaul
Abstract:
Exploration remains a central challenge for reinforcement learning (RL). Virtually all existing methods share the feature of a monolithic behaviour policy that changes only gradually (at best). In contrast, the exploratory behaviours of animals and humans exhibit a rich diversity, namely including forms of switching between modes. This paper presents an initial study of mode-switching, non-monolit…
▽ More
Exploration remains a central challenge for reinforcement learning (RL). Virtually all existing methods share the feature of a monolithic behaviour policy that changes only gradually (at best). In contrast, the exploratory behaviours of animals and humans exhibit a rich diversity, namely including forms of switching between modes. This paper presents an initial study of mode-switching, non-monolithic exploration for RL. We investigate different modes to switch between, at what timescales it makes sense to switch, and what signals make for good switching triggers. We also propose practical algorithmic components that make the switching mechanism adaptive and robust, which enables flexibility without an accompanying hyper-parameter-tuning burden. Finally, we report a promising and detailed analysis on Atari, using two-mode exploration and switching at sub-episodic time-scales.
△ Less
Submitted 4 March, 2022; v1 submitted 26 August, 2021;
originally announced August 2021.
-
Machine Translation Decoding beyond Beam Search
Authors:
Rémi Leblond,
Jean-Baptiste Alayrac,
Laurent Sifre,
Miruna Pislar,
Jean-Baptiste Lespiau,
Ioannis Antonoglou,
Karen Simonyan,
Oriol Vinyals
Abstract:
Beam search is the go-to method for decoding auto-regressive machine translation models. While it yields consistent improvements in terms of BLEU, it is only concerned with finding outputs with high model likelihood, and is thus agnostic to whatever end metric or score practitioners care about. Our aim is to establish whether beam search can be replaced by a more powerful metric-driven search tech…
▽ More
Beam search is the go-to method for decoding auto-regressive machine translation models. While it yields consistent improvements in terms of BLEU, it is only concerned with finding outputs with high model likelihood, and is thus agnostic to whatever end metric or score practitioners care about. Our aim is to establish whether beam search can be replaced by a more powerful metric-driven search technique. To this end, we explore numerous decoding algorithms, including some which rely on a value function parameterised by a neural network, and report results on a variety of metrics. Notably, we introduce a Monte-Carlo Tree Search (MCTS) based method and showcase its competitiveness. We provide a blueprint for how to use MCTS fruitfully in language applications, which opens promising future directions. We find that which algorithm is best heavily depends on the characteristics of the goal metric; we believe that our extensive experiments and analysis will inform further research in this area.
△ Less
Submitted 12 April, 2021;
originally announced April 2021.
-
Seeing Both the Forest and the Trees: Multi-head Attention for Joint Classification on Different Compositional Levels
Authors:
Miruna Pislar,
Marek Rei
Abstract:
In natural languages, words are used in association to construct sentences. It is not words in isolation, but the appropriate combination of hierarchical structures that conveys the meaning of the whole sentence. Neural networks can capture expressive language features; however, insights into the link between words and sentences are difficult to acquire automatically. In this work, we design a dee…
▽ More
In natural languages, words are used in association to construct sentences. It is not words in isolation, but the appropriate combination of hierarchical structures that conveys the meaning of the whole sentence. Neural networks can capture expressive language features; however, insights into the link between words and sentences are difficult to acquire automatically. In this work, we design a deep neural network architecture that explicitly wires lower and higher linguistic components; we then evaluate its ability to perform the same task at different hierarchical levels. Settling on broad text classification tasks, we show that our model, MHAL, learns to simultaneously solve them at different levels of granularity by fluidly transferring knowledge between hierarchies. Using a multi-head attention mechanism to tie the representations between single words and full sentences, MHAL systematically outperforms equivalent models that are not incentivized towards developing compositional representations. Moreover, we demonstrate that, with the proposed architecture, the sentence information flows naturally to individual words, allowing the model to behave like a sequence labeller (which is a lower, word-level task) even without any word supervision, in a zero-shot fashion.
△ Less
Submitted 1 November, 2020;
originally announced November 2020.
