-
StarCraft II: A New Challenge for Reinforcement Learning
Authors:
Oriol Vinyals,
Timo Ewalds,
Sergey Bartunov,
Petko Georgiev,
Alexander Sasha Vezhnevets,
Michelle Yeo,
Alireza Makhzani,
Heinrich Küttler,
John Agapiou,
Julian Schrittwieser,
John Quan,
Stephen Gaffney,
Stig Petersen,
Karen Simonyan,
Tom Schaul,
Hado van Hasselt,
David Silver,
Timothy Lillicrap,
Kevin Calderone,
Paul Keet,
Anthony Brunasso,
David Lawrence,
Anders Ekermo,
Jacob Repp,
Rodney Tsing
Abstract:
This paper introduces SC2LE (StarCraft II Learning Environment), a reinforcement learning environment based on the StarCraft II game. This domain poses a new grand challenge for reinforcement learning, representing a more difficult class of problems than considered in most prior work. It is a multi-agent problem with multiple players interacting; there is imperfect information due to a partially o…
▽ More
This paper introduces SC2LE (StarCraft II Learning Environment), a reinforcement learning environment based on the StarCraft II game. This domain poses a new grand challenge for reinforcement learning, representing a more difficult class of problems than considered in most prior work. It is a multi-agent problem with multiple players interacting; there is imperfect information due to a partially observed map; it has a large action space involving the selection and control of hundreds of units; it has a large state space that must be observed solely from raw input feature planes; and it has delayed credit assignment requiring long-term strategies over thousands of steps. We describe the observation, action, and reward specification for the StarCraft II domain and provide an open source Python-based interface for communicating with the game engine. In addition to the main game maps, we provide a suite of mini-games focusing on different elements of StarCraft II gameplay. For the main game maps, we also provide an accompanying dataset of game replay data from human expert players. We give initial baseline results for neural networks trained from this data to predict game outcomes and player actions. Finally, we present initial baseline results for canonical deep reinforcement learning agents applied to the StarCraft II domain. On the mini-games, these agents learn to achieve a level of play that is comparable to a novice player. However, when trained on the main game, these agents are unable to make significant progress. Thus, SC2LE offers a new and challenging environment for exploring deep reinforcement learning algorithms and architectures.
△ Less
Submitted 16 August, 2017;
originally announced August 2017.
-
DeepMind Lab
Authors:
Charles Beattie,
Joel Z. Leibo,
Denis Teplyashin,
Tom Ward,
Marcus Wainwright,
Heinrich Küttler,
Andrew Lefrancq,
Simon Green,
Víctor Valdés,
Amir Sadik,
Julian Schrittwieser,
Keith Anderson,
Sarah York,
Max Cant,
Adam Cain,
Adrian Bolton,
Stephen Gaffney,
Helen King,
Demis Hassabis,
Shane Legg,
Stig Petersen
Abstract:
DeepMind Lab is a first-person 3D game platform designed for research and development of general artificial intelligence and machine learning systems. DeepMind Lab can be used to study how autonomous artificial agents may learn complex tasks in large, partially observed, and visually diverse worlds. DeepMind Lab has a simple and flexible API enabling creative task-designs and novel AI-designs to b…
▽ More
DeepMind Lab is a first-person 3D game platform designed for research and development of general artificial intelligence and machine learning systems. DeepMind Lab can be used to study how autonomous artificial agents may learn complex tasks in large, partially observed, and visually diverse worlds. DeepMind Lab has a simple and flexible API enabling creative task-designs and novel AI-designs to be explored and quickly iterated upon. It is powered by a fast and widely recognised game engine, and tailored for effective use by the research community.
△ Less
Submitted 13 December, 2016; v1 submitted 12 December, 2016;
originally announced December 2016.
-
Probabilistic models for joint clustering and time-warping of multidimensional curves
Authors:
Darya Chudova,
Scott Gaffney,
Padhraic Smyth
Abstract:
In this paper we present a family of algorithms that can simultaneously align and cluster sets of multidimensional curves measured on a discrete time grid. Our approach is based on a generative mixture model that allows non-linear time warping of the observed curves relative to the mean curves within the clusters. We also allow for arbitrary discrete-valued translation of the time…
▽ More
In this paper we present a family of algorithms that can simultaneously align and cluster sets of multidimensional curves measured on a discrete time grid. Our approach is based on a generative mixture model that allows non-linear time warping of the observed curves relative to the mean curves within the clusters. We also allow for arbitrary discrete-valued translation of the time axis, random real-valued offsets of the measured curves, and additive measurement noise. The resulting model can be viewed as a dynamic Bayesian network with a special transition structure that allows effective inference and learning. The Expectation-Maximization (EM) algorithm can be used to simultaneously recover both the curve models for each cluster, and the most likely time warping, translation, offset, and cluster membership for each curve. We demonstrate how Bayesian estimation methods improve the results for smaller sample sizes by enforcing smoothness in the cluster mean curves. We evaluate the methodology on two real-world data sets, and show that the DBN models provide systematic improvements in predictive power over competing approaches.
△ Less
Submitted 19 October, 2012;
originally announced December 2012.
