Showing 1–50 of 194 results for author: Hughes, E

Regularizing infrared divergences in de Sitter spacetime

Authors: Javier Huenupi, Ellie Hughes, Gonzalo A. Palma, Spyros Sypsas

Abstract: Correlation functions of light scalars in de Sitter space, computed in standard perturbation theory, are hindered by time-dependent infrared divergences in the form of powers of $\ln a(t)$, where $a(t)$ is the scale factor describing the expansion of space. It has often been pointed out that loop corrections to these correlation functions make their divergence even stronger. In this note, we argue… ▽ More Correlation functions of light scalars in de Sitter space, computed in standard perturbation theory, are hindered by time-dependent infrared divergences in the form of powers of $\ln a(t)$, where $a(t)$ is the scale factor describing the expansion of space. It has often been pointed out that loop corrections to these correlation functions make their divergence even stronger. In this note, we argue that this is not the case: Loop corrections can be treated systematically with standard perturbative techniques (such as dimensional regularization) without necessarily introducing new $\ln a(t)$ dependencies. To be concrete, we focus on correlation functions represented by diagrams with a single vertex and an arbitrary number of loops. In this case, divergences from loops can be removed systematically with counterterms order by order, and one finds that observable loop-corrected correlation functions are indistinguishable from their tree-level form. By adopting a Wilsonian perspective, we further point out that our results favor the use of physical cutoffs (as opposed to comoving cutoffs) to regularize infrared divergences in general diagrams with an arbitrary number of loops and vertices. △ Less

Submitted 11 June, 2024; originally announced June 2024.

Comments: 11 pages + references. Comments welcome

Open-Endedness is Essential for Artificial Superhuman Intelligence

Authors: Edward Hughes, Michael Dennis, Jack Parker-Holder, Feryal Behbahani, Aditi Mavalankar, Yuge Shi, Tom Schaul, Tim Rocktaschel

Abstract: In recent years there has been a tremendous surge in the general capabilities of AI systems, mainly fuelled by training foundation models on internetscale data. Nevertheless, the creation of openended, ever self-improving AI remains elusive. In this position paper, we argue that the ingredients are now in place to achieve openendedness in AI systems with respect to a human observer. Furthermore, w… ▽ More In recent years there has been a tremendous surge in the general capabilities of AI systems, mainly fuelled by training foundation models on internetscale data. Nevertheless, the creation of openended, ever self-improving AI remains elusive. In this position paper, we argue that the ingredients are now in place to achieve openendedness in AI systems with respect to a human observer. Furthermore, we claim that such open-endedness is an essential property of any artificial superhuman intelligence (ASI). We begin by providing a concrete formal definition of open-endedness through the lens of novelty and learnability. We then illustrate a path towards ASI via open-ended systems built on top of foundation models, capable of making novel, humanrelevant discoveries. We conclude by examining the safety implications of generally-capable openended AI. We expect that open-ended foundation models will prove to be an increasingly fertile and safety-critical area of research in the near future. △ Less

Submitted 6 June, 2024; originally announced June 2024.

Artificial Generational Intelligence: Cultural Accumulation in Reinforcement Learning

Authors: Jonathan Cook, Chris Lu, Edward Hughes, Joel Z. Leibo, Jakob Foerster

Abstract: Cultural accumulation drives the open-ended and diverse progress in capabilities spanning human history. It builds an expanding body of knowledge and skills by combining individual exploration with inter-generational information transmission. Despite its widespread success among humans, the capacity for artificial learning agents to accumulate culture remains under-explored. In particular, approac… ▽ More Cultural accumulation drives the open-ended and diverse progress in capabilities spanning human history. It builds an expanding body of knowledge and skills by combining individual exploration with inter-generational information transmission. Despite its widespread success among humans, the capacity for artificial learning agents to accumulate culture remains under-explored. In particular, approaches to reinforcement learning typically strive for improvements over only a single lifetime. Generational algorithms that do exist fail to capture the open-ended, emergent nature of cultural accumulation, which allows individuals to trade-off innovation and imitation. Building on the previously demonstrated ability for reinforcement learning agents to perform social learning, we find that training setups which balance this with independent learning give rise to cultural accumulation. These accumulating agents outperform those trained for a single lifetime with the same cumulative experience. We explore this accumulation by constructing two models under two distinct notions of a generation: episodic generations, in which accumulation occurs via in-context learning and train-time generations, in which accumulation occurs via in-weights learning. In-context and in-weights cultural accumulation can be interpreted as analogous to knowledge and skill accumulation, respectively. To the best of our knowledge, this work is the first to present general models that achieve emergent cultural accumulation in reinforcement learning, opening up new avenues towards more open-ended learning systems, as well as presenting new opportunities for modelling human culture. △ Less

Submitted 1 June, 2024; originally announced June 2024.

Isotopic evidence of long-lived volcanism on Io

Authors: Katherine de Kleer, Ery C. Hughes, Francis Nimmo, John Eiler, Amy E. Hofmann, Statia Luszcz-Cook, Kathy Mandt

Abstract: Jupiter's moon Io hosts extensive volcanism driven by tidal heating. The isotopic composition of Io's inventory of volatile elements, including sulfur and chlorine, reflects its outgassing and mass loss history and provides an avenue for exploring its evolution. We used millimeter observations of Io's atmosphere to measure sulfur isotopes in gaseous SO2 and SO, and chlorine isotopes in gaseous NaC… ▽ More Jupiter's moon Io hosts extensive volcanism driven by tidal heating. The isotopic composition of Io's inventory of volatile elements, including sulfur and chlorine, reflects its outgassing and mass loss history and provides an avenue for exploring its evolution. We used millimeter observations of Io's atmosphere to measure sulfur isotopes in gaseous SO2 and SO, and chlorine isotopes in gaseous NaCl and KCl. We find $^{34}$S/$^{32}$S=0.0595$\pm$0.0038 ($δ^{34}$S=+347$\pm$86 per mille), which is highly enriched compared to average Solar System values and indicates that Io has lost 94 to 99% of its available sulfur. Our measurement of $^{37}$Cl/$^{35}$Cl=0.403$\pm$0.028 ($δ^{37}$Cl=+263$\pm$88 per mille) shows chlorine is similarly enriched. These measurements indicate that Io has been volcanically active for most or all of its history, with potentially higher outgassing and mass-loss rates at earlier times. △ Less

Submitted 28 May, 2024; originally announced May 2024.

Comments: This is the author's version of the work. It is posted here by permission of the AAAS for personal use, not for redistribution. The definitive version was published in Science on May 10, 2024, DOI: 10.1126/science.adj0625

Journal ref: Science, Volume 385, Issue 6696, pp. 682-687 (2024)

The Ethics of Advanced AI Assistants

Authors: Iason Gabriel, Arianna Manzini, Geoff Keeling, Lisa Anne Hendricks, Verena Rieser, Hasan Iqbal, Nenad Tomašev, Ira Ktena, Zachary Kenton, Mikel Rodriguez, Seliem El-Sayed, Sasha Brown, Canfer Akbulut, Andrew Trask, Edward Hughes, A. Stevie Bergman, Renee Shelby, Nahema Marchal, Conor Griffin, Juan Mateos-Garcia, Laura Weidinger, Winnie Street, Benjamin Lange, Alex Ingerman, Alison Lentz , et al. (32 additional authors not shown)

Abstract: This paper focuses on the opportunities and the ethical and societal risks posed by advanced AI assistants. We define advanced AI assistants as artificial agents with natural language interfaces, whose function is to plan and execute sequences of actions on behalf of a user, across one or more domains, in line with the user's expectations. The paper starts by considering the technology itself, pro… ▽ More This paper focuses on the opportunities and the ethical and societal risks posed by advanced AI assistants. We define advanced AI assistants as artificial agents with natural language interfaces, whose function is to plan and execute sequences of actions on behalf of a user, across one or more domains, in line with the user's expectations. The paper starts by considering the technology itself, providing an overview of AI assistants, their technical foundations and potential range of applications. It then explores questions around AI value alignment, well-being, safety and malicious uses. Extending the circle of inquiry further, we next consider the relationship between advanced AI assistants and individual users in more detail, exploring topics such as manipulation and persuasion, anthropomorphism, appropriate relationships, trust and privacy. With this analysis in place, we consider the deployment of advanced assistants at a societal scale, focusing on cooperation, equity and access, misinformation, economic impact, the environment and how best to evaluate advanced AI assistants. Finally, we conclude by providing a range of recommendations for researchers, developers, policymakers and public stakeholders. △ Less

Submitted 28 April, 2024; v1 submitted 24 April, 2024; originally announced April 2024.

Viblio: Introducing Credibility Signals and Citations to Video-Sharing Platforms

Authors: Emelia Hughes, Renee Wang, Prerna Juneja, Tony Li, Tanu Mitra, Amy Zhang

Abstract: As more users turn to video-sharing platforms like YouTube as an information source, they may consume misinformation despite their best efforts. In this work, we investigate ways that users can better assess the credibility of videos by first exploring how users currently determine credibility using existing signals on platforms and then by introducing and evaluating new credibility-based signals.… ▽ More As more users turn to video-sharing platforms like YouTube as an information source, they may consume misinformation despite their best efforts. In this work, we investigate ways that users can better assess the credibility of videos by first exploring how users currently determine credibility using existing signals on platforms and then by introducing and evaluating new credibility-based signals. We conducted 12 contextual inquiry interviews with YouTube users, determining that participants used a combination of existing signals, such as the channel name, the production quality, and prior knowledge, to evaluate credibility, yet sometimes stumbled in their efforts to do so. We then developed Viblio, a prototype system that enables YouTube users to view and add citations and related information while watching a video based on our participants' needs. From an evaluation with 12 people, all participants found Viblio to be intuitive and useful in the process of evaluating a video's credibility and could see themselves using Viblio in the future. △ Less

Submitted 27 February, 2024; originally announced February 2024.

Genie: Generative Interactive Environments

Authors: Jake Bruce, Michael Dennis, Ashley Edwards, Jack Parker-Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, Yusuf Aytar, Sarah Bechtle, Feryal Behbahani, Stephanie Chan, Nicolas Heess, Lucy Gonzalez, Simon Osindero, Sherjil Ozair, Scott Reed, Jingwei Zhang, Konrad Zolna, Jeff Clune, Nando de Freitas, Satinder Singh, Tim Rocktäschel

Abstract: We introduce Genie, the first generative interactive environment trained in an unsupervised manner from unlabelled Internet videos. The model can be prompted to generate an endless variety of action-controllable virtual worlds described through text, synthetic images, photographs, and even sketches. At 11B parameters, Genie can be considered a foundation world model. It is comprised of a spatiotem… ▽ More We introduce Genie, the first generative interactive environment trained in an unsupervised manner from unlabelled Internet videos. The model can be prompted to generate an endless variety of action-controllable virtual worlds described through text, synthetic images, photographs, and even sketches. At 11B parameters, Genie can be considered a foundation world model. It is comprised of a spatiotemporal video tokenizer, an autoregressive dynamics model, and a simple and scalable latent action model. Genie enables users to act in the generated environments on a frame-by-frame basis despite training without any ground-truth action labels or other domain-specific requirements typically found in the world model literature. Further the resulting learned latent action space facilitates training agents to imitate behaviors from unseen videos, opening the path for training generalist agents of the future. △ Less

Submitted 23 February, 2024; originally announced February 2024.

Comments: https://sites.google.com/corp/view/genie-2024/

Cool dark sector, concordance, and a low $σ_8$

Authors: Ellie Hughes, Fei Ge, Francis-Yan Cyr-Racine, Lloyd Knox, Srinivasan Raghunathan

Abstract: We investigate a cosmological model in which a fraction of the dark matter is atomic dark matter (ADM). This ADM consists of dark versions of the electron and of the proton, interacting with each other and with dark photons just as their light sector versions do, but interacting with everything else only gravitationally. We find constraints given current cosmic microwave background (CMB) and baryo… ▽ More We investigate a cosmological model in which a fraction of the dark matter is atomic dark matter (ADM). This ADM consists of dark versions of the electron and of the proton, interacting with each other and with dark photons just as their light sector versions do, but interacting with everything else only gravitationally. We find constraints given current cosmic microwave background (CMB) and baryon acoustic oscillation (BAO) data, with and without an $H_0$ prior, and with and without enforcing a big bang nucleosynthesis consistent helium abundance. We find that, at low dark photon temperature, one can have consistency with BAO and CMB data, with a fraction of dark matter that is ADM ($f_{\rm adm}$) as large as $\sim 0.1$. Such a large $f_{\rm adm}$ leads to a suppression of density fluctuations today on scales below about 60 Mpc that may be of relevance to the $σ_8$ tension. Our work motivates calculation of nonlinear corrections to matter power spectrum predictions in the ADM model. We forecast parameter constraints to come from future ground-based CMB surveys, and find that if ADM is indeed the cause of the $σ_8$ tension, the influence of the ADM, primarily on CMB lensing, will likely be detectable at high significance. △ Less

Submitted 10 May, 2024; v1 submitted 9 November, 2023; originally announced November 2023.

Comments: 16 pages + references, 9 figures. Published in PRD

Journal ref: Phys. Rev. D 109, 103516 (2024)

Towards a Pretrained Model for Restless Bandits via Multi-arm Generalization

Authors: Yunfan Zhao, Nikhil Behari, Edward Hughes, Edwin Zhang, Dheeraj Nagaraj, Karl Tuyls, Aparna Taneja, Milind Tambe

Abstract: Restless multi-arm bandits (RMABs), a class of resource allocation problems with broad application in areas such as healthcare, online advertising, and anti-poaching, have recently been studied from a multi-agent reinforcement learning perspective. Prior RMAB research suffers from several limitations, e.g., it fails to adequately address continuous states, and requires retraining from scratch when… ▽ More Restless multi-arm bandits (RMABs), a class of resource allocation problems with broad application in areas such as healthcare, online advertising, and anti-poaching, have recently been studied from a multi-agent reinforcement learning perspective. Prior RMAB research suffers from several limitations, e.g., it fails to adequately address continuous states, and requires retraining from scratch when arms opt-in and opt-out over time, a common challenge in many real world applications. We address these limitations by developing a neural network-based pre-trained model (PreFeRMAB) that has general zero-shot ability on a wide range of previously unseen RMABs, and which can be fine-tuned on specific instances in a more sample-efficient way than retraining from scratch. Our model also accommodates general multi-action settings and discrete or continuous state spaces. To enable fast generalization, we learn a novel single policy network model that utilizes feature information and employs a training procedure in which arms opt-in and out over time. We derive a new update rule for a crucial $λ$-network with theoretical convergence guarantees and empirically demonstrate the advantages of our approach on several challenging, real-world inspired problems. △ Less

Submitted 29 January, 2024; v1 submitted 22 October, 2023; originally announced October 2023.

Partial differential equation models for invasive species spread in the presence of spatial heterogeneity

Authors: Elliott Hughes, Miguel Moyers-Gonzalez, Rua Murray, Phillip L. Wilson

Abstract: Models of invasive species spread often assume that landscapes are spatially homogeneous; thus simplifying analysis but potentially reducing accuracy. We extend a recently developed partial differential equation model for invasive conifer spread to account for spatial heterogeneity in parameter values and introduce a method to obtain key outputs (e.g. spread rates) from computational simulations.… ▽ More Models of invasive species spread often assume that landscapes are spatially homogeneous; thus simplifying analysis but potentially reducing accuracy. We extend a recently developed partial differential equation model for invasive conifer spread to account for spatial heterogeneity in parameter values and introduce a method to obtain key outputs (e.g. spread rates) from computational simulations. Simulations produce patterns of spatial spread remarkably similar to observed patterns in grassland ecosystems invaded by exotic conifers, validating our spatially explicit strategy. We find that incorporating spatial variation in different parameters does not significantly affect the evolution of invasions (which are characterised by a long quiescent period followed by rapid evolution towards to a constant rate of invasion) but that distributional assumptions can have a significant impact on the spread rate of invasions. Our work demonstrates that spatial variation in site-suitability or other parameters can have a significant impact on invasions △ Less

Submitted 13 September, 2023; originally announced September 2023.

Comments: 13 pages, 18 figures

A Unified Transformer-based Network for multimodal Emotion Recognition

Authors: Kamran Ali, Charles E. Hughes

Abstract: The development of transformer-based models has resulted in significant advances in addressing various vision and NLP-based research challenges. However, the progress made in transformer-based methods has not been effectively applied to biosensing research. This paper presents a novel Unified Biosensor-Vision Multi-modal Transformer-based (UBVMT) method to classify emotions in an arousal-valence s… ▽ More The development of transformer-based models has resulted in significant advances in addressing various vision and NLP-based research challenges. However, the progress made in transformer-based methods has not been effectively applied to biosensing research. This paper presents a novel Unified Biosensor-Vision Multi-modal Transformer-based (UBVMT) method to classify emotions in an arousal-valence space by combining a 2D representation of an ECG/PPG signal with the face information. To achieve this goal, we first investigate and compare the unimodal emotion recognition performance of three image-based representations of the ECG/PPG signal. We then present our UBVMT network which is trained to perform emotion recognition by combining the 2D image-based representation of the ECG/PPG signal and the facial expression features. Our unified transformer model consists of homogeneous transformer blocks that take as an input the 2D representation of the ECG/PPG signal and the corresponding face frame for emotion representation learning with minimal modality-specific design. Our UBVMT model is trained by reconstructing masked patches of video frames and 2D images of ECG/PPG signals, and contrastive modeling to align face and ECG/PPG data. Extensive experiments on the MAHNOB-HCI and DEAP datasets show that our Unified UBVMT-based model produces comparable results to the state-of-the-art techniques. △ Less

Submitted 27 August, 2023; originally announced August 2023.

Comments: 12 pages

A Mathematically Robust Model of Exotic Pine Invasions

Authors: Elliott Hughes, Miguel Moyers-Gonzalez, Rua Murray, Phillip L. Wilson

Abstract: Invasive pine trees pose a threat to biodiversity in a variety of Southern Hemisphere countries, but understanding of the dynamics of invasions and the factors that retard or accelerate spread is limited. Here, we consider the past models of wilding pine spread and develop a new model of pine invasion. We show that many prior models feature parameter estimates which are not biologically supported… ▽ More Invasive pine trees pose a threat to biodiversity in a variety of Southern Hemisphere countries, but understanding of the dynamics of invasions and the factors that retard or accelerate spread is limited. Here, we consider the past models of wilding pine spread and develop a new model of pine invasion. We show that many prior models feature parameter estimates which are not biologically supported and rely on a conjecture to obtain an asymptotic spread speed of invasive pine populations, the main output of these models. In contrast to prior approaches, we use partial differential equations to model an invasion. We show that invasions are almost static for a significant period of time before rapidly accelerating to spread at a constant rate, matching observed behaviour in at least some field sites. Our work suggests that prior methods for estimating invasion speeds may not accurately predict spread and are sensitive to assumptions about the distribution of parameters. However, we present alternative estimation methods and suggest directions for further research. △ Less

Submitted 2 August, 2023; originally announced August 2023.

Comments: 36 pages, 9 figures

Heterogeneous Social Value Orientation Leads to Meaningful Diversity in Sequential Social Dilemmas

Authors: Udari Madhushani, Kevin R. McKee, John P. Agapiou, Joel Z. Leibo, Richard Everett, Thomas Anthony, Edward Hughes, Karl Tuyls, Edgar A. Duéñez-Guzmán

Abstract: In social psychology, Social Value Orientation (SVO) describes an individual's propensity to allocate resources between themself and others. In reinforcement learning, SVO has been instantiated as an intrinsic motivation that remaps an agent's rewards based on particular target distributions of group reward. Prior studies show that groups of agents endowed with heterogeneous SVO learn diverse poli… ▽ More In social psychology, Social Value Orientation (SVO) describes an individual's propensity to allocate resources between themself and others. In reinforcement learning, SVO has been instantiated as an intrinsic motivation that remaps an agent's rewards based on particular target distributions of group reward. Prior studies show that groups of agents endowed with heterogeneous SVO learn diverse policies in settings that resemble the incentive structure of Prisoner's dilemma. Our work extends this body of results and demonstrates that (1) heterogeneous SVO leads to meaningfully diverse policies across a range of incentive structures in sequential social dilemmas, as measured by task-specific diversity metrics; and (2) learning a best response to such policy diversity leads to better zero-shot generalization in some situations. We show that these best-response agents learn policies that are conditioned on their co-players, which we posit is the reason for improved zero-shot generalization results. △ Less

Submitted 1 May, 2023; originally announced May 2023.

Three-dimensional integration enables ultra-low-noise, isolator-free Si photonics

Authors: Chao Xiang, Warren Jin, Osama Terra, Bozhang Dong, Heming Wang, Lue Wu, Joel Guo, Theodore J. Morin, Eamonn Hughes, Jonathan Peters, Qing-Xin Ji, Avi Feshali, Mario Paniccia, Kerry J. Vahala, John E. Bowers

Abstract: While photonic integrated circuits (PICs) are being widely used in applications such as telecommunications and datacenter interconnects, PICs capable of replacing bulk optics and fibers in high-precision, highly-coherent applications will require ultra-low-noise laser sources to be integrated with other photonic components in a compact and robustly aligned format -- that is, on a single chip. Such… ▽ More While photonic integrated circuits (PICs) are being widely used in applications such as telecommunications and datacenter interconnects, PICs capable of replacing bulk optics and fibers in high-precision, highly-coherent applications will require ultra-low-noise laser sources to be integrated with other photonic components in a compact and robustly aligned format -- that is, on a single chip. Such PICs could offer superior scalability for complex functionalities and volume production, as well as improved stability and reliability over time. However, there are two major issues preventing the realization of such envisioned PICs: the high phase noise of semiconductor lasers, and the difficulty of integrating optical isolators directly on chip. PICs are still considered as inferior solutions in optical systems such as microwave synthesizers, optical gyroscopes and atomic clocks, despite their advantages in size, weight, power consumption and cost (SWaPC). Here, we challenge this convention by introducing three-dimensional (3D) integration in silicon photonics that results in ultra-low-noise, isolator-free PICs. Through multiple monolithic and heterogeneous processing sequences, direct on-chip integration of III-V gain and ultra-low-loss (ULL) silicon nitride (SiN) waveguides with optical loss around 0.5 dB/m are demonstrated. Consequently, the demonstrated PIC enters a new regime, such that an integrated ultra-high-Q cavity reduces the laser noise close to that of fiber lasers. Moreover, the cavity acts as an effective block for any downstream on-chip or off-chip reflection-induced destabilization, thus eliminating the need for optical isolators. We further showcase isolator-free, widely-tunable, low-noise, heterodyne microwave generation using two ultra-low-noise lasers on the same silicon chip. △ Less

Submitted 19 January, 2023; originally announced January 2023.

Human-Timescale Adaptation in an Open-Ended Task Space

Authors: Adaptive Agent Team, Jakob Bauer, Kate Baumli, Satinder Baveja, Feryal Behbahani, Avishkar Bhoopchand, Nathalie Bradley-Schmieg, Michael Chang, Natalie Clay, Adrian Collister, Vibhavari Dasagi, Lucy Gonzalez, Karol Gregor, Edward Hughes, Sheleem Kashem, Maria Loks-Thompson, Hannah Openshaw, Jack Parker-Holder, Shreya Pathak, Nicolas Perez-Nieves, Nemanja Rakicevic, Tim Rocktäschel, Yannick Schroecker, Jakub Sygnowski, Karl Tuyls , et al. (3 additional authors not shown)

Abstract: Foundation models have shown impressive adaptation and scalability in supervised and self-supervised learning problems, but so far these successes have not fully translated to reinforcement learning (RL). In this work, we demonstrate that training an RL agent at scale leads to a general in-context learning algorithm that can adapt to open-ended novel embodied 3D problems as quickly as humans. In a… ▽ More Foundation models have shown impressive adaptation and scalability in supervised and self-supervised learning problems, but so far these successes have not fully translated to reinforcement learning (RL). In this work, we demonstrate that training an RL agent at scale leads to a general in-context learning algorithm that can adapt to open-ended novel embodied 3D problems as quickly as humans. In a vast space of held-out environment dynamics, our adaptive agent (AdA) displays on-the-fly hypothesis-driven exploration, efficient exploitation of acquired knowledge, and can successfully be prompted with first-person demonstrations. Adaptation emerges from three ingredients: (1) meta-reinforcement learning across a vast, smooth and diverse task distribution, (2) a policy parameterised as a large-scale attention-based memory architecture, and (3) an effective automated curriculum that prioritises tasks at the frontier of an agent's capabilities. We demonstrate characteristic scaling laws with respect to network size, memory length, and richness of the training task distribution. We believe our results lay the foundation for increasingly general and adaptive RL agents that perform well across ever-larger open-ended domains. △ Less

Submitted 18 January, 2023; originally announced January 2023.

Dislocation-induced structural and luminescence degradation in InAs quantum dot emitters on silicon

Authors: Eamonn T. Hughes, Gunnar Kusch, Jennifer Selvidge, Bastien Bonef, Justin Norman, Chen Shang, John E. Bowers, Rachel A. Oliver, Kunal Mukherjee

Abstract: We probe the extent to which dislocations reduce carrier lifetimes and alter luminescence and growth morphology in InAs quantum dots (QD) grown on silicon. These heterostructures are key ingredients to achieving a highly reliable monolithically integrated light source on silicon necessary for photonic integrated circuits. We find up to 20-30% shorter carrier lifetimes at spatially resolved individ… ▽ More We probe the extent to which dislocations reduce carrier lifetimes and alter luminescence and growth morphology in InAs quantum dots (QD) grown on silicon. These heterostructures are key ingredients to achieving a highly reliable monolithically integrated light source on silicon necessary for photonic integrated circuits. We find up to 20-30% shorter carrier lifetimes at spatially resolved individual dislocations from both the QD ground and excited states at room temperature using time-resolved cathodoluminescence spectroscopy. These lifetimes are consistent with differences in the intensity measured under steady-state excitation suggesting that trap-assisted recombination limits the minority carrier lifetime, even away from dislocations. Our techniques also reveal the dramatic growth of misfit dislocations in these structures under carrier injection fueled by recombination-enhanced dislocation glide and III-V/Si residual strain. Beyond these direct effects of increased nonradiative recombination, we find the long-range strain field of misfit dislocations deeper in the defect filter layers employed during III-V/Si growth alter the QD growth environment and introduce a crosshatch-like variation in the QD emission color and intensity when the filter layer is positioned close to the QD emitter layer. Sessile threading dislocations generate even more egregious hillock defects that also reduce emission intensities by altering layer thicknesses, as measured by transmission electron microscopy and atom probe tomography. Our work presents a more complete picture of the impacts of dislocations relevant for the development of light sources for scalable silicon photonic integrated circuits. △ Less

Submitted 9 January, 2023; originally announced January 2023.

Comments: 15 pages, 6 figures

Versatile strain relief pathways in epitaxial films of (001)-oriented PbSe on III-V substrates

Authors: Brian B. Haidet, Jarod Meyer, Pooja Reddy, Eamonn T. Hughes, Kunal Mukherjee

Abstract: PbSe and related IV-VI rocksalt-structure semiconductors have important electronic properties that may be controlled by epitaxial strain and interfaces, thus harnessed in an emerging class of IV-VI/III-V heterostructures. The synthesis of such heterostructures and understanding mechanisms for strain-relief is central to achieving this goal. We show that a range of interfacial defects mediate latti… ▽ More PbSe and related IV-VI rocksalt-structure semiconductors have important electronic properties that may be controlled by epitaxial strain and interfaces, thus harnessed in an emerging class of IV-VI/III-V heterostructures. The synthesis of such heterostructures and understanding mechanisms for strain-relief is central to achieving this goal. We show that a range of interfacial defects mediate lattice mismatch in (001)-oriented epitaxial thin films of PbSe with III-V templates of GaAs, InAs, and GaSb. While the primary slip system {100}<110> for dislocation glide in PbSe is well-studied for its facile glide properties, it is inactive in (001)-oriented films used in our work. Yet, we obtain nearly relaxed PbSe films in the three heteroepitaxial systems studied with interfaces ranging from incoherent without localized misfit dislocations on 8.3% mismatched GaAs, a mixture of semi-coherent and incoherent patches on 1.5% mismatched InAs, to nearly coherent on 0.8% mismatched GaSb. The semi-coherent portions of the interfaces to InAs form by 60° misfit dislocations gliding on higher order {111}<110> slip systems. On the more closely lattice-matched GaSb, arrays of 90° (edge) misfit dislocations form via a climb process. The diversity of strain-relaxation mechanisms accessible to PbSe makes it a rich system for heteroepitaxial integration with III-V substrates. △ Less

Submitted 11 October, 2022; originally announced October 2022.

Comments: 13 pages, 8 figures

Developing, Evaluating and Scaling Learning Agents in Multi-Agent Environments

Authors: Ian Gemp, Thomas Anthony, Yoram Bachrach, Avishkar Bhoopchand, Kalesha Bullard, Jerome Connor, Vibhavari Dasagi, Bart De Vylder, Edgar Duenez-Guzman, Romuald Elie, Richard Everett, Daniel Hennes, Edward Hughes, Mina Khan, Marc Lanctot, Kate Larson, Guy Lever, Siqi Liu, Luke Marris, Kevin R. McKee, Paul Muller, Julien Perolat, Florian Strub, Andrea Tacchetti, Eugene Tarassov , et al. (2 additional authors not shown)

Abstract: The Game Theory & Multi-Agent team at DeepMind studies several aspects of multi-agent learning ranging from computing approximations to fundamental concepts in game theory to simulating social dilemmas in rich spatial environments and training 3-d humanoids in difficult team coordination tasks. A signature aim of our group is to use the resources and expertise made available to us at DeepMind in d… ▽ More The Game Theory & Multi-Agent team at DeepMind studies several aspects of multi-agent learning ranging from computing approximations to fundamental concepts in game theory to simulating social dilemmas in rich spatial environments and training 3-d humanoids in difficult team coordination tasks. A signature aim of our group is to use the resources and expertise made available to us at DeepMind in deep reinforcement learning to explore multi-agent systems in complex environments and use these benchmarks to advance our understanding. Here, we summarise the recent work of our team and present a taxonomy that we feel highlights many important open challenges in multi-agent research. △ Less

Submitted 22 September, 2022; originally announced September 2022.

Comments: Published in AI Communications 2022

Electrically pumped quantum-dot lasers grown on 300 mm patterned Si photonic wafers

Authors: Chen Shang, Kaiyin Feng, Eamonn T. Hughes, Andrew Clark, Mukul Debnath, Rosalyn Koscica, Gerald Leake, Joshua Herman, David Harame, Peter Ludewig, Yating Wan, John E. Bowers

Abstract: Monolithic integration of quantum dot (QD) gain materials onto Si photonic platforms via direct epitaxial growth is a promising solution for on-chip light sources. Recent developments have demonstrated superior device reliability in blanket hetero-epitaxy of III-V devices on Si at elevated temperatures. Yet, thick, defect management epi designs prevent vertical light coupling from the gain region… ▽ More Monolithic integration of quantum dot (QD) gain materials onto Si photonic platforms via direct epitaxial growth is a promising solution for on-chip light sources. Recent developments have demonstrated superior device reliability in blanket hetero-epitaxy of III-V devices on Si at elevated temperatures. Yet, thick, defect management epi designs prevent vertical light coupling from the gain region to the Si-on-Insulator (SOI) waveguides. Here, we demonstrate the first electrically pumped QD lasers grown on a 300 mm patterned (001) Si wafer with a butt-coupled configuration by molecular beam epitaxy (MBE). Unique growth and fabrication challenges imposed by the template architecture have been resolved, contributing to continuous wave lasing to 60 °C and a maximum double-side output power of 126.6 mW at 20 °C with a double-side wall plug efficiency of 8.6%. The potential for robust on-chip laser operation and efficient low-loss light coupling to Si photonic circuits makes this heteroepitaxial integration platform on Si promising for scalable and low-cost mass production. △ Less

Submitted 2 June, 2022; originally announced June 2022.

Comments: 11 pages including references, 6 figures

Semi-supervised Drifted Stream Learning with Short Lookback

Authors: Weijieying Ren, Pengyang Wang, Xiaolin Li, Charles E. Hughes, Yanjie Fu

Abstract: In many scenarios, 1) data streams are generated in real time; 2) labeled data are expensive and only limited labels are available in the beginning; 3) real-world data is not always i.i.d. and data drift over time gradually; 4) the storage of historical streams is limited and model updating can only be achieved based on a very short lookback window. This learning setting limits the applicability a… ▽ More In many scenarios, 1) data streams are generated in real time; 2) labeled data are expensive and only limited labels are available in the beginning; 3) real-world data is not always i.i.d. and data drift over time gradually; 4) the storage of historical streams is limited and model updating can only be achieved based on a very short lookback window. This learning setting limits the applicability and availability of many Machine Learning (ML) algorithms. We generalize the learning task under such setting as a semi-supervised drifted stream learning with short lookback problem (SDSL). SDSL imposes two under-addressed challenges on existing methods in semi-supervised learning, continuous learning, and domain adaptation: 1) robust pseudo-labeling under gradual shifts and 2) anti-forgetting adaptation with short lookback. To tackle these challenges, we propose a principled and generic generation-replay framework to solve SDSL. The framework is able to accomplish: 1) robust pseudo-labeling in the generation step; 2) anti-forgetting adaption in the replay step. To achieve robust pseudo-labeling, we develop a novel pseudo-label classification model to leverage supervised knowledge of previously labeled data, unsupervised knowledge of new data, and, structure knowledge of invariant label semantics. To achieve adaptive anti-forgetting model replay, we propose to view the anti-forgetting adaptation task as a flat region search problem. We propose a novel minimax game-based replay objective function to solve the flat region search problem and develop an effective optimization solver. Finally, we present extensive experiments to demonstrate our framework can effectively address the task of anti-forgetting learning in drifted streams with short lookback. △ Less

Submitted 25 May, 2022; originally announced May 2022.

Comments: To appear in KDD 2022

Emergent Bartering Behaviour in Multi-Agent Reinforcement Learning

Authors: Michael Bradley Johanson, Edward Hughes, Finbarr Timbers, Joel Z. Leibo

Abstract: Advances in artificial intelligence often stem from the development of new environments that abstract real-world situations into a form where research can be done conveniently. This paper contributes such an environment based on ideas inspired by elementary Microeconomics. Agents learn to produce resources in a spatially complex world, trade them with one another, and consume those that they prefe… ▽ More Advances in artificial intelligence often stem from the development of new environments that abstract real-world situations into a form where research can be done conveniently. This paper contributes such an environment based on ideas inspired by elementary Microeconomics. Agents learn to produce resources in a spatially complex world, trade them with one another, and consume those that they prefer. We show that the emergent production, consumption, and pricing behaviors respond to environmental conditions in the directions predicted by supply and demand shifts in Microeconomics. We also demonstrate settings where the agents' emergent prices for goods vary over space, reflecting the local abundance of goods. After the price disparities emerge, some agents then discover a niche of transporting goods between regions with different prevailing prices -- a profitable strategy because they can buy goods where they are cheap and sell them where they are expensive. Finally, in a series of ablation experiments, we investigate how choices in the environmental rewards, bartering actions, agent architecture, and ability to consume tradable goods can either aid or inhibit the emergence of this economic behavior. This work is part of the environment development branch of a research program that aims to build human-like artificial general intelligence through multi-agent interactions in simulated societies. By exploring which environment features are needed for the basic phenomena of elementary microeconomics to emerge automatically from learning, we arrive at an environment that differs from those studied in prior multi-agent reinforcement learning work along several dimensions. For example, the model incorporates heterogeneous tastes and physical abilities, and agents negotiate with one another as a grounded form of communication. △ Less

Submitted 13 May, 2022; originally announced May 2022.

Reinforced Imitative Graph Learning for Mobile User Profiling

Authors: Dongjie Wang, Pengyang Wang, Yanjie Fu, Kunpeng Liu, Hui Xiong, Charles E. Hughes

Abstract: Mobile user profiling refers to the efforts of extracting users' characteristics from mobile activities. In order to capture the dynamic varying of user characteristics for generating effective user profiling, we propose an imitation-based mobile user profiling framework. Considering the objective of teaching an autonomous agent to imitate user mobility based on the user's profile, the user profil… ▽ More Mobile user profiling refers to the efforts of extracting users' characteristics from mobile activities. In order to capture the dynamic varying of user characteristics for generating effective user profiling, we propose an imitation-based mobile user profiling framework. Considering the objective of teaching an autonomous agent to imitate user mobility based on the user's profile, the user profile is the most accurate when the agent can perfectly mimic the user behavior patterns. The profiling framework is formulated into a reinforcement learning task, where an agent is a next-visit planner, an action is a POI that a user will visit next, and the state of the environment is a fused representation of a user and spatial entities. An event in which a user visits a POI will construct a new state, which helps the agent predict users' mobility more accurately. In the framework, we introduce a spatial Knowledge Graph (KG) to characterize the semantics of user visits over connected spatial entities. Additionally, we develop a mutual-updating strategy to quantify the state that evolves over time. Along these lines, we develop a reinforcement imitative graph learning framework for mobile user profiling. Finally, we conduct extensive experiments to demonstrate the superiority of our approach. △ Less

Submitted 12 March, 2022; originally announced March 2022.

Comments: TKDE Under Review

Learning Robust Real-Time Cultural Transmission without Human Data

Authors: Cultural General Intelligence Team, Avishkar Bhoopchand, Bethanie Brownfield, Adrian Collister, Agustin Dal Lago, Ashley Edwards, Richard Everett, Alexandre Frechette, Yanko Gitahy Oliveira, Edward Hughes, Kory W. Mathewson, Piermaria Mendolicchio, Julia Pawar, Miruna Pislar, Alex Platonov, Evan Senter, Sukhdeep Singh, Alexander Zacherl, Lei M. Zhang

Abstract: Cultural transmission is the domain-general social skill that allows agents to acquire and use information from each other in real-time with high fidelity and recall. In humans, it is the inheritance process that powers cumulative cultural evolution, expanding our skills, tools and knowledge across generations. We provide a method for generating zero-shot, high recall cultural transmission in arti… ▽ More Cultural transmission is the domain-general social skill that allows agents to acquire and use information from each other in real-time with high fidelity and recall. In humans, it is the inheritance process that powers cumulative cultural evolution, expanding our skills, tools and knowledge across generations. We provide a method for generating zero-shot, high recall cultural transmission in artificially intelligent agents. Our agents succeed at real-time cultural transmission from humans in novel contexts without using any pre-collected human data. We identify a surprisingly simple set of ingredients sufficient for generating cultural transmission and develop an evaluation methodology for rigorously assessing it. This paves the way for cultural evolution as an algorithm for developing artificial general intelligence. △ Less

Submitted 1 March, 2022; originally announced March 2022.

The physics governing the upper truncation mass of the globular cluster mass function

Authors: Meghan E. Hughes, Joel L. Pfeffer, Nate Bastian, Marie Martig, J. M. Diederik Kruijssen, Robert A. Crain, Marta Reina-Campos, Sebastian Trujillo-Gomez

Abstract: The mass function of globular cluster (GC) populations is a fundamental observable that encodes the physical conditions under which these massive stellar clusters formed and evolved. The high-mass end of star cluster mass functions are commonly described using a Schechter function, with an exponential truncation mass $M_{c,*}$. For the GC mass functions in the Virgo galaxy cluster, this truncation… ▽ More The mass function of globular cluster (GC) populations is a fundamental observable that encodes the physical conditions under which these massive stellar clusters formed and evolved. The high-mass end of star cluster mass functions are commonly described using a Schechter function, with an exponential truncation mass $M_{c,*}$. For the GC mass functions in the Virgo galaxy cluster, this truncation mass increases with galaxy mass ($M_{*}$). In this paper we fit Schechter mass functions to the GCs in the most massive galaxy group ($M_{\mathrm{200}} = 5.14 \times 10^{13} M_{\odot}$) in the E-MOSAICS simulations. The fiducial cluster formation model in E-MOSAICS reproduces the observed trend of $M_{c,*}$ with $M_{*}$ for the Virgo cluster. We therefore examine the origin of the relation by fitting $M_{c,*}$ as a function of galaxy mass, with and without accounting for mass loss by two-body relaxation, tidal shocks and/or dynamical friction. In the absence of these mass-loss mechanisms, the $M_{c,*}$-$M_{*}$ relation is flat above $M_* > 10^{10} M_{\odot}$. It is therefore the disruption of high-mass GCs in galaxies with $M_{*}\sim 10^{10} M_{\odot}$ that lowers the $M_{c,*}$ in these galaxies. High-mass GCs are able to survive in more massive galaxies, since there are more mergers to facilitate their redistribution to less-dense environments. The $M_{c,*}-M_*$ relation is therefore a consequence of both the formation conditions of massive star clusters and their environmentally-dependent disruption mechanisms. △ Less

Submitted 3 December, 2021; originally announced December 2021.

Comments: Accepted to MNRAS

A novel measurement of initial-state gluon radiation in hadron collisions using Drell-Yan events

Authors: CDF Collaboration, T. Aaltonen, S. Amerio, D. Amidei, A. Anastassov, A. Annovi, J. Antos, G. Apollinari, J. A. Appel, T. Arisawa, A. Artikov, J. Asaadi, W. Ashmanskas, B. Auerbach, A. Aurisano, F. Azfar, W. Badgett, T. Bae, A. Barbaro-Galtieri, V. E. Barnes, B. A. Barnett, P. Barria, P. Bartos, M. Bauce, F. Bedeschi , et al. (375 additional authors not shown)

Abstract: A study of initial-state gluon radiation (ISR) in hadron collisions is presented using Drell-Yan (DY) events produced in proton-antiproton collisions by the Tevatron collider at a center-of-mass energy of 1.96 TeV. This paper adopts a novel approach which uses the mean value of the Z/$γ^*$ transverse momentum $<p_T^{DY}>$ in DY events as a powerful observable to characterize the effect of ISR. In… ▽ More A study of initial-state gluon radiation (ISR) in hadron collisions is presented using Drell-Yan (DY) events produced in proton-antiproton collisions by the Tevatron collider at a center-of-mass energy of 1.96 TeV. This paper adopts a novel approach which uses the mean value of the Z/$γ^*$ transverse momentum $<p_T^{DY}>$ in DY events as a powerful observable to characterize the effect of ISR. In a data sample corresponding to an integrated luminosity of 9.4 fb$^{-1}$ collected with the CDF Run II detector, $<p_T^{DY}>$ is measured as a function of the Z/$γ^*$ invariant mass. It is found that these two observables have a dependence, $<p_T^{DY}> = -8 + 2.2 \ln m_{DY}^2$ [GeV/c], where $m_{DY}$ is the value of the Z/$γ^*$ mass measured in units of GeV/$c^2$. This linear dependence is observed for the first time in this analysis. It may be exploited to model the effect of ISR and constrain its impact in other processes. △ Less

Submitted 28 October, 2021; v1 submitted 28 October, 2021; originally announced October 2021.

Comments: 14 pages, 14 figures

Collaborating with Humans without Human Data

Authors: DJ Strouse, Kevin R. McKee, Matt Botvinick, Edward Hughes, Richard Everett

Abstract: Collaborating with humans requires rapidly adapting to their individual strengths, weaknesses, and preferences. Unfortunately, most standard multi-agent reinforcement learning techniques, such as self-play (SP) or population play (PP), produce agents that overfit to their training partners and do not generalize well to humans. Alternatively, researchers can collect human data, train a human model… ▽ More Collaborating with humans requires rapidly adapting to their individual strengths, weaknesses, and preferences. Unfortunately, most standard multi-agent reinforcement learning techniques, such as self-play (SP) or population play (PP), produce agents that overfit to their training partners and do not generalize well to humans. Alternatively, researchers can collect human data, train a human model using behavioral cloning, and then use that model to train "human-aware" agents ("behavioral cloning play", or BCP). While such an approach can improve the generalization of agents to new human co-players, it involves the onerous and expensive step of collecting large amounts of human data first. Here, we study the problem of how to train agents that collaborate well with human partners without using human data. We argue that the crux of the problem is to produce a diverse set of training partners. Drawing inspiration from successful multi-agent approaches in competitive domains, we find that a surprisingly simple approach is highly effective. We train our agent partner as the best response to a population of self-play agents and their past checkpoints taken throughout training, a method we call Fictitious Co-Play (FCP). Our experiments focus on a two-player collaborative cooking simulator that has recently been proposed as a challenge problem for coordination with humans. We find that FCP agents score significantly higher than SP, PP, and BCP when paired with novel agent and human partners. Furthermore, humans also report a strong subjective preference to partnering with FCP agents over all baselines. △ Less

Submitted 7 January, 2022; v1 submitted 15 October, 2021; originally announced October 2021.

Comments: Accepted at NeurIPS 2021 (spotlight)

Bright mid-infrared photoluminescence from high dislocation density epitaxial PbSe films on GaAs

Authors: Jarod Meyer, Aaron J. Muhowski, Leland J. Nordin, Eamonn T. Hughes, Brian B. Haidet, Daniel Wasserman, Kunal Mukherjee

Abstract: We report on photoluminescence in the 3-7 $μ$m mid-wave infrared (MWIR) range from sub-100 nm strained thin films of rocksalt PbSe(001) grown on GaAs(001) substrates by molecular beam epitaxy. These bare films, grown epitaxially at temperatures below 400 °C, luminesce brightly at room temperature and have minority carrier lifetimes as long as 172 ns. The relatively long lifetimes in PbSe thin film… ▽ More We report on photoluminescence in the 3-7 $μ$m mid-wave infrared (MWIR) range from sub-100 nm strained thin films of rocksalt PbSe(001) grown on GaAs(001) substrates by molecular beam epitaxy. These bare films, grown epitaxially at temperatures below 400 °C, luminesce brightly at room temperature and have minority carrier lifetimes as long as 172 ns. The relatively long lifetimes in PbSe thin films are achievable despite threading dislocation densities exceeding $10^9$ $cm^{-2}$ arising from island growth on the nearly 8% lattice- and crystal-structure-mismatched GaAs substrate. Using quasi-continuous-wave and time-resolved photoluminescence, we show Shockley-Read-Hall recombination is slow in our high dislocation density PbSe films at room temperature, a hallmark of defect tolerance. Power-dependent photoluminescence and high injection excess carrier lifetimes at room temperature suggest that degenerate Auger recombination limits the efficiency of our films, though the Auger recombination rates are significantly lower than equivalent, III-V bulk materials and even a bit slower than expectations for bulk PbSe. Consequently, the combined effects of defect tolerance and low Auger recombination rates yield an estimated peak internal quantum efficiency of roughly 30% at room temperature, unparalleled in the MWIR for a severely lattice-mismatched thin film. We anticipate substantial opportunities for improving performance by optimizing crystal growth as well as understanding Auger processes in thin films. These results highlight the unique opportunity to harness the unusual chemical bonding in PbSe and related IV-VI semiconductors for heterogeneously integrated mid-infrared light sources constrained by tight thermal budgets in new device designs. △ Less

Submitted 28 August, 2021; originally announced August 2021.

Comments: 24 pages, 6 figures

Journal ref: APL Materials 9, 111112 (2021)

Measurement of the charge asymmetry of electrons from the decays of $W$ bosons produced in $p\bar{p}$ collisions at $\sqrt{s}=1.96$ TeV

Authors: CDF Collaboration, T. Aaltonen, S. Amerio, D. Amidei, A. Anastassov, A. Annovi, J. Antos, G. Apollinari, J. A. Appel, T. Arisawa, A. Artikov, J. Asaadi, W. Ashmanskas, B. Auerbach, A. Aurisano, F. Azfar, W. Badgett, T. Bae, A. Barbaro-Galtieri, V. E. Barnes, B. A. Barnett, P. Barria, P. Bartos, M. Bauce, F. Bedeschi , et al. (376 additional authors not shown)

Abstract: At the Fermilab Tevatron proton-antiproton ($p\bar{p}$) collider, high-mass electron-neutrino ($eν$) pairs are produced predominantly in the process $p \bar{p} \rightarrow W(\rightarrow eν) + X$. The asymmetry of the electron and positron yield as a function of their pseudorapidity constrain the slope of the ratio of the $u$- to $d$-quark parton distributions versus the fraction of the proton mome… ▽ More At the Fermilab Tevatron proton-antiproton ($p\bar{p}$) collider, high-mass electron-neutrino ($eν$) pairs are produced predominantly in the process $p \bar{p} \rightarrow W(\rightarrow eν) + X$. The asymmetry of the electron and positron yield as a function of their pseudorapidity constrain the slope of the ratio of the $u$- to $d$-quark parton distributions versus the fraction of the proton momentum carried by the quarks. This paper reports on the measurement of the electron-charge asymmetry using the full data set recorded by the Collider Detector at Fermilab in 2001--2011 and corresponding to 9.1~fb$^{-1}$ of integrated luminosity. The measurement significantly improves the precision of the Tevatron constraints on the parton-distribution functions of the proton. Numerical tables of the measurement are provided. △ Less

Submitted 2 November, 2021; v1 submitted 9 July, 2021; originally announced July 2021.

Comments: 27 pages, 25 figures. To be published in PRD

Report number: FERMILAB-PUB-21-293-E

Globular clusters as tracers of the dark matter halo: insights from the E-MOSAICS simulations

Authors: Marta Reina-Campos, Sebastian Trujillo-Gomez, Alis J. Deason, J. M. Diederik Kruijssen, Joel L. Pfeffer, Robert A. Crain, Nate Bastian, Meghan E. Hughes

Abstract: Globular clusters (GCs) are bright objects that span a wide range of galactocentric distances, and are thus probes of the structure of dark matter (DM) haloes. In this work, we explore whether the projected radial profiles of GCs can be used to infer the structural properties of their host DM haloes. We use the simulated GC populations in a sample of 166 central galaxies from the… ▽ More Globular clusters (GCs) are bright objects that span a wide range of galactocentric distances, and are thus probes of the structure of dark matter (DM) haloes. In this work, we explore whether the projected radial profiles of GCs can be used to infer the structural properties of their host DM haloes. We use the simulated GC populations in a sample of 166 central galaxies from the $(34.4~\rm cMpc)^3$ periodic volume of the E-MOSAICS project. We find that more massive galaxies host stellar and GC populations with shallower density profiles that are more radially extended. In addition, the metal-poor GC subpopulations tend to have shallower and more extended profiles than the metal-rich subsamples, which we relate to the preferentially accreted origin of the metal-poor GCs. We find strong correlations between the slopes and effective radii of the radial profiles of the GC populations and the structural properties of the DM haloes, such as their power-law slopes, scale radii, and concentration parameters. Accounting for a dependence on the galaxy stellar mass decreases the scatter of the two-dimensional relations. This suggests that the projected number counts of GCs, combined with their galaxy mass, trace the density profile of the DM halo of their host galaxy. When applied to extragalactic GC systems, we recover the scale radii and the extent of the DM haloes of a sample of ETGs with uncertainties smaller than $0.2~\rm dex$. Thus, extragalactic GC systems provide a novel avenue to explore the structure of DM haloes beyond the Local Group. △ Less

Submitted 14 June, 2021; originally announced June 2021.

Comments: 19 pages, 11 figures and 2 tables; submitted to MNRAS, comments and/or suggestions are welcomed!

Measurement of the Nucleon $F^n_2/F^p_2$ Structure Function Ratio by the Jefferson Lab MARATHON Tritium/Helium-3 Deep Inelastic Scattering Experiment

Authors: MARATHON Collaboration, D. Abrams, H. Albataineh, B. S. Aljawrneh, S. Alsalmi, K. Aniol, W. Armstrong, J. Arrington, H. Atac, T. Averett, C. Ayerbe Gayoso, X. Bai, J. Bane, S. Barcus, A. Beck, V. Bellini, H. Bhatt, D. Bhetuwal, D. Biswas, D. Blyth, W. Boeglin, D. Bulumulla, J. Butler, A. Camsonne, M. Carmignotto , et al. (107 additional authors not shown)

Abstract: The ratio of the nucleon $F_2$ structure functions, $F_2^n/F_2^p$, is determined by the MARATHON experiment from measurements of deep inelastic scattering of electrons from $^3$H and $^3$He nuclei. The experiment was performed in the Hall A Facility of Jefferson Lab and used two high resolution spectrometers for electron detection, and a cryogenic target system which included a low-activity tritiu… ▽ More The ratio of the nucleon $F_2$ structure functions, $F_2^n/F_2^p$, is determined by the MARATHON experiment from measurements of deep inelastic scattering of electrons from $^3$H and $^3$He nuclei. The experiment was performed in the Hall A Facility of Jefferson Lab and used two high resolution spectrometers for electron detection, and a cryogenic target system which included a low-activity tritium cell. The data analysis used a novel technique exploiting the mirror symmetry of the two nuclei, which essentially eliminates many theoretical uncertainties in the extraction of the ratio. The results, which cover the Bjorken scaling variable range $0.19 < x < 0.83$, represent a significant improvement compared to previous SLAC and Jefferson Lab measurements for the ratio. They are compared to recent theoretical calculations and empirical determinations of the $F_2^n/F_2^p$ ratio. △ Less

Submitted 9 June, 2021; v1 submitted 12 April, 2021; originally announced April 2021.

A multi-agent reinforcement learning model of reputation and cooperation in human groups

Authors: Kevin R. McKee, Edward Hughes, Tina O. Zhu, Martin J. Chadwick, Raphael Koster, Antonio Garcia Castaneda, Charlie Beattie, Thore Graepel, Matt Botvinick, Joel Z. Leibo

Abstract: Collective action demands that individuals efficiently coordinate how much, where, and when to cooperate. Laboratory experiments have extensively explored the first part of this process, demonstrating that a variety of social-cognitive mechanisms influence how much individuals choose to invest in group efforts. However, experimental research has been unable to shed light on how social cognitive me… ▽ More Collective action demands that individuals efficiently coordinate how much, where, and when to cooperate. Laboratory experiments have extensively explored the first part of this process, demonstrating that a variety of social-cognitive mechanisms influence how much individuals choose to invest in group efforts. However, experimental research has been unable to shed light on how social cognitive mechanisms contribute to the where and when of collective action. We build and test a computational model of human behavior in Clean Up, a social dilemma task popular in multi-agent reinforcement learning research. We show that human groups effectively cooperate in Clean Up when they can identify group members and track reputations over time, but fail to organize under conditions of anonymity. A multi-agent reinforcement learning model of reputation demonstrates the same difference in cooperation under conditions of identifiability and anonymity. In addition, the model accurately predicts spatial and temporal patterns of group behavior: in this public goods dilemma, the intrinsic motivation for reputation catalyzes the development of a non-territorial, turn-taking strategy to coordinate collective action. △ Less

Submitted 22 February, 2023; v1 submitted 8 March, 2021; originally announced March 2021.

Modelling Cooperation in Network Games with Spatio-Temporal Complexity

Authors: Michiel A. Bakker, Richard Everett, Laura Weidinger, Iason Gabriel, William S. Isaac, Joel Z. Leibo, Edward Hughes

Abstract: The real world is awash with multi-agent problems that require collective action by self-interested agents, from the routing of packets across a computer network to the management of irrigation systems. Such systems have local incentives for individuals, whose behavior has an impact on the global outcome for the group. Given appropriate mechanisms describing agent interaction, groups may achieve s… ▽ More The real world is awash with multi-agent problems that require collective action by self-interested agents, from the routing of packets across a computer network to the management of irrigation systems. Such systems have local incentives for individuals, whose behavior has an impact on the global outcome for the group. Given appropriate mechanisms describing agent interaction, groups may achieve socially beneficial outcomes, even in the face of short-term selfish incentives. In many cases, collective action problems possess an underlying graph structure, whose topology crucially determines the relationship between local decisions and emergent global effects. Such scenarios have received great attention through the lens of network games. However, this abstraction typically collapses important dimensions, such as geometry and time, relevant to the design of mechanisms promoting cooperation. In parallel work, multi-agent deep reinforcement learning has shown great promise in modelling the emergence of self-organized cooperation in complex gridworld domains. Here we apply this paradigm in graph-structured collective action problems. Using multi-agent deep reinforcement learning, we simulate an agent society for a variety of plausible mechanisms, finding clear transitions between different equilibria over time. We define analytic tools inspired by related literatures to measure the social outcomes, and use these to draw conclusions about the efficacy of different environmental interventions. Our methods have implications for mechanism design in both human and artificial agent systems. △ Less

Submitted 13 February, 2021; originally announced February 2021.

Comments: AAMAS 2021

Neural Recursive Belief States in Multi-Agent Reinforcement Learning

Authors: Pol Moreno, Edward Hughes, Kevin R. McKee, Bernardo Avila Pires, Théophane Weber

Abstract: In multi-agent reinforcement learning, the problem of learning to act is particularly difficult because the policies of co-players may be heavily conditioned on information only observed by them. On the other hand, humans readily form beliefs about the knowledge possessed by their peers and leverage beliefs to inform decision-making. Such abilities underlie individual success in a wide range of Ma… ▽ More In multi-agent reinforcement learning, the problem of learning to act is particularly difficult because the policies of co-players may be heavily conditioned on information only observed by them. On the other hand, humans readily form beliefs about the knowledge possessed by their peers and leverage beliefs to inform decision-making. Such abilities underlie individual success in a wide range of Markov games, from bluffing in Poker to conditional cooperation in the Prisoner's Dilemma, to convention-building in Bridge. Classical methods are usually not applicable to complex domains due to the intractable nature of hierarchical beliefs (i.e. beliefs of other agents' beliefs). We propose a scalable method to approximate these belief structures using recursive deep generative models, and to use the belief models to obtain representations useful to acting in complex tasks. Our agents trained with belief models outperform model-free baselines with equivalent representational capacity using common training paradigms. We also show that higher-order belief models outperform agents with lower-order models. △ Less

Submitted 3 February, 2021; originally announced February 2021.

What to expect when using globular clusters as tracers of the total mass distribution in Milky Way-mass galaxies

Authors: Meghan E. Hughes, Prashin Jethwa, Michael Hilker, Glenn van de Ven, Marie Martig, Joel L. Pfeffer, Nate Bastian, J. M. Diederik Kruijssen, Sebastian Trujillo-Gomez, Marta Reina-Campos, Robert A. Crain

Abstract: Dynamical models allow us to connect the motion of a set of tracers to the underlying gravitational potential, and thus to the total (luminous and dark) matter distribution. They are particularly useful for understanding the mass and spatial distribution of dark matter (DM) in a galaxy. Globular clusters (GCs) are an ideal tracer population in dynamical models, since they are bright and can be fou… ▽ More Dynamical models allow us to connect the motion of a set of tracers to the underlying gravitational potential, and thus to the total (luminous and dark) matter distribution. They are particularly useful for understanding the mass and spatial distribution of dark matter (DM) in a galaxy. Globular clusters (GCs) are an ideal tracer population in dynamical models, since they are bright and can be found far out into the halo of galaxies. We aim to test how well Jeans-Anisotropic-MGE (JAM) models using GCs (positions and line-of-sight velocities) as tracers can constrain the mass and radial distribution of DM halos. For this, we use the E-MOSAICS suite of 25 zoom-in simulations of L* galaxies. We find that the DM halo properties are reasonably well recovered by the JAM models. There is, however, a strong correlation between how well we recover the mass and the radial distribution of the DM and the number of GCs in the galaxy: the constraints get exponentially worse with fewer GCs, and at least 150 GCs are needed in order to guarantee that the JAM model will perform well. We find that while the data quality (uncertainty on the radial velocities) can be important, the number of GCs is the dominant factor in terms of the accuracy and precision of the measurements. This work shows promising results for these models to be used in extragalactic systems with a sample of more than 150 GCs. △ Less

Submitted 20 January, 2021; originally announced January 2021.

Comments: 18 pages, 13 figures. Accepted for publication in MNRAS

Open Problems in Cooperative AI

Authors: Allan Dafoe, Edward Hughes, Yoram Bachrach, Tantum Collins, Kevin R. McKee, Joel Z. Leibo, Kate Larson, Thore Graepel

Abstract: Problems of cooperation--in which agents seek ways to jointly improve their welfare--are ubiquitous and important. They can be found at scales ranging from our daily routines--such as driving on highways, scheduling meetings, and working collaboratively--to our global challenges--such as peace, commerce, and pandemic preparedness. Arguably, the success of the human species is rooted in our ability… ▽ More Problems of cooperation--in which agents seek ways to jointly improve their welfare--are ubiquitous and important. They can be found at scales ranging from our daily routines--such as driving on highways, scheduling meetings, and working collaboratively--to our global challenges--such as peace, commerce, and pandemic preparedness. Arguably, the success of the human species is rooted in our ability to cooperate. Since machines powered by artificial intelligence are playing an ever greater role in our lives, it will be important to equip them with the capabilities necessary to cooperate and to foster cooperation. We see an opportunity for the field of artificial intelligence to explicitly focus effort on this class of problems, which we term Cooperative AI. The objective of this research would be to study the many aspects of the problems of cooperation and to innovate in AI to contribute to solving these problems. Central goals include building machine agents with the capabilities needed for cooperation, building tools to foster cooperation in populations of (machine and/or human) agents, and otherwise conducting AI research for insight relevant to problems of cooperation. This research integrates ongoing work on multi-agent systems, game theory and social choice, human-machine interaction and alignment, natural-language processing, and the construction of social tools and platforms. However, Cooperative AI is not the union of these existing areas, but rather an independent bet about the productivity of specific kinds of conversations that involve these and other areas. We see opportunity to more explicitly focus on the problem of cooperation, to construct unified theory and vocabulary, and to build bridges with adjacent communities working on cooperation, including in the natural, social, and behavioural sciences. △ Less

Submitted 15 December, 2020; originally announced December 2020.

Linking globular cluster formation at low and high redshift through the age-metallicity relation in E-MOSAICS

Authors: Danny Horta, Meghan E. Hughes, Joel L. Pfeffer, Nate Bastian, J. M. Diederik Kruijssen, Marta Reina-Campos, Robert A. Crain

Abstract: We set out to compare the age-metallicity relation (AMR) of massive clusters from Magellanic Cloud mass galaxies in the E-MOSAICS suite of numerical cosmological simulations with an amalgamation of observational data of massive clusters in the Large and Small Magellanic Clouds (LMC/SMC). We aim to test if: i) star cluster formation proceeds according to universal physical processes, suggestive of… ▽ More We set out to compare the age-metallicity relation (AMR) of massive clusters from Magellanic Cloud mass galaxies in the E-MOSAICS suite of numerical cosmological simulations with an amalgamation of observational data of massive clusters in the Large and Small Magellanic Clouds (LMC/SMC). We aim to test if: i) star cluster formation proceeds according to universal physical processes, suggestive of a common formation mechanism for young-massive clusters (YMCs), intermediate-age clusters (IACs), and ancient globular clusters (GCs); ii) massive clusters of all ages trace a continuous AMR; iii) the AMRs of smaller mass galaxies show a shallower relation when compared to more massive galaxies. Our results show that, within the uncertainties, the predicted AMRs of L/SMC-mass galaxies with similar star formation histories to the L/SMC follow the same relation as observations. We also find that the metallicity at which the AMR saturates increases with galaxy mass, which is also found for the field star AMRs. This suggests that relatively low-metallicity clusters can still form in dwarfs galaxies. Given our results, we suggest that ancient GCs share their formation mechanism with IACs and YMCs, in which GCs are the result of a universal process of star cluster formation during the early episodes of star formation in their host galaxies. △ Less

Submitted 9 November, 2020; v1 submitted 20 October, 2020; originally announced October 2020.

Comments: Accepted for publication in MNRAS

Negotiating Team Formation Using Deep Reinforcement Learning

Authors: Yoram Bachrach, Richard Everett, Edward Hughes, Angeliki Lazaridou, Joel Z. Leibo, Marc Lanctot, Michael Johanson, Wojciech M. Czarnecki, Thore Graepel

Abstract: When autonomous agents interact in the same environment, they must often cooperate to achieve their goals. One way for agents to cooperate effectively is to form a team, make a binding agreement on a joint plan, and execute it. However, when agents are self-interested, the gains from team formation must be allocated appropriately to incentivize agreement. Various approaches for multi-agent negotia… ▽ More When autonomous agents interact in the same environment, they must often cooperate to achieve their goals. One way for agents to cooperate effectively is to form a team, make a binding agreement on a joint plan, and execute it. However, when agents are self-interested, the gains from team formation must be allocated appropriately to incentivize agreement. Various approaches for multi-agent negotiation have been proposed, but typically only work for particular negotiation protocols. More general methods usually require human input or domain-specific data, and so do not scale. To address this, we propose a framework for training agents to negotiate and form teams using deep reinforcement learning. Importantly, our method makes no assumptions about the specific negotiation protocol, and is instead completely experience driven. We evaluate our approach on both non-spatial and spatially extended team-formation negotiation environments, demonstrating that our agents beat hand-crafted bots and reach negotiation outcomes consistent with fair solutions predicted by cooperative game theory. Additionally, we investigate how the physical location of agents influences negotiation outcomes. △ Less

Submitted 20 October, 2020; originally announced October 2020.

ACM Class: I.2.6

Journal ref: Artificial Intelligence 288 (2020): 103356

Model-free conventions in multi-agent reinforcement learning with heterogeneous preferences

Authors: Raphael Köster, Kevin R. McKee, Richard Everett, Laura Weidinger, William S. Isaac, Edward Hughes, Edgar A. Duéñez-Guzmán, Thore Graepel, Matthew Botvinick, Joel Z. Leibo

Abstract: Game theoretic views of convention generally rest on notions of common knowledge and hyper-rational models of individual behavior. However, decades of work in behavioral economics have questioned the validity of both foundations. Meanwhile, computational neuroscience has contributed a modernized 'dual process' account of decision-making where model-free (MF) reinforcement learning trades off with… ▽ More Game theoretic views of convention generally rest on notions of common knowledge and hyper-rational models of individual behavior. However, decades of work in behavioral economics have questioned the validity of both foundations. Meanwhile, computational neuroscience has contributed a modernized 'dual process' account of decision-making where model-free (MF) reinforcement learning trades off with model-based (MB) reinforcement learning. The former captures habitual and procedural learning while the latter captures choices taken via explicit planning and deduction. Some conventions (e.g. international treaties) are likely supported by cognition that resonates with the game theoretic and MB accounts. However, convention formation may also occur via MF mechanisms like habit learning; though this possibility has been understudied. Here, we demonstrate that complex, large-scale conventions can emerge from MF learning mechanisms. This suggests that some conventions may be supported by habit-like cognition rather than explicit reasoning. We apply MF multi-agent reinforcement learning to a temporo-spatially extended game with incomplete information. In this game, large parts of the state space are reachable only by collective action. However, heterogeneity of tastes makes such coordinated action difficult: multiple equilibria are desirable for all players, but subgroups prefer a particular equilibrium over all others. This creates a coordination problem that can be solved by establishing a convention. We investigate start-up and free rider subproblems as well as the effects of group size, intensity of intrinsic preference, and salience on the emergence dynamics of coordination conventions. Results of our simulations show agents establish and switch between conventions, even working against their own preferred outcome when doing so is necessary for effective coordination. △ Less

Submitted 14 December, 2020; v1 submitted 18 October, 2020; originally announced October 2020.

Evidence and implications of abnormal predictive coding in dementia

Authors: Ece Kocagoncu, Anastasia Klimovich-Gray, Laura E Hughes, James B Rowe

Abstract: The diversity of cognitive deficits and neuropathological processes associated with dementias has encouraged divergence in pathophysiological explanations of disease. Here, we review an alternative framework that emphasises convergent critical features of pathophysiology, rather than the loss of memory centres or language centres, or singular neurotransmitter systems. Cognitive deficits are interp… ▽ More The diversity of cognitive deficits and neuropathological processes associated with dementias has encouraged divergence in pathophysiological explanations of disease. Here, we review an alternative framework that emphasises convergent critical features of pathophysiology, rather than the loss of memory centres or language centres, or singular neurotransmitter systems. Cognitive deficits are interpreted in the light of advances in normative accounts of brain function, based on predictive coding in hierarchical neural networks. The predicting coding rests on Bayesian integration of beliefs and sensory evidence, with hierarchical predictions and prediction errors, for memory, perception, speech and behaviour. We describe how analogous impairments in predictive coding in parallel neurocognitive systems can generate diverse clinical phenomena, in neurodegenerative dementias. The review presents evidence from behavioural and neurophysiological studies of perception, language, memory and decision-making. The re-formulation of cognitive deficits in dementia in terms of predictive coding has several advantages. It brings diverse clinical phenomena into a common framework, such as linking cognitive and movement disorders; and it makes specific predictions on cognitive physiology that support translational and experimental medicine studies. The insights into complex human cognitive disorders from the predictive coding model may therefore also inform future therapeutic strategies. △ Less

Submitted 11 June, 2020; originally announced June 2020.

Learning to Incentivize Other Learning Agents

Authors: Jiachen Yang, Ang Li, Mehrdad Farajtabar, Peter Sunehag, Edward Hughes, Hongyuan Zha

Abstract: The challenge of developing powerful and general Reinforcement Learning (RL) agents has received increasing attention in recent years. Much of this effort has focused on the single-agent setting, in which an agent maximizes a predefined extrinsic reward function. However, a long-term question inevitably arises: how will such independent agents cooperate when they are continually learning and actin… ▽ More The challenge of developing powerful and general Reinforcement Learning (RL) agents has received increasing attention in recent years. Much of this effort has focused on the single-agent setting, in which an agent maximizes a predefined extrinsic reward function. However, a long-term question inevitably arises: how will such independent agents cooperate when they are continually learning and acting in a shared multi-agent environment? Observing that humans often provide incentives to influence others' behavior, we propose to equip each RL agent in a multi-agent environment with the ability to give rewards directly to other agents, using a learned incentive function. Each agent learns its own incentive function by explicitly accounting for its impact on the learning of recipients and, through them, the impact on its own extrinsic objective. We demonstrate in experiments that such agents significantly outperform standard RL and opponent-shaping agents in challenging general-sum Markov games, often by finding a near-optimal division of labor. Our work points toward more opportunities and challenges along the path to ensure the common good in a multi-agent future. △ Less

Submitted 19 October, 2020; v1 submitted 10 June, 2020; originally announced June 2020.

Comments: 20 pages, 11 figures. To appear in 34th Conference on Neural Information Processing Systems (NeurIPS 2020)

Defect filtering for thermal expansion induced dislocations in III-V lasers on silicon

Authors: Jennifer Selvidge, Justin Norman, Eamonn T. Hughes, Chen Shang, Daehwan Jung, Aidan A. Taylor, MJ Kennedy, Robert Herrick, John E. Bowers, Kunal Mukherjee

Abstract: Epitaxially integrated III-V semiconductor lasers for silicon photonics have the potential to dramatically transform information networks, but currently, dislocations limit performance and reliability even in defect tolerant InAs quantum dot (QD) based lasers. Despite being below critical thickness, QD layers in these devices contain previously unexplained misfit dislocations, which facilitate non… ▽ More Epitaxially integrated III-V semiconductor lasers for silicon photonics have the potential to dramatically transform information networks, but currently, dislocations limit performance and reliability even in defect tolerant InAs quantum dot (QD) based lasers. Despite being below critical thickness, QD layers in these devices contain previously unexplained misfit dislocations, which facilitate non-radiative recombination. We demonstrate here that these misfit dislocations form during post-growth cooldown due to the combined effects of (1) thermal-expansion mismatch between the III-V layers and silicon and (2) precipitate and alloy hardening in the active region. By incorporating an additional sub-critical thickness, indium-alloyed misfit dislocation trapping layer, we leverage these mechanical hardening effects to our advantage, successfully displacing 95% of misfit dislocations from the QD layer in model structures. Unlike conventional dislocation mitigation strategies, the trapping layer reduces neither the number of threading dislocations nor the number of misfit dislocations. It simply shifts the position of misfit dislocations away from the QD layer, reducing the defects' impact on luminescence. In full lasers, adding a misfit dislocation trapping layer both above and below the QD active region displaces misfit dislocations and substantially improves performance: we measure a twofold reduction in lasing threshold currents and a greater than threefold increase in output power. Our results suggest that devices employing both traditional threading dislocation reduction techniques and optimized misfit dislocation trapping layers may finally lead to fully integrated, commercially viable silicon-based photonic integrated circuits. △ Less

Submitted 4 August, 2020; v1 submitted 12 May, 2020; originally announced May 2020.

Comments: 9 pages, 6 figures

Journal ref: Appl. Phys. Lett. 117 (2020) 122101

Where did the globular clusters of the Milky Way form? Insights from the E-MOSAICS simulations

Authors: Benjamin W. Keller, J. M. Diederik Kruijssen, Joel Pfeffer, Marta Reina-Campos, Nate Bastian, Sebastian Trujillo-Gomez, Meghan E. Hughes, Robert A. Crain

Abstract: Globular clusters (GCs) are typically old, with most having formed at z >~ 2. This makes understanding their birth environments difficult, as they are typically too distant to observe with sufficient angular resolution to resolve GC birth sites. Using 25 cosmological zoom-in simulations of Milky Way-like galaxies from the E-MOSAICS project, with physically-motivated models for star formation, feed… ▽ More Globular clusters (GCs) are typically old, with most having formed at z >~ 2. This makes understanding their birth environments difficult, as they are typically too distant to observe with sufficient angular resolution to resolve GC birth sites. Using 25 cosmological zoom-in simulations of Milky Way-like galaxies from the E-MOSAICS project, with physically-motivated models for star formation, feedback, and the formation, evolution, and disruption of GCs, we identify the birth environments of present-day GCs. We find roughly half of GCs in these galaxies formed in-situ (52.0 +/- 1.0 per cent) between z ~ 2 - 4, in turbulent, high-pressure discs fed by gas that was accreted without ever being strongly heated through a virial shock or feedback. A minority of GCs form during mergers (12.6 +/- 0.6 per cent in major mergers, and 7.2 +/- 0.5 per cent in minor mergers), but we find that mergers are important for preserving the GCs seen today by ejecting them from their natal, high density interstellar medium (ISM), where proto-GCs are rapidly destroyed due to tidal shocks from ISM substructure. This chaotic history of hierarchical galaxy assembly acts to mix the spatial and kinematic distribution of GCs formed through different channels, making it difficult to use observable GC properties to distinguish GCs formed in mergers from ones formed by smooth accretion, and similarly GCs formed in-situ from those formed ex-situ. These results suggest a simple picture of GC formation, in which GCs are a natural outcome of normal star formation in the typical, gas-rich galaxies that are the progenitors of present-day galaxies. △ Less

Submitted 11 May, 2020; originally announced May 2020.

Comments: 20 pages, 20 figures, resubmitted to MNRAS after accounting for referee's comments

The kinematics of globular cluster populations in the E-MOSAICS simulations and their implications for the assembly history of the Milky Way

Authors: Sebastian Trujillo-Gomez, J. M. Diederik Kruijssen, Marta Reina-Campos, Joel L. Pfeffer, Benjamin W. Keller, Robert A. Crain, Nate Bastian, Meghan E. Hughes

Abstract: We present a detailed comparison of the Milky Way (MW) globular cluster (GC) kinematics with the 25 Milky Way-mass cosmological simulations from the E-MOSAICS project. While the MW falls within the kinematic distribution of GCs spanned by the simulations, the relative kinematics of its metal-rich ($[\rm{Fe/H}]>-1.2$) versus metal-poor ($[\rm{Fe/H}]<-1.2$), and inner ($r<8\rm{kpc}$) versus outer (… ▽ More We present a detailed comparison of the Milky Way (MW) globular cluster (GC) kinematics with the 25 Milky Way-mass cosmological simulations from the E-MOSAICS project. While the MW falls within the kinematic distribution of GCs spanned by the simulations, the relative kinematics of its metal-rich ($[\rm{Fe/H}]>-1.2$) versus metal-poor ($[\rm{Fe/H}]<-1.2$), and inner ($r<8\rm{kpc}$) versus outer ($r>8\rm{kpc}$) populations are atypical for its mass. To understand the origins of these features, we perform a comprehensive statistical analysis of the simulations, and find 18 correlations describing the assembly of $L^*$ galaxies and their dark matter haloes based on their GC population kinematics. The correlations arise because the orbital distributions of accreted and in-situ GCs depend on the masses and accretion redshifts of accreted satellites, driven by the combined effects of dynamical fraction, tidal stripping, and dynamical heating. Because the kinematics of in-situ/accreted GCs are broadly traced by the metal-rich/metal-poor and inner/outer populations, the observed GC kinematics are a sensitive probe of galaxy assembly. We predict that relative to the population of $L^*$ galaxies, the MW assembled its dark matter and stellar mass rapidly through a combination of in-situ star formation, more than a dozen low-mass mergers, and $1.4\pm1.2$ early ($z=3.1\pm1.3$) major merger. The rapid assembly period ended early, limiting the fraction of accreted stars. We conclude by providing detailed quantitative predictions for the assembly history of the MW. △ Less

Submitted 30 March, 2021; v1 submitted 5 May, 2020; originally announced May 2020.

Comments: 24 pages, 20 figures. Published in MNRAS

Journal ref: Monthly Notices of the Royal Astronomical Society, Volume 503, Issue 1, pp.31-58, May 2021

An Efficient Integration of Disentangled Attended Expression and Identity FeaturesFor Facial Expression Transfer andSynthesis

Authors: Kamran Ali, Charles E. Hughes

Abstract: In this paper, we present an Attention-based Identity Preserving Generative Adversarial Network (AIP-GAN) to overcome the identity leakage problem from a source image to a generated face image, an issue that is encountered in a cross-subject facial expression transfer and synthesis process. Our key insight is that the identity preserving network should be able to disentangle and compose shape, app… ▽ More In this paper, we present an Attention-based Identity Preserving Generative Adversarial Network (AIP-GAN) to overcome the identity leakage problem from a source image to a generated face image, an issue that is encountered in a cross-subject facial expression transfer and synthesis process. Our key insight is that the identity preserving network should be able to disentangle and compose shape, appearance, and expression information for efficient facial expression transfer and synthesis. Specifically, the expression encoder of our AIP-GAN disentangles the expression information from the input source image by predicting its facial landmarks using our supervised spatial and channel-wise attention module. Similarly, the disentangled expression-agnostic identity features are extracted from the input target image by inferring its combined intrinsic-shape and appearance image employing our self-supervised spatial and channel-wise attention mod-ule. To leverage the expression and identity information encoded by the intermediate layers of both of our encoders, we combine these features with the features learned by the intermediate layers of our decoder using a cross-encoder bilinear pooling operation. Experimental results show the promising performance of our AIP-GAN based technique. △ Less

Submitted 1 May, 2020; originally announced May 2020.

Comments: 10 Pages, excluding references

Learning to Resolve Alliance Dilemmas in Many-Player Zero-Sum Games

Authors: Edward Hughes, Thomas W. Anthony, Tom Eccles, Joel Z. Leibo, David Balduzzi, Yoram Bachrach

Abstract: Zero-sum games have long guided artificial intelligence research, since they possess both a rich strategy space of best-responses and a clear evaluation metric. What's more, competition is a vital mechanism in many real-world multi-agent systems capable of generating intelligent innovations: Darwinian evolution, the market economy and the AlphaZero algorithm, to name a few. In two-player zero-sum… ▽ More Zero-sum games have long guided artificial intelligence research, since they possess both a rich strategy space of best-responses and a clear evaluation metric. What's more, competition is a vital mechanism in many real-world multi-agent systems capable of generating intelligent innovations: Darwinian evolution, the market economy and the AlphaZero algorithm, to name a few. In two-player zero-sum games, the challenge is usually viewed as finding Nash equilibrium strategies, safeguarding against exploitation regardless of the opponent. While this captures the intricacies of chess or Go, it avoids the notion of cooperation with co-players, a hallmark of the major transitions leading from unicellular organisms to human civilization. Beyond two players, alliance formation often confers an advantage; however this requires trust, namely the promise of mutual cooperation in the face of incentives to defect. Successful play therefore requires adaptation to co-players rather than the pursuit of non-exploitability. Here we argue that a systematic study of many-player zero-sum games is a crucial element of artificial intelligence research. Using symmetric zero-sum matrix games, we demonstrate formally that alliance formation may be seen as a social dilemma, and empirically that naïve multi-agent reinforcement learning therefore fails to form alliances. We introduce a toy model of economic competition, and show how reinforcement learning may be augmented with a peer-to-peer contract mechanism to discover and enforce alliances. Finally, we generalize our agent model to incorporate temporally-extended contracts, presenting opportunities for further work. △ Less

Submitted 27 February, 2020; originally announced March 2020.

Comments: Accepted for publication at AAMAS 2020

Predicting accreted satellite galaxy masses and accretion redshifts based on globular cluster orbits in the E-MOSAICS simulations

Authors: Joel L. Pfeffer, Sebastian Trujillo-Gomez, J. M. Diederik Kruijssen, Robert A. Crain, Meghan E. Hughes, Marta Reina-Campos, Nate Bastian

Abstract: The ages and metallicities of globular clusters (GCs) are known to be powerful tracers of the properties of their progenitor galaxies, enabling their use in determining the merger histories of galaxies. However, while useful in separating GCs into individual accretion events, the orbits of GC groups themselves have received less attention as probes of their progenitor galaxy properties. In this wo… ▽ More The ages and metallicities of globular clusters (GCs) are known to be powerful tracers of the properties of their progenitor galaxies, enabling their use in determining the merger histories of galaxies. However, while useful in separating GCs into individual accretion events, the orbits of GC groups themselves have received less attention as probes of their progenitor galaxy properties. In this work, we use simulations of galaxies and their GC systems from the E-MOSAICS project to explore how the present-day orbital properties of GCs are related to the properties of their progenitor galaxies. We find that the orbits of GCs deposited by accretion events are sensitive to the mass and merger redshift of the satellite galaxy. Earlier mergers and larger galaxy masses deposit GCs at smaller median apocentres and lower total orbital energy. The orbital properties of accreted groups of GCs can therefore be used to infer the properties of their progenitor galaxy, though there exists a degeneracy between galaxy mass and accretion time. Combining GC orbits with other tracers (GC ages, metallicities) will help to break the galaxy mass/accretion time degeneracy, enabling stronger constraints on the properties of their progenitor galaxy. In situ GCs generally orbit at lower energies (small apocentres) than accreted GCs, however they exhibit a large tail to high energies and even retrograde orbits (relative to the present-day disc), showing significant overlap with accreted GCs. Applying the results to Milky Way GCs groups suggests a merger redshift $z \sim 1.5$ for the Gaia Sausage/Enceladus and $z>2$ for the `low-energy'/Kraken group, adding further evidence that the Milky Way had two significant mergers in its past. △ Less

Submitted 20 October, 2020; v1 submitted 28 February, 2020; originally announced March 2020.

Comments: 13 pages, 7 figures. Accepted for publication in MNRAS (21 September 2020)

Social diversity and social preferences in mixed-motive reinforcement learning

Authors: Kevin R. McKee, Ian Gemp, Brian McWilliams, Edgar A. Duéñez-Guzmán, Edward Hughes, Joel Z. Leibo

Abstract: Recent research on reinforcement learning in pure-conflict and pure-common interest games has emphasized the importance of population heterogeneity. In contrast, studies of reinforcement learning in mixed-motive games have primarily leveraged homogeneous approaches. Given the defining characteristic of mixed-motive games--the imperfect correlation of incentives between group members--we study the… ▽ More Recent research on reinforcement learning in pure-conflict and pure-common interest games has emphasized the importance of population heterogeneity. In contrast, studies of reinforcement learning in mixed-motive games have primarily leveraged homogeneous approaches. Given the defining characteristic of mixed-motive games--the imperfect correlation of incentives between group members--we study the effect of population heterogeneity on mixed-motive reinforcement learning. We draw on interdependence theory from social psychology and imbue reinforcement learning agents with Social Value Orientation (SVO), a flexible formalization of preferences over group outcome distributions. We subsequently explore the effects of diversity in SVO on populations of reinforcement learning agents in two mixed-motive Markov games. We demonstrate that heterogeneity in SVO generates meaningful and complex behavioral variation among agents similar to that suggested by interdependence theory. Empirical results in these mixed-motive dilemmas suggest agents trained in heterogeneous populations develop particularly generalized, high-performing policies relative to those trained in homogeneous populations. △ Less

Submitted 12 February, 2020; v1 submitted 6 February, 2020; originally announced February 2020.

Comments: Proceedings of the 19th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2020)

The Rockerverse: Packages and Applications for Containerization with R

Authors: Daniel Nüst, Dirk Eddelbuettel, Dom Bennett, Robrecht Cannoodt, Dav Clark, Gergely Daroczi, Mark Edmondson, Colin Fay, Ellis Hughes, Lars Kjeldgaard, Sean Lopp, Ben Marwick, Heather Nolis, Jacqueline Nolis, Hong Ooi, Karthik Ram, Noam Ross, Lori Shepherd, Péter Sólymos, Tyson Lee Swetnam, Nitesh Turaga, Charlotte Van Petegem, Jason Williams, Craig Willis, Nan Xiao

Abstract: The Rocker Project provides widely used Docker images for R across different application scenarios. This article surveys downstream projects that build upon the Rocker Project images and presents the current state of R packages for managing Docker images and controlling containers. These use cases cover diverse topics such as package development, reproducible research, collaborative work, cloud-ba… ▽ More The Rocker Project provides widely used Docker images for R across different application scenarios. This article surveys downstream projects that build upon the Rocker Project images and presents the current state of R packages for managing Docker images and controlling containers. These use cases cover diverse topics such as package development, reproducible research, collaborative work, cloud-based data processing, and production deployment of services. The variety of applications demonstrates the power of the Rocker Project specifically and containerisation in general. Across the diverse ways to use containers, we identified common themes: reproducible environments, scalability and efficiency, and portability across clouds. We conclude that the current growth and diversification of use cases is likely to continue its positive impact, but see the need for consolidating the Rockerverse ecosystem of packages, developing common practices for applications, and exploring alternative containerisation software. △ Less

Submitted 17 August, 2020; v1 submitted 28 January, 2020; originally announced January 2020.

Comments: Source code for article available at https://github.com/nuest/rockerverse-paper/ Updated version includes some new paragraphs and corrections throughout the text; full diff available at https://github.com/nuest/rockerverse-paper/compare/preprint.v2...preprint.v3

MSC Class: 68N01 ACM Class: D.2.6; D.2.7; K.6.3

Journal ref: The R Journal (2020), 12:1, pages 437-461

Probing few-body nuclear dynamics via 3H and 3He (e,e'p)pn cross-section measurements

Authors: R. Cruz-Torres, D. Nguyen, F. Hauenstein, A. Schmidt, S. Li, D. Abrams, H. Albataineh, S. Alsalmi, D. Androic, K. Aniol, W. Armstrong, J. Arrington, H. Atac, T. Averett, C. Ayerbe Gayoso, X. Bai, J. Bane, S. Barcus, A. Beck, V. Bellini, F. Benmokhtar, H. Bhatt, D. Bhetuwal, D. Biswas, D. Blyth , et al. (103 additional authors not shown)

Abstract: We report the first measurement of the \eep three-body breakup reaction cross sections in helium-3 ($^3$He) and tritium ($^3$H) at large momentum transfer ($\langle Q^2 \rangle \approx 1.9$ (GeV/c)$^2$) and $x_B>1$ kinematics, where the cross section should be sensitive to quasielastic (QE) scattering from single nucleons. The data cover missing momenta $40 \le p_{miss} \le 500$ MeV/c that, in the… ▽ More We report the first measurement of the \eep three-body breakup reaction cross sections in helium-3 ($^3$He) and tritium ($^3$H) at large momentum transfer ($\langle Q^2 \rangle \approx 1.9$ (GeV/c)$^2$) and $x_B>1$ kinematics, where the cross section should be sensitive to quasielastic (QE) scattering from single nucleons. The data cover missing momenta $40 \le p_{miss} \le 500$ MeV/c that, in the QE limit with no rescattering, equals the initial momentum of the probed nucleon. The measured cross sections are compared with state-of-the-art ab-initio calculations. Overall good agreement, within $\pm20\%$, is observed between data and calculations for the full $p_{miss}$ range for $^3$H and for $100 \le p_{miss} \le 350$ MeV/c for $^3$He. Including the effects of rescattering of the outgoing nucleon improves agreement with the data at $p_{miss} > 250$ MeV/c and suggests contributions from charge-exchange (SCX) rescattering. The isoscalar sum of $^3$He plus $^3$H, which is largely insensitive to SCX, is described by calculations to within the accuracy of the data over the entire $p_{miss}$ range. This validates current models of the ground state of the three-nucleon system up to very high initial nucleon momenta of $500$ MeV/c. △ Less

Submitted 17 June, 2020; v1 submitted 20 January, 2020; originally announced January 2020.

Comments: Accepted for publication in PRL. 8 pages, 3 figures, and online supplementary materials

Journal ref: Phys. Rev. Lett. 124, 212501 (2020)

Smooth markets: A basic mechanism for organizing gradient-based learners

Authors: David Balduzzi, Wojciech M Czarnecki, Thomas W Anthony, Ian M Gemp, Edward Hughes, Joel Z Leibo, Georgios Piliouras, Thore Graepel

Abstract: With the success of modern machine learning, it is becoming increasingly important to understand and control how learning algorithms interact. Unfortunately, negative results from game theory show there is little hope of understanding or controlling general n-player games. We therefore introduce smooth markets (SM-games), a class of n-player games with pairwise zero sum interactions. SM-games codi… ▽ More With the success of modern machine learning, it is becoming increasingly important to understand and control how learning algorithms interact. Unfortunately, negative results from game theory show there is little hope of understanding or controlling general n-player games. We therefore introduce smooth markets (SM-games), a class of n-player games with pairwise zero sum interactions. SM-games codify a common design pattern in machine learning that includes (some) GANs, adversarial training, and other recent algorithms. We show that SM-games are amenable to analysis and optimization using first-order methods. △ Less

Submitted 18 January, 2020; v1 submitted 14 January, 2020; originally announced January 2020.

Comments: 18 pages, 3 figures

Journal ref: ICLR 2020