Showing 1–50 of 78 results for author: Reed, S

The iterative persuasion-polarization opinion dynamics and its mean-field analysis

Authors: Fei Cao, Stephanie Reed

Abstract: In this paper, we introduce the Iterative Persuasion-Polarization (IPP) model to study the dynamics of opinion formation and change within a population. The IPP model integrates mechanisms of persuasion and repulsion, where individuals influence each other through interactions that can either align opinions incrementally or lead to greater divergence. The probability of each interaction type is go… ▽ More In this paper, we introduce the Iterative Persuasion-Polarization (IPP) model to study the dynamics of opinion formation and change within a population. The IPP model integrates mechanisms of persuasion and repulsion, where individuals influence each other through interactions that can either align opinions incrementally or lead to greater divergence. The probability of each interaction type is governed by a parameter $α$, representing the population's receptiveness to persuasion. We investigate how these interaction dynamics shape the long-term distribution of opinions, examining conditions that promote consensus or polarization. By deriving a system of nonlinear and autonomous ordinary differential equations (ODEs), we provide a rigorous mathematical framework for analyzing the distributional behavior of opinions in large populations. Our findings contribute to a deeper understanding of social influence dynamics and their implications in complex social systems. △ Less

Submitted 31 July, 2024; originally announced August 2024.

Comments: 30 pages, 7 figures

MSC Class: 91D25; 91D30; 82C31; 82C22; 35Q91

Genie: Generative Interactive Environments

Authors: Jake Bruce, Michael Dennis, Ashley Edwards, Jack Parker-Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, Yusuf Aytar, Sarah Bechtle, Feryal Behbahani, Stephanie Chan, Nicolas Heess, Lucy Gonzalez, Simon Osindero, Sherjil Ozair, Scott Reed, Jingwei Zhang, Konrad Zolna, Jeff Clune, Nando de Freitas, Satinder Singh, Tim Rocktäschel

Abstract: We introduce Genie, the first generative interactive environment trained in an unsupervised manner from unlabelled Internet videos. The model can be prompted to generate an endless variety of action-controllable virtual worlds described through text, synthetic images, photographs, and even sketches. At 11B parameters, Genie can be considered a foundation world model. It is comprised of a spatiotem… ▽ More We introduce Genie, the first generative interactive environment trained in an unsupervised manner from unlabelled Internet videos. The model can be prompted to generate an endless variety of action-controllable virtual worlds described through text, synthetic images, photographs, and even sketches. At 11B parameters, Genie can be considered a foundation world model. It is comprised of a spatiotemporal video tokenizer, an autoregressive dynamics model, and a simple and scalable latent action model. Genie enables users to act in the generated environments on a frame-by-frame basis despite training without any ground-truth action labels or other domain-specific requirements typically found in the world model literature. Further the resulting learned latent action space facilitates training agents to imitate behaviors from unseen videos, opening the path for training generalist agents of the future. △ Less

Submitted 23 February, 2024; originally announced February 2024.

Comments: https://sites.google.com/corp/view/genie-2024/

A biased dollar exchange model involving bank and debt with discontinuous equilibrium

Authors: Fei Cao, Stephanie Reed

Abstract: In this work, we investigate a biased dollar exchange model with collective debt limit, in which agents picked at random (with a rate depending on the amount of dollars they have) give at random time a dollar to another agent being picked uniformly at random, as long as they have at least one dollar in their pockets or they can borrow a dollar from a central bank if the bank is not empty. This dyn… ▽ More In this work, we investigate a biased dollar exchange model with collective debt limit, in which agents picked at random (with a rate depending on the amount of dollars they have) give at random time a dollar to another agent being picked uniformly at random, as long as they have at least one dollar in their pockets or they can borrow a dollar from a central bank if the bank is not empty. This dynamics enjoys a mean-field type interaction and partially extends the recent work \cite{cao_uncovering_2022} on a related model. We perform a formal mean-field analysis as the number of agents grows to infinity and as a by-product we discover a two-phase (ODE) dynamics behind the underlying stochastic $N$-agents dynamics. Numerical experiments on the two-phase (ODE) dynamics are also conducted where we observe the convergence towards its unique equilibrium in the large time limit. △ Less

Submitted 13 November, 2023; originally announced November 2023.

Comments: 23 pages, 5 figures. arXiv admin note: substantial text overlap with arXiv:2208.11003

MSC Class: 91B70; 91B80; 82C22; 82C31

RoboCat: A Self-Improving Generalist Agent for Robotic Manipulation

Authors: Konstantinos Bousmalis, Giulia Vezzani, Dushyant Rao, Coline Devin, Alex X. Lee, Maria Bauza, Todor Davchev, Yuxiang Zhou, Agrim Gupta, Akhil Raju, Antoine Laurens, Claudio Fantacci, Valentin Dalibard, Martina Zambelli, Murilo Martins, Rugile Pevceviciute, Michiel Blokzijl, Misha Denil, Nathan Batchelor, Thomas Lampe, Emilio Parisotto, Konrad Żołna, Scott Reed, Sergio Gómez Colmenarejo, Jon Scholz , et al. (14 additional authors not shown)

Abstract: The ability to leverage heterogeneous robotic experience from different robots and tasks to quickly master novel skills and embodiments has the potential to transform robot learning. Inspired by recent advances in foundation models for vision and language, we propose a multi-embodiment, multi-task generalist agent for robotic manipulation. This agent, named RoboCat, is a visual goal-conditioned de… ▽ More The ability to leverage heterogeneous robotic experience from different robots and tasks to quickly master novel skills and embodiments has the potential to transform robot learning. Inspired by recent advances in foundation models for vision and language, we propose a multi-embodiment, multi-task generalist agent for robotic manipulation. This agent, named RoboCat, is a visual goal-conditioned decision transformer capable of consuming action-labelled visual experience. This data spans a large repertoire of motor control skills from simulated and real robotic arms with varying sets of observations and actions. With RoboCat, we demonstrate the ability to generalise to new tasks and robots, both zero-shot as well as through adaptation using only 100-1000 examples for the target task. We also show how a trained model itself can be used to generate data for subsequent training iterations, thus providing a basic building block for an autonomous improvement loop. We investigate the agent's capabilities, with large-scale evaluations both in simulation and on three different real robot embodiments. We find that as we grow and diversify its training data, RoboCat not only shows signs of cross-task transfer, but also becomes more efficient at adapting to new tasks. △ Less

Submitted 22 December, 2023; v1 submitted 20 June, 2023; originally announced June 2023.

Comments: Transactions on Machine Learning Research (12/2023)

GA-NIFS: Black hole and host galaxy properties of two z$\simeq$6.8 quasars from the NIRSpec IFU

Authors: Madeline A. Marshall, Michele Perna, Chris J. Willott, Roberto Maiolino, Jan Scholtz, Hannah Übler, Stefano Carniani, Santiago Arribas, Nora Lützgendorf, Andrew J. Bunker, Stephane Charlot, Pierre Ferruit, Peter Jakobsen, Hans-Walter Rix, Bruno Rodríguez Del Pino, Torsten Böker, Alex J. Cameron, Giovanni Cresci, Emma Curtis-Lake, Gareth C. Jones, Nimisha Kumari, Pablo G. Pérez-González, Sophie L. Reed

Abstract: Integral Field Spectroscopy (IFS) with JWST NIRSpec will significantly improve our understanding of the first quasars, by providing spatially resolved, infrared spectroscopic capabilities which cover key rest-frame optical emission lines that have been previously unobservable. Here we present our results from the first two z>6 quasars observed as a part of the Galaxy Assembly with NIRSpec IFS (GA-… ▽ More Integral Field Spectroscopy (IFS) with JWST NIRSpec will significantly improve our understanding of the first quasars, by providing spatially resolved, infrared spectroscopic capabilities which cover key rest-frame optical emission lines that have been previously unobservable. Here we present our results from the first two z>6 quasars observed as a part of the Galaxy Assembly with NIRSpec IFS (GA-NIFS) GTO program, DELS J0411-0907 at z=6.82 and VDES J0020-3653 at z=6.86. By observing the H$β$, [OIII], and H$α$ emission lines in these high-z quasars for the first time, we measure accurate black hole masses, $M_{\rm{BH}}=1.85e9$ and $2.9e9$M$_\odot$, corresponding to Eddington ratios of $λ_{\rm{Edd}}=0.8$ and 0.4 for DELS J0411-0907 and VDES J0020-3653 respectively. These provide a key comparison for existing estimates from the more uncertain MgII line. We perform quasar-host decomposition using models of the quasars' broad lines to measure the underlying host galaxies. We also discover multiple emission line regions surrounding each of the host galaxies, which are likely companion galaxies undergoing mergers with these hosts. We measure the star formation rates, excitation mechanisms, and dynamical masses of the hosts and companions, measuring the $M_{\rm{BH}}/M_{\rm{dyn}}$ ratios at high-z using these estimators for the first time. DELS J0411-0907 and VDES J0020-3653 both lie above the local black hole--host mass relation, and are consistent with the existing observations of $z\gtrsim6$ quasar host galaxies with ALMA. We detect ionized outflows in [OIII] and H$β$ from both quasars, with mass outflow rates of 58 and 525 M$_{\odot}$/yr for DELS J0411-0907 and VDES J0020-3653, much larger than their host star formation rates of <33 and <54 M$_\odot$/yr. This work highlights the exceptional capabilities of the JWST NIRSpec IFU for observing quasars in the early Universe. △ Less

Submitted 2 August, 2023; v1 submitted 9 February, 2023; originally announced February 2023.

Comments: 27 pages, 10 figures. Resubmitted to A&A after significant revisions. If you have cited values from our first version, please check this version and update accordingly, as many values have changed slightly thanks to improvements in our analysis

Journal ref: A&A 678, A191 (2023)

Development and Validation of ML-DQA -- a Machine Learning Data Quality Assurance Framework for Healthcare

Authors: Mark Sendak, Gaurav Sirdeshmukh, Timothy Ochoa, Hayley Premo, Linda Tang, Kira Niederhoffer, Sarah Reed, Kaivalya Deshpande, Emily Sterrett, Melissa Bauer, Laurie Snyder, Afreen Shariff, David Whellan, Jeffrey Riggio, David Gaieski, Kristin Corey, Megan Richards, Michael Gao, Marshall Nichols, Bradley Heintze, William Knechtle, William Ratliff, Suresh Balu

Abstract: The approaches by which the machine learning and clinical research communities utilize real world data (RWD), including data captured in the electronic health record (EHR), vary dramatically. While clinical researchers cautiously use RWD for clinical investigations, ML for healthcare teams consume public datasets with minimal scrutiny to develop new algorithms. This study bridges this gap by devel… ▽ More The approaches by which the machine learning and clinical research communities utilize real world data (RWD), including data captured in the electronic health record (EHR), vary dramatically. While clinical researchers cautiously use RWD for clinical investigations, ML for healthcare teams consume public datasets with minimal scrutiny to develop new algorithms. This study bridges this gap by developing and validating ML-DQA, a data quality assurance framework grounded in RWD best practices. The ML-DQA framework is applied to five ML projects across two geographies, different medical conditions, and different cohorts. A total of 2,999 quality checks and 24 quality reports were generated on RWD gathered on 247,536 patients across the five projects. Five generalizable practices emerge: all projects used a similar method to group redundant data element representations; all projects used automated utilities to build diagnosis and medication data elements; all projects used a common library of rules-based transformations; all projects used a unified approach to assign data quality checks to data elements; and all projects used a similar approach to clinical adjudication. An average of 5.8 individuals, including clinicians, data scientists, and trainees, were involved in implementing ML-DQA for each project and an average of 23.4 data elements per project were either transformed or removed in response to ML-DQA. This study demonstrates the importance role of ML-DQA in healthcare projects and provides teams a framework to conduct these essential activities. △ Less

Submitted 4 August, 2022; originally announced August 2022.

Comments: Presented at 2022 Machine Learning in Health Care Conference

Faro: A framework for measuring the scientific performance of petascale Rubin Observatory data products

Authors: Leanne P. Guy, Keith Bechtol, Jeffrey L. Carlin, Erik Dennihy, Peter S. Ferguson, K. Simon Krughoff, Robert H. Lupton, Colin T. Slater, Krzysztof Findeisen, Arun Kannawadi, Lee S. Kelvin, Nate B. Lust, Lauren A. MacArthur, Michael N. Martinez, Sophie L. Reed, Dan S. Taranu, W. Michael Wood-Vasey

Abstract: The Vera C. Rubin Observatory will advance many areas of astronomy over the next decade with its unique wide-fast-deep multi-color imaging survey, the Legacy Survey of Space and Time (LSST). The LSST will produce approximately 20TB of raw data per night, which will be automatically processed by the LSST Science Pipelines to generate science-ready data products -- processed images, catalogs and ale… ▽ More The Vera C. Rubin Observatory will advance many areas of astronomy over the next decade with its unique wide-fast-deep multi-color imaging survey, the Legacy Survey of Space and Time (LSST). The LSST will produce approximately 20TB of raw data per night, which will be automatically processed by the LSST Science Pipelines to generate science-ready data products -- processed images, catalogs and alerts. To ensure that these data products enable transformative science with LSST, stringent requirements have been placed on their quality and scientific fidelity, for example on image quality and depth, astrometric and photometric performance, and object recovery completeness. In this paper we introduce faro, a framework for automatically and efficiently computing scientific performance metrics on the LSST data products for units of data of varying granularity, ranging from single-detector to full-survey summary statistics. By measuring and monitoring metrics, we are able to evaluate trends in algorithmic performance and conduct regression testing during development, compare the performance of one algorithm against another, and verify that the LSST data products will meet performance requirements by comparing to specifications. We present initial results using faro to characterize the performance of the data products produced on simulated and precursor data sets, and discuss plans to use faro to verify the performance of the LSST commissioning data products. △ Less

Submitted 30 June, 2022; originally announced June 2022.

Comments: 12 pages, 9 figues, Proc. SPIE 12189, Software and Cyberinfrastructure for Astronomy VII, Montreal, CA, July 2022

Report number: SPIE Paper Number: 12189-24

A Generalist Agent

Authors: Scott Reed, Konrad Zolna, Emilio Parisotto, Sergio Gomez Colmenarejo, Alexander Novikov, Gabriel Barth-Maron, Mai Gimenez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, Tom Eccles, Jake Bruce, Ali Razavi, Ashley Edwards, Nicolas Heess, Yutian Chen, Raia Hadsell, Oriol Vinyals, Mahyar Bordbar, Nando de Freitas

Abstract: Inspired by progress in large-scale language modeling, we apply a similar approach towards building a single generalist agent beyond the realm of text outputs. The agent, which we refer to as Gato, works as a multi-modal, multi-task, multi-embodiment generalist policy. The same network with the same weights can play Atari, caption images, chat, stack blocks with a real robot arm and much more, dec… ▽ More Inspired by progress in large-scale language modeling, we apply a similar approach towards building a single generalist agent beyond the realm of text outputs. The agent, which we refer to as Gato, works as a multi-modal, multi-task, multi-embodiment generalist policy. The same network with the same weights can play Atari, caption images, chat, stack blocks with a real robot arm and much more, deciding based on its context whether to output text, joint torques, button presses, or other tokens. In this report we describe the model and the data, and document the current capabilities of Gato. △ Less

Submitted 11 November, 2022; v1 submitted 12 May, 2022; originally announced May 2022.

Comments: Published at TMLR, 42 pages

Journal ref: Transactions on Machine Learning Research, 11/2022, https://openreview.net/forum?id=1ikK0kHjvj

Distribution of money on connected graphs with multiple banks

Authors: Nicolas Lanchier, Stephanie Reed

Abstract: This paper studies an interacting particle system of interest in econophysics inspired from a model introduced in the physics literature. The original model consists of the customers of a single bank characterized by their capital, and the discrete-time dynamics consists of monetary transactions in which a random individual $x$ gives one coin to another random individual $y$, the transaction being… ▽ More This paper studies an interacting particle system of interest in econophysics inspired from a model introduced in the physics literature. The original model consists of the customers of a single bank characterized by their capital, and the discrete-time dynamics consists of monetary transactions in which a random individual $x$ gives one coin to another random individual $y$, the transaction being canceled when $x$ is in debt and there is no more coins to borrow from the bank. Using a combination of numerical simulations and heuristic arguments, physicists conjectured that the distribution of money (the distribution of the number of coins owned by a given individual) at equilibrium converges to an asymmetric Laplace distribution in the large population/temperature limit. In this paper, we prove and extend this conjecture to a more general model including multiple banks and interactions among customers across banks. More importantly, our model assumes that customers are located on a general undirected connected graph (as opposed to the complete graph in the original model) where neighbors are interpreted as business partners, and transactions occur along the edges, thus modeling the flow of money across a social network. We show the convergence to the asymmetric Laplace distribution in the large population/temperature limit for any graph, thus proving and extending the conjecture from the physicists, and derive an exact expression of the distribution of money for all population sizes and money temperatures. △ Less

Submitted 27 January, 2022; originally announced January 2022.

Comments: 22 pages, 2 figures

MSC Class: 60K35; 91B72

Shaking the foundations: delusions in sequence models for interaction and control

Authors: Pedro A. Ortega, Markus Kunesch, Grégoire Delétang, Tim Genewein, Jordi Grau-Moya, Joel Veness, Jonas Buchli, Jonas Degrave, Bilal Piot, Julien Perolat, Tom Everitt, Corentin Tallec, Emilio Parisotto, Tom Erez, Yutian Chen, Scott Reed, Marcus Hutter, Nando de Freitas, Shane Legg

Abstract: The recent phenomenal success of language models has reinvigorated machine learning research, and large sequence models such as transformers are being applied to a variety of domains. One important problem class that has remained relatively elusive however is purposeful adaptive behavior. Currently there is a common perception that sequence models "lack the understanding of the cause and effect of… ▽ More The recent phenomenal success of language models has reinvigorated machine learning research, and large sequence models such as transformers are being applied to a variety of domains. One important problem class that has remained relatively elusive however is purposeful adaptive behavior. Currently there is a common perception that sequence models "lack the understanding of the cause and effect of their actions" leading them to draw incorrect inferences due to auto-suggestive delusions. In this report we explain where this mismatch originates, and show that it can be resolved by treating actions as causal interventions. Finally, we show that in supervised learning, one can teach a system to condition or intervene on data by training with factual and counterfactual error signals respectively. △ Less

Submitted 20 October, 2021; originally announced October 2021.

Comments: DeepMind Tech Report, 16 pages, 4 figures

Third Data Release of the Hyper Suprime-Cam Subaru Strategic Program

Authors: Hiroaki Aihara, Yusra AlSayyad, Makoto Ando, Robert Armstrong, James Bosch, Eiichi Egami, Hisanori Furusawa, Junko Furusawa, Sumiko Harasawa, Yuichi Harikane, Bau-Ching Hsieh, Hiroyuki Ikeda, Kei Ito, Ikuru Iwata, Tadayuki Kodama, Michitaro Koike, Mitsuru Kokubo, Yutaka Komiyama, Xiangchong Li, Yongming Liang, Yen-Ting Lin, Robert H. Lupton, Nate B Lust, Lauren A. MacArthur, Ken Mawatari , et al. (42 additional authors not shown)

Abstract: The paper presents the third data release of Hyper Suprime-Cam Subaru Strategic Program (HSC-SSP), a wide-field multi-band imaging survey with the Subaru 8.2m telescope. HSC-SSP has three survey layers (Wide, Deep, and UltraDeep) with different area coverages and depths, designed to address a wide array of astrophysical questions. This third release from HSC-SSP includes data from 278 nights of ob… ▽ More The paper presents the third data release of Hyper Suprime-Cam Subaru Strategic Program (HSC-SSP), a wide-field multi-band imaging survey with the Subaru 8.2m telescope. HSC-SSP has three survey layers (Wide, Deep, and UltraDeep) with different area coverages and depths, designed to address a wide array of astrophysical questions. This third release from HSC-SSP includes data from 278 nights of observing time and covers about 670 square degrees in all five broad-band filters at the full depth ($\sim26$~mag at $5σ$) in the Wide layer. If we include partially observed area, the release covers 1,470 square degrees. The Deep and UltraDeep layers have $\sim80\%$ of the originally planned integration times, and are considered done, as we have slightly changed the observing strategy in order to compensate for various time losses. There are a number of updates in the image processing pipeline. Of particular importance is the change in the sky subtraction algorithm; we subtract the sky on small scales before the detection and measurement stages, which has significantly reduced false detections. Thanks to this and other updates, the overall quality of the processed data has improved since the previous release. However, there are limitations in the data (for example, the pipeline is not optimized for crowded fields), and we encourage the user to check the quality assurance plots as well as a list of known issues before exploiting the data. The data release website is https://hsc-release.mtk.nao.ac.jp/. △ Less

Submitted 30 August, 2021; originally announced August 2021.

Comments: 25 pages, 19 figures, submitted to PASJ. Data available at https://hsc-release.mtk.nao.ac.jp/

Effects of Steric Factors on Molecular Doping to MoS$_2$

Authors: Serrae N. Reed, Yifeng Chen, Milad Yarali, David J. Charboneau, Julia B. Curley, Nilay Hazari, Su Ying Quek, Judy J. Cha

Abstract: Surface functionalization of two-dimensional (2D) materials with organic electron donors (OEDs) is a powerful method to modulate the electronic properties of the material. However, our fundamental understanding of the doping mechanism is largely limited to the categorization of molecular dopants as n- or p-type based on the relative position of the molecule's redox potential in relation to the Fer… ▽ More Surface functionalization of two-dimensional (2D) materials with organic electron donors (OEDs) is a powerful method to modulate the electronic properties of the material. However, our fundamental understanding of the doping mechanism is largely limited to the categorization of molecular dopants as n- or p-type based on the relative position of the molecule's redox potential in relation to the Fermi level of the 2D host. Our limited knowledge about the impact of factors other than the redox properties of the molecule on doping makes it challenging to controllably use molecules to dope 2D materials and design new OEDs. Here, we functionalize monolayer MoS$_2$ using two molecular dopants, Me- and $^t$Bu-OED, which have the same redox potential but different steric properties to probe the effects of molecular size on the doping level of MoS$_2$. We show that, for the same functionalization conditions, the doping powers of Me- and $^t$Bu-OED are 0.22 - 0.44 and 0.11 electrons per molecule, respectively, demonstrating that the steric properties of the molecule critically affect doping levels. Using the stronger dopant, Me-OED, a carrier density of 1.10 +/- 0.37 x 10$^{14}$ cm$^{-2}$ is achieved in MoS$_2$, the highest doping level to date for MoS$_2$ using surface functionalization. Overall, we establish that tuning of the steric properties of the dopant is essential in the rational design of molecular dopants. △ Less

Submitted 5 August, 2021; v1 submitted 26 July, 2021; originally announced July 2021.

Comments: 5 figures and 1 table

Does Parking Matter? The Impact of Search Time for Parking on Last-Mile Delivery Optimization

Authors: Sara Reed, Ann Melissa Campbell, Barrett W. Thomas

Abstract: Parking is a necessary component of traditional last-mile delivery practices, but finding parking can be difficult. Yet, the routing literature largely does not account for the need to find parking. In this paper, we address this challenge of finding parking through the Capacitated Delivery Problem with Parking (CDPP). Unlike other models in the literature, the CDPP accounts for the search time fo… ▽ More Parking is a necessary component of traditional last-mile delivery practices, but finding parking can be difficult. Yet, the routing literature largely does not account for the need to find parking. In this paper, we address this challenge of finding parking through the Capacitated Delivery Problem with Parking (CDPP). Unlike other models in the literature, the CDPP accounts for the search time for parking in the objective and minimizes the completion time of the delivery tour. When we restrict the customer geography to a complete grid, we identify conditions for when a Traveling Salesman Problem (TSP) solution that parks at each customer is an optimal solution to the CDPP. We then determine when the search time for parking is large enough for the CDPP optimal solution to differ from this TSP solution. We also identify model improvements that allow reasonably-sized instances of the CDPP to be solved exactly. We introduce a heuristic for the CDPP that quickly finds high quality solutions to large instances. Computational experiments show that parking matters in last-mile delivery optimization. The CDPP outperforms industry practice and models in the literature showing the greatest advantage when the search time for parking is high. This analysis provides immediate ways to improve routing in last-mile delivery. △ Less

Submitted 31 January, 2022; v1 submitted 14 July, 2021; originally announced July 2021.

Comments: 48 pages, 14 figures

MSC Class: 90B06

Semi-supervised reward learning for offline reinforcement learning

Authors: Ksenia Konyushkova, Konrad Zolna, Yusuf Aytar, Alexander Novikov, Scott Reed, Serkan Cabi, Nando de Freitas

Abstract: In offline reinforcement learning (RL) agents are trained using a logged dataset. It appears to be the most natural route to attack real-life applications because in domains such as healthcare and robotics interactions with the environment are either expensive or unethical. Training agents usually requires reward functions, but unfortunately, rewards are seldom available in practice and their engi… ▽ More In offline reinforcement learning (RL) agents are trained using a logged dataset. It appears to be the most natural route to attack real-life applications because in domains such as healthcare and robotics interactions with the environment are either expensive or unethical. Training agents usually requires reward functions, but unfortunately, rewards are seldom available in practice and their engineering is challenging and laborious. To overcome this, we investigate reward learning under the constraint of minimizing human reward annotations. We consider two types of supervision: timestep annotations and demonstrations. We propose semi-supervised learning algorithms that learn from limited annotations and incorporate unlabelled data. In our experiments with a simulated robotic arm, we greatly improve upon behavioural cloning and closely approach the performance achieved with ground truth rewards. We further investigate the relationship between the quality of the reward model and the final policies. We notice, for example, that the reward models do not need to be perfect to result in useful policies. △ Less

Submitted 12 December, 2020; originally announced December 2020.

Comments: Accepted to Offline Reinforcement Learning Workshop at Neural Information Processing Systems (2020)

Offline Learning from Demonstrations and Unlabeled Experience

Authors: Konrad Zolna, Alexander Novikov, Ksenia Konyushkova, Caglar Gulcehre, Ziyu Wang, Yusuf Aytar, Misha Denil, Nando de Freitas, Scott Reed

Abstract: Behavior cloning (BC) is often practical for robot learning because it allows a policy to be trained offline without rewards, by supervised learning on expert demonstrations. However, BC does not effectively leverage what we will refer to as unlabeled experience: data of mixed and unknown quality without reward annotations. This unlabeled data can be generated by a variety of sources such as human… ▽ More Behavior cloning (BC) is often practical for robot learning because it allows a policy to be trained offline without rewards, by supervised learning on expert demonstrations. However, BC does not effectively leverage what we will refer to as unlabeled experience: data of mixed and unknown quality without reward annotations. This unlabeled data can be generated by a variety of sources such as human teleoperation, scripted policies and other agents on the same robot. Towards data-driven offline robot learning that can use this unlabeled experience, we introduce Offline Reinforced Imitation Learning (ORIL). ORIL first learns a reward function by contrasting observations from demonstrator and unlabeled trajectories, then annotates all data with the learned reward, and finally trains an agent via offline reinforcement learning. Across a diverse set of continuous control and simulated robotic manipulation tasks, we show that ORIL consistently outperforms comparable BC agents by effectively leveraging unlabeled experience. △ Less

Submitted 27 November, 2020; originally announced November 2020.

Comments: Accepted to Offline Reinforcement Learning Workshop at Neural Information Processing Systems (2020)

The visual lightcurve of comet C/1995 O1 (Hale-Bopp) from 1995-1999

Authors: M. Womack, O. Curtis, D. A. Rabson, O. Harrington Pinto, K. Wierzchos, S. Cruz Gonzalez, G. Sarid, C. Mentzer, N. Lastra, N. Pichette, N. Ruffini, T. Cox, I. Rivera, A. Micciche, C. Jackson, A. Homich, S. Rosslyn Escoto, T. Erdahl, Marcel P. Goldschen-Ohm, A. Tollison, S. Reed, J. Zilka, B. Henning, M. Spinar, W. T. Uhl

Abstract: The long-term brightness evolution of the great comet C/1995 O1 (Hale-Bopp) presented a remarkable opportunity to study the behavior of its coma over four years. We used approximately 2200 total visual magnitudes published in the International Comet Quarterly taken from 17 observers during the period of 1995 July - 1999 September to create a secular lightcurve. In order to account for observer dif… ▽ More The long-term brightness evolution of the great comet C/1995 O1 (Hale-Bopp) presented a remarkable opportunity to study the behavior of its coma over four years. We used approximately 2200 total visual magnitudes published in the International Comet Quarterly taken from 17 observers during the period of 1995 July - 1999 September to create a secular lightcurve. In order to account for observer differences, we present a novel algorithm to reduce scatter and increase precision in a lightcurve compiled from many sources. It is implemented in a publicly available code, ICQSPLITTER. This code addresses the differences among observers by using a self-consistent statistical approach, leading to a sharper lightcurve, and improving the precision of the measured slopes. To first order, the comet's lightcurve approximately follows a r$^{-4}$ response for both pre- and post-perihelion distances. Interestingly, the pre-perihelion data are better fit with a fifth-order polynomial with inflection points at 4.0, 2.6, 2.1 and 1.1 au. We analyze these specific regions and find that they are associated with physical phenomena in the comet's evolution. Contrary to other reports, the lightcurve shows no evidence for the comet having been in outburst at discovery. Afrho values derived from the visual lightcurve data are consistent with a r$^{-1.5}$ dependence on heliocentric distance, which is similar in shape to those derived from spectroscopy and narrow-band photometry. We present correlation equations for visual magnitudes and CO and H2O production rates, which are consistent with the pre-perihelion visual magnitudes increasing almost entirely due to CO outgassing until a heliocentric distance of about 2.6 - 3.0 au. We also present two correlation equations that should prove highly useful for observation planning and data analysis, and can be generalized to be applicable to other comets. △ Less

Submitted 15 August, 2020; originally announced August 2020.

Comments: 30 pages, 7 figures, 4 tables, submitted to the AAS Journals

The Effect of Mechanical Strain on Lithium Staging in Graphene

Authors: Joshua V. Pondick, Sajad Yazdani, Milad Yarali, Serrae N. Reed, David J. Hynek, Judy J. Cha

Abstract: Lithium intercalation into graphite is the foundation for the lithium-ion battery, and the thermodynamics of the lithiation of graphitic electrodes have been heavily investigated. Intercalated lithium in bulk graphite undergoes structural ordering known as staging to minimize electrostatic repulsions within the crystal lattice. While this process is well-understood for bulk graphite, confinement e… ▽ More Lithium intercalation into graphite is the foundation for the lithium-ion battery, and the thermodynamics of the lithiation of graphitic electrodes have been heavily investigated. Intercalated lithium in bulk graphite undergoes structural ordering known as staging to minimize electrostatic repulsions within the crystal lattice. While this process is well-understood for bulk graphite, confinement effects become important at the nanoscale, which can significantly impact the electrochemistry of nanostructured electrodes. Therefore, graphene offers a model platform to study intercalation dynamics at the nanoscale by combining on-chip device fabrication and electrochemical intercalation with in situ characterization. We show that microscale mechanical strain significantly affects the formation of ordered lithium phases in graphene. In situ Raman spectroscopy of graphene microflakes mechanically constrained at the edge during lithium intercalation reveals a thickness-dependent increase of up to 1.26 V in the electrochemical potential that induces lithium staging. While the induced mechanical strain energy increases with graphene thickness to the fourth power, its magnitude is small compared to the observed increase in electrochemical energy. We hypothesize that the mechanical strain energy increases a nucleation barrier for lithium staging, greatly delaying the formation of ordered lithium phases. Our results indicate that electrode assembly can critically impact lithium staging dynamics important for cycling rates and power generation for batteries. We demonstrate strain engineering in two-dimensional nanomaterials as an approach to manipulate phase transitions and chemical reactivity. △ Less

Submitted 6 August, 2020; originally announced August 2020.

Comments: 4 figures

Journal ref: Advanced Electronic Materials 2021

Near Unity Molecular Doping Efficiency in Monolayer MoS2

Authors: Milad Yarali, Yiren Zhong, Serrae N. Reed, Juefan Wang, Kanchan A. Ulman, David J. Charboneau, Julia B. Curley, David J. Hynek, Joshua V. Pondick, Sajad Yazdani, Nilay Hazari, Su Ying Quek, Hailiang Wang, Judy J. Cha

Abstract: Surface functionalization with organic electron donors (OEDs) is an effective doping strategy for two-dimensional (2D) materials, which can achieve doping levels beyond those possible with conventional electric field gating. While the effectiveness of surface functionalization has been demonstrated in many 2D systems, the doping efficiencies of OEDs have largely been unmeasured, which is in stark… ▽ More Surface functionalization with organic electron donors (OEDs) is an effective doping strategy for two-dimensional (2D) materials, which can achieve doping levels beyond those possible with conventional electric field gating. While the effectiveness of surface functionalization has been demonstrated in many 2D systems, the doping efficiencies of OEDs have largely been unmeasured, which is in stark contrast to their precision syntheses and tailored redox potentials. Here, using monolayer MoS2 as a model system and an organic reductant based on 4,4-bipyridine (DMAP-OED) as a strong organic dopant, we establish that the doping efficiency of DMAP-OED to MoS2 is in the range of 0.63 to 1.26 electrons per molecule. We also achieve the highest doping level to date in monolayer MoS2 by surface functionalization and demonstrate that DMAP-OED is a stronger dopant than benzyl viologen, which was the previous best OED dopant. The measured range of the doping efficiency is in good agreement with the values predicted from first-principles calculations. Our work provides a basis for the rational design of OEDs for high-level doping of 2D materials. △ Less

Submitted 7 July, 2020; originally announced July 2020.

Comments: 19 pages, 4 figures, 1 table

Journal ref: Advanced Electronic Materials 2021

Critic Regularized Regression

Authors: Ziyu Wang, Alexander Novikov, Konrad Zolna, Jost Tobias Springenberg, Scott Reed, Bobak Shahriari, Noah Siegel, Josh Merel, Caglar Gulcehre, Nicolas Heess, Nando de Freitas

Abstract: Offline reinforcement learning (RL), also known as batch RL, offers the prospect of policy optimization from large pre-recorded datasets without online environment interaction. It addresses challenges with regard to the cost of data collection and safety, both of which are particularly pertinent to real-world applications of RL. Unfortunately, most off-policy algorithms perform poorly when learnin… ▽ More Offline reinforcement learning (RL), also known as batch RL, offers the prospect of policy optimization from large pre-recorded datasets without online environment interaction. It addresses challenges with regard to the cost of data collection and safety, both of which are particularly pertinent to real-world applications of RL. Unfortunately, most off-policy algorithms perform poorly when learning from a fixed dataset. In this paper, we propose a novel offline RL algorithm to learn policies from data using a form of critic-regularized regression (CRR). We find that CRR performs surprisingly well and scales to tasks with high-dimensional state and action spaces -- outperforming several state-of-the-art offline RL algorithms by a significant margin on a wide range of benchmark tasks. △ Less

Submitted 22 September, 2021; v1 submitted 26 June, 2020; originally announced June 2020.

Comments: 24 pages; presented at NeurIPS 2020

X-ray properties of z>6.5 quasars

Authors: Estelle Pons, Richard G. McMahon, Manda Banerji, Sophie L. Reed

Abstract: We present XMM-Newton X-ray observations and analysis of three DES $z>6.5$ quasars (VDES J0020$-$3653 at $z=6.824$, VDES J0244$-$5008 at $z=6.724$ and VDES J0224$-$4711 at $z=6.526$) and six other quasars with $6.438 < z < 6.747$ from the XMM-Newton public archive. Two of the nine quasars are detected at a high ($>$4$σ$) significance level: VDES J0224$-$4711(z=6.53) at $9σ$ and PSO J159$-$02 (… ▽ More We present XMM-Newton X-ray observations and analysis of three DES $z>6.5$ quasars (VDES J0020$-$3653 at $z=6.824$, VDES J0244$-$5008 at $z=6.724$ and VDES J0224$-$4711 at $z=6.526$) and six other quasars with $6.438 < z < 6.747$ from the XMM-Newton public archive. Two of the nine quasars are detected at a high ($>$4$σ$) significance level: VDES J0224$-$4711(z=6.53) at $9σ$ and PSO J159$-$02 ($z=6.38$) at $8σ$. They have a photon index of $Γ=1.82^{+0.29}_{-0.27}$ and $Γ=1.94^{+0.31}_{-0.29}$ respectively, which is consistent with the mean value of $\sim1.9$ found for quasars at all redshifts. The rest-frame $2-10$ keV luminosity of VDES J0224$-$4711, is $L_{2-10\mathrm{keV}} = (2.92\pm0.43)\times10^{45}\;\mathrm{erg\;s^{-1}}$, which makes this quasar one of the most X-ray luminous quasars at $z>5.5$ and the most X-ray luminous quasar at $z>6.5$, with a luminosity 6 times and 2.5 times larger than ULAS J1120+0641 ($z=7.08$) and ULAS J1342+0928 ($z=7.54$) respectively. The X-ray-to-optical power-law slopes of the nine quasars are consistent with the previously observed anti-correlation of $α_{ox}$ with UV luminosity $L_{2500\mathrmÅ}$. We find no evidence for evolution of $α_{ox}$ with redshift when the anti-correlation with UV luminosity is taken into account. Similar to previous studies at z$\sim$6 we have found remarkably consistent X-ray spectral properties between low (z$\sim$1) and high-redshift quasars. Our results add further evidence to the picture that the observable properties of high luminosity quasars over the UV to X-ray spectral region have not evolved significantly from $z\sim7$ to the present day and that quasars comparable to local versions existed 800 million years after the big bang. △ Less

Submitted 9 October, 2019; originally announced October 2019.

Comments: 8 pages; resubmitted to MNRAS

Task-Relevant Adversarial Imitation Learning

Authors: Konrad Zolna, Scott Reed, Alexander Novikov, Sergio Gomez Colmenarejo, David Budden, Serkan Cabi, Misha Denil, Nando de Freitas, Ziyu Wang

Abstract: We show that a critical vulnerability in adversarial imitation is the tendency of discriminator networks to learn spurious associations between visual features and expert labels. When the discriminator focuses on task-irrelevant features, it does not provide an informative reward signal, leading to poor task performance. We analyze this problem in detail and propose a solution that outperforms sta… ▽ More We show that a critical vulnerability in adversarial imitation is the tendency of discriminator networks to learn spurious associations between visual features and expert labels. When the discriminator focuses on task-irrelevant features, it does not provide an informative reward signal, leading to poor task performance. We analyze this problem in detail and propose a solution that outperforms standard Generative Adversarial Imitation Learning (GAIL). Our proposed method, Task-Relevant Adversarial Imitation Learning (TRAIL), uses constrained discriminator optimization to learn informative rewards. In comprehensive experiments, we show that TRAIL can solve challenging robotic manipulation tasks from pixels by imitating human operators without access to any task rewards, and clearly outperforms comparable baseline imitation agents, including those trained via behaviour cloning and conventional GAIL. △ Less

Submitted 12 November, 2020; v1 submitted 2 October, 2019; originally announced October 2019.

Comments: Accepted to CoRL 2020 (see presentation here: https://youtu.be/ZgQvFGuEgFU )

Scaling data-driven robotics with reward sketching and batch reinforcement learning

Authors: Serkan Cabi, Sergio Gómez Colmenarejo, Alexander Novikov, Ksenia Konyushkova, Scott Reed, Rae Jeong, Konrad Zolna, Yusuf Aytar, David Budden, Mel Vecerik, Oleg Sushkov, David Barker, Jonathan Scholz, Misha Denil, Nando de Freitas, Ziyu Wang

Abstract: We present a framework for data-driven robotics that makes use of a large dataset of recorded robot experience and scales to several tasks using learned reward functions. We show how to apply this framework to accomplish three different object manipulation tasks on a real robot platform. Given demonstrations of a task together with task-agnostic recorded experience, we use a special form of human… ▽ More We present a framework for data-driven robotics that makes use of a large dataset of recorded robot experience and scales to several tasks using learned reward functions. We show how to apply this framework to accomplish three different object manipulation tasks on a real robot platform. Given demonstrations of a task together with task-agnostic recorded experience, we use a special form of human annotation as supervision to learn a reward function, which enables us to deal with real-world tasks where the reward signal cannot be acquired directly. Learned rewards are used in combination with a large dataset of experience from different tasks to learn a robot policy offline using batch RL. We show that using our approach it is possible to train agents to perform a variety of challenging manipulation tasks including stacking rigid objects and handling cloth. △ Less

Submitted 4 June, 2020; v1 submitted 26 September, 2019; originally announced September 2019.

Comments: Project website: https://sites.google.com/view/data-driven-robotics/

Journal ref: Robotics: Science and Systems Conference 2020

Learning Compositional Neural Programs with Recursive Tree Search and Planning

Authors: Thomas Pierrot, Guillaume Ligner, Scott Reed, Olivier Sigaud, Nicolas Perrin, Alexandre Laterre, David Kas, Karim Beguir, Nando de Freitas

Abstract: We propose a novel reinforcement learning algorithm, AlphaNPI, that incorporates the strengths of Neural Programmer-Interpreters (NPI) and AlphaZero. NPI contributes structural biases in the form of modularity, hierarchy and recursion, which are helpful to reduce sample complexity, improve generalization and increase interpretability. AlphaZero contributes powerful neural network guided search alg… ▽ More We propose a novel reinforcement learning algorithm, AlphaNPI, that incorporates the strengths of Neural Programmer-Interpreters (NPI) and AlphaZero. NPI contributes structural biases in the form of modularity, hierarchy and recursion, which are helpful to reduce sample complexity, improve generalization and increase interpretability. AlphaZero contributes powerful neural network guided search algorithms, which we augment with recursion. AlphaNPI only assumes a hierarchical program specification with sparse rewards: 1 when the program execution satisfies the specification, and 0 otherwise. Using this specification, AlphaNPI is able to train NPI models effectively with RL for the first time, completely eliminating the need for strong supervision in the form of execution traces. The experiments show that AlphaNPI can sort as well as previous strongly supervised NPI variants. The AlphaNPI agent is also trained on a Tower of Hanoi puzzle with two disks and is shown to generalize to puzzles with an arbitrary number of disk △ Less

Submitted 13 April, 2021; v1 submitted 30 May, 2019; originally announced May 2019.

Three new VHS-DES Quasars at 6.7 < z < 6.9 and Emission Line Properties at z > 6.5

Authors: S. L. Reed, M. Banerji, G. D. Becker, P. C. Hewett, P. Martini, R. G. McMahon, E. Pons, M. Rauch, T. M. C. Abbott, S. Allam, J. Annis, S. Avila, E. Bertin, D. Brooks, E. Buckley-Geer, A. Carnero Rosell, M. Carrasco Kind, J. Carretero, F. J. Castander, C. E. Cunha, C. B. D'Andrea, L. N. da Costa, J. De Vicente, S. Desai, H. T. Diehl , et al. (36 additional authors not shown)

Abstract: We report the results from a search for z > 6.5 quasars using the Dark Energy Survey (DES) Year 3 dataset combined with the VISTA Hemisphere Survey (VHS) and WISE All-Sky Survey. Our photometric selection method is shown to be highly efficient in identifying clean samples of high-redshift quasars leading to spectroscopic confirmation of three new quasars - VDESJ 0244-5008 (z=6.724), VDESJ 0020-365… ▽ More We report the results from a search for z > 6.5 quasars using the Dark Energy Survey (DES) Year 3 dataset combined with the VISTA Hemisphere Survey (VHS) and WISE All-Sky Survey. Our photometric selection method is shown to be highly efficient in identifying clean samples of high-redshift quasars leading to spectroscopic confirmation of three new quasars - VDESJ 0244-5008 (z=6.724), VDESJ 0020-3653 (z=6.834) and VDESJ 0246-5219 (z=6.90) - which were selected as the highest priority candidates in the survey data without any need for additional follow-up observations. The new quasars span the full range in luminosity covered by other z>6.5 quasar samples (J AB = 20.2 to 21.3; M1450 = -25.6 to -26.6). We have obtained spectroscopic observations in the near infrared for VDESJ 0244-5008 and VDESJ 0020-3653 as well as our previously identified quasar, VDESJ 0224-4711 at z=6.50 from Reed et al. (2017). We use the near infrared spectra to derive virial black-hole masses from the full-width-half-maximum of the MgII line. These black-hole masses are ~ 1 - 2 x 10$^9$M$_\odot$. Combining with the bolometric luminosities of these quasars of L$_{\rm{bol}}\simeq$ 1 - 3 x 10$^{47}$implies that the Eddington ratios are high - $\simeq$0.6-1.1. We consider the C\textrm{\textsc{IV}} emission line properties of the sample and demonstrate that our high-redshift quasars do not have unusual C\textrm{\textsc{IV}} line properties when compared to carefully matched low-redshift samples. Our new DES+VHS $z>6.5$ quasars now add to the growing census of luminous, rapidly accreting supermassive black-holes seen well into the epoch of reionisation. △ Less

Submitted 11 January, 2019; originally announced January 2019.

Comments: 12 pages, 9 figures

Rigorous results for the distribution of money on connected graphs (models with debts)

Authors: Nicolas Lanchier, Stephanie Reed

Abstract: In this paper, we continue our analysis of spatial versions of agent-based models for the dynamics of money that have been introduced in the statistical physics literature, focusing on two models with debts. Both models consist of systems of economical agents located on a finite connected graph representing a social network. Each agent is characterized by the number of coins she has, which can be… ▽ More In this paper, we continue our analysis of spatial versions of agent-based models for the dynamics of money that have been introduced in the statistical physics literature, focusing on two models with debts. Both models consist of systems of economical agents located on a finite connected graph representing a social network. Each agent is characterized by the number of coins she has, which can be negative in case she is in debt, and each monetary transaction consists in one coin moving from one agent to one of her neighbors. In the first model, that we name the model with individual debt limit, the agents are allowed to individually borrow up to a fixed number of coins. In the second model, that we name the model with collective debt limit, agents can borrow coins from a central bank as long as the bank is not empty, with reimbursements occurring each time an agent in debt receives a coin. Based on numerical simulations of the models on complete graphs, it was conjectured that, in the large population/temperature limits, the distribution of money converges to a shifted exponential distribution for the model with individual debt limit, and to an asymmetric Laplace distribution for the model with collective debt limit. In this paper, we prove exact formulas for the distribution of money that are valid for all possible social networks. Taking the large population/temperature limits in the formula found for the model with individual debt limit, we prove convergence to the shifted exponential distribution, thus establishing the first conjecture. Simplifying the formula found for the model with collective debt limit is more complicated, but using a computer to plot this formula shows an almost perfect fit with the Laplace distribution, which strongly supports the second conjecture. △ Less

Submitted 26 December, 2018; originally announced December 2018.

Comments: 19 pages, 2 figures

MSC Class: 60K35; 91B72

An Overview of the LSST Image Processing Pipelines

Authors: James Bosch, Yusra AlSayyad, Robert Armstrong, Eric Bellm, Hsin-Fang Chiang, Siegfried Eggl, Krzysztof Findeisen, Merlin Fisher-Levine, Leanne P. Guy, Augustin Guyonnet, Željko Ivezić, Tim Jenness, Gábor Kovács, K. Simon Krughoff, Robert H. Lupton, Nate B. Lust, Lauren A. MacArthur, Joshua Meyers, Fred Moolekamp, Christopher B. Morrison, Timothy D. Morton, William O'Mullane, John K. Parejko, Andrés A. Plazas, Paul A. Price , et al. (9 additional authors not shown)

Abstract: The Large Synoptic Survey Telescope (LSST) is an ambitious astronomical survey with a similarly ambitious Data Management component. Data Management for LSST includes processing on both nightly and yearly cadences to generate transient alerts, deep catalogs of the static sky, and forced photometry light-curves for billions of objects at hundreds of epochs, spanning at least a decade. The algorithm… ▽ More The Large Synoptic Survey Telescope (LSST) is an ambitious astronomical survey with a similarly ambitious Data Management component. Data Management for LSST includes processing on both nightly and yearly cadences to generate transient alerts, deep catalogs of the static sky, and forced photometry light-curves for billions of objects at hundreds of epochs, spanning at least a decade. The algorithms running in these pipelines are individually sophisticated and interact in subtle ways. This paper provides an overview of those pipelines, focusing more on those interactions than the details of any individual algorithm. △ Less

Submitted 7 December, 2018; originally announced December 2018.

Comments: Submitted to proceedings for ADASS XXVIII

A new bright z=6.82 quasar discovered with VISTA: VHS J0411-0907

Authors: Estelle Pons, Richard G. McMahon, Robert A. Simcoe, Manda Banerji, Paul C. Hewett, Sophie L. Reed

Abstract: We present the discovery of a new $z \sim 6.8$ quasar discovered with the near-IR VISTA Hemisphere Survey (VHS) which has been spectroscopically confirmed by the ESO New Technology Telescope (NTT) and the Magellan telescope. This quasar has been selected by spectral energy distribution (SED) classification using near infrared data from VISTA, optical data from Pan-STARRS, and mid-IR data from WISE… ▽ More We present the discovery of a new $z \sim 6.8$ quasar discovered with the near-IR VISTA Hemisphere Survey (VHS) which has been spectroscopically confirmed by the ESO New Technology Telescope (NTT) and the Magellan telescope. This quasar has been selected by spectral energy distribution (SED) classification using near infrared data from VISTA, optical data from Pan-STARRS, and mid-IR data from WISE. The SED classification algorithm is used to statistically rank two classes; foreground Galactic low-mass stars and high redshift quasars, prior to spectroscopic observation. Forced photometry on Pan-STARRS pixels for VHS J0411-0907 allows to improve the SED classification reduced-$χ^2$ and photometric redshift. VHS J0411-0907 ($z=6.82$, $y_{AB} = 20.1$ mag, $J_{AB} = 20.0$ mag) has the brightest J-band continuum magnitude of the nine known quasars at $z > 6.7$ and is currently the highest redshift quasar detected in the Pan-STARRS survey. This quasar has one of the lowest black hole mass ($M_{\rm{BH}}= (6.13 \pm 0.51)\times 10^8\:\mathrm{M_{\odot}}$) and the highest Eddington ratio ($2.37\pm0.22$) of the known quasars at $z>6.5$. The high Eddington ratio indicates that some very high-$z$ quasars are undergoing super Eddington accretion. We also present coefficients of the best polynomials fits for colours vs spectral type on the Pan-STARRS, VISTA and WISE system for MLT dwarfs and present a forecast for the expected numbers of quasars at $z>6.5$. △ Less

Submitted 23 January, 2019; v1 submitted 6 December, 2018; originally announced December 2018.

Comments: 14 pages, accepted for publication by MNRAS

One-Shot High-Fidelity Imitation: Training Large-Scale Deep Nets with RL

Authors: Tom Le Paine, Sergio Gómez Colmenarejo, Ziyu Wang, Scott Reed, Yusuf Aytar, Tobias Pfaff, Matt W. Hoffman, Gabriel Barth-Maron, Serkan Cabi, David Budden, Nando de Freitas

Abstract: Humans are experts at high-fidelity imitation -- closely mimicking a demonstration, often in one attempt. Humans use this ability to quickly solve a task instance, and to bootstrap learning of new tasks. Achieving these abilities in autonomous agents is an open problem. In this paper, we introduce an off-policy RL algorithm (MetaMimic) to narrow this gap. MetaMimic can learn both (i) policies for… ▽ More Humans are experts at high-fidelity imitation -- closely mimicking a demonstration, often in one attempt. Humans use this ability to quickly solve a task instance, and to bootstrap learning of new tasks. Achieving these abilities in autonomous agents is an open problem. In this paper, we introduce an off-policy RL algorithm (MetaMimic) to narrow this gap. MetaMimic can learn both (i) policies for high-fidelity one-shot imitation of diverse novel skills, and (ii) policies that enable the agent to solve tasks more efficiently than the demonstrators. MetaMimic relies on the principle of storing all experiences in a memory and replaying these to learn massive deep neural network policies by off-policy RL. This paper introduces, to the best of our knowledge, the largest existing neural networks for deep RL and shows that larger networks with normalization are needed to achieve one-shot high-fidelity imitation on a challenging manipulation task. The results also show that both types of policy can be learned from vision, in spite of the task rewards being sparse, and without access to demonstrator actions. △ Less

Submitted 11 October, 2018; originally announced October 2018.

Sample Efficient Adaptive Text-to-Speech

Authors: Yutian Chen, Yannis Assael, Brendan Shillingford, David Budden, Scott Reed, Heiga Zen, Quan Wang, Luis C. Cobo, Andrew Trask, Ben Laurie, Caglar Gulcehre, Aäron van den Oord, Oriol Vinyals, Nando de Freitas

Abstract: We present a meta-learning approach for adaptive text-to-speech (TTS) with few data. During training, we learn a multi-speaker model using a shared conditional WaveNet core and independent learned embeddings for each speaker. The aim of training is not to produce a neural network with fixed weights, which is then deployed as a TTS system. Instead, the aim is to produce a network that requires few… ▽ More We present a meta-learning approach for adaptive text-to-speech (TTS) with few data. During training, we learn a multi-speaker model using a shared conditional WaveNet core and independent learned embeddings for each speaker. The aim of training is not to produce a neural network with fixed weights, which is then deployed as a TTS system. Instead, the aim is to produce a network that requires few data at deployment time to rapidly adapt to new speakers. We introduce and benchmark three strategies: (i) learning the speaker embedding while keeping the WaveNet core fixed, (ii) fine-tuning the entire architecture with stochastic gradient descent, and (iii) predicting the speaker embedding with a trained neural network encoder. The experiments show that these approaches are successful at adapting the multi-speaker neural network to new speakers, obtaining state-of-the-art results in both sample naturalness and voice similarity with merely a few minutes of audio data from new speakers. △ Less

Submitted 16 January, 2019; v1 submitted 27 September, 2018; originally announced September 2018.

Comments: Accepted by ICLR 2019

Neural Arithmetic Logic Units

Authors: Andrew Trask, Felix Hill, Scott Reed, Jack Rae, Chris Dyer, Phil Blunsom

Abstract: Neural networks can learn to represent and manipulate numerical information, but they seldom generalize well outside of the range of numerical values encountered during training. To encourage more systematic numerical extrapolation, we propose an architecture that represents numerical quantities as linear activations which are manipulated using primitive arithmetic operators, controlled by learned… ▽ More Neural networks can learn to represent and manipulate numerical information, but they seldom generalize well outside of the range of numerical values encountered during training. To encourage more systematic numerical extrapolation, we propose an architecture that represents numerical quantities as linear activations which are manipulated using primitive arithmetic operators, controlled by learned gates. We call this module a neural arithmetic logic unit (NALU), by analogy to the arithmetic logic unit in traditional processors. Experiments show that NALU-enhanced neural networks can learn to track time, perform arithmetic over images of numbers, translate numerical language into real-valued scalars, execute computer code, and count objects in images. In contrast to conventional architectures, we obtain substantially better generalization both inside and outside of the range of numerical values encountered during training, often extrapolating orders of magnitude beyond trained numerical ranges. △ Less

Submitted 1 August, 2018; originally announced August 2018.

New constraints on Lyman-α opacity with a sample of 62 quasars at z > 5.7

Authors: Sarah E. I. Bosman, Xiaohui Fan, Linhua Jiang, Sophie L. Reed, Yoshiki Matsuoka, George D. Becker, Martin G. Haehnelt

Abstract: We present measurements of the mean and scatter of the IGM Lyman-α opacity at 4.9 < z < 6.1 along the lines of sight of 62 quasars at z > 5.7, the largest sample assembled at these redshifts to date by a factor of two. The sample size enables us to sample cosmic variance at these redshifts more robustly than ever before. The spectra used here were obtained by the SDSS, DES-VHS and SHELLQs collabor… ▽ More We present measurements of the mean and scatter of the IGM Lyman-α opacity at 4.9 < z < 6.1 along the lines of sight of 62 quasars at z > 5.7, the largest sample assembled at these redshifts to date by a factor of two. The sample size enables us to sample cosmic variance at these redshifts more robustly than ever before. The spectra used here were obtained by the SDSS, DES-VHS and SHELLQs collaborations, drawn from the ESI and X-Shooter archives, reused from previous studies or observed specifically for this work. We measure the effective optical depth of Lyman-α in bins of 10, 30, 50 and 70 cMpc h-1, construct cumulative distribution functions under two treatments of upper limits on flux and explore an empirical analytic fit to residual Lyman-α transmission. We verify the consistency of our results with those of previous studies via bootstrap re-sampling and confirm the existence of tails towards high values in the opacity distributions, which may persist down to z = 5.2. Comparing our results with predictions from cosmological simulations, we find further strong evidence against models that include a spatially uniform ionizing background and temperature-density relation. We also compare to IGM models that include either a fluctuating UVB dominated by rare quasars or temperature fluctuations due to patchy reionization. Although both models produce better agreement with the observations, neither fully captures the observed scatter in IGM opacity. Our sample of 62 z > 5.7 quasar spectra opens many avenues for future study of the reionisation epoch. △ Less

Submitted 24 May, 2018; v1 submitted 22 February, 2018; originally announced February 2018.

Comments: 23 pages, updated to match MNRAS accepted version

UV-Luminous, Star-Forming Hosts of z~2 Reddened Quasars in the Dark Energy Survey

Authors: C. F. Wethers, M. Banerji, P. C. Hewett, C. A. Lemon, R. G. McMahon, S. L. Reed, Y. Shen, F. B. Abdalla, A. Benoit-Lévy, D. Brooks, E. Buckley-Geer, D. Capozzi, A. Carnero Rosell, M. CarrascoKind, J. Carretero, C. E. Cunha, C. B. D'Andrea, L. N. da Costa, D. L. DePoy, S. Desai, P. Doel, B. Flaugher, P. Fosalba, J. Frieman, J. García-Bellido , et al. (33 additional authors not shown)

Abstract: We present the first rest-frame UV population study of 17 heavily reddened, high-luminosity (E(B-V)$_{\rm{QSO}}\gtrsim$ 0.5; L$_{\rm{bol}}>$ 10$^{46}$ergs$^{-1}$) broad-line quasars at $1.5 < z < 2.7$. We combine the first year of deep, optical, ground-based observations from the Dark Energy Survey (DES) with the near infrared VISTA Hemisphere Survey (VHS) and UKIDSS Large Area Survey (ULAS) data,… ▽ More We present the first rest-frame UV population study of 17 heavily reddened, high-luminosity (E(B-V)$_{\rm{QSO}}\gtrsim$ 0.5; L$_{\rm{bol}}>$ 10$^{46}$ergs$^{-1}$) broad-line quasars at $1.5 < z < 2.7$. We combine the first year of deep, optical, ground-based observations from the Dark Energy Survey (DES) with the near infrared VISTA Hemisphere Survey (VHS) and UKIDSS Large Area Survey (ULAS) data, from which the reddened quasars were initially identified. We demonstrate that the significant dust reddening towards the quasar in our sample allows host galaxy emission to be detected at the rest-frame UV wavelengths probed by the DES photometry. By exploiting this reddening effect, we disentangle the quasar emission from that of the host galaxy via spectral energy distribution (SED) fitting. We find evidence for a relatively unobscured, star-forming host galaxy in at least ten quasars, with a further three quasars exhibiting emission consistent with either star formation or scattered light. From the rest-frame UV emission, we derive instantaneous, dust-corrected star formation rates (SFRs) in the range 25 < SFR$_{\rm{UV}}$ < 365 M$_{\odot}$yr$^{-1}$, with an average SFR$_{\rm{UV}}$ = 130 $\pm$ 95 M$_{\odot}$yr$^{-1}$. We find a broad correlation between SFR$_{\rm{UV}}$ and the bolometric quasar luminosity. Overall, our results show evidence for coeval star formation and black hole accretion occurring in luminous, reddened quasars at the peak epoch of galaxy formation. △ Less

Submitted 8 January, 2018; originally announced January 2018.

Comments: 21 pages, 8 figures, MNRAS Accepted

Rigorous results for the distribution of money on connected graphs

Authors: Nicolas Lanchier, Stephanie Reed

Abstract: This paper is concerned with general spatially explicit versions of three stochastic models for the dynamics of money that have been introduced and studied numerically by statistical physicists: the uniform reshuffling model, the immediate exchange model and the model with saving propensity. All three models consist of systems of economical agents that consecutively engage in pairwise monetary tra… ▽ More This paper is concerned with general spatially explicit versions of three stochastic models for the dynamics of money that have been introduced and studied numerically by statistical physicists: the uniform reshuffling model, the immediate exchange model and the model with saving propensity. All three models consist of systems of economical agents that consecutively engage in pairwise monetary transactions. Computer simulations performed in the physics literature suggest that, when the number of agents and the average amount of money per agent are large, the distribution of money at equilibrium approaches the exponential distribution for the first model, the gamma distribution with shape parameter two for the second model and a gamma distribution whose shape parameter depends on the saving propensity for the third model. The main objective of this paper is to give rigorous proofs of and extend these conjectures to generalizations of the first two models and a variant of the third model that include local rather than global interactions, i.e., instead of choosing the two interacting agents uniformly at random from the system, the agents are located on the vertex set of a general connected graph and can only interact with their neighbors. △ Less

Submitted 1 January, 2018; originally announced January 2018.

Comments: 17 pages, 2 figures

MSC Class: 60K35; 91B72

ScanComplete: Large-Scale Scene Completion and Semantic Segmentation for 3D Scans

Authors: Angela Dai, Daniel Ritchie, Martin Bokeloh, Scott Reed, Jürgen Sturm, Matthias Nießner

Abstract: We introduce ScanComplete, a novel data-driven approach for taking an incomplete 3D scan of a scene as input and predicting a complete 3D model along with per-voxel semantic labels. The key contribution of our method is its ability to handle large scenes with varying spatial extent, managing the cubic growth in data size as scene size increases. To this end, we devise a fully-convolutional generat… ▽ More We introduce ScanComplete, a novel data-driven approach for taking an incomplete 3D scan of a scene as input and predicting a complete 3D model along with per-voxel semantic labels. The key contribution of our method is its ability to handle large scenes with varying spatial extent, managing the cubic growth in data size as scene size increases. To this end, we devise a fully-convolutional generative 3D CNN model whose filter kernels are invariant to the overall scene size. The model can be trained on scene subvolumes but deployed on arbitrarily large scenes at test time. In addition, we propose a coarse-to-fine inference strategy in order to produce high-resolution output while also leveraging large input context sizes. In an extensive series of experiments, we carefully evaluate different model design choices, considering both deterministic and probabilistic models for completion and semantic inference. Our results show that we outperform other methods not only in the size of the environments handled and processing efficiency, but also with regard to completion quality and semantic segmentation performance by a significant margin. △ Less

Submitted 27 March, 2018; v1 submitted 29 December, 2017; originally announced December 2017.

Comments: Video: https://youtu.be/5s5s8iH0NF8

DIAPHANE: a Portable Radiation Transport Library for Astrophysical Applications

Authors: Darren S. Reed, Tim Dykes, Ruben Cabezon, Claudio Gheller, Lucio Mayer

Abstract: One of the most computationally demanding aspects of the hydrodynamical modelling of Astrophysical phenomena is the transport of energy by radiation or relativistic particles. Physical processes involving energy transport are ubiquitous and of capital importance in many scenarios ranging from planet formation to cosmic structure evolution, including explosive events like core collapse supernova or… ▽ More One of the most computationally demanding aspects of the hydrodynamical modelling of Astrophysical phenomena is the transport of energy by radiation or relativistic particles. Physical processes involving energy transport are ubiquitous and of capital importance in many scenarios ranging from planet formation to cosmic structure evolution, including explosive events like core collapse supernova or gamma-ray bursts. Moreover, the ability to model and hence understand these processes has often been limited by the approximations and incompleteness in the treatment of radiation and relativistic particles. The DIAPHANE project has focused in developing a portable and scalable library that handles the transport of radiation and particles (in particular neutrinos) independently of the underlying hydrodynamic code. In this work, we present the computational framework and the functionalities of the first version of the DIAPHANE library, which has been successfully ported to three different smoothed-particle hydrodynamic codes, GADGET2, GASOLINE and SPHYNX. We also present validation of different modules solving the equations of radiation and neutrino transport using different numerical schemes. △ Less

Submitted 11 December, 2017; originally announced December 2017.

Comments: Accepted for publication in Computer Physics Communications

Few-shot Autoregressive Density Estimation: Towards Learning to Learn Distributions

Authors: Scott Reed, Yutian Chen, Thomas Paine, Aäron van den Oord, S. M. Ali Eslami, Danilo Rezende, Oriol Vinyals, Nando de Freitas

Abstract: Deep autoregressive models have shown state-of-the-art performance in density estimation for natural images on large-scale datasets such as ImageNet. However, such models require many thousands of gradient-based weight updates and unique image examples for training. Ideally, the models would rapidly learn visual concepts from only a handful of examples, similar to the manner in which humans learns… ▽ More Deep autoregressive models have shown state-of-the-art performance in density estimation for natural images on large-scale datasets such as ImageNet. However, such models require many thousands of gradient-based weight updates and unique image examples for training. Ideally, the models would rapidly learn visual concepts from only a handful of examples, similar to the manner in which humans learns across many vision tasks. In this paper, we show how 1) neural attention and 2) meta learning techniques can be used in combination with autoregressive models to enable effective few-shot density estimation. Our proposed modifications to PixelCNN result in state-of-the art few-shot density estimation on the Omniglot dataset. Furthermore, we visualize the learned attention policy and find that it learns intuitive algorithms for simple tasks such as image mirroring on ImageNet and handwriting on Omniglot without supervision. Finally, we extend the model to natural images and demonstrate few-shot image generation on the Stanford Online Products dataset. △ Less

Submitted 28 February, 2018; v1 submitted 27 October, 2017; originally announced October 2017.

The Discovery of a Five-Image Lensed Quasar at z = 3.34 using PanSTARRS1 and Gaia

Authors: Fernanda Ostrovski, Cameron A. Lemon, Matthew W. Auger, Richard G. McMahon, Christopher D. Fassnacht, Geoff C. -F. Chen, Andrew J. Connolly, Sergey E. Koposov, Estelle Pons, Sophie L. Reed, Cristian E. Rusu

Abstract: We report the discovery, spectroscopic confirmation, and mass modelling of the gravitationally lensed quasar system PS J0630-1201. The lens was discovered by matching a photometric quasar catalogue compiled from Pan-STARRS and WISE photometry to the Gaia DR1 catalogue, exploiting the high spatial resolution of the latter (FWHM $\sim $0.1") to identify the three brightest components of the lens. Fo… ▽ More We report the discovery, spectroscopic confirmation, and mass modelling of the gravitationally lensed quasar system PS J0630-1201. The lens was discovered by matching a photometric quasar catalogue compiled from Pan-STARRS and WISE photometry to the Gaia DR1 catalogue, exploiting the high spatial resolution of the latter (FWHM $\sim $0.1") to identify the three brightest components of the lens. Follow-up spectroscopic observations with the WHT confirm the multiple objects are quasars at redshift $z_{q}=3.34$. Further follow-up with Keck AO high-resolution imaging reveals that the system is composed of two lensing galaxies and the quasar is lensed into a $\sim$2.8" separation four-image cusp configuration with a fifth image clearly visible, and a 1.0" arc due to the lensed quasar host galaxy. The system is well-modelled with two singular isothermal ellipsoids, reproducing the position of the fifth image. We discuss future prospects for measuring time delays between the images and constraining any offset between mass and light using the faintly detected Einstein arcs associated with the quasar host galaxy. △ Less

Submitted 16 October, 2017; v1 submitted 26 September, 2017; originally announced September 2017.

Comments: Submitted to MNRAS Letters, 6 pages, 3 figures, 4 tables

OzDES multifibre spectroscopy for the Dark Energy Survey: Three year results and first data release

Authors: M. J. Childress, C. Lidman, T. M. Davis, B. E. Tucker, J. Asorey, F. Yuan, T. M. C. Abbott, F. B. Abdalla, S. Allam, J. Annis, M. Banerji, A. Benoit-Levy, S. R. Bernard, E. Bertin, D. Brooks, E. Buckley-Geer, D. L. Burke, A. Carnero Rosell, D. Carollo, M. Carrasco Kind, J. Carretero, F. J. Castander, C. E. Cunha, L. N. da Costa, C. B. D'Andrea , et al. (76 additional authors not shown)

Abstract: We present results for the first three years of OzDES, a six-year programme to obtain redshifts for objects in the Dark Energy Survey (DES) supernova fields using the 2dF fibre positioner and AAOmega spectrograph on the Anglo-Australian Telescope. OzDES is a multi-object spectroscopic survey targeting multiple types of targets at multiple epochs over a multi-year baseline, and is one of the first… ▽ More We present results for the first three years of OzDES, a six-year programme to obtain redshifts for objects in the Dark Energy Survey (DES) supernova fields using the 2dF fibre positioner and AAOmega spectrograph on the Anglo-Australian Telescope. OzDES is a multi-object spectroscopic survey targeting multiple types of targets at multiple epochs over a multi-year baseline, and is one of the first multi-object spectroscopic surveys to dynamically include transients into the target list soon after their discovery. At the end of three years, OzDES has spectroscopically confirmed almost 100 supernovae, and has measured redshifts for 17,000 objects, including the redshifts of 2,566 supernova hosts. We examine how our ability to measure redshifts for targets of various types depends on signal-to-noise, magnitude, and exposure time, finding that our redshift success rate increases significantly at a signal-to-noise of 2 to 3 per 1-Angstrom bin. We also find that the change in signal-to-noise with exposure time closely matches the Poisson limit for stacked exposures as long as 10 hours. We use these results to predict the redshift yield of the full OzDES survey, as well as the potential yields of future surveys on other facilities such as the 4m Multi-Object Spectroscopic Telescope (4MOST), the Subaru Prime Focus Spectrograph (PFS), and the Maunakea Spectroscopic Explorer (MSE). This work marks the first OzDES data release, comprising 14,693 redshifts. OzDES is on target to obtain over a yield of approximately 5,700 supernova host-galaxy redshifts. △ Less

Submitted 15 August, 2017; originally announced August 2017.

Comments: Accepted for publication in MNRAS. Redshift data release is available at http://www.mso.anu.edu.au/ozdes/DR1

Robust Imitation of Diverse Behaviors

Authors: Ziyu Wang, Josh Merel, Scott Reed, Greg Wayne, Nando de Freitas, Nicolas Heess

Abstract: Deep generative models have recently shown great promise in imitation learning for motor control. Given enough data, even supervised approaches can do one-shot imitation learning; however, they are vulnerable to cascading failures when the agent trajectory diverges from the demonstrations. Compared to purely supervised methods, Generative Adversarial Imitation Learning (GAIL) can learn more robust… ▽ More Deep generative models have recently shown great promise in imitation learning for motor control. Given enough data, even supervised approaches can do one-shot imitation learning; however, they are vulnerable to cascading failures when the agent trajectory diverges from the demonstrations. Compared to purely supervised methods, Generative Adversarial Imitation Learning (GAIL) can learn more robust controllers from fewer demonstrations, but is inherently mode-seeking and more difficult to train. In this paper, we show how to combine the favourable aspects of these two approaches. The base of our model is a new type of variational autoencoder on demonstration trajectories that learns semantic policy embeddings. We show that these embeddings can be learned on a 9 DoF Jaco robot arm in reaching tasks, and then smoothly interpolated with a resulting smooth interpolation of reaching behavior. Leveraging these policy representations, we develop a new version of GAIL that (1) is much more robust than the purely-supervised controller, especially with few demonstrations, and (2) avoids mode collapse, capturing many diverse behaviors when GAIL on its own does not. We demonstrate our approach on learning diverse gaits from demonstration on a 2D biped and a 62 DoF 3D humanoid in the MuJoCo physics environment. △ Less

Submitted 14 July, 2017; v1 submitted 10 July, 2017; originally announced July 2017.

The role of cooperation in spatially explicit economical systems

Authors: Nicolas Lanchier, Stephanie Reed

Abstract: This paper is concerned with a model in econophysics, the subfield of statistical physics that applies concepts from traditional physics to economics. In our model, economical agents are represented by the vertices of a connected graph and are characterized by the number of coins they possess. Agents independently spend one coin at rate one for their basic need, earn one coin at a rate chosen inde… ▽ More This paper is concerned with a model in econophysics, the subfield of statistical physics that applies concepts from traditional physics to economics. In our model, economical agents are represented by the vertices of a connected graph and are characterized by the number of coins they possess. Agents independently spend one coin at rate one for their basic need, earn one coin at a rate chosen independently from a fixed distribution $φ$ and exchange money at rate $μ$ with one of their nearest neighbors, with the richest neighbor giving one coin to the other neighbor. If an agent needs to spend one coin when her fortune is at zero, she dies, i.e., the corresponding vertex is removed from the graph. Our first results focus on the two extreme cases of lack of cooperation $μ= 0$ and perfect cooperation $μ= \infty$ for finite connected graphs. These results suggest that, when overall the agents earn more than they spend, cooperation is beneficial for the survival of the population, whereas when overall the agents earn less than they spend, cooperation becomes detrimental. The infinite one-dimensional system is also studied. In this case, when the agents earn less than they spend in average, the density of agents that die eventually is bounded from below by a positive constant that does not depend on the initial number of coins per agent or the level of cooperation. △ Less

Submitted 22 April, 2017; originally announced April 2017.

Comments: 17 pages, 3 figures

MSC Class: 60K35; 91B72

Parallel Multiscale Autoregressive Density Estimation

Authors: Scott Reed, Aäron van den Oord, Nal Kalchbrenner, Sergio Gómez Colmenarejo, Ziyu Wang, Dan Belov, Nando de Freitas

Abstract: PixelCNN achieves state-of-the-art results in density estimation for natural images. Although training is fast, inference is costly, requiring one network evaluation per pixel; O(N) for N pixels. This can be sped up by caching activations, but still involves generating each pixel sequentially. In this work, we propose a parallelized PixelCNN that allows more efficient inference by modeling certain… ▽ More PixelCNN achieves state-of-the-art results in density estimation for natural images. Although training is fast, inference is costly, requiring one network evaluation per pixel; O(N) for N pixels. This can be sped up by caching activations, but still involves generating each pixel sequentially. In this work, we propose a parallelized PixelCNN that allows more efficient inference by modeling certain pixel groups as conditionally independent. Our new PixelCNN model achieves competitive density estimation and orders of magnitude speedup - O(log N) sampling instead of O(N) - enabling the practical generation of 512x512 images. We evaluate the model on class-conditional image generation, text-to-image synthesis, and action-conditional video generation, showing that our model achieves the best results among non-pixel-autoregressive density models that allow efficient sampling. △ Less

Submitted 10 March, 2017; originally announced March 2017.

Eight new luminous z > 6 quasars selected via SED model fitting of VISTA, WISE and Dark Energy Survey Year 1 Observations

Authors: S. L. Reed, R. G. McMahon, P. Martini, M. Banerji, M. Auger, P. C. Hewett, S. E. Koposov, S. L. J. Gibbons, E. Gonzalez-Solares, F. Ostrovski, S. S. Tie, F. B. Abdalla, S. Allam, A. Benoit-Levy, E. Bertin, D. Brooks, E. Buckley-Geer, D. L. Burke, A. Carnero Rosell, M. Carrasco Kind, J. Carretero, L. N. da Costa, D. L. DePoy, S. Desai, H. T. Diehl , et al. (38 additional authors not shown)

Abstract: We present the discovery and spectroscopic confirmation with the ESO NTT and Gemini South telescopes of eight new 6.0 < z < 6.5 quasars with z$_{AB}$ < 21.0. These quasars were photometrically selected without any star-galaxy morphological criteria from 1533 deg$^{2}$ using SED model fitting to photometric data from the Dark Energy Survey (g, r, i, z, Y), the VISTA Hemisphere Survey (J, H, K) and… ▽ More We present the discovery and spectroscopic confirmation with the ESO NTT and Gemini South telescopes of eight new 6.0 < z < 6.5 quasars with z$_{AB}$ < 21.0. These quasars were photometrically selected without any star-galaxy morphological criteria from 1533 deg$^{2}$ using SED model fitting to photometric data from the Dark Energy Survey (g, r, i, z, Y), the VISTA Hemisphere Survey (J, H, K) and the Wide-Field Infrared Survey Explorer (W1, W2). The photometric data was fitted with a grid of quasar model SEDs with redshift dependent Lyman-α forest absorption and a range of intrinsic reddening as well as a series of low mass cool star models. Candidates were ranked using on a SED-model based $χ^{2}$-statistic, which is extendable to other future imaging surveys (e.g. LSST, Euclid). Our spectral confirmation success rate is 100% without the need for follow-up photometric observations as used in other studies of this type. Combined with automatic removal of the main types of non-astrophysical contaminants the method allows large data sets to be processed without human intervention and without being over run by spurious false candidates. We also present a robust parametric redshift estimating technique that gives comparable accuracy to MgII and CO based redshift estimators. We find two z $\sim$ 6.2 quasars with HII near zone sizes < 3 proper Mpc which could indicate that these quasars may be young with ages < 10$^6$ - 10$^7$ years or lie in over dense regions of the IGM. The z = 6.5 quasar VDESJ0224-4711 has J$_{AB}$ = 19.75 is the second most luminous quasar known with z > 6.5. △ Less

Submitted 17 January, 2017; originally announced January 2017.

Comments: 19 pages, 15 figures

Hints against the cold and collisionless nature of dark matter from the galaxy velocity function

Authors: Aurel Schneider, Sebastian Trujillo-Gomez, Emmanouil Papastergis, Darren S. Reed, George Lake

Abstract: The observed number of dwarf galaxies as a function of rotation velocity is significantly smaller than predicted by the standard model of cosmology. This discrepancy cannot be simply solved by assuming strong baryonic feedback processes, since they would violate the observed relation between maximum circular velocity ($v_{\rm max}$) and baryon mass of galaxies. A speculative but tantalising possib… ▽ More The observed number of dwarf galaxies as a function of rotation velocity is significantly smaller than predicted by the standard model of cosmology. This discrepancy cannot be simply solved by assuming strong baryonic feedback processes, since they would violate the observed relation between maximum circular velocity ($v_{\rm max}$) and baryon mass of galaxies. A speculative but tantalising possibility is that the mismatch between observation and theory points towards the existence of non-cold or non-collisionless dark matter (DM). In this paper, we investigate the effects of warm, mixed (i.e warm plus cold), and self-interacting DM scenarios on the abundance of dwarf galaxies and the relation between observed HI line-width and maximum circular velocity. Both effects have the potential to alleviate the apparent mismatch between the observed and theoretical abundance of galaxies as a function of $v_{\rm max}$. For the case of warm and mixed DM, we show that the discrepancy disappears, even for luke-warm models that evade stringent bounds from the Lyman-$α$ forest. Self-interacting DM scenarios can also provide a solution as long as they lead to extended ($\gtrsim 1.5$ kpc) dark matter cores in the density profiles of dwarf galaxies. Only models with velocity-dependent cross sections can yield such cores without violating other observational constraints at larger scales. △ Less

Submitted 28 June, 2017; v1 submitted 28 November, 2016; originally announced November 2016.

Comments: Matches published version

A Study of Quasar Selection in the Dark Energy Survey Supernova fields

Authors: S. S. Tie, P. Martini, D. Mudd, F. Ostrovski, S. L. Reed, C. Lidman, C. Kochanek, T. M. Davis, R. Sharp, S. Uddin, A. King, W. Wester, B. E. Tucker, D. L. Tucker, E. Buckley-Geer, D. Carollo, M. Childress, K. Glazebrook, S. R. Hinton, G. Lewis, E. Macaulay, C. R. O'Neill, T. M. C. Abbott, F. B. Abdalla, J. Annis , et al. (53 additional authors not shown)

Abstract: We present a study of quasar selection using the DES supernova fields. We used a quasar catalog from an overlapping portion of the SDSS Stripe 82 region to quantify the completeness and efficiency of selection methods involving color, probabilistic modeling, variability, and combinations of color/probabilistic modeling with variability. We only considered objects that appear as point sources in th… ▽ More We present a study of quasar selection using the DES supernova fields. We used a quasar catalog from an overlapping portion of the SDSS Stripe 82 region to quantify the completeness and efficiency of selection methods involving color, probabilistic modeling, variability, and combinations of color/probabilistic modeling with variability. We only considered objects that appear as point sources in the DES images. We examine color selection methods based on the WISE mid-IR W1-W2 color, a mixture of WISE and DES colors (g-i and i-W1) and a mixture of VHS and DES colors (g-i and i-K). For probabilistic quasar selection, we used XDQSOz, an algorithm that employs an empirical multi-wavelength flux model of quasars to assign quasar probabilities. Our variability selection uses the multi-band chi2-probability that sources are constant in the DES Year 1 griz-band light curves. The completeness and efficiency are calculated relative to an underlying sample of point sources that are detected in the required selection bands and pass our data quality and photometric error cuts. We conduct our analyses at two magnitude limits, i<19.8 mag and i<22 mag. For sources with W1 and W2 detections, the W1-W2 color or XDQSOz method combined with variability gives the highest completenesses of >85% for both i-band magnitude limits and efficiencies of >80% to the bright limit and >60% to the faint limit; however, the giW1 and giW1+variability methods give the highest quasar surface densities. The XDQSOz method and combinations of W1W2/giW1/XDQSOz with variability are among the better selection methods when both high completeness and high efficiency are desired. We also present the OzDES Quasar Catalog of 1,263 spectroscopically-confirmed quasars taken by the OzDES survey. The catalog includes quasars with redshifts up to z~4 and brighter than i=22 mag, although the catalog is not complete up this magnitude limit. △ Less

Submitted 16 November, 2016; originally announced November 2016.

Comments: 16 pages, 12 figures, 3 tables; Submitted to ApJ. The OzDES Quasar Catalog can be downloaded at http://bit.ly/2fWsK9t . For a video summary of the paper, please see https://www.youtube.com/watch?v=JeQNdUATze0

Another baryon miracle? Testing solutions to the "missing dwarfs" problem

Authors: Sebastian Trujillo-Gomez, Aurel Schneider, Emmanouil Papastergis, Darren S. Reed, George Lake

Abstract: The dearth of dwarf galaxies in the local universe is hard to reconcile with the large number of low mass haloes expected within the concordance $Λ$CDM paradigm. In this paper we perform a systematic evaluation of the uncertainties affecting the measurement of DM halo abundance using galaxy kinematics. Using a large sample of dwarf galaxies with spatially resolved kinematic data we derive a correc… ▽ More The dearth of dwarf galaxies in the local universe is hard to reconcile with the large number of low mass haloes expected within the concordance $Λ$CDM paradigm. In this paper we perform a systematic evaluation of the uncertainties affecting the measurement of DM halo abundance using galaxy kinematics. Using a large sample of dwarf galaxies with spatially resolved kinematic data we derive a correction to obtain the observed abundance of galaxies as a function of their halo maximum circular velocity from the line-of-sight velocity function in the Local Volume. This estimate provides a direct means of comparing the predictions of theoretical models and simulations (including nonstandard cosmologies and novel galaxy formation physics) to the observational constraints. The new "galactic $V_{max}$" function is steeper than the line-of-sight velocity function but still shallower than the theoretical CDM expectation, showing that some unaccounted physical process is necessary to reduce the abundance of galaxies and/or drastically modify their density profiles compared to CDM haloes. Using this new galactic $V_{max}$ function, we investigate the viability of baryonic solutions such as feedback-powered outflows and photoevaporation of gas from an ionising radiation background. At the 3-$σ$ confidence level neither energetic feedback nor photoevaporation are effective enough to reconcile the disagreement. In the case of maximum baryonic effects, the theoretical estimate still deviates significantly from the observations for $V_{max} < 60$ km/s. CDM predicts at least 1.8 times more galaxies with $V_{max} = 50$ km/s and 2.5 times more than observed at $30$ km/s. Recent hydrodynamic simulations seem to resolve the discrepancy but disagree with the properties of observed galaxies with resolved kinematics. (abridged) △ Less

Submitted 12 September, 2017; v1 submitted 28 October, 2016; originally announced October 2016.

Comments: 17 pages, 22 figures; major revisions include clarification of the method, expanded comparison with simulations with a new figure, analysis of uncertainties in model as well as pressure support corrections, and a new table with nomenclature

Learning What and Where to Draw

Authors: Scott Reed, Zeynep Akata, Santosh Mohan, Samuel Tenka, Bernt Schiele, Honglak Lee

Abstract: Generative Adversarial Networks (GANs) have recently demonstrated the capability to synthesize compelling real-world images, such as room interiors, album covers, manga, faces, birds, and flowers. While existing models can synthesize images based on global constraints such as a class label or caption, they do not provide control over pose or object location. We propose a new model, the Generative… ▽ More Generative Adversarial Networks (GANs) have recently demonstrated the capability to synthesize compelling real-world images, such as room interiors, album covers, manga, faces, birds, and flowers. While existing models can synthesize images based on global constraints such as a class label or caption, they do not provide control over pose or object location. We propose a new model, the Generative Adversarial What-Where Network (GAWWN), that synthesizes images given instructions describing what content to draw in which location. We show high-quality 128 x 128 image synthesis on the Caltech-UCSD Birds dataset, conditioned on both informal text descriptions and also object location. Our system exposes control over both the bounding box around the bird and its constituent parts. By modeling the conditional distributions over part locations, our system also enables conditioning on arbitrary subsets of parts (e.g. only the beak and tail), yielding an efficient interface for picking part locations. We also show preliminary results on the more challenging domain of text- and location-controllable synthesis of images of human actions on the MPII Human Pose dataset. △ Less

Submitted 7 October, 2016; originally announced October 2016.

Comments: In NIPS 2016

VDES J2325-5229 a z=2.7 gravitationally lensed quasar discovered using morphology independent supervised machine learning

Authors: Fernanda Ostrovski, Richard G. McMahon, Andrew J. Connolly, Cameron A. Lemon, Matthew W. Auger, Manda Banerji, Johnathan M. Hung, Sergey E. Koposov, Christopher E. Lidman, Sophie L. Reed, Sahar Allam, Aurélien Benoit-Lévy, Emmanuel Bertin, David Brooks, Elizabeth Buckley-Geer, Aurelio Carnero Rosell, Matias Carrasco Kind, Jorge Carretero, Carlos E. Cunha, Luiz N. da Costa, Shantanu Desai, H. Thomas Diehl, Jörg P. Dietrich, August E. Evrard, David A. Finley , et al. (34 additional authors not shown)

Abstract: We present the discovery and preliminary characterization of a gravitationally lensed quasar with a source redshift $z_{s}=2.74$ and image separation of $2.9"$ lensed by a foreground $z_{l}=0.40$ elliptical galaxy. Since the images of gravitationally lensed quasars are the superposition of multiple point sources and a foreground lensing galaxy, we have developed a morphology independent multi-wave… ▽ More We present the discovery and preliminary characterization of a gravitationally lensed quasar with a source redshift $z_{s}=2.74$ and image separation of $2.9"$ lensed by a foreground $z_{l}=0.40$ elliptical galaxy. Since the images of gravitationally lensed quasars are the superposition of multiple point sources and a foreground lensing galaxy, we have developed a morphology independent multi-wavelength approach to the photometric selection of lensed quasar candidates based on Gaussian Mixture Models (GMM) supervised machine learning. Using this technique and $gi$ multicolour photometric observations from the Dark Energy Survey (DES), near IR $JK$ photometry from the VISTA Hemisphere Survey (VHS) and WISE mid IR photometry, we have identified a candidate system with two catalogue components with $i_{AB}=18.61$ and $i_{AB}=20.44$ comprised of an elliptical galaxy and two blue point sources. Spectroscopic follow-up with NTT and the use of an archival AAT spectrum show that the point sources can be identified as a lensed quasar with an emission line redshift of $z=2.739\pm0.003$ and a foreground early type galaxy with $z=0.400\pm0.002$. We model the system as a single isothermal ellipsoid and find the Einstein radius $θ_E \sim 1.47"$, enclosed mass $M_{enc} \sim 4 \times 10^{11}$M$_{\odot}$ and a time delay of $\sim$52 days. The relatively wide separation, month scale time delay duration and high redshift make this an ideal system for constraining the expansion rate beyond a redshift of 1. △ Less

Submitted 15 November, 2016; v1 submitted 5 July, 2016; originally announced July 2016.

Comments: 11 pages, 7 figures, 7 tables, MNRAS accepted

Generative Adversarial Text to Image Synthesis

Authors: Scott Reed, Zeynep Akata, Xinchen Yan, Lajanugen Logeswaran, Bernt Schiele, Honglak Lee

Abstract: Automatic synthesis of realistic images from text would be interesting and useful, but current AI systems are still far from this goal. However, in recent years generic and powerful recurrent neural network architectures have been developed to learn discriminative text feature representations. Meanwhile, deep convolutional generative adversarial networks (GANs) have begun to generate highly compel… ▽ More Automatic synthesis of realistic images from text would be interesting and useful, but current AI systems are still far from this goal. However, in recent years generic and powerful recurrent neural network architectures have been developed to learn discriminative text feature representations. Meanwhile, deep convolutional generative adversarial networks (GANs) have begun to generate highly compelling images of specific categories, such as faces, album covers, and room interiors. In this work, we develop a novel deep architecture and GAN formulation to effectively bridge these advances in text and image model- ing, translating visual concepts from characters to pixels. We demonstrate the capability of our model to generate plausible images of birds and flowers from detailed text descriptions. △ Less

Submitted 5 June, 2016; v1 submitted 17 May, 2016; originally announced May 2016.

Comments: ICML 2016

Learning Deep Representations of Fine-grained Visual Descriptions

Authors: Scott Reed, Zeynep Akata, Bernt Schiele, Honglak Lee

Abstract: State-of-the-art methods for zero-shot visual recognition formulate learning as a joint embedding problem of images and side information. In these formulations the current best complement to visual features are attributes: manually encoded vectors describing shared characteristics among categories. Despite good performance, attributes have limitations: (1) finer-grained recognition requires commen… ▽ More State-of-the-art methods for zero-shot visual recognition formulate learning as a joint embedding problem of images and side information. In these formulations the current best complement to visual features are attributes: manually encoded vectors describing shared characteristics among categories. Despite good performance, attributes have limitations: (1) finer-grained recognition requires commensurately more attributes, and (2) attributes do not provide a natural language interface. We propose to overcome these limitations by training neural language models from scratch; i.e. without pre-training and only consuming words and characters. Our proposed models train end-to-end to align with the fine-grained and category-specific content of images. Natural language provides a flexible and compact way of encoding only the salient visual aspects for distinguishing categories. By training on raw text, our model can do inference on raw text as well, providing humans a familiar mode both for annotation and retrieval. Our model achieves strong performance on zero-shot text-based image retrieval and significantly outperforms the attribute-based state-of-the-art for zero-shot classification on the Caltech UCSD Birds 200-2011 dataset. △ Less

Submitted 17 May, 2016; originally announced May 2016.

Comments: CVPR 2016

Weakly-supervised Disentangling with Recurrent Transformations for 3D View Synthesis

Authors: Jimei Yang, Scott Reed, Ming-Hsuan Yang, Honglak Lee

Abstract: An important problem for both graphics and vision is to synthesize novel views of a 3D object from a single image. This is particularly challenging due to the partial observability inherent in projecting a 3D object onto the image space, and the ill-posedness of inferring object shape and pose. However, we can train a neural network to address the problem if we restrict our attention to specific o… ▽ More An important problem for both graphics and vision is to synthesize novel views of a 3D object from a single image. This is particularly challenging due to the partial observability inherent in projecting a 3D object onto the image space, and the ill-posedness of inferring object shape and pose. However, we can train a neural network to address the problem if we restrict our attention to specific object categories (in our case faces and chairs) for which we can gather ample training data. In this paper, we propose a novel recurrent convolutional encoder-decoder network that is trained end-to-end on the task of rendering rotated objects starting from a single image. The recurrent structure allows our model to capture long-term dependencies along a sequence of transformations. We demonstrate the quality of its predictions for human faces on the Multi-PIE dataset and for a dataset of 3D chair models, and also show its ability to disentangle latent factors of variation (e.g., identity and pose) without using full supervision. △ Less

Submitted 4 January, 2016; originally announced January 2016.

Comments: This was published in NIPS 2015 conference