Zum Hauptinhalt springen

Showing 1–16 of 16 results for author: Spencer, S

Searching in archive cs. Search in all archives.
.
  1. arXiv:2403.05530  [pdf, other

    cs.CL cs.AI

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context

    Authors: Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, Soroosh Mariooryad, Yifan Ding, Xinyang Geng, Fred Alcober, Roy Frostig, Mark Omernick, Lexi Walker, Cosmin Paduraru, Christina Sorokin, Andrea Tacchetti, Colin Gaffney, Samira Daruki, Olcan Sercinoglu, Zach Gleicher, Juliette Love , et al. (1110 additional authors not shown)

    Abstract: In this report, we introduce the Gemini 1.5 family of models, representing the next generation of highly compute-efficient multimodal models capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio. The family includes two new models: (1) an updated Gemini 1.5 Pro, which exceeds the February… ▽ More

    Submitted 8 August, 2024; v1 submitted 8 March, 2024; originally announced March 2024.

  2. arXiv:2312.09187  [pdf, other

    cs.LG

    Vision-Language Models as a Source of Rewards

    Authors: Kate Baumli, Satinder Baveja, Feryal Behbahani, Harris Chan, Gheorghe Comanici, Sebastian Flennerhag, Maxime Gazeau, Kristian Holsheimer, Dan Horgan, Michael Laskin, Clare Lyle, Hussain Masoom, Kay McKinney, Volodymyr Mnih, Alexander Neitz, Dmitry Nikulin, Fabio Pardo, Jack Parker-Holder, John Quan, Tim Rocktäschel, Himanshu Sahni, Tom Schaul, Yannick Schroecker, Stephen Spencer, Richie Steigerwald , et al. (2 additional authors not shown)

    Abstract: Building generalist agents that can accomplish many goals in rich open-ended environments is one of the research frontiers for reinforcement learning. A key limiting factor for building generalist agents with RL has been the need for a large number of reward functions for achieving different goals. We investigate the feasibility of using off-the-shelf vision-language models, or VLMs, as sources of… ▽ More

    Submitted 12 July, 2024; v1 submitted 14 December, 2023; originally announced December 2023.

    Comments: 10 pages, 5 figures

  3. arXiv:2308.16848  [pdf, other

    physics.comp-ph cs.LG physics.chem-ph quant-ph

    Natural Quantum Monte Carlo Computation of Excited States

    Authors: David Pfau, Simon Axelrod, Halvard Sutterud, Ingrid von Glehn, James S. Spencer

    Abstract: We present a variational Monte Carlo algorithm for estimating the lowest excited states of a quantum system which is a natural generalization of the estimation of ground states. The method has no free parameters and requires no explicit orthogonalization of the different states, instead transforming the problem of finding excited states of a given system into that of finding the ground state of an… ▽ More

    Submitted 12 February, 2024; v1 submitted 31 August, 2023; originally announced August 2023.

    Comments: Added funding acknowledgment

  4. arXiv:2305.06989  [pdf, other

    cond-mat.quant-gas cond-mat.supr-con cs.LG physics.comp-ph

    Neural Wave Functions for Superfluids

    Authors: Wan Tong Lou, Halvard Sutterud, Gino Cassella, W. M. C. Foulkes, Johannes Knolle, David Pfau, James S. Spencer

    Abstract: Understanding superfluidity remains a major goal of condensed matter physics. Here we tackle this challenge utilizing the recently developed Fermionic neural network (FermiNet) wave function Ansatz [D. Pfau et al., Phys. Rev. Res. 2, 033429 (2020).] for variational Monte Carlo calculations. We study the unitary Fermi gas, a system with strong, short-range, two-body interactions known to possess a… ▽ More

    Submitted 10 June, 2024; v1 submitted 11 May, 2023; originally announced May 2023.

    Comments: 19 pages, 8 figures. Talk presented at the 2023 APS March Meeting, March 5-10, 2023, Las Vegas, Nevada, United States

    Journal ref: Phys. Rev. X 14, 021030 (2024)

  5. arXiv:2303.17853  [pdf, other

    physics.pop-ph astro-ph.HE cs.CL

    Can AI Put Gamma-Ray Astrophysicists Out of a Job?

    Authors: Samuel T. Spencer, Vikas Joshi, Alison M. W. Mitchell

    Abstract: In what will likely be a litany of generative-model-themed arXiv submissions celebrating April the 1st, we evaluate the capacity of state-of-the-art transformer models to create a paper detailing the detection of a Pulsar Wind Nebula with a non-existent Imaging Atmospheric Cherenkov Telescope (IACT) Array. We do this to evaluate the ability of such models to interpret astronomical observations and… ▽ More

    Submitted 4 April, 2023; v1 submitted 31 March, 2023; originally announced March 2023.

  6. arXiv:2211.13672  [pdf, other

    physics.chem-ph cs.LG physics.comp-ph

    A Self-Attention Ansatz for Ab-initio Quantum Chemistry

    Authors: Ingrid von Glehn, James S. Spencer, David Pfau

    Abstract: We present a novel neural network architecture using self-attention, the Wavefunction Transformer (Psiformer), which can be used as an approximation (or Ansatz) for solving the many-electron Schrödinger equation, the fundamental equation for quantum chemistry and material science. This equation can be solved from first principles, requiring no external training data. In recent years, deep neural n… ▽ More

    Submitted 19 April, 2023; v1 submitted 24 November, 2022; originally announced November 2022.

  7. arXiv:2210.14215  [pdf, other

    cs.LG cs.AI

    In-context Reinforcement Learning with Algorithm Distillation

    Authors: Michael Laskin, Luyu Wang, Junhyuk Oh, Emilio Parisotto, Stephen Spencer, Richie Steigerwald, DJ Strouse, Steven Hansen, Angelos Filos, Ethan Brooks, Maxime Gazeau, Himanshu Sahni, Satinder Singh, Volodymyr Mnih

    Abstract: We propose Algorithm Distillation (AD), a method for distilling reinforcement learning (RL) algorithms into neural networks by modeling their training histories with a causal sequence model. Algorithm Distillation treats learning to reinforcement learn as an across-episode sequential prediction problem. A dataset of learning histories is generated by a source RL algorithm, and then a causal transf… ▽ More

    Submitted 25 October, 2022; originally announced October 2022.

  8. arXiv:2209.12466  [pdf, other

    cond-mat.mtrl-sci cs.LG physics.comp-ph

    Learned Force Fields Are Ready For Ground State Catalyst Discovery

    Authors: Michael Schaarschmidt, Morgane Riviere, Alex M. Ganose, James S. Spencer, Alexander L. Gaunt, James Kirkpatrick, Simon Axelrod, Peter W. Battaglia, Jonathan Godwin

    Abstract: We present evidence that learned density functional theory (``DFT'') force fields are ready for ground state catalyst discovery. Our key finding is that relaxation using forces from a learned potential yields structures with similar or lower energy to those relaxed using the RPBE functional in over 50\% of evaluated systems, despite the fact that the predicted forces differ significantly from the… ▽ More

    Submitted 26 September, 2022; originally announced September 2022.

  9. arXiv:2202.05183  [pdf, other

    physics.comp-ph cond-mat.other cond-mat.str-el cs.LG

    Discovering Quantum Phase Transitions with Fermionic Neural Networks

    Authors: G. Cassella, H. Sutterud, S. Azadi, N. D. Drummond, D. Pfau, J. S. Spencer, W. M. C. Foulkes

    Abstract: Deep neural networks have been extremely successful as highly accurate wave function ansätze for variational Monte Carlo calculations of molecular ground states. We present an extension of one such ansatz, FermiNet, to calculations of the ground states of periodic Hamiltonians, and study the homogeneous electron gas. FermiNet calculations of the ground-state energies of small electron gas systems… ▽ More

    Submitted 5 July, 2022; v1 submitted 10 February, 2022; originally announced February 2022.

    Comments: 12 pages, 3 figures

  10. arXiv:2110.15331  [pdf, other

    cs.LG cs.AI

    Wasserstein Distance Maximizing Intrinsic Control

    Authors: Ishan Durugkar, Steven Hansen, Stephen Spencer, Volodymyr Mnih

    Abstract: This paper deals with the problem of learning a skill-conditioned policy that acts meaningfully in the absence of a reward signal. Mutual information based objectives have shown some success in learning skills that reach a diverse set of states in this setting. These objectives include a KL-divergence term, which is maximized by visiting distinct states even if those states are not far apart in th… ▽ More

    Submitted 28 October, 2021; originally announced October 2021.

  11. arXiv:2103.06054  [pdf, other

    astro-ph.IM astro-ph.HE cs.LG

    Deep learning with photosensor timing information as a background rejection method for the Cherenkov Telescope Array

    Authors: Samuel Spencer, Thomas Armstrong, Jason Watson, Salvatore Mangano, Yves Renier, Garret Cotter

    Abstract: New deep learning techniques present promising new analysis methods for Imaging Atmospheric Cherenkov Telescopes (IACTs) such as the upcoming Cherenkov Telescope Array (CTA). In particular, the use of Convolutional Neural Networks (CNNs) could provide a direct event classification method that uses the entire information contained within the Cherenkov shower image, bypassing the need to Hillas para… ▽ More

    Submitted 10 March, 2021; originally announced March 2021.

    Comments: Full paper accepted in Astroparticle Physics. 39 Pages with 11 Figures. Minimal code to reproduce results in the paper available at: https://github.com/STSpencer/wavelearn_release. Some early results previously presented at ICRC2019 (doi:10.22323/1.358.0798)

    Report number: 102579

    Journal ref: Astroparticle Physics 129C (2021) 102579

  12. arXiv:2011.07125  [pdf, other

    physics.comp-ph cs.LG physics.chem-ph

    Better, Faster Fermionic Neural Networks

    Authors: James S. Spencer, David Pfau, Aleksandar Botev, W. M. C. Foulkes

    Abstract: The Fermionic Neural Network (FermiNet) is a recently-developed neural network architecture that can be used as a wavefunction Ansatz for many-electron systems, and has already demonstrated high accuracy on small systems. Here we present several improvements to the FermiNet that allow us to set new records for speed and accuracy on challenging systems. We find that increasing the size of the netwo… ▽ More

    Submitted 13 November, 2020; originally announced November 2020.

    Comments: To appear at the 3rd NeurIPS Workshop on Machine Learning and Physical Science

  13. arXiv:2003.01250  [pdf, ps, other

    cs.NE cs.LG

    Explicitly Trained Spiking Sparsity in Spiking Neural Networks with Backpropagation

    Authors: Jason M. Allred, Steven J. Spencer, Gopalakrishnan Srinivasan, Kaushik Roy

    Abstract: Spiking Neural Networks (SNNs) are being explored for their potential energy efficiency resulting from sparse, event-driven computations. Many recent works have demonstrated effective backpropagation for deep Spiking Neural Networks (SNNs) by approximating gradients over discontinuous neuron spikes or firing events. A beneficial side-effect of these surrogate gradient spiking backpropagation algor… ▽ More

    Submitted 2 March, 2020; originally announced March 2020.

  14. arXiv:1909.02487  [pdf, other

    physics.chem-ph cs.LG physics.comp-ph

    Ab-Initio Solution of the Many-Electron Schrödinger Equation with Deep Neural Networks

    Authors: David Pfau, James S. Spencer, Alexander G. de G. Matthews, W. M. C. Foulkes

    Abstract: Given access to accurate solutions of the many-electron Schrödinger equation, nearly all chemistry could be derived from first principles. Exact wavefunctions of interesting chemical systems are out of reach because they are NP-hard to compute in general, but approximations can be found using polynomially-scaling algorithms. The key challenge for many of these algorithms is the choice of wavefunct… ▽ More

    Submitted 25 March, 2021; v1 submitted 5 September, 2019; originally announced September 2019.

    Comments: Final proof for Physical Review Research

    Journal ref: Phys. Rev. Research 2, 033429 (2020)

  15. arXiv:1604.03499  [pdf, ps, other

    math.CA cs.IT

    Noisy 1-Bit Compressed Sensing Embeddings Enjoy a Restricted Isometry Property

    Authors: Scott Spencer

    Abstract: We investigate the sign-linear embeddings of 1-bit compressed sensing given by Gaussian measurements. One can give short arguments concerning a Restricted Isometry Property of such maps using Vapnik-Chervonenkis dimension of sparse hemispheres. This approach has a natural extension to the presence of additive white noise prior to quantization. Noisy one-bit mappings are shown to satisfy an RIP whe… ▽ More

    Submitted 12 April, 2016; originally announced April 2016.

    Comments: 10 pages

  16. Open-source development experiences in scientific software: the HANDE quantum Monte Carlo project

    Authors: J. S. Spencer, N. S. Blunt, W. A. Vigor, F. D. Malone, W. M. C. Foulkes, James J. Shepherd, A. J. W. Thom

    Abstract: The HANDE quantum Monte Carlo project offers accessible stochastic algorithms for general use for scientists in the field of quantum chemistry. HANDE is an ambitious and general high-performance code developed by a geographically-dispersed team with a variety of backgrounds in computational science. In the course of preparing a public, open-source release, we have taken this opportunity to step ba… ▽ More

    Submitted 14 November, 2015; v1 submitted 21 July, 2014; originally announced July 2014.

    Comments: 6 pages. Submission to WSSSPE2

    Journal ref: Journal of Open Research Software, 3, e9, 2015