-
EFT Workshop at Notre Dame
Authors:
Nick Smith,
Daniel Spitzbart,
Jennet Dickinson,
Jon Wilson,
Lindsey Gray,
Kelci Mohrman,
Saptaparna Bhattacharya,
Andrea Piccinelli,
Titas Roy,
Garyfallia Paspalaki,
Duarte Fontes,
Adam Martin,
William Shepherd,
Sergio Sánchez Cruz,
Dorival Goncalves,
Andrei Gritsan,
Harrison Prosper,
Tom Junk,
Kyle Cranmer,
Michael Peskin,
Andrew Gilbert,
Jonathon Langford,
Frank Petriello,
Luca Mantani,
Andrew Wightman
, et al. (5 additional authors not shown)
Abstract:
The LPC EFT workshop was held April 25-26, 2024 at the University of Notre Dame. The workshop was organized into five thematic sessions: "how far beyond linear" discusses issues of truncation and validity in interpretation of results with an eye towards practicality; "reconstruction-level results" visits the question of how best to design analyses directly targeting inference of EFT parameters; "l…
▽ More
The LPC EFT workshop was held April 25-26, 2024 at the University of Notre Dame. The workshop was organized into five thematic sessions: "how far beyond linear" discusses issues of truncation and validity in interpretation of results with an eye towards practicality; "reconstruction-level results" visits the question of how best to design analyses directly targeting inference of EFT parameters; "logistics of combining likelihoods" addresses the challenges of bringing a diverse array of measurements into a cohesive whole; "unfolded results" tackles the question of designing fiducial measurements for later use in EFT interpretations, and the benefits and limitations of unfolding; and "building a sample library" addresses how best to generate simulation samples for use in data analysis. This document serves as a summary of presentations, subsequent discussions, and actionable items identified over the course of the workshop.
△ Less
Submitted 20 August, 2024;
originally announced August 2024.
-
Exploring the Quantum Universe: Pathways to Innovation and Discovery in Particle Physics
Authors:
Shoji Asai,
Amalia Ballarino,
Tulika Bose,
Kyle Cranmer,
Francis-Yan Cyr-Racine,
Sarah Demers,
Cameron Geddes,
Yuri Gershtein,
Karsten Heeger,
Beate Heinemann,
JoAnne Hewett,
Patrick Huber,
Kendall Mahn,
Rachel Mandelbaum,
Jelena Maricic,
Petra Merkel,
Christopher Monahan,
Hitoshi Murayama,
Peter Onyisi,
Mark Palmer,
Tor Raubenheimer,
Mayly Sanchez,
Richard Schnee,
Sally Seidel,
Seon-Hee Seo
, et al. (7 additional authors not shown)
Abstract:
This is the report from the 2023 Particle Physics Project Prioritization Panel (P5) approved by High Energy Physics Advisory Panel (HEPAP) on December 8, 2023. The final version was made public on May 8, 2024 and submitted to DOE SC and NSF MPS.
This is the report from the 2023 Particle Physics Project Prioritization Panel (P5) approved by High Energy Physics Advisory Panel (HEPAP) on December 8, 2023. The final version was made public on May 8, 2024 and submitted to DOE SC and NSF MPS.
△ Less
Submitted 27 July, 2024;
originally announced July 2024.
-
Transforming the Bootstrap: Using Transformers to Compute Scattering Amplitudes in Planar N = 4 Super Yang-Mills Theory
Authors:
Tianji Cai,
Garrett W. Merz,
François Charton,
Niklas Nolte,
Matthias Wilhelm,
Kyle Cranmer,
Lance J. Dixon
Abstract:
We pursue the use of deep learning methods to improve state-of-the-art computations in theoretical high-energy physics. Planar N = 4 Super Yang-Mills theory is a close cousin to the theory that describes Higgs boson production at the Large Hadron Collider; its scattering amplitudes are large mathematical expressions containing integer coefficients. In this paper, we apply Transformers to predict t…
▽ More
We pursue the use of deep learning methods to improve state-of-the-art computations in theoretical high-energy physics. Planar N = 4 Super Yang-Mills theory is a close cousin to the theory that describes Higgs boson production at the Large Hadron Collider; its scattering amplitudes are large mathematical expressions containing integer coefficients. In this paper, we apply Transformers to predict these coefficients. The problem can be formulated in a language-like representation amenable to standard cross-entropy training objectives. We design two related experiments and show that the model achieves high accuracy (> 98%) on both tasks. Our work shows that Transformers can be applied successfully to problems in theoretical physics that require exact solutions.
△ Less
Submitted 9 May, 2024;
originally announced May 2024.
-
Robust Anomaly Detection for Particle Physics Using Multi-Background Representation Learning
Authors:
Abhijith Gandrakota,
Lily Zhang,
Aahlad Puli,
Kyle Cranmer,
Jennifer Ngadiuba,
Rajesh Ranganath,
Nhan Tran
Abstract:
Anomaly, or out-of-distribution, detection is a promising tool for aiding discoveries of new particles or processes in particle physics. In this work, we identify and address two overlooked opportunities to improve anomaly detection for high-energy physics. First, rather than train a generative model on the single most dominant background process, we build detection algorithms using representation…
▽ More
Anomaly, or out-of-distribution, detection is a promising tool for aiding discoveries of new particles or processes in particle physics. In this work, we identify and address two overlooked opportunities to improve anomaly detection for high-energy physics. First, rather than train a generative model on the single most dominant background process, we build detection algorithms using representation learning from multiple background types, thus taking advantage of more information to improve estimation of what is relevant for detection. Second, we generalize decorrelation to the multi-background setting, thus directly enforcing a more complete definition of robustness for anomaly detection. We demonstrate the benefit of the proposed robust multi-background anomaly detection algorithms on a high-dimensional dataset of particle decays at the Large Hadron Collider.
△ Less
Submitted 16 January, 2024;
originally announced January 2024.
-
Advances in machine-learning-based sampling motivated by lattice quantum chromodynamics
Authors:
Kyle Cranmer,
Gurtej Kanwar,
Sébastien Racanière,
Danilo J. Rezende,
Phiala E. Shanahan
Abstract:
Sampling from known probability distributions is a ubiquitous task in computational science, underlying calculations in domains from linguistics to biology and physics. Generative machine-learning (ML) models have emerged as a promising tool in this space, building on the success of this approach in applications such as image, text, and audio generation. Often, however, generative tasks in scienti…
▽ More
Sampling from known probability distributions is a ubiquitous task in computational science, underlying calculations in domains from linguistics to biology and physics. Generative machine-learning (ML) models have emerged as a promising tool in this space, building on the success of this approach in applications such as image, text, and audio generation. Often, however, generative tasks in scientific domains have unique structures and features -- such as complex symmetries and the requirement of exactness guarantees -- that present both challenges and opportunities for ML. This Perspective outlines the advances in ML-based sampling motivated by lattice quantum field theory, in particular for the theory of quantum chromodynamics. Enabling calculations of the structure and interactions of matter from our most fundamental understanding of particle physics, lattice quantum chromodynamics is one of the main consumers of open-science supercomputing worldwide. The design of ML algorithms for this application faces profound challenges, including the necessity of scaling custom ML architectures to the largest supercomputers, but also promises immense benefits, and is spurring a wave of development in ML-based sampling more broadly. In lattice field theory, if this approach can realize its early promise it will be a transformative step towards first-principles physics calculations in particle, nuclear and condensed matter physics that are intractable with traditional approaches.
△ Less
Submitted 3 September, 2023;
originally announced September 2023.
-
Normalizing flows for lattice gauge theory in arbitrary space-time dimension
Authors:
Ryan Abbott,
Michael S. Albergo,
Aleksandar Botev,
Denis Boyda,
Kyle Cranmer,
Daniel C. Hackett,
Gurtej Kanwar,
Alexander G. D. G. Matthews,
Sébastien Racanière,
Ali Razavi,
Danilo J. Rezende,
Fernando Romero-López,
Phiala E. Shanahan,
Julian M. Urban
Abstract:
Applications of normalizing flows to the sampling of field configurations in lattice gauge theory have so far been explored almost exclusively in two space-time dimensions. We report new algorithmic developments of gauge-equivariant flow architectures facilitating the generalization to higher-dimensional lattice geometries. Specifically, we discuss masked autoregressive transformations with tracta…
▽ More
Applications of normalizing flows to the sampling of field configurations in lattice gauge theory have so far been explored almost exclusively in two space-time dimensions. We report new algorithmic developments of gauge-equivariant flow architectures facilitating the generalization to higher-dimensional lattice geometries. Specifically, we discuss masked autoregressive transformations with tractable and unbiased Jacobian determinants, a key ingredient for scalable and asymptotically exact flow-based sampling algorithms. For concreteness, results from a proof-of-principle application to SU(3) lattice gauge theory in four space-time dimensions are reported.
△ Less
Submitted 3 May, 2023;
originally announced May 2023.
-
Scaling MadMiner with a deployment on REANA
Authors:
Irina Espejo,
Sinclert Pérez,
Kenyi Hurtado,
Lukas Heinrich,
Kyle Cranmer
Abstract:
MadMiner is a Python package that implements a powerful family of multivariate inference techniques that leverage matrix element information and machine learning. This multivariate approach neither requires the reduction of high-dimensional data to summary statistics nor any simplifications to the underlying physics or detector response. In this paper, we address some of the challenges arising fro…
▽ More
MadMiner is a Python package that implements a powerful family of multivariate inference techniques that leverage matrix element information and machine learning. This multivariate approach neither requires the reduction of high-dimensional data to summary statistics nor any simplifications to the underlying physics or detector response. In this paper, we address some of the challenges arising from deploying MadMiner in a real-scale HEP analysis with the goal of offering a new tool in HEP that is easily accessible. The proposed approach encapsulates a typical MadMiner pipeline into a parametrized yadage workflow described in YAML files. The general workflow is split into two yadage sub-workflows, one dealing with the physics simulations and the other with the ML inference. After that, the workflow is deployed using REANA, a reproducible research data analysis platform that takes care of flexibility, scalability, reusability, and reproducibility features. To test the performance of our method, we performed scaling experiments for a MadMiner workflow on the National Energy Research Scientific Computer (NERSC) cluster with an HT-Condor back-end. All the stages of the physics sub-workflow had a linear dependency between resources or wall time and the number of events generated. This trend has allowed us to run a typical MadMiner workflow, consisting of 11M events, in 5 hours compared to days in the original study.
△ Less
Submitted 12 April, 2023;
originally announced April 2023.
-
AI for Science: An Emerging Agenda
Authors:
Philipp Berens,
Kyle Cranmer,
Neil D. Lawrence,
Ulrike von Luxburg,
Jessica Montgomery
Abstract:
This report documents the programme and the outcomes of Dagstuhl Seminar 22382 "Machine Learning for Science: Bridging Data-Driven and Mechanistic Modelling". Today's scientific challenges are characterised by complexity. Interconnected natural, technological, and human systems are influenced by forces acting across time- and spatial-scales, resulting in complex interactions and emergent behaviour…
▽ More
This report documents the programme and the outcomes of Dagstuhl Seminar 22382 "Machine Learning for Science: Bridging Data-Driven and Mechanistic Modelling". Today's scientific challenges are characterised by complexity. Interconnected natural, technological, and human systems are influenced by forces acting across time- and spatial-scales, resulting in complex interactions and emergent behaviours. Understanding these phenomena -- and leveraging scientific advances to deliver innovative solutions to improve society's health, wealth, and well-being -- requires new ways of analysing complex systems. The transformative potential of AI stems from its widespread applicability across disciplines, and will only be achieved through integration across research domains. AI for science is a rendezvous point. It brings together expertise from $\mathrm{AI}$ and application domains; combines modelling knowledge with engineering know-how; and relies on collaboration across disciplines and between humans and machines. Alongside technical advances, the next wave of progress in the field will come from building a community of machine learning researchers, domain experts, citizen scientists, and engineers working together to design and deploy effective AI tools. This report summarises the discussions from the seminar and provides a roadmap to suggest how different communities can collaborate to deliver a new wave of progress in AI and its application for scientific discovery.
△ Less
Submitted 7 March, 2023;
originally announced March 2023.
-
Configurable calorimeter simulation for AI applications
Authors:
Francesco Armando Di Bello,
Anton Charkin-Gorbulin,
Kyle Cranmer,
Etienne Dreyer,
Sanmay Ganguly,
Eilam Gross,
Lukas Heinrich,
Lorenzo Santi,
Marumi Kado,
Nilotpal Kakati,
Patrick Rieck,
Matteo Tusoni
Abstract:
A configurable calorimeter simulation for AI (COCOA) applications is presented, based on the Geant4 toolkit and interfaced with the Pythia event generator. This open-source project is aimed to support the development of machine learning algorithms in high energy physics that rely on realistic particle shower descriptions, such as reconstruction, fast simulation, and low-level analysis. Specificati…
▽ More
A configurable calorimeter simulation for AI (COCOA) applications is presented, based on the Geant4 toolkit and interfaced with the Pythia event generator. This open-source project is aimed to support the development of machine learning algorithms in high energy physics that rely on realistic particle shower descriptions, such as reconstruction, fast simulation, and low-level analysis. Specifications such as the granularity and material of its nearly hermetic geometry are user-configurable. The tool is supplemented with simple event processing including topological clustering, jet algorithms, and a nearest-neighbors graph construction. Formatting is also provided to visualise events using the Phoenix event display software.
△ Less
Submitted 8 March, 2023; v1 submitted 3 March, 2023;
originally announced March 2023.
-
LHC EFT WG Report: Experimental Measurements and Observables
Authors:
N. Castro,
K. Cranmer,
A. V. Gritsan,
J. Howarth,
G. Magni,
K. Mimasu,
J. Rojo,
J. Roskes,
E. Vryonidou,
T. You
Abstract:
The LHC effective field theory working group gathers members of the LHC experiments and the theory community to provide a framework for the interpretation of LHC data in the context of EFT. In this note we discuss experimental observables and corresponding measurements in analysis of the Higgs, top, and electroweak data at the LHC. We review the relationship between operators and measurements rele…
▽ More
The LHC effective field theory working group gathers members of the LHC experiments and the theory community to provide a framework for the interpretation of LHC data in the context of EFT. In this note we discuss experimental observables and corresponding measurements in analysis of the Higgs, top, and electroweak data at the LHC. We review the relationship between operators and measurements relevant for the interpretation of experimental data in the context of a global SMEFT analysis. One of the goals of ongoing effort is bridging the gap between theory and experimental communities working on EFT, and in particular concerning optimised analyses. This note serves as a guide to experimental measurements and observables leading to EFT fits and establishes good practice, but does not present authoritative guidelines how those measurements should be performed.
△ Less
Submitted 16 November, 2022; v1 submitted 15 November, 2022;
originally announced November 2022.
-
Aspects of scaling and scalability for flow-based sampling of lattice QCD
Authors:
Ryan Abbott,
Michael S. Albergo,
Aleksandar Botev,
Denis Boyda,
Kyle Cranmer,
Daniel C. Hackett,
Alexander G. D. G. Matthews,
Sébastien Racanière,
Ali Razavi,
Danilo J. Rezende,
Fernando Romero-López,
Phiala E. Shanahan,
Julian M. Urban
Abstract:
Recent applications of machine-learned normalizing flows to sampling in lattice field theory suggest that such methods may be able to mitigate critical slowing down and topological freezing. However, these demonstrations have been at the scale of toy models, and it remains to be determined whether they can be applied to state-of-the-art lattice quantum chromodynamics calculations. Assessing the vi…
▽ More
Recent applications of machine-learned normalizing flows to sampling in lattice field theory suggest that such methods may be able to mitigate critical slowing down and topological freezing. However, these demonstrations have been at the scale of toy models, and it remains to be determined whether they can be applied to state-of-the-art lattice quantum chromodynamics calculations. Assessing the viability of sampling algorithms for lattice field theory at scale has traditionally been accomplished using simple cost scaling laws, but as we discuss in this work, their utility is limited for flow-based approaches. We conclude that flow-based approaches to sampling are better thought of as a broad family of algorithms with different scaling properties, and that scalability must be assessed experimentally.
△ Less
Submitted 14 November, 2022;
originally announced November 2022.
-
The Future of High Energy Physics Software and Computing
Authors:
V. Daniel Elvira,
Steven Gottlieb,
Oliver Gutsche,
Benjamin Nachman,
S. Bailey,
W. Bhimji,
P. Boyle,
G. Cerati,
M. Carrasco Kind,
K. Cranmer,
G. Davies,
V. D. Elvira,
R. Gardner,
K. Heitmann,
M. Hildreth,
W. Hopkins,
T. Humble,
M. Lin,
P. Onyisi,
J. Qiang,
K. Pedro,
G. Perdue,
A. Roberts,
M. Savage,
P. Shanahan
, et al. (3 additional authors not shown)
Abstract:
Software and Computing (S&C) are essential to all High Energy Physics (HEP) experiments and many theoretical studies. The size and complexity of S&C are now commensurate with that of experimental instruments, playing a critical role in experimental design, data acquisition/instrumental control, reconstruction, and analysis. Furthermore, S&C often plays a leading role in driving the precision of th…
▽ More
Software and Computing (S&C) are essential to all High Energy Physics (HEP) experiments and many theoretical studies. The size and complexity of S&C are now commensurate with that of experimental instruments, playing a critical role in experimental design, data acquisition/instrumental control, reconstruction, and analysis. Furthermore, S&C often plays a leading role in driving the precision of theoretical calculations and simulations. Within this central role in HEP, S&C has been immensely successful over the last decade. This report looks forward to the next decade and beyond, in the context of the 2021 Particle Physics Community Planning Exercise ("Snowmass") organized by the Division of Particles and Fields (DPF) of the American Physical Society.
△ Less
Submitted 8 November, 2022; v1 submitted 11 October, 2022;
originally announced October 2022.
-
Reinterpretation and Long-Term Preservation of Data and Code
Authors:
Stephen Bailey,
K. S. Cranmer,
Matthew Feickert,
Rob Fine,
Sabine Kraml,
Clemens Lange
Abstract:
Careful preservation of experimental data, simulations, analysis products, and theoretical work maximizes their long-term scientific return on investment by enabling new analyses and reinterpretation of the results in the future. Key infrastructure and technical developments needed for some high-value science targets are not in scope for the operations program of the large experiments and are ofte…
▽ More
Careful preservation of experimental data, simulations, analysis products, and theoretical work maximizes their long-term scientific return on investment by enabling new analyses and reinterpretation of the results in the future. Key infrastructure and technical developments needed for some high-value science targets are not in scope for the operations program of the large experiments and are often not effectively funded. Increasingly, the science goals of our projects require contributions that span the boundaries between individual experiments and surveys, and between the theoretical and experimental communities. Furthermore, the computational requirements and technical sophistication of this work is increasing. As a result, it is imperative that the funding agencies create programs that can devote significant resources to these efforts outside of the context of the operations of individual major experiments, including smaller experiments and theory/simulation work. In this Snowmass 2021 Computational Frontier topical group report (CompF7: Reinterpretation and long-term preservation of data and code), we summarize the current state of the field and make recommendations for the future.
△ Less
Submitted 16 September, 2022;
originally announced September 2022.
-
Sampling QCD field configurations with gauge-equivariant flow models
Authors:
Ryan Abbott,
Michael S. Albergo,
Aleksandar Botev,
Denis Boyda,
Kyle Cranmer,
Daniel C. Hackett,
Gurtej Kanwar,
Alexander G. D. G. Matthews,
Sébastien Racanière,
Ali Razavi,
Danilo J. Rezende,
Fernando Romero-López,
Phiala E. Shanahan,
Julian M. Urban
Abstract:
Machine learning methods based on normalizing flows have been shown to address important challenges, such as critical slowing-down and topological freezing, in the sampling of gauge field configurations in simple lattice field theories. A critical question is whether this success will translate to studies of QCD. This Proceedings presents a status update on advances in this area. In particular, it…
▽ More
Machine learning methods based on normalizing flows have been shown to address important challenges, such as critical slowing-down and topological freezing, in the sampling of gauge field configurations in simple lattice field theories. A critical question is whether this success will translate to studies of QCD. This Proceedings presents a status update on advances in this area. In particular, it is illustrated how recently developed algorithmic components may be combined to construct flow-based sampling algorithms for QCD in four dimensions. The prospects and challenges for future use of this approach in at-scale applications are summarized.
△ Less
Submitted 20 August, 2022; v1 submitted 7 August, 2022;
originally announced August 2022.
-
Gauge-equivariant flow models for sampling in lattice field theories with pseudofermions
Authors:
Ryan Abbott,
Michael S. Albergo,
Denis Boyda,
Kyle Cranmer,
Daniel C. Hackett,
Gurtej Kanwar,
Sébastien Racanière,
Danilo J. Rezende,
Fernando Romero-López,
Phiala E. Shanahan,
Betsy Tian,
Julian M. Urban
Abstract:
This work presents gauge-equivariant architectures for flow-based sampling in fermionic lattice field theories using pseudofermions as stochastic estimators for the fermionic determinant. This is the default approach in state-of-the-art lattice field theory calculations, making this development critical to the practical application of flow models to theories such as QCD. Methods by which flow-base…
▽ More
This work presents gauge-equivariant architectures for flow-based sampling in fermionic lattice field theories using pseudofermions as stochastic estimators for the fermionic determinant. This is the default approach in state-of-the-art lattice field theory calculations, making this development critical to the practical application of flow models to theories such as QCD. Methods by which flow-based sampling approaches can be improved via standard techniques such as even/odd preconditioning and the Hasenbusch factorization are also outlined. Numerical demonstrations in two-dimensional U(1) and SU(3) gauge theories with $N_f=2$ flavors of fermions are provided.
△ Less
Submitted 16 October, 2022; v1 submitted 18 July, 2022;
originally announced July 2022.
-
Data and Analysis Preservation, Recasting, and Reinterpretation
Authors:
Stephen Bailey,
Christian Bierlich,
Andy Buckley,
Jon Butterworth,
Kyle Cranmer,
Matthew Feickert,
Lukas Heinrich,
Axel Huebl,
Sabine Kraml,
Anders Kvellestad,
Clemens Lange,
Andre Lessa,
Kati Lassila-Perini,
Christine Nattrass,
Mark S. Neubauer,
Sezen Sekmen,
Giordon Stark,
Graeme Watt
Abstract:
We make the case for the systematic, reliable preservation of event-wise data, derived data products, and executable analysis code. This preservation enables the analyses' long-term future reuse, in order to maximise the scientific impact of publicly funded particle-physics experiments. We cover the needs of both the experimental and theoretical particle physics communities, and outline the goals…
▽ More
We make the case for the systematic, reliable preservation of event-wise data, derived data products, and executable analysis code. This preservation enables the analyses' long-term future reuse, in order to maximise the scientific impact of publicly funded particle-physics experiments. We cover the needs of both the experimental and theoretical particle physics communities, and outline the goals and benefits that are uniquely enabled by analysis recasting and reinterpretation. We also discuss technical challenges and infrastructure needs, as well as sociological challenges and changes, and give summary recommendations to the particle-physics community.
△ Less
Submitted 18 March, 2022;
originally announced March 2022.
-
Broadening the scope of Education, Career and Open Science in HEP
Authors:
Sudhir Malik,
David DeMuth,
Sijbrand de Jong,
Randal Ruchti,
Savannah Thais,
Guillermo Fidalgo,
Ken Heller,
Mathew Muether,
Minerba Betancourt,
Meenakshi Narain,
Tiffany R. Lewis,
Kyle Cranmer,
Gordon Watts
Abstract:
High Energy Particle Physics (HEP) faces challenges over the coming decades with a need to attract young people to the field and STEM careers, as well as a need to recognize, promote and sustain those in the field who are making important contributions to the research effort across the many specialties needed to deliver the science. Such skills can also serve as attractors for students who may not…
▽ More
High Energy Particle Physics (HEP) faces challenges over the coming decades with a need to attract young people to the field and STEM careers, as well as a need to recognize, promote and sustain those in the field who are making important contributions to the research effort across the many specialties needed to deliver the science. Such skills can also serve as attractors for students who may not want to pursue a PhD in HEP but use them as a springboard to other STEM careers. This paper reviews the challenges and develops strategies to correct the disparities to help transform the particle physics field into a stronger and more diverse ecosystem of talent and expertise, with the expectation of long-lasting scientific and societal benefits.
△ Less
Submitted 15 March, 2022;
originally announced March 2022.
-
Analysis Facilities for HL-LHC
Authors:
Doug Benjamin,
Kenneth Bloom,
Brian Bockelman,
Lincoln Bryant,
Kyle Cranmer,
Rob Gardner,
Chris Hollowell,
Burt Holzman,
Eric Lançon,
Ofer Rind,
Oksana Shadura,
Wei Yang
Abstract:
The HL-LHC presents significant challenges for the HEP analysis community. The number of events in each analysis is expected to increase by an order of magnitude and new techniques are expected to be required; both challenges necessitate new services and approaches for analysis facilities. These services are expected to provide new capabilities, a larger scale, and different access modalities (com…
▽ More
The HL-LHC presents significant challenges for the HEP analysis community. The number of events in each analysis is expected to increase by an order of magnitude and new techniques are expected to be required; both challenges necessitate new services and approaches for analysis facilities. These services are expected to provide new capabilities, a larger scale, and different access modalities (complementing -- but distinct from -- traditional batch-oriented approaches). To facilitate this transition, the US-LHC community is actively investing in analysis facilities to provide a testbed for those developing new analysis systems and to demonstrate new techniques for service delivery. This whitepaper outlines the existing activities within the US LHC community in this R&D area, the short- to medium-term goals, and the outline of common goals and milestones.
△ Less
Submitted 16 March, 2022; v1 submitted 15 March, 2022;
originally announced March 2022.
-
Machine Learning and LHC Event Generation
Authors:
Anja Butter,
Tilman Plehn,
Steffen Schumann,
Simon Badger,
Sascha Caron,
Kyle Cranmer,
Francesco Armando Di Bello,
Etienne Dreyer,
Stefano Forte,
Sanmay Ganguly,
Dorival Gonçalves,
Eilam Gross,
Theo Heimel,
Gudrun Heinrich,
Lukas Heinrich,
Alexander Held,
Stefan Höche,
Jessica N. Howard,
Philip Ilten,
Joshua Isaacson,
Timo Janßen,
Stephen Jones,
Marumi Kado,
Michael Kagan,
Gregor Kasieczka
, et al. (26 additional authors not shown)
Abstract:
First-principle simulations are at the heart of the high-energy physics research program. They link the vast data output of multi-purpose detectors with fundamental theory predictions and interpretation. This review illustrates a wide range of applications of modern machine learning to event generation and simulation-based inference, including conceptional developments driven by the specific requi…
▽ More
First-principle simulations are at the heart of the high-energy physics research program. They link the vast data output of multi-purpose detectors with fundamental theory predictions and interpretation. This review illustrates a wide range of applications of modern machine learning to event generation and simulation-based inference, including conceptional developments driven by the specific requirements of particle physics. New ideas and tools developed at the interface of particle physics and machine learning will improve the speed and precision of forward simulations, handle the complexity of collision data, and enhance inference as an inverse simulation problem.
△ Less
Submitted 28 December, 2022; v1 submitted 14 March, 2022;
originally announced March 2022.
-
Flow-based sampling in the lattice Schwinger model at criticality
Authors:
Michael S. Albergo,
Denis Boyda,
Kyle Cranmer,
Daniel C. Hackett,
Gurtej Kanwar,
Sébastien Racanière,
Danilo J. Rezende,
Fernando Romero-López,
Phiala E. Shanahan,
Julian M. Urban
Abstract:
Recent results suggest that flow-based algorithms may provide efficient sampling of field distributions for lattice field theory applications, such as studies of quantum chromodynamics and the Schwinger model. In this work, we provide a numerical demonstration of robust flow-based sampling in the Schwinger model at the critical value of the fermion mass. In contrast, at the same parameters, conven…
▽ More
Recent results suggest that flow-based algorithms may provide efficient sampling of field distributions for lattice field theory applications, such as studies of quantum chromodynamics and the Schwinger model. In this work, we provide a numerical demonstration of robust flow-based sampling in the Schwinger model at the critical value of the fermion mass. In contrast, at the same parameters, conventional methods fail to sample all parts of configuration space, leading to severely underestimated uncertainties.
△ Less
Submitted 23 February, 2022;
originally announced February 2022.
-
The Quantum Trellis: A classical algorithm for sampling the parton shower with interference effects
Authors:
Sebastian Macaluso,
Kyle Cranmer
Abstract:
Simulations of high-energy particle collisions, such as those used at the Large Hadron Collider, are based on quantum field theory; however, many approximations are made in practice. For example, the simulation of the parton shower, which gives rise to objects called `jets', is based on a semi-classical approximation that neglects various interference effects. While there is a desire to incorporat…
▽ More
Simulations of high-energy particle collisions, such as those used at the Large Hadron Collider, are based on quantum field theory; however, many approximations are made in practice. For example, the simulation of the parton shower, which gives rise to objects called `jets', is based on a semi-classical approximation that neglects various interference effects. While there is a desire to incorporate interference effects, new computational techniques are needed to cope with the exponential growth in complexity associated to quantum processes. We present a classical algorithm called the quantum trellis to efficiently compute the un-normalized probability density over N-body phase space including all interference effects, and we pair this with an MCMC-based sampling strategy. This provides a potential path forward for classical computers and a strong baseline for approaches based on quantum computing.
△ Less
Submitted 23 December, 2021;
originally announced December 2021.
-
Simulation Intelligence: Towards a New Generation of Scientific Methods
Authors:
Alexander Lavin,
David Krakauer,
Hector Zenil,
Justin Gottschlich,
Tim Mattson,
Johann Brehmer,
Anima Anandkumar,
Sanjay Choudry,
Kamil Rocki,
Atılım Güneş Baydin,
Carina Prunkl,
Brooks Paige,
Olexandr Isayev,
Erik Peterson,
Peter L. McMahon,
Jakob Macke,
Kyle Cranmer,
Jiaxin Zhang,
Haruko Wainwright,
Adi Hanuka,
Manuela Veloso,
Samuel Assefa,
Stephan Zheng,
Avi Pfeffer
Abstract:
The original "Seven Motifs" set forth a roadmap of essential methods for the field of scientific computing, where a motif is an algorithmic method that captures a pattern of computation and data movement. We present the "Nine Motifs of Simulation Intelligence", a roadmap for the development and integration of the essential algorithms necessary for a merger of scientific computing, scientific simul…
▽ More
The original "Seven Motifs" set forth a roadmap of essential methods for the field of scientific computing, where a motif is an algorithmic method that captures a pattern of computation and data movement. We present the "Nine Motifs of Simulation Intelligence", a roadmap for the development and integration of the essential algorithms necessary for a merger of scientific computing, scientific simulation, and artificial intelligence. We call this merger simulation intelligence (SI), for short. We argue the motifs of simulation intelligence are interconnected and interdependent, much like the components within the layers of an operating system. Using this metaphor, we explore the nature of each layer of the simulation intelligence operating system stack (SI-stack) and the motifs therein: (1) Multi-physics and multi-scale modeling; (2) Surrogate modeling and emulation; (3) Simulation-based inference; (4) Causal modeling and inference; (5) Agent-based modeling; (6) Probabilistic programming; (7) Differentiable programming; (8) Open-ended optimization; (9) Machine programming. We believe coordinated efforts between motifs offers immense opportunity to accelerate scientific discovery, from solving inverse problems in synthetic biology and climate science, to directing nuclear energy experiments and predicting emergent behavior in socioeconomic settings. We elaborate on each layer of the SI-stack, detailing the state-of-art methods, presenting examples to highlight challenges and opportunities, and advocating for specific ways to advance the motifs and the synergies from their combinations. Advancing and integrating these technologies can enable a robust and efficient hypothesis-simulation-analysis type of scientific method, which we introduce with several use-cases for human-machine teaming and automated science.
△ Less
Submitted 27 November, 2022; v1 submitted 6 December, 2021;
originally announced December 2021.
-
A neural simulation-based inference approach for characterizing the Galactic Center $γ$-ray excess
Authors:
Siddharth Mishra-Sharma,
Kyle Cranmer
Abstract:
The nature of the Fermi gamma-ray Galactic Center Excess (GCE) has remained a persistent mystery for over a decade. Although the excess is broadly compatible with emission expected due to dark matter annihilation, an explanation in terms of a population of unresolved astrophysical point sources e.g., millisecond pulsars, remains viable. The effort to uncover the origin of the GCE is hampered in pa…
▽ More
The nature of the Fermi gamma-ray Galactic Center Excess (GCE) has remained a persistent mystery for over a decade. Although the excess is broadly compatible with emission expected due to dark matter annihilation, an explanation in terms of a population of unresolved astrophysical point sources e.g., millisecond pulsars, remains viable. The effort to uncover the origin of the GCE is hampered in particular by an incomplete understanding of diffuse emission of Galactic origin. This can lead to spurious features that make it difficult to robustly differentiate smooth emission, as expected for a dark matter origin, from more "clumpy" emission expected for a population of relatively bright, unresolved point sources. We use recent advancements in the field of simulation-based inference, in particular density estimation techniques using normalizing flows, in order to characterize the contribution of modeled components, including unresolved point source populations, to the GCE. Compared to traditional techniques based on the statistical distribution of photon counts, our machine learning-based method is able to utilize more of the information contained in a given model of the Galactic Center emission, and in particular can perform posterior parameter estimation while accounting for pixel-to-pixel spatial correlations in the gamma-ray map. This makes the method demonstrably more resilient to certain forms of model misspecification. On application to Fermi data, the method generically attributes a smaller fraction of the GCE flux to unresolved point sources when compared to traditional approaches. We nevertheless infer such a contribution to make up a non-negligible fraction of the GCE across all analysis variations considered, with at least $38^{+9}_{-19}\%$ of the excess attributed to unresolved point sources in our baseline analysis.
△ Less
Submitted 27 March, 2022; v1 submitted 13 October, 2021;
originally announced October 2021.
-
Publishing statistical models: Getting the most out of particle physics experiments
Authors:
Kyle Cranmer,
Sabine Kraml,
Harrison B. Prosper,
Philip Bechtle,
Florian U. Bernlochner,
Itay M. Bloch,
Enzo Canonero,
Marcin Chrzaszcz,
Andrea Coccaro,
Jan Conrad,
Glen Cowan,
Matthew Feickert,
Nahuel Ferreiro Iachellini,
Andrew Fowlie,
Lukas Heinrich,
Alexander Held,
Thomas Kuhr,
Anders Kvellestad,
Maeve Madigan,
Farvah Mahmoudi,
Knut Dundas Morå,
Mark S. Neubauer,
Maurizio Pierini,
Juan Rojo,
Sezen Sekmen
, et al. (8 additional authors not shown)
Abstract:
The statistical models used to derive the results of experimental analyses are of incredible scientific value and are essential information for analysis preservation and reuse. In this paper, we make the scientific case for systematically publishing the full statistical models and discuss the technical developments that make this practical. By means of a variety of physics cases -- including parto…
▽ More
The statistical models used to derive the results of experimental analyses are of incredible scientific value and are essential information for analysis preservation and reuse. In this paper, we make the scientific case for systematically publishing the full statistical models and discuss the technical developments that make this practical. By means of a variety of physics cases -- including parton distribution functions, Higgs boson measurements, effective field theory interpretations, direct searches for new physics, heavy flavor physics, direct dark matter detection, world averages, and beyond the Standard Model global fits -- we illustrate how detailed information on the statistical modelling can enhance the short- and long-term impact of experimental results.
△ Less
Submitted 10 September, 2021;
originally announced September 2021.
-
Flow-based sampling for multimodal distributions in lattice field theory
Authors:
Daniel C. Hackett,
Chung-Chun Hsieh,
Michael S. Albergo,
Denis Boyda,
Jiunn-Wei Chen,
Kai-Feng Chen,
Kyle Cranmer,
Gurtej Kanwar,
Phiala E. Shanahan
Abstract:
Recent results have demonstrated that samplers constructed with flow-based generative models are a promising new approach for configuration generation in lattice field theory. In this paper, we present a set of methods to construct flow models for targets with multiple separated modes (i.e. theories with multiple vacua). We demonstrate the application of these methods to modeling two-dimensional r…
▽ More
Recent results have demonstrated that samplers constructed with flow-based generative models are a promising new approach for configuration generation in lattice field theory. In this paper, we present a set of methods to construct flow models for targets with multiple separated modes (i.e. theories with multiple vacua). We demonstrate the application of these methods to modeling two-dimensional real scalar field theory in its symmetry-broken phase. In this context we investigate the performance of different flow-based sampling algorithms, including a composite sampling algorithm where flow-based proposals are occasionally augmented by applying updates using traditional algorithms like HMC.
△ Less
Submitted 1 July, 2021;
originally announced July 2021.
-
Flow-based sampling for fermionic lattice field theories
Authors:
Michael S. Albergo,
Gurtej Kanwar,
Sébastien Racanière,
Danilo J. Rezende,
Julian M. Urban,
Denis Boyda,
Kyle Cranmer,
Daniel C. Hackett,
Phiala E. Shanahan
Abstract:
Algorithms based on normalizing flows are emerging as promising machine learning approaches to sampling complicated probability distributions in a way that can be made asymptotically exact. In the context of lattice field theory, proof-of-principle studies have demonstrated the effectiveness of this approach for scalar theories, gauge theories, and statistical systems. This work develops approache…
▽ More
Algorithms based on normalizing flows are emerging as promising machine learning approaches to sampling complicated probability distributions in a way that can be made asymptotically exact. In the context of lattice field theory, proof-of-principle studies have demonstrated the effectiveness of this approach for scalar theories, gauge theories, and statistical systems. This work develops approaches that enable flow-based sampling of theories with dynamical fermions, which is necessary for the technique to be applied to lattice field theory studies of the Standard Model of particle physics and many condensed matter systems. As a practical demonstration, these methods are applied to the sampling of field configurations for a two-dimensional theory of massless staggered fermions coupled to a scalar field via a Yukawa interaction.
△ Less
Submitted 28 December, 2021; v1 submitted 10 June, 2021;
originally announced June 2021.
-
Reframing Jet Physics with New Computational Methods
Authors:
Kyle Cranmer,
Matthew Drnevich,
Sebastian Macaluso,
Duccio Pappadopulo
Abstract:
We reframe common tasks in jet physics in probabilistic terms, including jet reconstruction, Monte Carlo tuning, matrix element - parton shower matching for large jet multiplicity, and efficient event generation of jets in complex, signal-like regions of phase space. We also introduce Ginkgo, a simplified, generative model for jets, that facilitates research into these tasks with techniques from s…
▽ More
We reframe common tasks in jet physics in probabilistic terms, including jet reconstruction, Monte Carlo tuning, matrix element - parton shower matching for large jet multiplicity, and efficient event generation of jets in complex, signal-like regions of phase space. We also introduce Ginkgo, a simplified, generative model for jets, that facilitates research into these tasks with techniques from statistics, machine learning, and combinatorial optimization. We review some of the recent research in this direction that has been enabled with Ginkgo. We show how probabilistic programming can be used to efficiently sample the showering process, how a novel trellis algorithm can be used to efficiently marginalize over the enormous number of clustering histories for the same observed particles, and how dynamic programming, A* search, and reinforcement learning can be used to find the maximum likelihood clustering in this enormous search space. This work builds bridges with work in hierarchical clustering, statistics, combinatorial optmization, and reinforcement learning.
△ Less
Submitted 21 May, 2021;
originally announced May 2021.
-
Exact and Approximate Hierarchical Clustering Using A*
Authors:
Craig S. Greenberg,
Sebastian Macaluso,
Nicholas Monath,
Avinava Dubey,
Patrick Flaherty,
Manzil Zaheer,
Amr Ahmed,
Kyle Cranmer,
Andrew McCallum
Abstract:
Hierarchical clustering is a critical task in numerous domains. Many approaches are based on heuristics and the properties of the resulting clusterings are studied post hoc. However, in several applications, there is a natural cost function that can be used to characterize the quality of the clustering. In those cases, hierarchical clustering can be seen as a combinatorial optimization problem. To…
▽ More
Hierarchical clustering is a critical task in numerous domains. Many approaches are based on heuristics and the properties of the resulting clusterings are studied post hoc. However, in several applications, there is a natural cost function that can be used to characterize the quality of the clustering. In those cases, hierarchical clustering can be seen as a combinatorial optimization problem. To that end, we introduce a new approach based on A* search. We overcome the prohibitively large search space by combining A* with a novel \emph{trellis} data structure. This combination results in an exact algorithm that scales beyond previous state of the art, from a search space with $10^{12}$ trees to $10^{15}$ trees, and an approximate algorithm that improves over baselines, even in enormous search spaces that contain more than $10^{1000}$ trees. We empirically demonstrate that our method achieves substantially higher quality results than baselines for a particle physics use case and other clustering benchmarks. We describe how our method provides significantly improved theoretical bounds on the time and space complexity of A* for clustering.
△ Less
Submitted 14 April, 2021;
originally announced April 2021.
-
A deep search for decaying dark matter with XMM-Newton blank-sky observations
Authors:
Joshua W. Foster,
Marius Kongsore,
Christopher Dessert,
Yujin Park,
Nicholas L. Rodd,
Kyle Cranmer,
Benjamin R. Safdi
Abstract:
Sterile neutrinos with masses in the keV range are well-motivated extensions to the Standard Model that could explain the observed neutrino masses while also making up the dark matter (DM) of the Universe. If sterile neutrinos are DM then they may slowly decay into active neutrinos and photons, giving rise to the possibility of their detection through narrow spectral features in astrophysical X-ra…
▽ More
Sterile neutrinos with masses in the keV range are well-motivated extensions to the Standard Model that could explain the observed neutrino masses while also making up the dark matter (DM) of the Universe. If sterile neutrinos are DM then they may slowly decay into active neutrinos and photons, giving rise to the possibility of their detection through narrow spectral features in astrophysical X-ray data sets. In this work, we perform the most sensitive search to date for this and other decaying DM scenarios across the mass range from 5 to 16 keV using archival XMM-Newton data. We reduce 547 Ms of data from both the MOS and PN instruments using observations taken across the full sky and then use this data to search for evidence of DM decay in the ambient halo of the Milky Way. We determine the instrumental and astrophysical baselines with data taken far away from the Galactic Center, and use Gaussian Process modeling to capture additional continuum background contributions. No evidence is found for unassociated X-ray lines, leading us to produce the strongest constraints to date on decaying DM in this mass range.
△ Less
Submitted 4 May, 2021; v1 submitted 3 February, 2021;
originally announced February 2021.
-
Introduction to Normalizing Flows for Lattice Field Theory
Authors:
Michael S. Albergo,
Denis Boyda,
Daniel C. Hackett,
Gurtej Kanwar,
Kyle Cranmer,
Sébastien Racanière,
Danilo Jimenez Rezende,
Phiala E. Shanahan
Abstract:
This notebook tutorial demonstrates a method for sampling Boltzmann distributions of lattice field theories using a class of machine learning models known as normalizing flows. The ideas and approaches proposed in arXiv:1904.12072, arXiv:2002.02428, and arXiv:2003.06413 are reviewed and a concrete implementation of the framework is presented. We apply this framework to a lattice scalar field theor…
▽ More
This notebook tutorial demonstrates a method for sampling Boltzmann distributions of lattice field theories using a class of machine learning models known as normalizing flows. The ideas and approaches proposed in arXiv:1904.12072, arXiv:2002.02428, and arXiv:2003.06413 are reviewed and a concrete implementation of the framework is presented. We apply this framework to a lattice scalar field theory and to U(1) gauge theory, explicitly encoding gauge symmetries in the flow-based approach to the latter. This presentation is intended to be interactive and working with the attached Jupyter notebook is recommended.
△ Less
Submitted 6 August, 2021; v1 submitted 20 January, 2021;
originally announced January 2021.
-
Hierarchical clustering in particle physics through reinforcement learning
Authors:
Johann Brehmer,
Sebastian Macaluso,
Duccio Pappadopulo,
Kyle Cranmer
Abstract:
Particle physics experiments often require the reconstruction of decay patterns through a hierarchical clustering of the observed final-state particles. We show that this task can be phrased as a Markov Decision Process and adapt reinforcement learning algorithms to solve it. In particular, we show that Monte-Carlo Tree Search guided by a neural policy can construct high-quality hierarchical clust…
▽ More
Particle physics experiments often require the reconstruction of decay patterns through a hierarchical clustering of the observed final-state particles. We show that this task can be phrased as a Markov Decision Process and adapt reinforcement learning algorithms to solve it. In particular, we show that Monte-Carlo Tree Search guided by a neural policy can construct high-quality hierarchical clusterings and outperform established greedy and beam search baselines.
△ Less
Submitted 18 December, 2020; v1 submitted 16 November, 2020;
originally announced November 2020.
-
Semi-parametric $γ$-ray modeling with Gaussian processes and variational inference
Authors:
Siddharth Mishra-Sharma,
Kyle Cranmer
Abstract:
Mismodeling the uncertain, diffuse emission of Galactic origin can seriously bias the characterization of astrophysical gamma-ray data, particularly in the region of the Inner Milky Way where such emission can make up over 80% of the photon counts observed at ~GeV energies. We introduce a novel class of methods that use Gaussian processes and variational inference to build flexible background and…
▽ More
Mismodeling the uncertain, diffuse emission of Galactic origin can seriously bias the characterization of astrophysical gamma-ray data, particularly in the region of the Inner Milky Way where such emission can make up over 80% of the photon counts observed at ~GeV energies. We introduce a novel class of methods that use Gaussian processes and variational inference to build flexible background and signal models for gamma-ray analyses with the goal of enabling a more robust interpretation of the make-up of the gamma-ray sky, particularly focusing on characterizing potential signals of dark matter in the Galactic Center with data from the Fermi telescope.
△ Less
Submitted 20 October, 2020;
originally announced October 2020.
-
Simulation-based inference methods for particle physics
Authors:
Johann Brehmer,
Kyle Cranmer
Abstract:
Our predictions for particle physics processes are realized in a chain of complex simulators. They allow us to generate high-fidelity simulated data, but they are not well-suited for inference on the theory parameters with observed data. We explain why the likelihood function of high-dimensional LHC data cannot be explicitly evaluated, why this matters for data analysis, and reframe what the field…
▽ More
Our predictions for particle physics processes are realized in a chain of complex simulators. They allow us to generate high-fidelity simulated data, but they are not well-suited for inference on the theory parameters with observed data. We explain why the likelihood function of high-dimensional LHC data cannot be explicitly evaluated, why this matters for data analysis, and reframe what the field has traditionally done to circumvent this problem. We then review new simulation-based inference methods that let us directly analyze high-dimensional data by combining machine learning techniques and information from the simulator. Initial studies indicate that these techniques have the potential to substantially improve the precision of LHC measurements. Finally, we discuss probabilistic programming, an emerging paradigm that lets us extend inference to the latent process of the simulator.
△ Less
Submitted 2 November, 2020; v1 submitted 13 October, 2020;
originally announced October 2020.
-
Sampling using $SU(N)$ gauge equivariant flows
Authors:
Denis Boyda,
Gurtej Kanwar,
Sébastien Racanière,
Danilo Jimenez Rezende,
Michael S. Albergo,
Kyle Cranmer,
Daniel C. Hackett,
Phiala E. Shanahan
Abstract:
We develop a flow-based sampling algorithm for $SU(N)$ lattice gauge theories that is gauge-invariant by construction. Our key contribution is constructing a class of flows on an $SU(N)$ variable (or on a $U(N)$ variable by a simple alternative) that respect matrix conjugation symmetry. We apply this technique to sample distributions of single $SU(N)$ variables and to construct flow-based samplers…
▽ More
We develop a flow-based sampling algorithm for $SU(N)$ lattice gauge theories that is gauge-invariant by construction. Our key contribution is constructing a class of flows on an $SU(N)$ variable (or on a $U(N)$ variable by a simple alternative) that respect matrix conjugation symmetry. We apply this technique to sample distributions of single $SU(N)$ variables and to construct flow-based samplers for $SU(2)$ and $SU(3)$ lattice gauge theory in two dimensions.
△ Less
Submitted 18 September, 2020; v1 submitted 12 August, 2020;
originally announced August 2020.
-
Secondary Vertex Finding in Jets with Neural Networks
Authors:
Jonathan Shlomi,
Sanmay Ganguly,
Eilam Gross,
Kyle Cranmer,
Yaron Lipman,
Hadar Serviansky,
Haggai Maron,
Nimrod Segol
Abstract:
Jet classification is an important ingredient in measurements and searches for new physics at particle coliders, and secondary vertex reconstruction is a key intermediate step in building powerful jet classifiers. We use a neural network to perform vertex finding inside jets in order to improve the classification performance, with a focus on separation of bottom vs. charm flavor tagging. We implem…
▽ More
Jet classification is an important ingredient in measurements and searches for new physics at particle coliders, and secondary vertex reconstruction is a key intermediate step in building powerful jet classifiers. We use a neural network to perform vertex finding inside jets in order to improve the classification performance, with a focus on separation of bottom vs. charm flavor tagging. We implement a novel, universal set-to-graph model, which takes into account information from all tracks in a jet to determine if pairs of tracks originated from a common vertex. We explore different performance metrics and find our method to outperform traditional approaches in accurate secondary vertex reconstruction. We also find that improved vertex finding leads to a significant improvement in jet classification performance.
△ Less
Submitted 27 May, 2021; v1 submitted 6 August, 2020;
originally announced August 2020.
-
Discovering Symbolic Models from Deep Learning with Inductive Biases
Authors:
Miles Cranmer,
Alvaro Sanchez-Gonzalez,
Peter Battaglia,
Rui Xu,
Kyle Cranmer,
David Spergel,
Shirley Ho
Abstract:
We develop a general approach to distill symbolic representations of a learned deep model by introducing strong inductive biases. We focus on Graph Neural Networks (GNNs). The technique works as follows: we first encourage sparse latent representations when we train a GNN in a supervised setting, then we apply symbolic regression to components of the learned model to extract explicit physical rela…
▽ More
We develop a general approach to distill symbolic representations of a learned deep model by introducing strong inductive biases. We focus on Graph Neural Networks (GNNs). The technique works as follows: we first encourage sparse latent representations when we train a GNN in a supervised setting, then we apply symbolic regression to components of the learned model to extract explicit physical relations. We find the correct known equations, including force laws and Hamiltonians, can be extracted from the neural network. We then apply our method to a non-trivial cosmology example-a detailed dark matter simulation-and discover a new analytic formula which can predict the concentration of dark matter from the mass distribution of nearby cosmic structures. The symbolic expressions extracted from the GNN using our technique also generalized to out-of-distribution data better than the GNN itself. Our approach offers alternative directions for interpreting neural networks and discovering novel physical principles from the representations they learn.
△ Less
Submitted 17 November, 2020; v1 submitted 19 June, 2020;
originally announced June 2020.
-
Flows for simultaneous manifold learning and density estimation
Authors:
Johann Brehmer,
Kyle Cranmer
Abstract:
We introduce manifold-learning flows (M-flows), a new class of generative models that simultaneously learn the data manifold as well as a tractable probability density on that manifold. Combining aspects of normalizing flows, GANs, autoencoders, and energy-based models, they have the potential to represent datasets with a manifold structure more faithfully and provide handles on dimensionality red…
▽ More
We introduce manifold-learning flows (M-flows), a new class of generative models that simultaneously learn the data manifold as well as a tractable probability density on that manifold. Combining aspects of normalizing flows, GANs, autoencoders, and energy-based models, they have the potential to represent datasets with a manifold structure more faithfully and provide handles on dimensionality reduction, denoising, and out-of-distribution detection. We argue why such models should not be trained by maximum likelihood alone and present a new training algorithm that separates manifold and density updates. In a range of experiments we demonstrate how M-flows learn the data manifold and allow for better inference than standard flows in the ambient data space.
△ Less
Submitted 13 November, 2020; v1 submitted 30 March, 2020;
originally announced March 2020.
-
Equivariant flow-based sampling for lattice gauge theory
Authors:
Gurtej Kanwar,
Michael S. Albergo,
Denis Boyda,
Kyle Cranmer,
Daniel C. Hackett,
Sébastien Racanière,
Danilo Jimenez Rezende,
Phiala E. Shanahan
Abstract:
We define a class of machine-learned flow-based sampling algorithms for lattice gauge theories that are gauge-invariant by construction. We demonstrate the application of this framework to U(1) gauge theory in two spacetime dimensions, and find that near critical points in parameter space the approach is orders of magnitude more efficient at sampling topological quantities than more traditional sa…
▽ More
We define a class of machine-learned flow-based sampling algorithms for lattice gauge theories that are gauge-invariant by construction. We demonstrate the application of this framework to U(1) gauge theory in two spacetime dimensions, and find that near critical points in parameter space the approach is orders of magnitude more efficient at sampling topological quantities than more traditional sampling procedures such as Hybrid Monte Carlo and Heat Bath.
△ Less
Submitted 13 March, 2020;
originally announced March 2020.
-
Data Structures & Algorithms for Exact Inference in Hierarchical Clustering
Authors:
Craig S. Greenberg,
Sebastian Macaluso,
Nicholas Monath,
Ji-Ah Lee,
Patrick Flaherty,
Kyle Cranmer,
Andrew McGregor,
Andrew McCallum
Abstract:
Hierarchical clustering is a fundamental task often used to discover meaningful structures in data, such as phylogenetic trees, taxonomies of concepts, subtypes of cancer, and cascades of particle decays in particle physics. Typically approximate algorithms are used for inference due to the combinatorial number of possible hierarchical clusterings. In contrast to existing methods, we present novel…
▽ More
Hierarchical clustering is a fundamental task often used to discover meaningful structures in data, such as phylogenetic trees, taxonomies of concepts, subtypes of cancer, and cascades of particle decays in particle physics. Typically approximate algorithms are used for inference due to the combinatorial number of possible hierarchical clusterings. In contrast to existing methods, we present novel dynamic-programming algorithms for \emph{exact} inference in hierarchical clustering based on a novel trellis data structure, and we prove that we can exactly compute the partition function, maximum likelihood hierarchy, and marginal probabilities of sub-hierarchies and clusters. Our algorithms scale in time and space proportional to the powerset of $N$ elements which is super-exponentially more efficient than explicitly considering each of the (2N-3)!! possible hierarchies. Also, for larger datasets where our exact algorithms become infeasible, we introduce an approximate algorithm based on a sparse trellis that compares well to other benchmarks. Exact methods are relevant to data analyses in particle physics and for finding correlations among gene expression in cancer genomics, and we give examples in both areas, where our algorithms outperform greedy and beam search baselines. In addition, we consider Dasgupta's cost with synthetic data.
△ Less
Submitted 22 October, 2020; v1 submitted 26 February, 2020;
originally announced February 2020.
-
Set2Graph: Learning Graphs From Sets
Authors:
Hadar Serviansky,
Nimrod Segol,
Jonathan Shlomi,
Kyle Cranmer,
Eilam Gross,
Haggai Maron,
Yaron Lipman
Abstract:
Many problems in machine learning can be cast as learning functions from sets to graphs, or more generally to hypergraphs; in short, Set2Graph functions. Examples include clustering, learning vertex and edge features on graphs, and learning features on triplets in a collection. A natural approach for building Set2Graph models is to characterize all linear equivariant set-to-hypergraph layers and s…
▽ More
Many problems in machine learning can be cast as learning functions from sets to graphs, or more generally to hypergraphs; in short, Set2Graph functions. Examples include clustering, learning vertex and edge features on graphs, and learning features on triplets in a collection. A natural approach for building Set2Graph models is to characterize all linear equivariant set-to-hypergraph layers and stack them with non-linear activations. This poses two challenges: (i) the expressive power of these networks is not well understood; and (ii) these models would suffer from high, often intractable computational and memory complexity, as their dimension grows exponentially. This paper advocates a family of neural network models for learning Set2Graph functions that is both practical and of maximal expressive power (universal), that is, can approximate arbitrary continuous Set2Graph functions over compact sets. Testing these models on different machine learning tasks, mainly an application to particle physics, we find them favorable to existing baselines.
△ Less
Submitted 26 November, 2020; v1 submitted 20 February, 2020;
originally announced February 2020.
-
Normalizing Flows on Tori and Spheres
Authors:
Danilo Jimenez Rezende,
George Papamakarios,
Sébastien Racanière,
Michael S. Albergo,
Gurtej Kanwar,
Phiala E. Shanahan,
Kyle Cranmer
Abstract:
Normalizing flows are a powerful tool for building expressive distributions in high dimensions. So far, most of the literature has concentrated on learning flows on Euclidean spaces. Some problems however, such as those involving angles, are defined on spaces with more complex geometries, such as tori or spheres. In this paper, we propose and compare expressive and numerically stable flows on such…
▽ More
Normalizing flows are a powerful tool for building expressive distributions in high dimensions. So far, most of the literature has concentrated on learning flows on Euclidean spaces. Some problems however, such as those involving angles, are defined on spaces with more complex geometries, such as tori or spheres. In this paper, we propose and compare expressive and numerically stable flows on such spaces. Our flows are built recursively on the dimension of the space, starting from flows on circles, closed intervals or spheres.
△ Less
Submitted 1 July, 2020; v1 submitted 6 February, 2020;
originally announced February 2020.
-
The frontier of simulation-based inference
Authors:
Kyle Cranmer,
Johann Brehmer,
Gilles Louppe
Abstract:
Many domains of science have developed complex simulations to describe phenomena of interest. While these simulations provide high-fidelity models, they are poorly suited for inference and lead to challenging inverse problems. We review the rapidly developing field of simulation-based inference and identify the forces giving new momentum to the field. Finally, we describe how the frontier is expan…
▽ More
Many domains of science have developed complex simulations to describe phenomena of interest. While these simulations provide high-fidelity models, they are poorly suited for inference and lead to challenging inverse problems. We review the rapidly developing field of simulation-based inference and identify the forces giving new momentum to the field. Finally, we describe how the frontier is expanding so that a broad audience can appreciate the profound change these developments may have on science.
△ Less
Submitted 2 April, 2020; v1 submitted 4 November, 2019;
originally announced November 2019.
-
Extending RECAST for Truth-Level Reinterpretations
Authors:
Alex Schuy,
Lukas Heinrich,
Kyle Cranmer,
Shih-Chieh Hsu
Abstract:
RECAST is an analysis reinterpretation framework; since analyses are often sensitive to a range of models, RECAST can be used to constrain the plethora of theoretical models without the significant investment required for a new analysis. However, experiment-specific full simulation is still computationally expensive. Thus, to facilitate rapid exploration, RECAST has been extended to truth-level re…
▽ More
RECAST is an analysis reinterpretation framework; since analyses are often sensitive to a range of models, RECAST can be used to constrain the plethora of theoretical models without the significant investment required for a new analysis. However, experiment-specific full simulation is still computationally expensive. Thus, to facilitate rapid exploration, RECAST has been extended to truth-level reinterpretations, interfacing with existing systems such as RIVET.
△ Less
Submitted 22 October, 2019;
originally announced October 2019.
-
Hamiltonian Graph Networks with ODE Integrators
Authors:
Alvaro Sanchez-Gonzalez,
Victor Bapst,
Kyle Cranmer,
Peter Battaglia
Abstract:
We introduce an approach for imposing physically informed inductive biases in learned simulation models. We combine graph networks with a differentiable ordinary differential equation integrator as a mechanism for predicting future states, and a Hamiltonian as an internal representation. We find that our approach outperforms baselines without these biases in terms of predictive accuracy, energy ac…
▽ More
We introduce an approach for imposing physically informed inductive biases in learned simulation models. We combine graph networks with a differentiable ordinary differential equation integrator as a mechanism for predicting future states, and a Hamiltonian as an internal representation. We find that our approach outperforms baselines without these biases in terms of predictive accuracy, energy accuracy, and zero-shot generalization to time-step sizes and integrator orders not experienced during training. This advances the state-of-the-art of learned simulation, and in principle is applicable beyond physical domains.
△ Less
Submitted 27 September, 2019;
originally announced September 2019.
-
Mining for Dark Matter Substructure: Inferring subhalo population properties from strong lenses with machine learning
Authors:
Johann Brehmer,
Siddharth Mishra-Sharma,
Joeri Hermans,
Gilles Louppe,
Kyle Cranmer
Abstract:
The subtle and unique imprint of dark matter substructure on extended arcs in strong lensing systems contains a wealth of information about the properties and distribution of dark matter on small scales and, consequently, about the underlying particle physics. However, teasing out this effect poses a significant challenge since the likelihood function for realistic simulations of population-level…
▽ More
The subtle and unique imprint of dark matter substructure on extended arcs in strong lensing systems contains a wealth of information about the properties and distribution of dark matter on small scales and, consequently, about the underlying particle physics. However, teasing out this effect poses a significant challenge since the likelihood function for realistic simulations of population-level parameters is intractable. We apply recently-developed simulation-based inference techniques to the problem of substructure inference in galaxy-galaxy strong lenses. By leveraging additional information extracted from the simulator, neural networks are efficiently trained to estimate likelihood ratios associated with population-level parameters characterizing substructure. Through proof-of-principle application to simulated data, we show that these methods can provide an efficient and principled way to simultaneously analyze an ensemble of strong lenses, and can be used to mine the large sample of lensing images deliverable by near-future surveys for signatures of dark matter substructure.
△ Less
Submitted 17 October, 2019; v1 submitted 4 September, 2019;
originally announced September 2019.
-
MadMiner: Machine learning-based inference for particle physics
Authors:
Johann Brehmer,
Felix Kling,
Irina Espejo,
Kyle Cranmer
Abstract:
Precision measurements at the LHC often require analyzing high-dimensional event data for subtle kinematic signatures, which is challenging for established analysis methods. Recently, a powerful family of multivariate inference techniques that leverage both matrix element information and machine learning has been developed. This approach neither requires the reduction of high-dimensional data to s…
▽ More
Precision measurements at the LHC often require analyzing high-dimensional event data for subtle kinematic signatures, which is challenging for established analysis methods. Recently, a powerful family of multivariate inference techniques that leverage both matrix element information and machine learning has been developed. This approach neither requires the reduction of high-dimensional data to summary statistics nor any simplifications to the underlying physics or detector response. In this paper we introduce MadMiner, a Python module that streamlines the steps involved in this procedure. Wrapping around MadGraph5_aMC and Pythia 8, it supports almost any physics process and model. To aid phenomenological studies, the tool also wraps around Delphes 3, though it is extendable to a full Geant4-based detector simulation. We demonstrate the use of MadMiner in an example analysis of dimension-six operators in ttH production, finding that the new techniques substantially increase the sensitivity to new physics.
△ Less
Submitted 20 January, 2020; v1 submitted 24 July, 2019;
originally announced July 2019.
-
Etalumis: Bringing Probabilistic Programming to Scientific Simulators at Scale
Authors:
Atılım Güneş Baydin,
Lei Shao,
Wahid Bhimji,
Lukas Heinrich,
Lawrence Meadows,
Jialin Liu,
Andreas Munk,
Saeid Naderiparizi,
Bradley Gram-Hansen,
Gilles Louppe,
Mingfei Ma,
Xiaohui Zhao,
Philip Torr,
Victor Lee,
Kyle Cranmer,
Prabhat,
Frank Wood
Abstract:
Probabilistic programming languages (PPLs) are receiving widespread attention for performing Bayesian inference in complex generative models. However, applications to science remain limited because of the impracticability of rewriting complex scientific simulators in a PPL, the computational cost of inference, and the lack of scalable implementations. To address these, we present a novel PPL frame…
▽ More
Probabilistic programming languages (PPLs) are receiving widespread attention for performing Bayesian inference in complex generative models. However, applications to science remain limited because of the impracticability of rewriting complex scientific simulators in a PPL, the computational cost of inference, and the lack of scalable implementations. To address these, we present a novel PPL framework that couples directly to existing scientific simulators through a cross-platform probabilistic execution protocol and provides Markov chain Monte Carlo (MCMC) and deep-learning-based inference compilation (IC) engines for tractable inference. To guide IC inference, we perform distributed training of a dynamic 3DCNN--LSTM architecture with a PyTorch-MPI-based framework on 1,024 32-core CPU nodes of the Cori supercomputer with a global minibatch size of 128k: achieving a performance of 450 Tflop/s through enhancements to PyTorch. We demonstrate a Large Hadron Collider (LHC) use-case with the C++ Sherpa simulator and achieve the largest-scale posterior inference in a Turing-complete PPL.
△ Less
Submitted 27 August, 2019; v1 submitted 7 July, 2019;
originally announced July 2019.
-
Effective LHC measurements with matrix elements and machine learning
Authors:
Johann Brehmer,
Kyle Cranmer,
Irina Espejo,
Felix Kling,
Gilles Louppe,
Juan Pavez
Abstract:
One major challenge for the legacy measurements at the LHC is that the likelihood function is not tractable when the collected data is high-dimensional and the detector response has to be modeled. We review how different analysis strategies solve this issue, including the traditional histogram approach used in most particle physics analyses, the Matrix Element Method, Optimal Observables, and mode…
▽ More
One major challenge for the legacy measurements at the LHC is that the likelihood function is not tractable when the collected data is high-dimensional and the detector response has to be modeled. We review how different analysis strategies solve this issue, including the traditional histogram approach used in most particle physics analyses, the Matrix Element Method, Optimal Observables, and modern techniques based on neural density estimation. We then discuss powerful new inference methods that use a combination of matrix element information and machine learning to accurately estimate the likelihood function. The MadMiner package automates all necessary data-processing steps. In first studies we find that these new techniques have the potential to substantially improve the sensitivity of the LHC legacy measurements.
△ Less
Submitted 4 June, 2019;
originally announced June 2019.
-
Inferring the quantum density matrix with machine learning
Authors:
Kyle Cranmer,
Siavash Golkar,
Duccio Pappadopulo
Abstract:
We introduce two methods for estimating the density matrix for a quantum system: Quantum Maximum Likelihood and Quantum Variational Inference. In these methods, we construct a variational family to model the density matrix of a mixed quantum state. We also introduce quantum flows, the quantum analog of normalizing flows, which can be used to increase the expressivity of this variational family. Th…
▽ More
We introduce two methods for estimating the density matrix for a quantum system: Quantum Maximum Likelihood and Quantum Variational Inference. In these methods, we construct a variational family to model the density matrix of a mixed quantum state. We also introduce quantum flows, the quantum analog of normalizing flows, which can be used to increase the expressivity of this variational family. The eigenstates and eigenvalues of interest are then derived by optimizing an appropriate loss function. The approach is qualitatively different than traditional lattice techniques that rely on the time dependence of correlation functions that summarize the lattice configurations. The resulting estimate of the density matrix can then be used to evaluate the expectation of an arbitrary operator, which opens the door to new possibilities.
△ Less
Submitted 11 April, 2019;
originally announced April 2019.
-
Machine learning and the physical sciences
Authors:
Giuseppe Carleo,
Ignacio Cirac,
Kyle Cranmer,
Laurent Daudet,
Maria Schuld,
Naftali Tishby,
Leslie Vogt-Maranto,
Lenka Zdeborová
Abstract:
Machine learning encompasses a broad range of algorithms and modeling tools used for a vast array of data processing tasks, which has entered most scientific disciplines in recent years. We review in a selective way the recent research on the interface between machine learning and physical sciences. This includes conceptual developments in machine learning (ML) motivated by physical insights, appl…
▽ More
Machine learning encompasses a broad range of algorithms and modeling tools used for a vast array of data processing tasks, which has entered most scientific disciplines in recent years. We review in a selective way the recent research on the interface between machine learning and physical sciences. This includes conceptual developments in machine learning (ML) motivated by physical insights, applications of machine learning techniques to several domains in physics, and cross-fertilization between the two fields. After giving basic notion of machine learning methods and principles, we describe examples of how statistical physics is used to understand methods in ML. We then move to describe applications of ML methods in particle physics and cosmology, quantum many body physics, quantum computing, and chemical and material physics. We also highlight research and development into novel computing architectures aimed at accelerating ML. In each of the sections we describe recent successes as well as domain-specific methodology and challenges.
△ Less
Submitted 6 December, 2019; v1 submitted 25 March, 2019;
originally announced March 2019.