-
SHAPER: Can You Hear the Shape of a Jet?
Authors:
Demba Ba,
Akshunna S. Dogra,
Rikab Gambhir,
Abiy Tasissa,
Jesse Thaler
Abstract:
The identification of interesting substructures within jets is an important tool for searching for new physics and probing the Standard Model at colliders. Many of these substructure tools have previously been shown to take the form of optimal transport problems, in particular the Energy Mover's Distance (EMD). In this work, we show that the EMD is in fact the natural structure for comparing colli…
▽ More
The identification of interesting substructures within jets is an important tool for searching for new physics and probing the Standard Model at colliders. Many of these substructure tools have previously been shown to take the form of optimal transport problems, in particular the Energy Mover's Distance (EMD). In this work, we show that the EMD is in fact the natural structure for comparing collider events, which accounts for its recent success in understanding event and jet substructure. We then present a Shape Hunting Algorithm using Parameterized Energy Reconstruction (SHAPER), which is a general framework for defining and computing shape-based observables. SHAPER generalizes N-jettiness from point clusters to any extended, parametrizable shape. This is accomplished by efficiently minimizing the EMD between events and parameterized manifolds of energy flows representing idealized shapes, implemented using the dual-potential Sinkhorn approximation of the Wasserstein metric. We show how the geometric language of observables as manifolds can be used to define novel observables with built-in infrared-and-collinear safety. We demonstrate the efficacy of the SHAPER framework by performing empirical jet substructure studies using several examples of new shape-based observables.
△ Less
Submitted 20 July, 2023; v1 submitted 23 February, 2023;
originally announced February 2023.
-
Universality of Winning Tickets: A Renormalization Group Perspective
Authors:
William T. Redman,
Tianlong Chen,
Zhangyang Wang,
Akshunna S. Dogra
Abstract:
Foundational work on the Lottery Ticket Hypothesis has suggested an exciting corollary: winning tickets found in the context of one task can be transferred to similar tasks, possibly even across different architectures. This has generated broad interest, but methods to study this universality are lacking. We make use of renormalization group theory, a powerful tool from theoretical physics, to add…
▽ More
Foundational work on the Lottery Ticket Hypothesis has suggested an exciting corollary: winning tickets found in the context of one task can be transferred to similar tasks, possibly even across different architectures. This has generated broad interest, but methods to study this universality are lacking. We make use of renormalization group theory, a powerful tool from theoretical physics, to address this need. We find that iterative magnitude pruning, the principal algorithm used for discovering winning tickets, is a renormalization group scheme, and can be viewed as inducing a flow in parameter space. We demonstrate that ResNet-50 models with transferable winning tickets have flows with common properties, as would be expected from the theory. Similar observations are made for BERT models, with evidence that their flows are near fixed points. Additionally, we leverage our framework to study winning tickets transferred across ResNet architectures, observing that smaller models have flows with more uniform properties than larger models, complicating transfer between them.
△ Less
Submitted 15 June, 2022; v1 submitted 7 October, 2021;
originally announced October 2021.
-
Local error quantification for Neural Network Differential Equation solvers
Authors:
Akshunna S. Dogra,
William T Redman
Abstract:
Neural networks have been identified as powerful tools for the study of complex systems. A noteworthy example is the neural network differential equation (NN DE) solver, which can provide functional approximations to the solutions of a wide variety of differential equations. Such solvers produce robust functional expressions, are well suited for further manipulations on the quantities of interest…
▽ More
Neural networks have been identified as powerful tools for the study of complex systems. A noteworthy example is the neural network differential equation (NN DE) solver, which can provide functional approximations to the solutions of a wide variety of differential equations. Such solvers produce robust functional expressions, are well suited for further manipulations on the quantities of interest (for example, taking derivatives), and capable of leveraging the modern advances in parallelization and computing power. However, there is a lack of work on the role precise error quantification can play in their predictions: usually, the focus is on ambiguous and/or global measures of performance like the loss function and/or obtaining global bounds on the errors associated with the predictions. Precise, local error quantification is seldom possible without external means or outright knowledge of the true solution. We address these concerns in the context of dynamical system NN DE solvers, leveraging learnt information within the NN DE solvers to develop methods that allow them to be more accurate and efficient, while still pursuing an unsupervised approach that does not rely on external tools or data. We achieve this via methods that can precisely estimate NN DE solver prediction errors point-wise, thus allowing the user the capacity for efficient and targeted error correction. We exemplify the utility of our methods by testing them on a nonlinear and a chaotic system each.
△ Less
Submitted 28 January, 2021; v1 submitted 24 August, 2020;
originally announced August 2020.
-
Error Estimation and Correction from within Neural Network Differential Equation Solvers
Authors:
Akshunna S. Dogra
Abstract:
Neural Network Differential Equation (NN DE) solvers have surged in popularity due to a combination of factors: computational advances making their optimization more tractable, their capacity to handle high dimensional problems, easy interpret-ability of their models, etc. However, almost all NN DE solvers suffer from a fundamental limitation: they are trained using loss functions that depend only…
▽ More
Neural Network Differential Equation (NN DE) solvers have surged in popularity due to a combination of factors: computational advances making their optimization more tractable, their capacity to handle high dimensional problems, easy interpret-ability of their models, etc. However, almost all NN DE solvers suffer from a fundamental limitation: they are trained using loss functions that depend only implicitly on the error associated with the estimate. As such, validation and error analysis of solution estimates requires knowledge of the true solution. Indeed, if the true solution is unknown, we are often reduced to simply hoping that a "low enough" loss implies "small enough" errors, since explicit relationships between the two are not available/well defined. In this work, we describe a general strategy for efficiently constructing error estimates and corrections for Neural Network Differential Equation solvers. Our methods do not require advance knowledge of the true solutions and obtain explicit relationships between loss functions and the error associated with solution estimates. In turn, these explicit relationships directly allow us to estimate and correct for the errors.
△ Less
Submitted 18 November, 2021; v1 submitted 9 July, 2020;
originally announced July 2020.
-
Optimizing Neural Networks via Koopman Operator Theory
Authors:
Akshunna S. Dogra,
William T Redman
Abstract:
Koopman operator theory, a powerful framework for discovering the underlying dynamics of nonlinear dynamical systems, was recently shown to be intimately connected with neural network training. In this work, we take the first steps in making use of this connection. As Koopman operator theory is a linear theory, a successful implementation of it in evolving network weights and biases offers the pro…
▽ More
Koopman operator theory, a powerful framework for discovering the underlying dynamics of nonlinear dynamical systems, was recently shown to be intimately connected with neural network training. In this work, we take the first steps in making use of this connection. As Koopman operator theory is a linear theory, a successful implementation of it in evolving network weights and biases offers the promise of accelerated training, especially in the context of deep networks, where optimization is inherently a non-convex problem. We show that Koopman operator theoretic methods allow for accurate predictions of weights and biases of feedforward, fully connected deep networks over a non-trivial range of training time. During this window, we find that our approach is >10x faster than various gradient descent based methods (e.g. Adam, Adadelta, Adagrad), in line with our complexity analysis. We end by highlighting open questions in this exciting intersection between dynamical systems and neural network theory. We highlight additional methods by which our results could be expanded to broader classes of networks and larger training intervals, which shall be the focus of future work.
△ Less
Submitted 21 October, 2020; v1 submitted 3 June, 2020;
originally announced June 2020.
-
Dynamical Systems and Neural Networks
Authors:
Akshunna S. Dogra
Abstract:
Neural Networks (NNs) have been identified as a potentially powerful tool in the study of complex dynamical systems. A good example is the NN differential equation (DE) solver, which provides closed form, differentiable, functional approximations for the evolution of a wide variety of dynamical systems. A major disadvantage of such NN solvers can be the amount of computational resources needed to…
▽ More
Neural Networks (NNs) have been identified as a potentially powerful tool in the study of complex dynamical systems. A good example is the NN differential equation (DE) solver, which provides closed form, differentiable, functional approximations for the evolution of a wide variety of dynamical systems. A major disadvantage of such NN solvers can be the amount of computational resources needed to achieve accuracy comparable to existing numerical solvers. We present new strategies for existing dynamical system NN DE solvers, making efficient use of the \textit{learnt} information, to speed up their training process, while still pursuing a completely unsupervised approach. We establish a fundamental connection between NN theory and dynamical systems theory via Koopman Operator Theory (KOT), by showing that the usual training processes for Neural Nets are fertile ground for identifying multiple Koopman operators of interest. We end by illuminating certain applications that KOT might have for NNs in general.
△ Less
Submitted 20 April, 2020;
originally announced April 2020.
-
Hamiltonian neural networks for solving equations of motion
Authors:
Marios Mattheakis,
David Sondak,
Akshunna S. Dogra,
Pavlos Protopapas
Abstract:
There has been a wave of interest in applying machine learning to study dynamical systems. We present a Hamiltonian neural network that solves the differential equations that govern dynamical systems. This is an equation-driven machine learning method where the optimization process of the network depends solely on the predicted functions without using any ground truth data. The model learns soluti…
▽ More
There has been a wave of interest in applying machine learning to study dynamical systems. We present a Hamiltonian neural network that solves the differential equations that govern dynamical systems. This is an equation-driven machine learning method where the optimization process of the network depends solely on the predicted functions without using any ground truth data. The model learns solutions that satisfy, up to an arbitrarily small error, Hamilton's equations and, therefore, conserve the Hamiltonian invariants. The choice of an appropriate activation function drastically improves the predictability of the network. Moreover, an error analysis is derived and states that the numerical errors depend on the overall network performance. The Hamiltonian network is then employed to solve the equations for the nonlinear oscillator and the chaotic Henon-Heiles dynamical system. In both systems, a symplectic Euler integrator requires two orders more evaluation points than the Hamiltonian network in order to achieve the same order of the numerical error in the predicted phase space trajectories.
△ Less
Submitted 26 April, 2022; v1 submitted 29 January, 2020;
originally announced January 2020.
-
Impact of ionizing radiation on superconducting qubit coherence
Authors:
Antti Vepsäläinen,
Amir H. Karamlou,
John L. Orrell,
Akshunna S. Dogra,
Ben Loer,
Francisca Vasconcelos,
David K. Kim,
Alexander J. Melville,
Bethany M. Niedzielski,
Jonilyn L. Yoder,
Simon Gustavsson,
Joseph A. Formaggio,
Brent A. VanDevender,
William D. Oliver
Abstract:
The practical viability of any qubit technology stands on long coherence times and high-fidelity operations, with the superconducting qubit modality being a leading example. However, superconducting qubit coherence is impacted by broken Cooper pairs, referred to as quasiparticles, with a density that is empirically observed to be orders of magnitude greater than the value predicted for thermal equ…
▽ More
The practical viability of any qubit technology stands on long coherence times and high-fidelity operations, with the superconducting qubit modality being a leading example. However, superconducting qubit coherence is impacted by broken Cooper pairs, referred to as quasiparticles, with a density that is empirically observed to be orders of magnitude greater than the value predicted for thermal equilibrium by the Bardeen-Cooper-Schrieffer (BCS) theory of superconductivity. Previous work has shown that infrared photons significantly increase the quasiparticle density, yet even in the best isolated systems, it still remains higher than expected, suggesting that another generation mechanism exists. In this Letter, we provide evidence that ionizing radiation from environmental radioactive materials and cosmic rays contributes to this observed difference, leading to an elevated quasiparticle density that would ultimately limit superconducting qubits of the type measured here to coherence times in the millisecond regime. We further demonstrate that introducing radiation shielding reduces the flux of ionizing radiation and positively correlates with increased coherence time. Albeit a small effect for today's qubits, reducing or otherwise mitigating the impact of ionizing radiation will be critical for realizing fault-tolerant superconducting quantum computers.
△ Less
Submitted 27 August, 2020; v1 submitted 24 January, 2020;
originally announced January 2020.
-
Optimal Presentations of Mathematical Objects
Authors:
Akshunna Shaurya Dogra
Abstract:
We discuss the optimal presentations of mathematical objects under well defined symbol libraries. We shall examine what light our chosen symbol libraries and syntax shed upon the objects they represent. A major part of this work will focus on discrete sets, particularly the natural numbers, with results that describe the presentation of the natural numbers under specific symbol libraries and what…
▽ More
We discuss the optimal presentations of mathematical objects under well defined symbol libraries. We shall examine what light our chosen symbol libraries and syntax shed upon the objects they represent. A major part of this work will focus on discrete sets, particularly the natural numbers, with results that describe the presentation of the natural numbers under specific symbol libraries and what those presentations may reveal about the properties of the natural numbers themselves. We shall present bounds and constraints on the length and shape of presentations, connect already existing problems in other fields of mathematics to questions relevant to these presentations and otherwise illuminate why such a study can produce exciting results.
△ Less
Submitted 3 December, 2018;
originally announced December 2018.
-
Geometry of transient chaos in streamwise-localized pipe flow turbulence
Authors:
Nazmi Burak Budanur,
Akshunna S. Dogra,
Björn Hof
Abstract:
In pipes and channels, the onset of turbulence is initially dominated by localized transients, which lead to sustained turbulence through their collective dynamics. In the present work, we study the localized turbulence in pipe flow numerically and elucidate a state space structure that gives rise to transient chaos. Starting from the basin boundary separating laminar and turbulent flow, we identi…
▽ More
In pipes and channels, the onset of turbulence is initially dominated by localized transients, which lead to sustained turbulence through their collective dynamics. In the present work, we study the localized turbulence in pipe flow numerically and elucidate a state space structure that gives rise to transient chaos. Starting from the basin boundary separating laminar and turbulent flow, we identify transverse homoclinic orbits, the presence of which necessitates a homoclinic tangle and chaos. A direct consequence of the homoclinic tangle is the fractal nature of the laminar-turbulent boundary, which was conjectured in various earlier studies. By mapping the transverse intersections between the stable and unstable manifold of a periodic orbit, we identify the 'gateways' that promote an escape from turbulence.
△ Less
Submitted 21 October, 2019; v1 submitted 4 October, 2018;
originally announced October 2018.
-
Minimal Representations of Natural Numbers Under a Set of Operators
Authors:
Akshunna Shaurya Dogra
Abstract:
This paper studies the minimal length representation of the natural numbers. Let O be a fixed set of integer-valued functions (primarily hyperoperations). For each n, what is the shortest way of expressing n as a combinations of functions in O to the constant 1? For example, if O contains the two functions Sx (successor of x) and *xy (x times y) then the shortest representation of 12 is *SSS1SS1,…
▽ More
This paper studies the minimal length representation of the natural numbers. Let O be a fixed set of integer-valued functions (primarily hyperoperations). For each n, what is the shortest way of expressing n as a combinations of functions in O to the constant 1? For example, if O contains the two functions Sx (successor of x) and *xy (x times y) then the shortest representation of 12 is *SSS1SS1, with 8 symbols. This is taken to mean that 8 is complexity of 12 under O. We make a study of such minimal representations and complexities in this paper, proving and/or rightly predicting bounds on complexities, discussing some relevant patterns in the complexities and minimal representations of the natural numbers and listing the results gleaned from computational analysis. Computationally, the first 4.5 million natural numbers were probed to verify our mathematically obtained results. Due to the finiteness of the problem, we used the method of exhaustion of possibilities to state some other results as well.
△ Less
Submitted 2 January, 2018;
originally announced January 2018.
