-
Quantum HyperNetworks: Training Binary Neural Networks in Quantum Superposition
Authors:
Juan Carrasquilla,
Mohamed Hibat-Allah,
Estelle Inack,
Alireza Makhzani,
Kirill Neklyudov,
Graham W. Taylor,
Giacomo Torlai
Abstract:
Binary neural networks, i.e., neural networks whose parameters and activations are constrained to only two possible values, offer a compelling avenue for the deployment of deep learning models on energy- and memory-limited devices. However, their training, architectural design, and hyperparameter tuning remain challenging as these involve multiple computationally expensive combinatorial optimizati…
▽ More
Binary neural networks, i.e., neural networks whose parameters and activations are constrained to only two possible values, offer a compelling avenue for the deployment of deep learning models on energy- and memory-limited devices. However, their training, architectural design, and hyperparameter tuning remain challenging as these involve multiple computationally expensive combinatorial optimization problems. Here we introduce quantum hypernetworks as a mechanism to train binary neural networks on quantum computers, which unify the search over parameters, hyperparameters, and architectures in a single optimization loop. Through classical simulations, we demonstrate that of our approach effectively finds optimal parameters, hyperparameters and architectural choices with high probability on classification problems including a two-dimensional Gaussian dataset and a scaled-down version of the MNIST handwritten digits. We represent our quantum hypernetworks as variational quantum circuits, and find that an optimal circuit depth maximizes the probability of finding performant binary neural networks. Our unified approach provides an immense scope for other applications in the field of machine learning.
△ Less
Submitted 19 January, 2023;
originally announced January 2023.
-
Neural annealing and visualization of autoregressive neural networks in the Newman-Moore model
Authors:
Estelle M. Inack,
Stewart Morawetz,
Roger G. Melko
Abstract:
Artificial neural networks have been widely adopted as ansatzes to study classical and quantum systems. However, some notably hard systems such as those exhibiting glassiness and frustration have mainly achieved unsatisfactory results despite their representational power and entanglement content, thus, suggesting a potential conservation of computational complexity in the learning process. We expl…
▽ More
Artificial neural networks have been widely adopted as ansatzes to study classical and quantum systems. However, some notably hard systems such as those exhibiting glassiness and frustration have mainly achieved unsatisfactory results despite their representational power and entanglement content, thus, suggesting a potential conservation of computational complexity in the learning process. We explore this possibility by implementing the neural annealing method with autoregressive neural networks on a model that exhibits glassy and fractal dynamics: the two-dimensional Newman-Moore model on a triangular lattice. We find that the annealing dynamics is globally unstable because of highly chaotic loss landscapes. Furthermore, even when the correct ground state energy is found, the neural network generally cannot find degenerate ground-state configurations due to mode collapse. These findings indicate that the glassy dynamics exhibited by the Newman-Moore model caused by the presence of fracton excitations in the configurational space likely manifests itself through trainability issues and mode collapse in the optimization landscape.
△ Less
Submitted 24 April, 2022;
originally announced April 2022.
-
Coherence resonance and stochastic synchronization in a small-world neural network: An interplay in the presence of spike-timing-dependent plasticity
Authors:
Marius E. Yamakou,
Estelle M. Inack
Abstract:
Coherence resonance (CR), stochastic synchronization (SS), and spike-timing-dependent plasticity (STDP) are ubiquitous dynamical processes in biological neural networks. Whether there exists an optimal network and STDP configuration at which CR and SS are both pronounced is a fundamental question of interest that is still elusive. We expect such a configuration to enable the brain to make synergis…
▽ More
Coherence resonance (CR), stochastic synchronization (SS), and spike-timing-dependent plasticity (STDP) are ubiquitous dynamical processes in biological neural networks. Whether there exists an optimal network and STDP configuration at which CR and SS are both pronounced is a fundamental question of interest that is still elusive. We expect such a configuration to enable the brain to make synergistic and optimal use of these phenomena to process information efficiently. This paper considers a small-world network of excitable Hodgkin-Huxley neurons driven by channel noise and STDP with an asymmetric Hebbian time window. Numerical results indicate specific network topology and STDP parameter intervals in which CR and SS can be simultaneously enhanced. Our results imply that an optimally tuned inherent background noise, STDP rule, and network topology can play a constructive role in enhancing both the time precision of firing and the synchronization in neural systems.
△ Less
Submitted 1 January, 2023; v1 submitted 14 January, 2022;
originally announced January 2022.
-
Variational Neural Annealing
Authors:
Mohamed Hibat-Allah,
Estelle M. Inack,
Roeland Wiersema,
Roger G. Melko,
Juan Carrasquilla
Abstract:
Many important challenges in science and technology can be cast as optimization problems. When viewed in a statistical physics framework, these can be tackled by simulated annealing, where a gradual cooling procedure helps search for groundstate solutions of a target Hamiltonian. While powerful, simulated annealing is known to have prohibitively slow sampling dynamics when the optimization landsca…
▽ More
Many important challenges in science and technology can be cast as optimization problems. When viewed in a statistical physics framework, these can be tackled by simulated annealing, where a gradual cooling procedure helps search for groundstate solutions of a target Hamiltonian. While powerful, simulated annealing is known to have prohibitively slow sampling dynamics when the optimization landscape is rough or glassy. Here we show that by generalizing the target distribution with a parameterized model, an analogous annealing framework based on the variational principle can be used to search for groundstate solutions. Modern autoregressive models such as recurrent neural networks provide ideal parameterizations since they can be exactly sampled without slow dynamics even when the model encodes a rough landscape. We implement this procedure in the classical and quantum settings on several prototypical spin glass Hamiltonians, and find that it significantly outperforms traditional simulated annealing in the asymptotic limit, illustrating the potential power of this yet unexplored route to optimization.
△ Less
Submitted 25 January, 2021;
originally announced January 2021.
-
Tunneling in projective quantum Monte Carlo simulations with guiding wave functions
Authors:
T. Parolini,
E. M. Inack,
G. Giudici,
S. Pilati
Abstract:
Quantum tunneling is a valuable resource exploited by quantum annealers to solve complex optimization problems. Tunneling events also occur during projective quantum Monte Carlo (PQMC) simulations, and in a class of problems characterized by a double-well energy landscape their rate was found to scale linearly with the first energy gap, i.e., even more favorably than in physical quantum annealers,…
▽ More
Quantum tunneling is a valuable resource exploited by quantum annealers to solve complex optimization problems. Tunneling events also occur during projective quantum Monte Carlo (PQMC) simulations, and in a class of problems characterized by a double-well energy landscape their rate was found to scale linearly with the first energy gap, i.e., even more favorably than in physical quantum annealers, where the rate scales with the gap squared. Here we investigate how a guiding wave function --- which is essential to make many-body PQMC simulations computationally feasible --- affects the tunneling rate. The chosen testbeds are a continuous-space double-well problem, the ferromagnetic quantum Ising chain, and the recently introduced shamrock model. As guiding wave function, we consider an approximate Boltzmann-type ansatz, the numerically-exact ground state of the double-well model, and a neural-network wave function based on a Boltzmann machine. Remarkably, for each ansatz we find the same asymptotic linear scaling of the tunneling rate that was previously found in the PQMC simulations performed without a guiding wave function. We also provide a semiclassical theory for the double-well with exact guiding wave function that explains the observed linear scaling. These findings suggest that PQMC simulations guided by an accurate ansatz represent a valuable benchmark for physical quantum annealers and a potentially competitive quantum-inspired optimization technique.
△ Less
Submitted 12 December, 2019; v1 submitted 27 August, 2019;
originally announced August 2019.
-
Self-learning projective quantum Monte Carlo simulations guided by restricted Boltzmann machines
Authors:
S. Pilati,
E. M. Inack,
P. Pieri
Abstract:
The projective quantum Monte Carlo (PQMC) algorithms are among the most powerful computational techniques to simulate the ground state properties of quantum many-body systems. However, they are efficient only if a sufficiently accurate trial wave function is used to guide the simulation. In the standard approach, this guiding wave function is obtained in a separate simulation that performs a varia…
▽ More
The projective quantum Monte Carlo (PQMC) algorithms are among the most powerful computational techniques to simulate the ground state properties of quantum many-body systems. However, they are efficient only if a sufficiently accurate trial wave function is used to guide the simulation. In the standard approach, this guiding wave function is obtained in a separate simulation that performs a variational minimization. Here we show how to perform PQMC simulations guided by an adaptive wave function based on a restricted Boltzmann machine. This adaptive wave function is optimized along the PQMC simulation via unsupervised machine learning, avoiding the need of a separate variational optimization. As a byproduct, this technique provides an accurate ansatz for the ground state wave function, which is obtained by minimizing the Kullback-Leibler divergence with respect to the PQMC samples, rather than by minimizing the energy expectation value as in standard variational optimizations. The high accuracy of this self-learning PQMC technique is demonstrated for a paradigmatic sign-problem-free model, namely, the ferromagnetic quantum Ising chain, showing very precise agreement with the predictions of the Jordan-Wigner theory and of loop quantum Monte Carlo simulations performed in the low-temperature limit.
△ Less
Submitted 1 July, 2019;
originally announced July 2019.
-
Projective quantum Monte Carlo simulations guided by unrestricted neural network states
Authors:
E. M. Inack,
G. E. Santoro,
L. Dell'Anna,
S. Pilati
Abstract:
We investigate the use of variational wave-functions that mimic stochastic recurrent neural networks, specifically, unrestricted Boltzmann machines, as guiding functions in projective quantum Monte Carlo (PQMC) simulations of quantum spin models. As a preliminary step, we investigate the accuracy of such unrestricted neural network states as variational Ansätze for the ground state of the ferromag…
▽ More
We investigate the use of variational wave-functions that mimic stochastic recurrent neural networks, specifically, unrestricted Boltzmann machines, as guiding functions in projective quantum Monte Carlo (PQMC) simulations of quantum spin models. As a preliminary step, we investigate the accuracy of such unrestricted neural network states as variational Ansätze for the ground state of the ferromagnetic quantum Ising chain. We find that by optimizing just three variational parameters, independently on the system size, accurate ground-state energies are obtained, comparable to those previously obtained using restricted Boltzmann machines with few variational parameters per spin. Chiefly, we show that if one uses optimized unrestricted neural network states as guiding functions for importance sampling the efficiency of the PQMC algorithms is greatly enhanced, drastically reducing the most relevant systematic bias, namely that due to the finite random-walker population. The scaling of the computational cost with the system size changes from the exponential scaling characteristic of PQMC simulations performed without importance sampling, to a polynomial scaling, even at the ferromagnetic quantum critical point. The important role of the protocol chosen to sample hidden-spins configurations, in particular at the critical point, is analyzed. We discuss the implications of these findings for what concerns the problem of simulating adiabatic quantum optimization using stochastic algorithms on classical computers.
△ Less
Submitted 10 September, 2018;
originally announced September 2018.
-
Understanding Quantum Tunneling using Diffusion Monte Carlo Simulations
Authors:
E. M. Inack,
G. Giudici,
T. Parolini,
G. Santoro,
S. Pilati
Abstract:
In simple ferromagnetic quantum Ising models characterized by an effective double-well energy landscape the characteristic tunneling time of path-integral Monte Carlo (PIMC) simulations has been shown to scale as the incoherent quantum-tunneling time, i.e., as $1/Δ^2$, where $Δ$ is the tunneling gap. Since incoherent quantum tunneling is employed by quantum annealers (QAs) to solve optimization pr…
▽ More
In simple ferromagnetic quantum Ising models characterized by an effective double-well energy landscape the characteristic tunneling time of path-integral Monte Carlo (PIMC) simulations has been shown to scale as the incoherent quantum-tunneling time, i.e., as $1/Δ^2$, where $Δ$ is the tunneling gap. Since incoherent quantum tunneling is employed by quantum annealers (QAs) to solve optimization problems, this result suggests there is no quantum advantage in using QAs w.r.t. quantum Monte Carlo (QMC) simulations. A counterexample is the recently introduced shamrock model, where topological obstructions cause an exponential slowdown of the PIMC tunneling dynamics with respect to incoherent quantum tunneling, leaving the door open for potential quantum speedup, even for stoquastic models. In this work, we investigate the tunneling time of projective QMC simulations based on the diffusion Monte Carlo (DMC) algorithm without guiding functions, showing that it scales as $1/Δ$, i.e., even more favorably than the incoherent quantum-tunneling time, both in a simple ferromagnetic system and in the more challenging shamrock model. However a careful comparison between the DMC ground-state energies and the exact solution available for the transverse-field Ising chain points at an exponential scaling of the computational cost required to keep a fixed relative error as the system size increases.
△ Less
Submitted 21 November, 2017;
originally announced November 2017.
-
Simulated quantum annealing of double-well and multi-well potentials
Authors:
E. M. Inack,
S. Pilati
Abstract:
We analyze the performance of quantum annealing as a heuristic optimization method to find the absolute minimum of various continuous models, including landscapes with only two wells and also models with many competing minima and with disorder. The simulations performed using a projective quantum Monte Carlo (QMC) algorithm are compared with those based on the finite-temperature path-integral QMC…
▽ More
We analyze the performance of quantum annealing as a heuristic optimization method to find the absolute minimum of various continuous models, including landscapes with only two wells and also models with many competing minima and with disorder. The simulations performed using a projective quantum Monte Carlo (QMC) algorithm are compared with those based on the finite-temperature path-integral QMC technique and with classical annealing. We show that the projective QMC algorithm is more efficient than the finite-temperature QMC technique, and that both are inferior to classical annealing if this is performed with appropriate long-range moves. However, as the difficulty of the optimization problem increases, classical annealing looses efficiency, while the projective QMC algorithm keeps stable performance and is finally the most effective optimization tool. We discuss the implications of our results for the outstanding problem of testing the efficiency of adiabatic quantum computers using stochastic simulations performed on classical computers.
△ Less
Submitted 15 October, 2015;
originally announced October 2015.
