-
Exact minimax entropy models of large-scale neuronal activity
Authors:
Christopher W. Lynn,
Qiwei Yu,
Rich Pang,
Stephanie E. Palmer,
William Bialek
Abstract:
In the brain, fine-scale correlations combine to produce macroscopic patterns of activity. However, as experiments record from larger and larger populations, we approach a fundamental bottleneck: the number of correlations one would like to include in a model grows larger than the available data. In this undersampled regime, one must focus on a sparse subset of correlations; the optimal choice con…
▽ More
In the brain, fine-scale correlations combine to produce macroscopic patterns of activity. However, as experiments record from larger and larger populations, we approach a fundamental bottleneck: the number of correlations one would like to include in a model grows larger than the available data. In this undersampled regime, one must focus on a sparse subset of correlations; the optimal choice contains the maximum information about patterns of activity or, equivalently, minimizes the entropy of the inferred maximum entropy model. Applying this ``minimax entropy" principle is generally intractable, but here we present an exact and scalable solution for pairwise correlations that combine to form a tree (a network without loops). Applying our method to over one thousand neurons in the mouse hippocampus, we find that the optimal tree of correlations reduces our uncertainty about the population activity by 14% (over 50 times more than a random tree). Despite containing only 0.1% of all pairwise correlations, this minimax entropy model accurately predicts the observed large-scale synchrony in neural activity and becomes even more accurate as the population grows. The inferred Ising model is almost entirely ferromagnetic (with positive interactions) and exhibits signatures of thermodynamic criticality. These results suggest that a sparse backbone of excitatory interactions may play an important role in driving collective neuronal activity.
△ Less
Submitted 18 December, 2023;
originally announced February 2024.
-
Exactly solvable statistical physics models for large neuronal populations
Authors:
Christopher W. Lynn,
Qiwei Yu,
Rich Pang,
William Bialek,
Stephanie E. Palmer
Abstract:
Maximum entropy methods provide a principled path connecting measurements of neural activity directly to statistical physics models, and this approach has been successful for populations of $N\sim 100$ neurons. As $N$ increases in new experiments, we enter an undersampled regime where we have to choose which observables should be constrained in the maximum entropy construction. The best choice is…
▽ More
Maximum entropy methods provide a principled path connecting measurements of neural activity directly to statistical physics models, and this approach has been successful for populations of $N\sim 100$ neurons. As $N$ increases in new experiments, we enter an undersampled regime where we have to choose which observables should be constrained in the maximum entropy construction. The best choice is the one that provides the greatest reduction in entropy, defining a "minimax entropy" principle. This principle becomes tractable if we restrict attention to correlations among pairs of neurons that link together into a tree; we can find the best tree efficiently, and the underlying statistical physics models are exactly solved. We use this approach to analyze experiments on $N\sim 1500$ neurons in the mouse hippocampus, and show that the resulting model captures the distribution of synchronous activity in the network.
△ Less
Submitted 16 October, 2023;
originally announced October 2023.
-
Multi-Relevance: Coexisting but Distinct Notions of Scale in Large Systems
Authors:
Adam G. Kline,
Stephanie E. Palmer
Abstract:
Renormalization group (RG) methods are emerging as tools in biology and computer science to support the search for simplifying structure in distributions over high-dimensional spaces. We show that mixture models can be thought of as having multiple coexisting, exactly independent RG flows, each with its own notion of scale. We define this property as ``multi-relevance''. As an example, we construc…
▽ More
Renormalization group (RG) methods are emerging as tools in biology and computer science to support the search for simplifying structure in distributions over high-dimensional spaces. We show that mixture models can be thought of as having multiple coexisting, exactly independent RG flows, each with its own notion of scale. We define this property as ``multi-relevance''. As an example, we construct a model that has two distinct notions of scale, each corresponding to the state of an unobserved categorical variable. In the regime where this latent variable can be inferred using a linear classifier, the vertex expansion approach in non-perturbative RG can be applied successfully but will give different answers depending the choice of expansion point in state space. In the regime where linear estimation of the latent state fails, we show that the vertex expansion predicts a decrease in the total number of relevant couplings from four to three and does not admit a good polynomial truncation scheme. This indicates oversimplification. One consequence of this is that principal component analysis (PCA) may be a poor choice of coarse-graining scheme in multi-relevant systems, since it imposes a notion of scale which is incorrect from the RG perspective. Taken together, our results indicate that RG and PCA can lead to oversimplification when multi-relevance is present and not accounted for.
△ Less
Submitted 7 February, 2024; v1 submitted 18 May, 2023;
originally announced May 2023.
-
Gaussian Information Bottleneck and the Non-Perturbative Renormalization Group
Authors:
Adam G. Kline,
Stephanie E. Palmer
Abstract:
The renormalization group (RG) is a class of theoretical techniques used to explain the collective physics of interacting, many-body systems. It has been suggested that the RG formalism may be useful in finding and interpreting emergent low-dimensional structure in complex systems outside of the traditional physics context, such as in biology or computer science. In such contexts, one common dimen…
▽ More
The renormalization group (RG) is a class of theoretical techniques used to explain the collective physics of interacting, many-body systems. It has been suggested that the RG formalism may be useful in finding and interpreting emergent low-dimensional structure in complex systems outside of the traditional physics context, such as in biology or computer science. In such contexts, one common dimensionality-reduction framework already in use is information bottleneck (IB), in which the goal is to compress an ``input'' signal $X$ while maximizing its mutual information with some stochastic ``relevance'' variable $Y$. IB has been applied in the vertebrate and invertebrate processing systems to characterize optimal encoding of the future motion of the external world. Other recent work has shown that the RG scheme for the dimer model could be ``discovered'' by a neural network attempting to solve an IB-like problem. This manuscript explores whether IB and any existing formulation of RG are formally equivalent. A class of soft-cutoff non-perturbative RG techniques are defined by families of non-deterministic coarsening maps, and hence can be formally mapped onto IB, and vice versa. For concreteness, this discussion is limited entirely to Gaussian statistics (GIB), for which IB has exact, closed-form solutions. Under this constraint, GIB has a semigroup structure, in which successive transformations remain IB-optimal. Further, the RG cutoff scheme associated with GIB can be identified. Our results suggest that IB can be used to impose a notion of ``large scale'' structure, such as biological function, on an RG procedure.
△ Less
Submitted 28 July, 2021;
originally announced July 2021.
-
Inferring couplings in networks across order-disorder phase transitions
Authors:
Vudtiwat Ngampruetikorn,
Vedant Sachdeva,
Johanna Torrence,
Jan Humplik,
David J. Schwab,
Stephanie E. Palmer
Abstract:
Statistical inference is central to many scientific endeavors, yet how it works remains unresolved. Answering this requires a quantitative understanding of the intrinsic interplay between statistical models, inference methods and data structure. To this end, we characterize the efficacy of direct coupling analysis (DCA)--a highly successful method for analyzing amino acid sequence data--in inferri…
▽ More
Statistical inference is central to many scientific endeavors, yet how it works remains unresolved. Answering this requires a quantitative understanding of the intrinsic interplay between statistical models, inference methods and data structure. To this end, we characterize the efficacy of direct coupling analysis (DCA)--a highly successful method for analyzing amino acid sequence data--in inferring pairwise interactions from samples of ferromagnetic Ising models on random graphs. Our approach allows for physically motivated exploration of qualitatively distinct data regimes separated by phase transitions. We show that inference quality depends strongly on the nature of generative models: optimal accuracy occurs at an intermediate temperature where the detrimental effects from macroscopic order and thermal noise are minimal. Importantly our results indicate that DCA does not always outperform its local-statistics-based predecessors; while DCA excels at low temperatures, it becomes inferior to simple correlation thresholding at virtually all temperatures when data are limited. Our findings offer new insights into the regime in which DCA operates so successfully and more broadly how inference interacts with data structure.
△ Less
Submitted 25 August, 2021; v1 submitted 4 June, 2021;
originally announced June 2021.
-
What makes it possible to learn probability distributions in the natural world?
Authors:
William Bialek,
Stephanie E. Palmer,
David J. Schwab
Abstract:
Organisms and algorithms learn probability distributions from previous observations, either over evolutionary time or on the fly. In the absence of regularities, estimating the underlying distribution from data would require observing each possible outcome many times. Here we show that two conditions allow us to escape this infeasible requirement. First, the mutual information between two halves o…
▽ More
Organisms and algorithms learn probability distributions from previous observations, either over evolutionary time or on the fly. In the absence of regularities, estimating the underlying distribution from data would require observing each possible outcome many times. Here we show that two conditions allow us to escape this infeasible requirement. First, the mutual information between two halves of the system should be consistently sub-extensive. Second, this shared information should be compressible, so that it can be represented by a number of bits proportional to the information rather than to the entropy. Under these conditions, a distribution can be described with a number of parameters that grows linearly with system size. These conditions are borne out in natural images and in models from statistical physics, respectively.
△ Less
Submitted 21 February, 2021; v1 submitted 27 August, 2020;
originally announced August 2020.
-
Quantum disorder in the two-dimensional pyrochlore Heisenberg antiferromagnet
Authors:
S. E. Palmer,
J. T. Chalker
Abstract:
We present the results of an exact diagonalization study of the spin-1/2 Heisenberg antiferromagnet on a two-dimensional version of the pyrochlore lattice, also known as the square lattice with crossings or the checkerboard lattice. Examining the low energy spectra for systems of up to 24 spins, we find that all clusters studied have non-degenerate ground states with total spin zero, and big ene…
▽ More
We present the results of an exact diagonalization study of the spin-1/2 Heisenberg antiferromagnet on a two-dimensional version of the pyrochlore lattice, also known as the square lattice with crossings or the checkerboard lattice. Examining the low energy spectra for systems of up to 24 spins, we find that all clusters studied have non-degenerate ground states with total spin zero, and big energy gaps to states with higher total spin. We also find a large number of non-magnetic excitations at energies within this spin gap. Spin-spin and spin-Peierls correlation functions appear to be short-ranged, and we suggest that the ground state is a spin liquid.
△ Less
Submitted 12 April, 2001; v1 submitted 24 February, 2001;
originally announced February 2001.
-
Order induced by dipolar interactions in a geometrically frustrated antiferromagnet
Authors:
S. E. Palmer,
J. T. Chalker
Abstract:
We study the classical Heisenberg model for spins on a pyrochlore lattice interacting via long range dipole-dipole forces and nearest neighbor exchange. Antiferromagnetic exchange alone is known not to induce ordering in this system. We analyze low temperature order resulting from the combined interactions, both by using a mean-field approach and by examining the energy cost of fluctuations abou…
▽ More
We study the classical Heisenberg model for spins on a pyrochlore lattice interacting via long range dipole-dipole forces and nearest neighbor exchange. Antiferromagnetic exchange alone is known not to induce ordering in this system. We analyze low temperature order resulting from the combined interactions, both by using a mean-field approach and by examining the energy cost of fluctuations about an ordered state. We discuss behavior as a function of the ratio of the dipolar and exchange interaction strengths and find two types of ordered phase. We relate our results to the recent experimental work and reproduce and extend the theoretical calculations on the pyrochlore compound, Gd$_2$Ti$_2$O$_7$, by Raju \textit{et al.}, Phys. Rev. B {\bf 59}, 14489 (1999).
△ Less
Submitted 31 December, 1999;
originally announced December 1999.
