-
A rate-distortion framework for MCMC algorithms: geometry and factorization of multivariate Markov chains
Authors:
Michael C. H. Choi,
Youjia Wang,
Geoffrey Wolfer
Abstract:
We introduce a framework rooted in a rate distortion problem for Markov chains, and show how a suite of commonly used Markov Chain Monte Carlo (MCMC) algorithms are specific instances within it, where the target stationary distribution is controlled by the distortion function. Our approach offers a unified variational view on the optimality of algorithms such as Metropolis-Hastings, Glauber dynami…
▽ More
We introduce a framework rooted in a rate distortion problem for Markov chains, and show how a suite of commonly used Markov Chain Monte Carlo (MCMC) algorithms are specific instances within it, where the target stationary distribution is controlled by the distortion function. Our approach offers a unified variational view on the optimality of algorithms such as Metropolis-Hastings, Glauber dynamics, the swapping algorithm and Feynman-Kac path models. Along the way, we analyze factorizability and geometry of multivariate Markov chains. Specifically, we demonstrate that induced chains on factors of a product space can be regarded as information projections with respect to a particular divergence. This perspective yields Han--Shearer type inequalities for Markov chains as well as applications in the context of large deviations and mixing time comparison.
△ Less
Submitted 18 April, 2024;
originally announced April 2024.
-
Markov chain entropy games and the geometry of their Nash equilibria
Authors:
Michael C. H. Choi,
Geoffrey Wolfer
Abstract:
Consider the following two-person mixed strategy game of a probabilist against Nature with respect to the parameters $(f, \mathcal{B},π)$, where $f$ is a convex function satisfying certain regularity conditions, $\mathcal{B}$ is either the set $\{L_i\}_{i=1}^n$ or its convex hull with each $L_i$ being a Markov infinitesimal generator on a finite state space $\mathcal{X}$ and $π$ is a given positiv…
▽ More
Consider the following two-person mixed strategy game of a probabilist against Nature with respect to the parameters $(f, \mathcal{B},π)$, where $f$ is a convex function satisfying certain regularity conditions, $\mathcal{B}$ is either the set $\{L_i\}_{i=1}^n$ or its convex hull with each $L_i$ being a Markov infinitesimal generator on a finite state space $\mathcal{X}$ and $π$ is a given positive discrete distribution on $\mathcal{X}$. The probabilist chooses a prior measure $μ$ within the set of probability measures on $\mathcal{B}$ denoted by $\mathcal{P}(\mathcal{B})$ and picks a $L \in \mathcal{B}$ at random according to $μ$, whereas Nature follows a pure strategy to select $M \in \mathcal{L}(π)$, the set of $π$-reversible Markov generators on $\mathcal{X}$. Nature pays an amount $D_f(M||L)$, the $f$-divergence from $L$ to $M$, to the probabilist. We prove that a mixed strategy Nash equilibrium always exists, and establish a minimax result on the expected payoff of the game. This also contrasts with the pure strategy version of the game where we show a Nash equilibrium may not exist. To find approximately a mixed strategy Nash equilibrium, we propose and develop a simple projected subgradient algorithm that provably converges with a rate of $\mathcal{O}(1/\sqrt{t})$, where $t$ is the number of iterations. In addition, we elucidate the relationships of Nash equilibrium with other seemingly disparate notions such as weighted information centroid, Chebyshev center and Bayes risk. This article generalizes the two-person game of a statistician against Nature developed in the literature, and highlights the powerful interplay and synergy between modern Markov chains theory and geometry, information theory, game theory, optimization and mathematical statistics.
△ Less
Submitted 6 October, 2023;
originally announced October 2023.
-
Systematic approaches to generate reversiblizations of Markov chains
Authors:
Michael C. H. Choi,
Geoffrey Wolfer
Abstract:
Given a target distribution $π$ and an arbitrary Markov infinitesimal generator $L$ on a finite state space $\mathcal{X}$, we develop three structured and inter-related approaches to generate new reversiblizations from $L$. The first approach hinges on a geometric perspective, in which we view reversiblizations as projections onto the space of $π$-reversible generators under suitable information d…
▽ More
Given a target distribution $π$ and an arbitrary Markov infinitesimal generator $L$ on a finite state space $\mathcal{X}$, we develop three structured and inter-related approaches to generate new reversiblizations from $L$. The first approach hinges on a geometric perspective, in which we view reversiblizations as projections onto the space of $π$-reversible generators under suitable information divergences such as $f$-divergences. With different choices of functions $f$, we not only recover nearly all established reversiblizations but also unravel and generate new reversiblizations. Along the way, we unveil interesting geometric results such as bisection properties, Pythagorean identities, parallelogram laws and a Markov chain counterpart of the arithmetic-geometric-harmonic mean inequality governing these reversiblizations. This further serves as motivation for introducing the notion of information centroids of a sequence of Markov chains and to give conditions for their existence and uniqueness. Building upon the first approach, we view reversiblizations as generalized means. In this second approach, we construct new reversiblizations via different natural notions of generalized means such as the Cauchy mean or the dual mean. In the third approach, we combine the recently introduced locally-balanced Markov processes framework and the notion of convex $*$-conjugate in the study of $f$-divergence. The latter offers a rich source of balancing functions to generate new reversiblizations.
△ Less
Submitted 10 September, 2023; v1 submitted 6 March, 2023;
originally announced March 2023.
-
A Geometric Reduction Approach for Identity Testing of Reversible Markov Chains
Authors:
Geoffrey Wolfer,
Shun Watanabe
Abstract:
We consider the problem of testing the identity of a reversible Markov chain against a reference from a single trajectory of observations. Employing the recently introduced notion of a lumping-congruent Markov embedding, we show that, at least in a mildly restricted setting, testing identity to a reversible chain reduces to testing to a symmetric chain over a larger state space and recover state-o…
▽ More
We consider the problem of testing the identity of a reversible Markov chain against a reference from a single trajectory of observations. Employing the recently introduced notion of a lumping-congruent Markov embedding, we show that, at least in a mildly restricted setting, testing identity to a reversible chain reduces to testing to a symmetric chain over a larger state space and recover state-of-the-art sample complexity for the problem.
△ Less
Submitted 15 February, 2023;
originally announced February 2023.
-
Variance-Aware Estimation of Kernel Mean Embedding
Authors:
Geoffrey Wolfer,
Pierre Alquier
Abstract:
An important feature of kernel mean embeddings (KME) is that the rate of convergence of the empirical KME to the true distribution KME can be bounded independently of the dimension of the space, properties of the distribution and smoothness features of the kernel. We show how to speed-up convergence by leveraging variance information in the RKHS. Furthermore, we show that even when such informatio…
▽ More
An important feature of kernel mean embeddings (KME) is that the rate of convergence of the empirical KME to the true distribution KME can be bounded independently of the dimension of the space, properties of the distribution and smoothness features of the kernel. We show how to speed-up convergence by leveraging variance information in the RKHS. Furthermore, we show that even when such information is a priori unknown, we can efficiently estimate it from the data, recovering the desiderata of a distribution agnostic bound that enjoys acceleration in fortuitous settings. We illustrate our methods in the context of hypothesis testing and robust parametric estimation.
△ Less
Submitted 12 October, 2022;
originally announced October 2022.
-
Geometric Aspects of Data-Processing of Markov Chains
Authors:
Geoffrey Wolfer,
Shun Watanabe
Abstract:
We examine data-processing of Markov chains through the lens of information geometry. We first establish a theory of congruent Markov morphisms within the framework of stochastic matrices. Specifically, we introduce and justify the concept of a linear right inverse (congruent embedding) for lumping, a well-known operation used in Markov chains to extract coarse information. Furthermore, we inspect…
▽ More
We examine data-processing of Markov chains through the lens of information geometry. We first establish a theory of congruent Markov morphisms within the framework of stochastic matrices. Specifically, we introduce and justify the concept of a linear right inverse (congruent embedding) for lumping, a well-known operation used in Markov chains to extract coarse information. Furthermore, we inspect information projections onto geodesically convex sets of stochastic matrices, and show that under some conditions, projecting (m-projection) onto doubly convex submanifolds can be regarded as a form of data-processing. Finally, we show that the family of lumpable stochastic matrices can be meaningfully endowed with the structure of a foliated manifold and motivate our construction in the context of embedded models and inference.
△ Less
Submitted 20 December, 2023; v1 submitted 9 March, 2022;
originally announced March 2022.
-
Information Geometry of Reversible Markov Chains
Authors:
Geoffrey Wolfer,
Shun Watanabe
Abstract:
We analyze the information geometric structure of time reversibility for parametric families of irreducible transition kernels of Markov chains. We define and characterize reversible exponential families of Markov kernels, and show that irreducible and reversible Markov kernels form both a mixture family and, perhaps surprisingly, an exponential family in the set of all stochastic kernels. We prop…
▽ More
We analyze the information geometric structure of time reversibility for parametric families of irreducible transition kernels of Markov chains. We define and characterize reversible exponential families of Markov kernels, and show that irreducible and reversible Markov kernels form both a mixture family and, perhaps surprisingly, an exponential family in the set of all stochastic kernels. We propose a parametrization of the entire manifold of reversible kernels, and inspect reversible geodesics. We define information projections onto the reversible manifold, and derive closed-form expressions for the e-projection and m-projection, along with Pythagorean identities with respect to information divergence, leading to some new notion of reversiblization of Markov kernels. We show the family of edge measures pertaining to irreducible and reversible kernels also forms an exponential family among distributions over pairs. We further explore geometric properties of the reversible family, by comparing them with other remarkable families of stochastic matrices. Finally, we show that reversible kernels are, in a sense we define, the minimal exponential family generated by the m-family of symmetric kernels, and the smallest mixture family that comprises the e-family of memoryless kernels.
△ Less
Submitted 20 October, 2021; v1 submitted 10 June, 2021;
originally announced June 2021.
-
On the $α$-lazy version of Markov chains in estimation and testing problems
Authors:
Sela Fried,
Geoffrey Wolfer
Abstract:
Given access to a single long trajectory generated by an unknown irreducible Markov chain $M$, we simulate an $α$-lazy version of $M$ which is ergodic. This enables us to generalize recent results on estimation and identity testing that were stated for ergodic Markov chains in a way that allows fully empirical inference. In particular, our approach shows that the pseudo spectral gap introduced by…
▽ More
Given access to a single long trajectory generated by an unknown irreducible Markov chain $M$, we simulate an $α$-lazy version of $M$ which is ergodic. This enables us to generalize recent results on estimation and identity testing that were stated for ergodic Markov chains in a way that allows fully empirical inference. In particular, our approach shows that the pseudo spectral gap introduced by Paulin [2015] and defined for ergodic Markov chains may be given a meaning already in the case of irreducible but possibly periodic Markov chains.
△ Less
Submitted 29 October, 2021; v1 submitted 20 May, 2021;
originally announced May 2021.
-
Dimension-Free Empirical Entropy Estimation
Authors:
Doron Cohen,
Aryeh Kontorovich,
Aaron Koolyk,
Geoffrey Wolfer
Abstract:
We seek an entropy estimator for discrete distributions with fully empirical accuracy bounds. As stated, this goal is infeasible without some prior assumptions on the distribution. We discover that a certain information moment assumption renders the problem feasible. We argue that the moment assumption is natural and, in some sense, {\em minimalistic} -- weaker than finite support or tail decay co…
▽ More
We seek an entropy estimator for discrete distributions with fully empirical accuracy bounds. As stated, this goal is infeasible without some prior assumptions on the distribution. We discover that a certain information moment assumption renders the problem feasible. We argue that the moment assumption is natural and, in some sense, {\em minimalistic} -- weaker than finite support or tail decay conditions. Under the moment assumption, we provide the first finite-sample entropy estimates for infinite alphabets, nearly recovering the known minimax rates. Moreover, we demonstrate that our empirical bounds are significantly sharper than the state-of-the-art bounds, for various natural distributions and non-trivial sample regimes. Along the way, we give a dimension-free analogue of the Cover-Thomas result on entropy continuity (with respect to total variation distance) for finite alphabets, which may be of independent interest. Additionally, we resolve all of the open problems posed by Jürgensen and Matthews, 2010.
△ Less
Submitted 26 December, 2022; v1 submitted 16 May, 2021;
originally announced May 2021.
-
Identity testing of reversible Markov chains
Authors:
Sela Fried,
Geoffrey Wolfer
Abstract:
We consider the problem of identity testing of Markov chain transition matrices based on a single trajectory of observations under the distance notion introduced by Daskalakis et al. [2018a] and further analyzed by Cherapanamjeri and Bartlett [2019]. Both works made the restrictive assumption that the Markov chains under consideration are symmetric. In this work we relax the symmetry assumption an…
▽ More
We consider the problem of identity testing of Markov chain transition matrices based on a single trajectory of observations under the distance notion introduced by Daskalakis et al. [2018a] and further analyzed by Cherapanamjeri and Bartlett [2019]. Both works made the restrictive assumption that the Markov chains under consideration are symmetric. In this work we relax the symmetry assumption and show that it is possible to perform identity testing under the much weaker assumption of reversibility, provided that the stationary distributions of the reference and of the unknown Markov chains are close under a distance notion related to the separation distance. Additionally, we provide intuition on the distance notion of Daskalakis et al. [2018a] by showing how it behaves under several natural operations. In particular, we address some of their open questions.
△ Less
Submitted 11 March, 2022; v1 submitted 13 May, 2021;
originally announced May 2021.
-
Empirical and Instance-Dependent Estimation of Markov Chain and Mixing Time
Authors:
Geoffrey Wolfer
Abstract:
We address the problem of estimating the mixing time of a Markov chain from a single trajectory of observations. Unlike most previous works which employed Hilbert space methods to estimate spectral gaps, we opt for an approach based on contraction with respect to total variation. Specifically, we estimate the contraction coefficient introduced in Wolfer [2020], inspired from Dobrushin's. This quan…
▽ More
We address the problem of estimating the mixing time of a Markov chain from a single trajectory of observations. Unlike most previous works which employed Hilbert space methods to estimate spectral gaps, we opt for an approach based on contraction with respect to total variation. Specifically, we estimate the contraction coefficient introduced in Wolfer [2020], inspired from Dobrushin's. This quantity, unlike the spectral gap, controls the mixing time up to strong universal constants and remains applicable to non-reversible chains. We improve existing fully data-dependent confidence intervals around this contraction coefficient, which are both easier to compute and thinner than spectral counterparts. Furthermore, we introduce a novel analysis beyond the worst-case scenario by leveraging additional information about the transition matrix. This allows us to derive instance-dependent rates for estimating the matrix with respect to the induced uniform norm, and some of its mixing properties.
△ Less
Submitted 12 September, 2023; v1 submitted 14 December, 2019;
originally announced December 2019.
-
Estimating the Mixing Time of Ergodic Markov Chains
Authors:
Geoffrey Wolfer,
Aryeh Kontorovich
Abstract:
We address the problem of estimating the mixing time $t_{\mathsf{mix}}$ of an arbitrary ergodic finite-state Markov chain from a single trajectory of length $m$. The reversible case was addressed by Hsu et al. [2019], who left the general case as an open problem. In the reversible case, the analysis is greatly facilitated by the fact that the Markov operator is self-adjoint, and Weyl's inequality…
▽ More
We address the problem of estimating the mixing time $t_{\mathsf{mix}}$ of an arbitrary ergodic finite-state Markov chain from a single trajectory of length $m$. The reversible case was addressed by Hsu et al. [2019], who left the general case as an open problem. In the reversible case, the analysis is greatly facilitated by the fact that the Markov operator is self-adjoint, and Weyl's inequality allows for a dimension-free perturbation analysis of the empirical eigenvalues. As Hsu et al. point out, in the absence of reversibility (which induces asymmetric pair probabilities matrices), the existing perturbation analysis has a worst-case exponential dependence on the number of states $d$. Furthermore, even if an eigenvalue perturbation analysis with better dependence on $d$ were available, in the non-reversible case the connection between the spectral gap and the mixing time is not nearly as straightforward as in the reversible case. Our key insight is to estimate the pseudo-spectral gap $γ_{\mathsf{ps}}$ instead, which allows us to overcome the loss of symmetry and to achieve a polynomial dependence on the minimal stationary probability $π_\star$ and $γ_{\mathsf{ps}}$. Additionally, in the reversible case, we obtain simultaneous nearly (up to logarithmic factors) minimax rates in $t_{\mathsf{mix}}$ and precision $\varepsilon$, closing a gap in Hsu et al., who treated $\varepsilon$ as constant in the lower bounds. Finally, we construct fully empirical confidence intervals for $γ_{\mathsf{ps}}$, which shrink to zero at a rate of roughly $1/\sqrt{m}$, and improve the state of the art in even the reversible case.
△ Less
Submitted 16 August, 2022; v1 submitted 1 February, 2019;
originally announced February 2019.
-
Minimax Testing of Identity to a Reference Ergodic Markov Chain
Authors:
Geoffrey Wolfer,
Aryeh Kontorovich
Abstract:
We exhibit an efficient procedure for testing, based on a single long state sequence, whether an unknown Markov chain is identical to or $\varepsilon$-far from a given reference chain. We obtain nearly matching (up to logarithmic factors) upper and lower sample complexity bounds for our notion of distance, which is based on total variation. Perhaps surprisingly, we discover that the sample complex…
▽ More
We exhibit an efficient procedure for testing, based on a single long state sequence, whether an unknown Markov chain is identical to or $\varepsilon$-far from a given reference chain. We obtain nearly matching (up to logarithmic factors) upper and lower sample complexity bounds for our notion of distance, which is based on total variation. Perhaps surprisingly, we discover that the sample complexity depends solely on the properties of the known reference chain and does not involve the unknown chain at all, which is not even assumed to be ergodic.
△ Less
Submitted 24 September, 2019; v1 submitted 31 January, 2019;
originally announced February 2019.
-
Statistical Estimation of Ergodic Markov Chain Kernel over Discrete State Space
Authors:
Geoffrey Wolfer,
Aryeh Kontorovich
Abstract:
We investigate the statistical complexity of estimating the parameters of a discrete-state Markov chain kernel from a single long sequence of state observations. In the finite case, we characterize (modulo logarithmic factors) the minimax sample complexity of estimation with respect to the operator infinity norm, while in the countably infinite case, we analyze the problem with respect to a natura…
▽ More
We investigate the statistical complexity of estimating the parameters of a discrete-state Markov chain kernel from a single long sequence of state observations. In the finite case, we characterize (modulo logarithmic factors) the minimax sample complexity of estimation with respect to the operator infinity norm, while in the countably infinite case, we analyze the problem with respect to a natural entry-wise norm derived from total variation. We show that in both cases, the sample complexity is governed by the mixing properties of the unknown chain, for which, in the finite-state case, there are known finite-sample estimators with fully empirical confidence intervals.
△ Less
Submitted 13 August, 2020; v1 submitted 13 September, 2018;
originally announced September 2018.
