Showing 1–50 of 52 results for author: Webber, J

Dynamical mixture modeling with fast, automatic determination of Markov chains

Authors: Christopher E. Miles, Robert J. Webber

Abstract: Markov state modeling has gained popularity in various scientific fields due to its ability to reduce complex time series data into transitions between a few states. Yet, current frameworks are limited by assuming a single Markov chain describes the data, and they suffer an inability to discern heterogeneities. As a solution, this paper proposes a variational expectation-maximization algorithm tha… ▽ More Markov state modeling has gained popularity in various scientific fields due to its ability to reduce complex time series data into transitions between a few states. Yet, current frameworks are limited by assuming a single Markov chain describes the data, and they suffer an inability to discern heterogeneities. As a solution, this paper proposes a variational expectation-maximization algorithm that identifies a mixture of Markov chains in a time-series data set. The method is agnostic to the definition of the Markov states, whether data-driven (e.g. by spectral clustering) or based on domain knowledge. Variational EM efficiently and organically identifies the number of Markov chains and dynamics of each chain without expensive model comparisons or posterior sampling. The approach is supported by a theoretical analysis and numerical experiments, including simulated and observational data sets based on ${\tt Last.fm}$ music listening, ultramarathon running, and gene expression. The results show the new algorithm is competitive with contemporary mixture modeling approaches and powerful in identifying meaningful heterogeneities in time series data. △ Less

Submitted 7 June, 2024; originally announced June 2024.

The fast committor machine: Interpretable prediction with kernels

Authors: D. Aristoff, M. Johnson, G. Simpson, R. J. Webber

Abstract: In the study of stochastic systems, the committor function describes the probability that a system starting from an initial configuration $x$ will reach a set $B$ before a set $A$. This paper introduces an efficient and interpretable algorithm for approximating the committor, called the "fast committor machine" (FCM). The FCM uses simulated trajectory data to build a kernel-based model of the comm… ▽ More In the study of stochastic systems, the committor function describes the probability that a system starting from an initial configuration $x$ will reach a set $B$ before a set $A$. This paper introduces an efficient and interpretable algorithm for approximating the committor, called the "fast committor machine" (FCM). The FCM uses simulated trajectory data to build a kernel-based model of the committor. The kernel function is constructed to emphasize low-dimensional subspaces that optimally describe the $A$ to $B$ transitions. The coefficients in the kernel model are determined using randomized linear algebra, leading to a runtime that scales linearly in the number of data points. In numerical experiments involving a triple-well potential and alanine dipeptide, the FCM yields higher accuracy and trains more quickly than a neural network with the same number of parameters. The FCM is also more interpretable than the neural net. △ Less

Submitted 10 August, 2024; v1 submitted 16 May, 2024; originally announced May 2024.

Comments: 10 pages, 7 figures

MSC Class: 82C31; 82C32; 65C30; 65C40

Local reconstruction analysis of inverting the Radon transform in the plane from noisy discrete data

Authors: Anuj Abhishek, Alexander Katsevich, James W. Webber

Abstract: In this paper, we investigate the reconstruction error, $N_\e^{\text{rec}}(x)$, when a linear, filtered back-projection (FBP) algorithm is applied to noisy, discrete Radon transform data with sampling step size $ε$ in two-dimensions. Specifically, we analyze $N_\e^{\text{rec}}(x)$ for $x$ in small, $O(\e)$-sized neighborhoods around a generic fixed point, $x_0$, in the plane, where the measurement… ▽ More In this paper, we investigate the reconstruction error, $N_\e^{\text{rec}}(x)$, when a linear, filtered back-projection (FBP) algorithm is applied to noisy, discrete Radon transform data with sampling step size $ε$ in two-dimensions. Specifically, we analyze $N_\e^{\text{rec}}(x)$ for $x$ in small, $O(\e)$-sized neighborhoods around a generic fixed point, $x_0$, in the plane, where the measurement noise values, $η_{k,j}$ (i.e., the errors in the sinogram space), are random variables. The latter are independent, but not necessarily identically distributed. We show, under suitable assumptions on the first three moments of the $η_{k,j}$, that the following limit exists: $N^{\text{rec}}(\chx;x_0) = \lim_{\e\to0}N_\e^{\text{rec}}(x_0+\e\chx)$, for $\check x$ in a bounded domain. Here, $N_\e^{\text{rec}}$ and $ N^{\text{rec}}$ are viewed as continuous random variables, and the limit is understood in the sense of distributions. Once the limit is established, we prove that $N^{\text{rec}}$ is a zero mean Gaussian random field and compute explicitly its covariance. In addition, we validate our theory using numerical simulations and pseudo random noise. △ Less

Submitted 19 March, 2024; originally announced March 2024.

AI can identify Solar System instability billions of years in advance

Authors: Dorian S. Abbot, J. D. Laurence-Chasen, Robert J. Webber, David M. Hernandez, Jonathan Weare

Abstract: Rare event schemes require an approximation of the probability of the rare event as a function of system state. Finding an appropriate reaction coordinate is typically the most challenging aspect of applying a rare event scheme. Here we develop an artificial intelligence (AI) based reaction coordinate that effectively predicts which of a limited number of simulations of the Solar System will go un… ▽ More Rare event schemes require an approximation of the probability of the rare event as a function of system state. Finding an appropriate reaction coordinate is typically the most challenging aspect of applying a rare event scheme. Here we develop an artificial intelligence (AI) based reaction coordinate that effectively predicts which of a limited number of simulations of the Solar System will go unstable using a convolutional neural network classifier. The performance of the algorithm does not degrade significantly even 3.5 billion years before the instability. We overcome the class imbalance intrinsic to rare event problems using a combination of minority class oversampling, increased minority class weighting, and pulling multiple non-overlapping training sequences from simulations. Our success suggests that AI may provide a promising avenue for developing reaction coordinates without detailed theoretical knowledge of the system. △ Less

Submitted 15 January, 2024; originally announced January 2024.

Comments: 3 pages, 1 figure

Journal ref: Res. Notes AAS 8 3 (2024)

Surface of revolution Radon transforms with centers on generalized surfaces in $\mathbb{R}^n$

Authors: James W. Webber, Sean Holman, Eric Todd Quinto

Abstract: We present a novel analysis of a Radon transform, $R$, which maps an $L^2$ function of compact support to its integrals over smooth surfaces of revolution with centers on an embedded hypersurface in $\mathbb{R}^n$. Using microlocal analysis, we derive necessary and sufficient conditions relating to $R$ for the Bolker condition to hold, which has implications regarding the existence and location of… ▽ More We present a novel analysis of a Radon transform, $R$, which maps an $L^2$ function of compact support to its integrals over smooth surfaces of revolution with centers on an embedded hypersurface in $\mathbb{R}^n$. Using microlocal analysis, we derive necessary and sufficient conditions relating to $R$ for the Bolker condition to hold, which has implications regarding the existence and location of image artifacts. We present a general inversion framework based on Volterra equation theory and known results on the spherical Radon transform, and we prove injectivity results for $R$. Several example applications of our theory are discussed in the context of, e.g., Compton Scatter Tomography (CST) and Ultrasound Reflection Tomography (URT). In addition, using the proposed inversion framework, we validate our microlocal theory via simulation, and present simulated image reconstructions of image phantoms with added noise. △ Less

Submitted 25 December, 2023; originally announced December 2023.

Comments: 23 pages, 7 figures

Randomly sparsified Richardson iteration is really fast

Authors: Jonathan Weare, Robert J. Webber

Abstract: Recently, a class of algorithms combining classical fixed point iterations with repeated random sparsification of approximate solution vectors has been successfully applied to eigenproblems with matrices as large as $10^{108} \times 10^{108}$. So far, a complete mathematical explanation for their success has proven elusive. Additionally, the methods have not been extended to linear system solves.… ▽ More Recently, a class of algorithms combining classical fixed point iterations with repeated random sparsification of approximate solution vectors has been successfully applied to eigenproblems with matrices as large as $10^{108} \times 10^{108}$. So far, a complete mathematical explanation for their success has proven elusive. Additionally, the methods have not been extended to linear system solves. In this paper we propose a new scheme based on repeated random sparsification that is capable of solving linear systems in extremely high dimensions. We provide a complete mathematical analysis of this new algorithm. Our analysis establishes a faster-than-Monte Carlo convergence rate and justifies use of the scheme even when the solution vector itself is too large to store. △ Less

Submitted 17 November, 2023; v1 submitted 29 September, 2023; originally announced September 2023.

Comments: 27 pages, 2 figures

On a cylindrical scanning modality in three-dimensional Compton scatter tomography

Authors: James W. Webber

Abstract: We present injectivity and microlocal analyses of a new generalized Radon transform, $\mathcal{R}$, which has applications to a novel scanner design in three-dimensional Compton Scattering Tomography (CST), which we also introduce here. Using Fourier decomposition and Volterra equation theory, we prove that $\mathcal{R}$ is injective and show that the image solution is unique. Using microlocal ana… ▽ More We present injectivity and microlocal analyses of a new generalized Radon transform, $\mathcal{R}$, which has applications to a novel scanner design in three-dimensional Compton Scattering Tomography (CST), which we also introduce here. Using Fourier decomposition and Volterra equation theory, we prove that $\mathcal{R}$ is injective and show that the image solution is unique. Using microlocal analysis, we prove that $\mathcal{R}$ satisfies the Bolker condition, and we investigate the edge detection capabilities of $\mathcal{R}$. This has important implications regarding the stability of inversion and the amplification of measurement noise. In addition, we present simulated 3-D image reconstructions from $\mathcal{R}f$ data, where $f$ is a 3-D density, with varying levels of added Gaussian noise. This paper provides the theoretical groundwork for 3-D CST using the proposed scanner design. △ Less

Submitted 8 July, 2023; originally announced July 2023.

Comments: 23 pages, 9 figures, 1 table

Randomized algorithms for low-rank matrix approximation: Design, analysis, and applications

Authors: Joel A. Tropp, Robert J. Webber

Abstract: This survey explores modern approaches for computing low-rank approximations of high-dimensional matrices by means of the randomized SVD, randomized subspace iteration, and randomized block Krylov iteration. The paper compares the procedures via theoretical analyses and numerical studies to highlight how the best choice of algorithm depends on spectral properties of the matrix and the computationa… ▽ More This survey explores modern approaches for computing low-rank approximations of high-dimensional matrices by means of the randomized SVD, randomized subspace iteration, and randomized block Krylov iteration. The paper compares the procedures via theoretical analyses and numerical studies to highlight how the best choice of algorithm depends on spectral properties of the matrix and the computational resources available. Despite superior performance for many problems, randomized block Krylov iteration has not been widely adopted in computational science. The paper strengthens the case for this method in three ways. First, it presents new pseudocode that can significantly reduce computational costs. Second, it provides a new analysis that yields simple, precise, and informative error bounds. Last, it showcases applications to challenging scientific problems, including principal component analysis for genetic data and spectral clustering for molecular dynamics data. △ Less

Submitted 21 September, 2023; v1 submitted 21 June, 2023; originally announced June 2023.

Comments: 60 pages, 14 figures

MSC Class: 68W20; 65F10; 65F55

Mercury's chaotic secular evolution as a subdiffusive process

Authors: Dorian S. Abbot, Robert J. Webber, David M. Hernandez, Sam Hadden, Jonathan Weare

Abstract: Mercury's orbit can destabilize, generally resulting in a collision with either Venus or the Sun. Chaotic evolution can cause g1 to decrease to the approximately constant value of g5 and create a resonance. Previous work has approximated the variation in g1 as stochastic diffusion, which leads to a phenomological model that can reproduce the Mercury instability statistics of secular and N-body mod… ▽ More Mercury's orbit can destabilize, generally resulting in a collision with either Venus or the Sun. Chaotic evolution can cause g1 to decrease to the approximately constant value of g5 and create a resonance. Previous work has approximated the variation in g1 as stochastic diffusion, which leads to a phenomological model that can reproduce the Mercury instability statistics of secular and N-body models on timescales longer than 10 Gyr. Here we show that the diffusive model underpredicts the Mercury instability probability by a factor of 3-10,000 on timescales less than 5 Gyr, the remaining lifespan of the Solar System. This is because g1 exhibits larger variations on short timescales than the diffusive model would suggest. To better model the variations on short timescales, we build a new subdiffusive phenomological model for g1. Subdiffusion is similar to diffusion but exhibits larger displacements on short timescales and smaller displacements on long timescales. We choose model parameters based on the behavior of the g1 trajectories in the N-body simulations, leading to a tuned model that can reproduce Mercury instability statistics from 1-40 Gyr. This work motivates fundamental questions in Solar System dynamics: Why does subdiffusion better approximate the variation in g1 than standard diffusion? Why is there an upper bound on g1, but not a lower bound that would prevent it from reaching g5? △ Less

Submitted 12 April, 2024; v1 submitted 20 June, 2023; originally announced June 2023.

Comments: accepted at ApJ

Generalized Abel equations and applications to translation invariant Radon transforms

Authors: James W. Webber

Abstract: Generalized Abel equations have been employed in the recent literature to invert Radon transforms which arise in a number of important imaging applications, including Compton Scatter Tomography (CST), Ultrasound Reflection Tomography (URT), and X-ray CT. In this paper, we present novel injectivity results and inversion methods for Generalized Abel operators. We apply our theory to a new Radon tran… ▽ More Generalized Abel equations have been employed in the recent literature to invert Radon transforms which arise in a number of important imaging applications, including Compton Scatter Tomography (CST), Ultrasound Reflection Tomography (URT), and X-ray CT. In this paper, we present novel injectivity results and inversion methods for Generalized Abel operators. We apply our theory to a new Radon transform, $\mathcal{R}_j$, of interest in URT, which integrates a square integrable function of compact support, $f$, over ellipsoid and hyperboloid surfaces with centers on a plane. Using our newly established theory on generalized Abel equations, we show that $\mathcal{R}_j$ is injective and provide an inversion method based on Neumann series. In addition, using algebraic methods, we present image phantom reconstructions from $\mathcal{R}_jf$ data with added pseudo random noise. △ Less

Submitted 13 June, 2023; originally announced June 2023.

Comments: 23 pages, 6 figures, 2 tables

Robust, randomized preconditioning for kernel ridge regression

Authors: Mateo Díaz, Ethan N. Epperly, Zachary Frangella, Joel A. Tropp, Robert J. Webber

Abstract: This paper investigates two randomized preconditioning techniques for solving kernel ridge regression (KRR) problems with a medium to large number of data points ($10^4 \leq N \leq 10^7$), and it introduces two new methods with state-of-the-art performance. The first method, RPCholesky preconditioning, accurately solves the full-data KRR problem in $O(N^2)$ arithmetic operations, assuming sufficie… ▽ More This paper investigates two randomized preconditioning techniques for solving kernel ridge regression (KRR) problems with a medium to large number of data points ($10^4 \leq N \leq 10^7$), and it introduces two new methods with state-of-the-art performance. The first method, RPCholesky preconditioning, accurately solves the full-data KRR problem in $O(N^2)$ arithmetic operations, assuming sufficiently rapid polynomial decay of the kernel matrix eigenvalues. The second method, KRILL preconditioning, offers an accurate solution to a restricted version of the KRR problem involving $k \ll N$ selected data centers at a cost of $O((N + k^2) k \log k)$ operations. The proposed methods solve a broad range of KRR problems, making them ideal for practical applications. △ Less

Submitted 10 July, 2024; v1 submitted 24 April, 2023; originally announced April 2023.

Comments: 29 pages, 11 figures

MSC Class: 68W20; 65F10; 65F55

XTrace: Making the most of every sample in stochastic trace estimation

Authors: Ethan N. Epperly, Joel A. Tropp, Robert J. Webber

Abstract: The implicit trace estimation problem asks for an approximation of the trace of a square matrix, accessed via matrix-vector products (matvecs). This paper designs new randomized algorithms, XTrace and XNysTrace, for the trace estimation problem by exploiting both variance reduction and the exchangeability principle. For a fixed budget of matvecs, numerical experiments show that the new methods can… ▽ More The implicit trace estimation problem asks for an approximation of the trace of a square matrix, accessed via matrix-vector products (matvecs). This paper designs new randomized algorithms, XTrace and XNysTrace, for the trace estimation problem by exploiting both variance reduction and the exchangeability principle. For a fixed budget of matvecs, numerical experiments show that the new methods can achieve errors that are orders of magnitude smaller than existing algorithms, such as the Girard-Hutchinson estimator or the Hutch++ estimator. A theoretical analysis confirms the benefits by offering a precise description of the performance of these algorithms as a function of the spectrum of the input matrix. The paper also develops an exchangeable estimator, XDiag, for approximating the diagonal of a square matrix using matvecs. △ Less

Submitted 5 January, 2024; v1 submitted 18 January, 2023; originally announced January 2023.

Comments: 31 pages, 8 figures

MSC Class: 65C05; 65F30; 68W20

Journal ref: SIAM Journal on Matrix Analysis and Applications, 45(1), 1-23 (2024)

Localized covariance estimation: A Bayesian perspective

Authors: Robert J. Webber, Matthias Morzfeld

Abstract: A major problem in numerical weather prediction (NWP) is the estimation of high-dimensional covariance matrices from a small number of samples. Maximum likelihood estimators cannot provide reliable estimates when the overall dimension is much larger than the number of samples. Fortunately, NWP practitioners have found ingenious ways to boost the accuracy of their covariance estimators by leveragin… ▽ More A major problem in numerical weather prediction (NWP) is the estimation of high-dimensional covariance matrices from a small number of samples. Maximum likelihood estimators cannot provide reliable estimates when the overall dimension is much larger than the number of samples. Fortunately, NWP practitioners have found ingenious ways to boost the accuracy of their covariance estimators by leveraging the assumption that the correlations decay with spatial distance. In this work, Bayesian statistics is used to provide a new justification and analysis of the practical NWP covariance estimators. The Bayesian framework involves manipulating distributions over symmetric positive definite matrices, and it leads to two main findings: (i) the commonly used "hybrid estimator" for the covariance matrix has a naturally Bayesian interpretation; (ii) the very commonly used "Schur product estimator" is not Bayesian, but it can be studied and understood within the Bayesian framework. As practical implications, the Bayesian framework shows how to reduce the amount of tuning required for covariance estimation, and it suggests that efficient covariance estimation should be rooted in understanding and penalizing conditional correlations, rather than correlations. △ Less

Submitted 12 January, 2023; originally announced January 2023.

Comments: 20 pages, 4 figures

MSC Class: 62H10; 65C20; 86-10

Simple physics and integrators accurately reproduce Mercury instability statistics

Authors: Dorian S. Abbot, David M. Hernandez, Sam Hadden, Robert J. Webber, Georgios P. Afentakis, Jonathan Weare

Abstract: The long-term stability of the Solar System is an issue of significant scientific and philosophical interest. The mechanism leading to instability is Mercury's eccentricity being pumped up so high that Mercury either collides with Venus or is scattered into the Sun. Previously, only three five-billion-year $N$-body ensembles of the Solar System with thousands of simulations have been run to assess… ▽ More The long-term stability of the Solar System is an issue of significant scientific and philosophical interest. The mechanism leading to instability is Mercury's eccentricity being pumped up so high that Mercury either collides with Venus or is scattered into the Sun. Previously, only three five-billion-year $N$-body ensembles of the Solar System with thousands of simulations have been run to assess long-term stability. We generate two additional ensembles, each with 2750 members, and make them publicly available at \texttt{https://archive.org/details/@dorianabbot}. We find that accurate Mercury instability statistics can be obtained by (1) including only the Sun and the 8 planets, (2) using a simple Wisdom-Holman scheme without correctors, (3) using a basic representation of general relativity, and (4) using a time step of 3.16 days. By combining our Solar System ensembles with previous ensembles we form a 9,601-member ensemble of ensembles. In this ensemble of ensembles, the logarithm of the frequency of a Mercury instability event increases linearly with time between 1.3 and 5 Gyr, suggesting that a single mechanism is responsible for Mercury instabilities in this time range and that this mechanism becomes more active as time progresses. Our work provides a robust estimate of Mercury instability statistics over the next five billion years, outlines methodologies that may be useful for exoplanet system investigations, and provides two large ensembles of publicly available Solar System integrations that can serve as testbeds for theoretical ideas as well as training sets for artificial intelligence schemes. △ Less

Submitted 21 February, 2023; v1 submitted 30 December, 2022; originally announced December 2022.

Comments: accepted at ApJ

Journal ref: Ap.J. 944 (2023) 190

Multi-coil MRI by analytic continuation

Authors: James W. Webber

Abstract: We present novel reconstruction and stability analysis methodologies for two-dimensional, multi-coil MRI, based on analytic continuation ideas. We show that the 2-D, limited-data MRI inverse problem, whereby the missing parts of $\textbf{k}$-space (Fourier space) are lines parallel to either $k_1$ or $k_2$ (i.e., the $\textbf{k}$-space axis), can be reduced to a set of 1-D Fredholm type inverse pr… ▽ More We present novel reconstruction and stability analysis methodologies for two-dimensional, multi-coil MRI, based on analytic continuation ideas. We show that the 2-D, limited-data MRI inverse problem, whereby the missing parts of $\textbf{k}$-space (Fourier space) are lines parallel to either $k_1$ or $k_2$ (i.e., the $\textbf{k}$-space axis), can be reduced to a set of 1-D Fredholm type inverse problems. The Fredholm equations are then solved to recover the 2-D image on 1-D line profiles (``slice-by-slice" imaging). The technique is tested on a range of medical in vivo images (e.g., brain, spine, cardiac), and phantom data. Our method is shown to offer optimal performance, in terms of structural similarity, when compared against similar methods from the literature, and when the $\textbf{k}$-space data is sub-sampled at random so as to simulate motion corruption. In addition, we present a Singular Value Decomposition (SVD) and stability analysis of the Fredholm operators, and compare the stability properties of different $\textbf{k}$-space sub-sampling schemes (e.g., random vs uniform accelerated sampling). △ Less

Submitted 20 December, 2022; originally announced December 2022.

Comments: 20 pages, 9 figures

Ellipsoidal and hyperbolic Radon transforms; microlocal properties and injectivity

Authors: James W. Webber, Sean Holman, Eric Todd Quinto

Abstract: We present novel microlocal and injectivity analyses of ellipsoid and hyperboloid Radon transforms. We introduce a new Radon transform, $R$, which defines the integrals of a compactly supported $L^2$ function, $f$, over ellipsoids and hyperboloids with centers on a smooth connected surface, $S$. $R$ is shown to be a Fourier Integral Operator (FIO) and in our main theorem we prove that $R$ satisfie… ▽ More We present novel microlocal and injectivity analyses of ellipsoid and hyperboloid Radon transforms. We introduce a new Radon transform, $R$, which defines the integrals of a compactly supported $L^2$ function, $f$, over ellipsoids and hyperboloids with centers on a smooth connected surface, $S$. $R$ is shown to be a Fourier Integral Operator (FIO) and in our main theorem we prove that $R$ satisfies the Bolker condition if the support of $f$ is connected and not intersected by any plane tangent to $S$. Under certain conditions, this is an equivalence. We give examples where our theory can be applied. Focusing specifically on a cylindrical geometry of interest in Ultrasound Reflection Tomography (URT), we prove injectivity results and investigate the visible singularities. In addition, we present example reconstructions of image phantoms in two-dimensions, and validate our microlocal theory. △ Less

Submitted 30 November, 2022; originally announced December 2022.

Comments: 26 pages, 11 figures

Understanding and eliminating spurious modes in variational Monte Carlo using collective variables

Authors: Huan Zhang, Robert J. Webber, Michael Lindsey, Timothy C. Berkelbach, Jonathan Weare

Abstract: The use of neural network parametrizations to represent the ground state in variational Monte Carlo (VMC) calculations has generated intense interest in recent years. However, as we demonstrate in the context of the periodic Heisenberg spin chain, this approach can produce unreliable wave function approximations. One of the most obvious signs of failure is the occurrence of random, persistent spik… ▽ More The use of neural network parametrizations to represent the ground state in variational Monte Carlo (VMC) calculations has generated intense interest in recent years. However, as we demonstrate in the context of the periodic Heisenberg spin chain, this approach can produce unreliable wave function approximations. One of the most obvious signs of failure is the occurrence of random, persistent spikes in the energy estimate during training. These energy spikes are caused by regions of configuration space that are over-represented by the wave function density, which are called ``spurious modes'' in the machine learning literature. After exploring these spurious modes in detail, we demonstrate that a collective-variable-based penalization yields a substantially more robust training procedure, preventing the formation of spurious modes and improving the accuracy of energy estimates. Because the penalization scheme is cheap to implement and is not specific to the particular model studied here, it can be extended to other applications of VMC where a reasonable choice of collective variable is available. △ Less

Submitted 11 November, 2022; originally announced November 2022.

Comments: 12 pages, 13 figures

Journal ref: Phys.Rev.Research 5 (2023) 023101

Autovocoder: Fast Waveform Generation from a Learned Speech Representation using Differentiable Digital Signal Processing

Authors: Jacob J Webber, Cassia Valentini-Botinhao, Evelyn Williams, Gustav Eje Henter, Simon King

Abstract: Most state-of-the-art Text-to-Speech systems use the mel-spectrogram as an intermediate representation, to decompose the task into acoustic modelling and waveform generation. A mel-spectrogram is extracted from the waveform by a simple, fast DSP operation, but generating a high-quality waveform from a mel-spectrogram requires computationally expensive machine learning: a neural vocoder. Our prop… ▽ More Most state-of-the-art Text-to-Speech systems use the mel-spectrogram as an intermediate representation, to decompose the task into acoustic modelling and waveform generation. A mel-spectrogram is extracted from the waveform by a simple, fast DSP operation, but generating a high-quality waveform from a mel-spectrogram requires computationally expensive machine learning: a neural vocoder. Our proposed ``autovocoder'' reverses this arrangement. We use machine learning to obtain a representation that replaces the mel-spectrogram, and that can be inverted back to a waveform using simple, fast operations including a differentiable implementation of the inverse STFT. The autovocoder generates a waveform 5 times faster than the DSP-based Griffin-Lim algorithm, and 14 times faster than the neural vocoder HiFi-GAN. We provide perceptual listening test results to confirm that the speech is of comparable quality to HiFi-GAN in the copy synthesis task. △ Less

Submitted 24 May, 2023; v1 submitted 13 November, 2022; originally announced November 2022.

Comments: Accepted to the 2023 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2023)

Journal ref: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Rhodes Island, Greece, 2023, pp. 1-5

Randomly pivoted Cholesky: Practical approximation of a kernel matrix with few entry evaluations

Authors: Yifan Chen, Ethan N. Epperly, Joel A. Tropp, Robert J. Webber

Abstract: The randomly pivoted partial Cholesky algorithm (RPCholesky) computes a factorized rank-k approximation of an N x N positive-semidefinite (psd) matrix. RPCholesky requires only (k + 1) N entry evaluations and O(k^2 N) additional arithmetic operations, and it can be implemented with just a few lines of code. The method is particularly useful for approximating a kernel matrix. This paper offers a… ▽ More The randomly pivoted partial Cholesky algorithm (RPCholesky) computes a factorized rank-k approximation of an N x N positive-semidefinite (psd) matrix. RPCholesky requires only (k + 1) N entry evaluations and O(k^2 N) additional arithmetic operations, and it can be implemented with just a few lines of code. The method is particularly useful for approximating a kernel matrix. This paper offers a thorough new investigation of the empirical and theoretical behavior of this fundamental algorithm. For matrix approximation problems that arise in scientific machine learning, experiments show that RPCholesky matches or beats the performance of alternative algorithms. Moreover, RPCholesky provably returns low-rank approximations that are nearly optimal. The simplicity, effectiveness, and robustness of RPCholesky strongly support its use in scientific computing and machine learning applications. △ Less

Submitted 12 December, 2023; v1 submitted 13 July, 2022; originally announced July 2022.

Comments: 38 pages, 4 figures

MSC Class: 65F55; 65C99; 68T05

Weighted ensemble: Recent mathematical developments

Authors: D. Aristoff, J. Copperman, G. Simpson, R. J. Webber, D. M. Zuckerman

Abstract: The weighted ensemble (WE) method, an enhanced sampling approach based on periodically replicating and pruning trajectories in a set of parallel simulations, has grown increasingly popular for computational biochemistry problems, due in part to improved hardware and the availability of modern software. Algorithmic and analytical improvements have also played an important role, and progress has acc… ▽ More The weighted ensemble (WE) method, an enhanced sampling approach based on periodically replicating and pruning trajectories in a set of parallel simulations, has grown increasingly popular for computational biochemistry problems, due in part to improved hardware and the availability of modern software. Algorithmic and analytical improvements have also played an important role, and progress has accelerated in recent years. Here, we discuss and elaborate on the WE method from a mathematical perspective, highlighting recent results which have begun to yield greater computational efficiency. Notable among these innovations are variance reduction approaches that optimize trajectory management for systems of arbitrary dimensionality. △ Less

Submitted 29 June, 2022; originally announced June 2022.

Comments: 12 pages, 10 figures

Journal ref: J.Chem.Phys. 158 (2023) 014108

Full Configuration Interaction Excited-State Energies in Large Active Spaces from Subspace Iteration with Repeated Random Sparsification

Authors: Samuel M. Greene, Robert J. Webber, James E. T. Smith, Jonathan Weare, Timothy C. Berkelbach

Abstract: We present a stable and systematically improvable quantum Monte Carlo (QMC) approach to calculating excited-state energies, which we implement using our fast randomized iteration method for the full configuration interaction problem (FCI-FRI). Unlike previous excited-state quantum Monte Carlo methods, our approach, which is an asymmetric variant of subspace iteration, avoids the use of dot product… ▽ More We present a stable and systematically improvable quantum Monte Carlo (QMC) approach to calculating excited-state energies, which we implement using our fast randomized iteration method for the full configuration interaction problem (FCI-FRI). Unlike previous excited-state quantum Monte Carlo methods, our approach, which is an asymmetric variant of subspace iteration, avoids the use of dot products of random vectors and instead relies upon trial vectors to maintain orthogonality and estimate eigenvalues. By leveraging recent advances, we apply our method to calculate ground- and excited-state energies of strongly correlated molecular systems in large active spaces, including the carbon dimer with 8 electrons in 108 orbitals (8e,108o), an oxo-Mn(salen) transition metal complex (28e,28o), ozone (18e,87o), and butadiene (22e,82o). In the majority of these test cases, our approach yields total excited-state energies that agree with those from state-of-the-art methods -- including heat-bath CI, the density matrix renormalization group approach, and FCIQMC -- to within sub-milliHartree accuracy. In all cases, estimated excitation energies agree to within about 0.1 eV. △ Less

Submitted 12 October, 2022; v1 submitted 28 January, 2022; originally announced January 2022.

Comments: 16 pages, 5 figures, 3 tables

Microlocal properties of seven-dimensional lemon and apple Radon transforms with applications in Compton scattering tomography

Authors: James W. Webber, Eric Todd Quinto

Abstract: We present a microlocal analysis of two novel Radon transforms of interest in Compton Scattering Tomography (CST), which map compactly supported $L^2$ functions to their integrals over seven-dimensional sets of apple and lemon surfaces. Specifically, we show that the apple and lemon transforms are elliptic Fourier Integral Operators (FIO), which satisfy the Bolker condition. After an analysis of t… ▽ More We present a microlocal analysis of two novel Radon transforms of interest in Compton Scattering Tomography (CST), which map compactly supported $L^2$ functions to their integrals over seven-dimensional sets of apple and lemon surfaces. Specifically, we show that the apple and lemon transforms are elliptic Fourier Integral Operators (FIO), which satisfy the Bolker condition. After an analysis of the full seven-dimensional case, we focus our attention on $n$-D subsets of apple and lemon surfaces with fixed central axis, where $n<7$. Such subsets of surface integrals have applications in airport baggage and security screening. When the data dimensionality is restricted, the apple transform is shown to violate the Bolker condition, and there are artifacts which occur on apple-cylinder intersections. The lemon transform is shown to satisfy the Bolker condition, when the support of the function is restricted to the strip $\{0<z<1\}$. △ Less

Submitted 11 January, 2022; originally announced January 2022.

Comments: 29 pages, 6 figures

Data-driven transition path analysis yields a statistical understanding of sudden stratospheric warming events in an idealized model

Authors: Justin Finkel, Robert J. Webber, Edwin P. Gerber, Dorian S. Abbot, Jonathan Weare

Abstract: Atmospheric regime transitions are highly impactful as drivers of extreme weather events, but pose two formidable modeling challenges: predicting the next event (weather forecasting), and characterizing the statistics of events of a given severity (the risk climatology). Each event has a different duration and spatial structure, making it hard to define an objective "average event." We argue here… ▽ More Atmospheric regime transitions are highly impactful as drivers of extreme weather events, but pose two formidable modeling challenges: predicting the next event (weather forecasting), and characterizing the statistics of events of a given severity (the risk climatology). Each event has a different duration and spatial structure, making it hard to define an objective "average event." We argue here that transition path theory (TPT), a stochastic process framework, is an appropriate tool for the task. We demonstrate TPT's capacities on a wave-mean flow model of sudden stratospheric warmings (SSWs) developed by Holton and Mass (1976), which is idealized enough for transparent TPT analysis but complex enough to demonstrate computational scalability. Whereas a recent article (Finkel et al. 2021) studied near-term SSW predictability, the present article uses TPT to link predictability to long-term SSW frequency. This requires not only forecasting forward in time from an initial condition, but also \emph{backward in time} to assess the probability of the initial conditions themselves. TPT enables one to condition the dynamics on the regime transition occurring, and thus visualize its physical drivers with a vector field called the \emph{reactive current}. The reactive current shows that before an SSW, dissipation and stochastic forcing drive a slow decay of vortex strength at lower altitudes. The response of upper-level winds is late and sudden, occurring only after the transition is almost complete from a probabilistic point of view. This case study demonstrates that TPT quantities, visualized in a space of physically meaningful variables, can help one understand the dynamics of regime transitions. △ Less

Submitted 19 October, 2022; v1 submitted 28 August, 2021; originally announced August 2021.

Comments: 18 pages, 7 figures (main text), 19 pages, 1 figure (supplement). Accepted for publication in the Journal of the Atmospheric Sciences

Rayleigh-Gauss-Newton optimization with enhanced sampling for variational Monte Carlo

Authors: Robert J. Webber, Michael Lindsey

Abstract: Variational Monte Carlo (VMC) is an approach for computing ground-state wavefunctions that has recently become more powerful due to the introduction of neural network-based wavefunction parametrizations. However, efficiently training neural wavefunctions to converge to an energy minimum remains a difficult problem. In this work, we analyze optimization and sampling methods used in VMC and introduc… ▽ More Variational Monte Carlo (VMC) is an approach for computing ground-state wavefunctions that has recently become more powerful due to the introduction of neural network-based wavefunction parametrizations. However, efficiently training neural wavefunctions to converge to an energy minimum remains a difficult problem. In this work, we analyze optimization and sampling methods used in VMC and introduce alterations to improve their performance. First, based on theoretical convergence analysis in a noiseless setting, we motivate a new optimizer that we call the Rayleigh-Gauss-Newton method, which can improve upon gradient descent and natural gradient descent to achieve superlinear convergence at no more than twice the computational cost. Second, in order to realize this favorable comparison in the presence of stochastic noise, we analyze the effect of sampling error on VMC parameter updates and experimentally demonstrate that it can be reduced by the parallel tempering method. In particular, we demonstrate that RGN can be made robust to energy spikes that occur when the sampler moves between metastable regions of configuration space. Finally, putting theory into practice, we apply our enhanced optimization and sampling methods to the transverse-field Ising and XXZ models on large lattices, yielding ground-state energy estimates with remarkably high accuracy after just 200 parameter updates. △ Less

Submitted 19 July, 2022; v1 submitted 19 June, 2021; originally announced June 2021.

Comments: 12 pages, 8 figures

Rare Event Sampling Improves Mercury Instability Statistics

Authors: Dorian S. Abbot, Robert J. Webber, Sam Hadden, Darryl Seligman, Jonathan Weare

Abstract: Due to the chaotic nature of planetary dynamics, there is a non-zero probability that Mercury's orbit will become unstable in the future. Previous efforts have estimated the probability of this happening between 3 and 5 billion years in the future using a large number of direct numerical simulations with an N-body code, but were not able to obtain accurate estimates before 3 billion years in the f… ▽ More Due to the chaotic nature of planetary dynamics, there is a non-zero probability that Mercury's orbit will become unstable in the future. Previous efforts have estimated the probability of this happening between 3 and 5 billion years in the future using a large number of direct numerical simulations with an N-body code, but were not able to obtain accurate estimates before 3 billion years in the future because Mercury instability events are too rare. In this paper we use a new rare event sampling technique, Quantile Diffusion Monte Carlo (QDMC), to estimate that the probability of a Mercury instability event in the next 2 billion years is approximately $10^{-4}$ in the REBOUND N-body code. We show that QDMC provides unbiased probability estimates at a computational cost of up to 100 times less than direct numerical simulation. QDMC is easy to implement and could be applied to many problems in planetary dynamics in which it is necessary to estimate the probability of a rare event. △ Less

Submitted 27 December, 2021; v1 submitted 16 June, 2021; originally announced June 2021.

Comments: Abbot et al 2021 ApJ 923 236

Approximating matrix eigenvalues by subspace iteration with repeated random sparsification

Authors: Samuel M. Greene, Robert J. Webber, Timothy C. Berkelbach, Jonathan Weare

Abstract: Traditional numerical methods for calculating matrix eigenvalues are prohibitively expensive for high-dimensional problems. Iterative random sparsification methods allow for the estimation of a single dominant eigenvalue at reduced cost by leveraging repeated random sampling and averaging. We present a general approach to extending such methods for the estimation of multiple eigenvalues and demons… ▽ More Traditional numerical methods for calculating matrix eigenvalues are prohibitively expensive for high-dimensional problems. Iterative random sparsification methods allow for the estimation of a single dominant eigenvalue at reduced cost by leveraging repeated random sampling and averaging. We present a general approach to extending such methods for the estimation of multiple eigenvalues and demonstrate its performance for several benchmark problems in quantum chemistry. △ Less

Submitted 2 March, 2022; v1 submitted 22 March, 2021; originally announced March 2021.

Comments: 31 pages, 8 figures

Learning forecasts of rare stratospheric transitions from short simulations

Authors: Justin Finkel, Robert J. Webber, Dorian S. Abbot, Edwin P. Gerber, Jonathan Weare

Abstract: Rare events arising in nonlinear atmospheric dynamics remain hard to predict and attribute. We address the problem of forecasting rare events in a prototypical example, Sudden Stratospheric Warmings (SSWs). Approximately once every other winter, the boreal stratospheric polar vortex rapidly breaks down, shifting midlatitude surface weather patterns for months. We focus on two key quantities of int… ▽ More Rare events arising in nonlinear atmospheric dynamics remain hard to predict and attribute. We address the problem of forecasting rare events in a prototypical example, Sudden Stratospheric Warmings (SSWs). Approximately once every other winter, the boreal stratospheric polar vortex rapidly breaks down, shifting midlatitude surface weather patterns for months. We focus on two key quantities of interest: the probability of an SSW occurring, and the expected lead time if it does occur, as functions of initial condition. These \emph{optimal forecasts} concretely measure the event's progress. Direct numerical simulation can estimate them in principle, but is prohibitively expensive in practice: each rare event requires a long integration to observe, and the cost of each integration grows with model complexity. We describe an alternative approach using integrations that are \emph{short} compared to the timescale of the warming event. We compute the probability and lead time efficiently by solving equations involving the transition operator, which encodes all information about the dynamics. We relate these optimal forecasts to a small number of interpretable physical variables, suggesting optimal measurements for forecasting. We illustrate the methodology on a prototype SSW model developed by Holton and Mass (1976) and modified by stochastic forcing. While highly idealized, this model captures the essential nonlinear dynamics of SSWs and exhibits the key forecasting challenge: the dramatic separation in timescales between a single event and the return time between successive events. Our methodology is designed to fully exploit high-dimensional data from models and observations, and has the potential to identify detailed predictors of many complex rare events in meteorology. △ Less

Submitted 28 August, 2021; v1 submitted 15 February, 2021; originally announced February 2021.

Comments: 26 pages, 7 figures, major revision after original. Accepted to Monthly Weather Review, American Meteorological Society

A novel reconstruction technique for two-dimensional Bragg scatter imaging

Authors: James W. Webber, Eric L. Miller

Abstract: Here we introduce a new reconstruction technique for two-dimensional Bragg Scattering Tomography (BST), based on the Radon transform models of [arXiv preprint, arXiv:2004.10961 (2020)]. Our method uses a combination of ideas from multibang control and microlocal analysis to construct an objective function which can regularize the BST artifacts; specifically the boundary artifacts due to sharp cuto… ▽ More Here we introduce a new reconstruction technique for two-dimensional Bragg Scattering Tomography (BST), based on the Radon transform models of [arXiv preprint, arXiv:2004.10961 (2020)]. Our method uses a combination of ideas from multibang control and microlocal analysis to construct an objective function which can regularize the BST artifacts; specifically the boundary artifacts due to sharp cutoff in sinogram space (as observed in [arXiv preprint, arXiv:2007.00208 (2020)]), and artifacts arising from approximations made in constructing the model used for inversion. We then test our algorithm in a variety of Monte Carlo (MC) simulated examples of practical interest in airport baggage screening and threat detection. The data used in our studies is generated with a novel Monte-Carlo code presented here. The model, which is available from the authors upon request, captures both the Bragg scatter effects described by BST as well as beam attenuation and Compton scatter. △ Less

Submitted 24 January, 2021; originally announced January 2021.

Comments: 26 pages, 17 figures

A splitting method to reduce MCMC variance

Authors: Robert J. Webber, David Aristoff, Gideon Simpson

Abstract: We explore whether splitting and killing methods can improve the accuracy of Markov chain Monte Carlo (MCMC) estimates of rare event probabilities, and we make three contributions. First, we prove that "weighted ensemble" is the only splitting and killing method that provides asymptotically consistent estimates when combined with MCMC. Second, we prove a lower bound on the asymptotic variance of w… ▽ More We explore whether splitting and killing methods can improve the accuracy of Markov chain Monte Carlo (MCMC) estimates of rare event probabilities, and we make three contributions. First, we prove that "weighted ensemble" is the only splitting and killing method that provides asymptotically consistent estimates when combined with MCMC. Second, we prove a lower bound on the asymptotic variance of weighted ensemble's estimates. Third, we give a constructive proof and numerical examples to show that weighted ensemble can approach this optimal variance bound, in many cases reducing the variance of MCMC estimates by multiple orders of magnitude. △ Less

Submitted 16 December, 2020; v1 submitted 27 November, 2020; originally announced November 2020.

Comments: 30 pages, 9 figures

MSC Class: 65C05; 65C40; 82C80

Ensemble sampler for infinite-dimensional inverse problems

Authors: Jeremie Coullon, Robert J Webber

Abstract: We introduce a new Markov chain Monte Carlo (MCMC) sampler for infinite-dimensional inverse problems. Our new sampler is based on the affine invariant ensemble sampler, which uses interacting walkers to adapt to the covariance structure of the target distribution. We extend this ensemble sampler for the first time to infinite-dimensional function spaces, yielding a highly efficient gradient-free M… ▽ More We introduce a new Markov chain Monte Carlo (MCMC) sampler for infinite-dimensional inverse problems. Our new sampler is based on the affine invariant ensemble sampler, which uses interacting walkers to adapt to the covariance structure of the target distribution. We extend this ensemble sampler for the first time to infinite-dimensional function spaces, yielding a highly efficient gradient-free MCMC algorithm. Because our new ensemble sampler does not require gradients or posterior covariance estimates, it is simple to implement and broadly applicable. △ Less

Submitted 14 February, 2021; v1 submitted 28 October, 2020; originally announced October 2020.

Stokes drift through corals

Authors: Joseph J. Webber, Herbert E. Huppert

Abstract: We investigate the all-penetrating drift velocities, due to surface wave motion in an effectively inviscid fluid that overlies a saturated porous bed of finite depth. Previous work in this area either neglects the large-scale flow between layers [Phillips (1991)] or only considers the drift above the porous layer [(Monismith (2007)]. We propose a model where flow is described by a velocity potenti… ▽ More We investigate the all-penetrating drift velocities, due to surface wave motion in an effectively inviscid fluid that overlies a saturated porous bed of finite depth. Previous work in this area either neglects the large-scale flow between layers [Phillips (1991)] or only considers the drift above the porous layer [(Monismith (2007)]. We propose a model where flow is described by a velocity potential above the porous layer, and by Darcy's law in the porous bed, with derived matching conditions at the interface between the two layers. The damping effect of the porous bed requires a complex wavenumber k and both a vertical and horizontal Stokes drift of the fluid, unlike the solely horizontal drift first derived by Stokes Stokes (1847) in a pure fluid layer. Our work provides a physical model for coral reefs in shallow seas, where fluid drift both above and within the reef is vitally important for maintaining a healthy reef ecosystem [Koehl et al. (1997), Monismith (2007)]. We compare our model with measurements by Koehl \& Hadfield (2004) and also explain the vertical drift effects described in Koehl et al. (2007), who measured the exchange between a coral reef layer and the (relatively shallow) sea above. △ Less

Submitted 16 September, 2020; originally announced September 2020.

MSC Class: 86A05 (Primary); 76S05 (Secondary)

Journal ref: Environ. Fluid. Mech. (2021)

Integrated VAC: A robust strategy for identifying eigenfunctions of dynamical operators

Authors: Chatipat Lorpaiboon, Erik Henning Thiede, Robert J. Webber, Jonathan Weare, Aaron R. Dinner

Abstract: One approach to analyzing the dynamics of a physical system is to search for long-lived patterns in its motions. This approach has been particularly successful for molecular dynamics data, where slowly decorrelating patterns can indicate large-scale conformational changes. Detecting such patterns is the central objective of the variational approach to conformational dynamics (VAC), as well as the… ▽ More One approach to analyzing the dynamics of a physical system is to search for long-lived patterns in its motions. This approach has been particularly successful for molecular dynamics data, where slowly decorrelating patterns can indicate large-scale conformational changes. Detecting such patterns is the central objective of the variational approach to conformational dynamics (VAC), as well as the related methods of time-lagged independent component analysis and Markov state modeling. In VAC, the search for slowly decorrelating patterns is formalized as a variational problem solved by the eigenfunctions of the system's transition operator. VAC computes solutions to this variational problem by optimizing a linear or nonlinear model of the eigenfunctions using time series data. Here, we build on VAC's success by addressing two practical limitations. First, VAC can give poor eigenfunction estimates when the lag time parameter is chosen poorly. Second, VAC can overfit when using flexible parameterizations such as artificial neural networks with insufficient regularization. To address these issues, we propose an extension that we call integrated VAC (IVAC). IVAC integrates over multiple lag times before solving the variational problem, making its results more robust and reproducible than VAC's. △ Less

Submitted 9 September, 2020; v1 submitted 15 July, 2020; originally announced July 2020.

Comments: 16 pages, 7 figures

Microlocal analysis of generalized Radon transforms from scattering tomography

Authors: James W. Webber, Eric Todd Quinto

Abstract: Here we present a novel microlocal analysis of generalized Radon transforms which describe the integrals of $L^2$ functions of compact support over surfaces of revolution of $C^{\infty}$ curves $q$. We show that the Radon transforms are elliptic Fourier Integral Operators (FIO) and provide an analysis of the left projections $Π_L$. Our main theorem shows that $Π_L$ satisfies the semi-global Bolker… ▽ More Here we present a novel microlocal analysis of generalized Radon transforms which describe the integrals of $L^2$ functions of compact support over surfaces of revolution of $C^{\infty}$ curves $q$. We show that the Radon transforms are elliptic Fourier Integral Operators (FIO) and provide an analysis of the left projections $Π_L$. Our main theorem shows that $Π_L$ satisfies the semi-global Bolker assumption if and only if $g=q'/q$ is an immersion. An analysis of the visible singularities is presented, after which we derive novel Sobolev smoothness estimates for the Radon FIO. Our theory has specific applications of interest in Compton Scattering Tomography (CST) and Bragg Scattering Tomography (BST). We show that the CST and BST integration curves satisfy the Bolker assumption and provide simulated reconstructions from CST and BST data. Additionally we give example "sinusoidal" integration curves which do not satisfy Bolker and provide simulations of the image artefacts. The observed artefacts in reconstruction are shown to align exactly with our predictions. △ Less

Submitted 30 June, 2020; originally announced July 2020.

Comments: 24 pages, 9 figures

300 GHz generation based on a Kerr microresonator frequency comb stabilized to a low noise microwave reference

Authors: Tomohiro Tetsumoto, Fumiya Ayano, Mark Yeo, Julian Webber, Tadao Nagatsuma, Antoine Rolland

Abstract: In this letter, we experimentally demonstrate low noise 300GHz wave generation based on a Kerr microresonator frequency comb operating in soliton regime. The spectral purity of a 10GHz GPS-disciplined dielectric resonant oscillator is transferred to the 300GHz repetition rate frequency of the soliton comb through an optoelectronic phase-locked loop. Two adjacent comb lines beat on a uni-travelling… ▽ More In this letter, we experimentally demonstrate low noise 300GHz wave generation based on a Kerr microresonator frequency comb operating in soliton regime. The spectral purity of a 10GHz GPS-disciplined dielectric resonant oscillator is transferred to the 300GHz repetition rate frequency of the soliton comb through an optoelectronic phase-locked loop. Two adjacent comb lines beat on a uni-travelling carrier photodiode emitting the 300GHz millimeter-wave signal into a waveguide. In an out-of-loop measurement we have measured the 300GHz power spectral density of phase noise to be -88dBc/Hz, -105dBc/Hz at 10kHz, 1MHz Fourier frequency, respectively. The free-running fractional frequency instability at 300GHz is $1 \times 10^{-9}$ at 1 second averaging time. Stabilized to a GPS signal, we report an in-loop residual instability of $2 \times 10^{-15}$ at 1 second which averages down to < $1 \times 10^{-17}$ at 1000 seconds. Such system provides a promising path to the realization of compact, low power consumption millimeter-wave oscillators with low noise performance for out-of-the-lab applications. △ Less

Submitted 16 June, 2020; originally announced June 2020.

Comments: 4 pages, 4 figures

Journal ref: Optics Letters Vol. 45, Issue 16, pp. 4377-4380 (2020)

Error bounds for dynamical spectral estimation

Authors: Robert J. Webber, Erik H. Thiede, Douglas Dow, Aaron R. Dinner, Jonathan Weare

Abstract: Dynamical spectral estimation is a well-established numerical approach for estimating eigenvalues and eigenfunctions of the Markov transition operator from trajectory data. Although the approach has been widely applied in biomolecular simulations, its error properties remain poorly understood. Here we analyze the error of a dynamical spectral estimation method called "the variational approach to c… ▽ More Dynamical spectral estimation is a well-established numerical approach for estimating eigenvalues and eigenfunctions of the Markov transition operator from trajectory data. Although the approach has been widely applied in biomolecular simulations, its error properties remain poorly understood. Here we analyze the error of a dynamical spectral estimation method called "the variational approach to conformational dynamics" (VAC). We bound the approximation error and estimation error for VAC estimates. Our analysis establishes VAC's convergence properties and suggests new strategies for tuning VAC to improve accuracy. △ Less

Submitted 24 September, 2020; v1 submitted 5 May, 2020; originally announced May 2020.

Comments: 34 pages, 7 figures

MSC Class: 65C05; 60J35; 65N30

Improved Fast Randomized Iteration Approach to Full Configuration Interaction

Authors: Samuel M. Greene, Robert J. Webber, Jonathan Weare, Timothy C. Berkelbach

Abstract: We present three modifications to our recently introduced fast randomized iteration method for full configuration interaction (FCI-FRI) and investigate their effects on the method's performance for Ne, H$_2$O, and N$_2$. The initiator approximation, originally developed for full configuration interaction quantum Monte Carlo, significantly reduces statistical error in FCI-FRI when few samples are u… ▽ More We present three modifications to our recently introduced fast randomized iteration method for full configuration interaction (FCI-FRI) and investigate their effects on the method's performance for Ne, H$_2$O, and N$_2$. The initiator approximation, originally developed for full configuration interaction quantum Monte Carlo, significantly reduces statistical error in FCI-FRI when few samples are used in compression operations, enabling its application to larger chemical systems. The semi-stochastic extension, which involves exactly preserving a fixed subset of elements in each compression, improves statistical efficiency in some cases but reduces it in others. We also developed a new approach to sampling excitations that yields consistent improvements in statistical efficiency and reductions in computational cost. We discuss possible strategies based on our findings for improving the performance of stochastic quantum chemistry methods more generally. △ Less

Submitted 20 July, 2020; v1 submitted 1 May, 2020; originally announced May 2020.

Comments: 13 pages, 5 figures

Bragg scattering tomography

Authors: James W. Webber, Eric L. Miller

Abstract: Here we introduce a new forward model and imaging modality for Bragg Scattering Tomography (BST). The model we propose is based on an X-ray portal scanner with linear detector collimation, currently being developed for use in airport baggage screening. The geometry under consideration leads us to a novel two-dimensional inverse problem, where we aim to reconstruct the Bragg scattering differential… ▽ More Here we introduce a new forward model and imaging modality for Bragg Scattering Tomography (BST). The model we propose is based on an X-ray portal scanner with linear detector collimation, currently being developed for use in airport baggage screening. The geometry under consideration leads us to a novel two-dimensional inverse problem, where we aim to reconstruct the Bragg scattering differential cross section function from its integrals over a set of symmetric $C^2$ curves in the plane. The integral transform which describes the forward problem in BST is a new type of Radon transform, which we introduce and denote as the Bragg transform. We provide new injectivity results for the Bragg transform here, and describe how the conditions of our theorems can be applied to assist in the machine design of the portal scanner. Further we provide an extension of our results to $n$-dimensions, where a generalization of the Bragg transform is introduced. Here we aim to reconstruct a real valued function on $\mathbb{R}^{n+1}$ from its integrals over $n$-dimensional surfaces of revolution of $C^2$ curves embedded in $\mathbb{R}^{n+1}$. Injectivity proofs are provided also for the generalized Bragg transform. △ Less

Submitted 2 December, 2020; v1 submitted 23 April, 2020; originally announced April 2020.

Comments: 39 pages, 13 figures

A joint reconstruction and lambda tomography regularization technique for energy-resolved X-ray imaging

Authors: James Webber, Eric Todd Quinto, Eric L. Miller

Abstract: Here we present new joint reconstruction and regularization techniques inspired by ideas in microlocal analysis and lambda tomography, for the simultaneous reconstruction of the attenuation coefficient and electron density from X-ray transmission (i.e., X-ray CT) and backscattered data (assumed to be primarily Compton scattered). To demonstrate our theory and reconstruction methods, we consider th… ▽ More Here we present new joint reconstruction and regularization techniques inspired by ideas in microlocal analysis and lambda tomography, for the simultaneous reconstruction of the attenuation coefficient and electron density from X-ray transmission (i.e., X-ray CT) and backscattered data (assumed to be primarily Compton scattered). To demonstrate our theory and reconstruction methods, we consider the "parallel line segment" acquisition geometry of Webber and Miller ("Compton scattering tomography in translational geometries." Inverse Problems 36, no. 2 (2020): 025007), which is motivated by system architectures currently under development for airport security screening. We first present a novel microlocal analysis of the parallel line geometry which explains the nature of image artefacts when the attenuation coefficient and electron density are reconstructed separately. We next introduce a new joint reconstruction scheme for low effective $Z$ (atomic number) imaging ($Z<20$) characterized by a regularization strategy whose structure is derived from lambda tomography principles and motivated directly by the microlocal analytic results. Finally we show the effectiveness of our method in combating noise and image artefacts on simulated phantoms. △ Less

Submitted 6 April, 2020; v1 submitted 13 February, 2020; originally announced February 2020.

Comments: 32 pages, 15 figures

Compton scattering tomography in translational geometries

Authors: James Webber, Eric Miller

Abstract: Here we present new $L^2$ injectivity results for 2-D and 3-D Compton scattering tomography (CST) problems in translational geometries. The results are proven through the explicit inversion of a new toric section and apple Radon transform, which describe novel 2-D and 3-D acquisition geometries in CST. The geometry considered has potential applications in airport baggage screening and threat detec… ▽ More Here we present new $L^2$ injectivity results for 2-D and 3-D Compton scattering tomography (CST) problems in translational geometries. The results are proven through the explicit inversion of a new toric section and apple Radon transform, which describe novel 2-D and 3-D acquisition geometries in CST. The geometry considered has potential applications in airport baggage screening and threat detection. We also present a generalization of our injectivity results in 3-D to Radon transforms which describe the integrals of the charge density over the surfaces of revolution of a class of $C^1$ curves. △ Less

Submitted 30 June, 2019; originally announced July 2019.

Comments: 16 pages, 2 figures

Beyond Walkers in Stochastic Quantum Chemistry: Reducing Error using Fast Randomized Iteration

Authors: Samuel M. Greene, Robert J. Webber, Jonathan Weare, Timothy C. Berkelbach

Abstract: We introduce a family of methods for the full configuration interaction problem in quantum chemistry, based on the fast randomized iteration (FRI) framework [L.-H. Lim and J. Weare, SIAM Rev. 59, 547 (2017)]. These methods, which we term "FCI-FRI," stochastically impose sparsity during iterations of the power method and can be viewed as a generalization of full configuration interaction quantum Mo… ▽ More We introduce a family of methods for the full configuration interaction problem in quantum chemistry, based on the fast randomized iteration (FRI) framework [L.-H. Lim and J. Weare, SIAM Rev. 59, 547 (2017)]. These methods, which we term "FCI-FRI," stochastically impose sparsity during iterations of the power method and can be viewed as a generalization of full configuration interaction quantum Monte Carlo (FCIQMC) without walkers. In addition to the multinomial scheme commonly used to sample excitations in FCIQMC, we present a systematic scheme where excitations are not sampled independently. Performing ground-state calculations on five small molecules at fixed cost, we find that the systematic FCI-FRI scheme is 11 to 45 times more statistically efficient than the multinomial FCI-FRI scheme, which is in turn 1.4 to 178 times more statistically efficient than the original FCIQMC algorithm. △ Less

Submitted 9 July, 2019; v1 submitted 2 May, 2019; originally announced May 2019.

Comments: 19 pages, 7 figures

Maximizing simulated tropical cyclone intensity with action minimization

Authors: David A. Plotkin, Robert J. Webber, Morgan E O'Neill, Jonathan Weare, Dorian S. Abbot

Abstract: Direct computer simulation of intense tropical cyclones (TCs) in weather models is limited by computational expense. Intense TCs are rare and have small-scale structures, making it difficult to produce large ensembles of storms at high resolution. Further, models often fail to capture the process of rapid intensification, which is a distinguishing feature of many intense TCs. Understanding rapid i… ▽ More Direct computer simulation of intense tropical cyclones (TCs) in weather models is limited by computational expense. Intense TCs are rare and have small-scale structures, making it difficult to produce large ensembles of storms at high resolution. Further, models often fail to capture the process of rapid intensification, which is a distinguishing feature of many intense TCs. Understanding rapid intensification is especially important in the context of global warming, which may increase the frequency of intense TCs. To better leverage computational resources for the study of rapid intensification, we introduce an action minimization algorithm applied to the WRF and WRFPLUS models. Action minimization nudges the model into forming more intense TCs than it otherwise would; it does so via the maximum likelihood path in a stochastic formulation of the model, thereby allowing targeted study of intensification mechanisms. We apply action minimization to simulations of Hurricanes Danny (2015) and Fred (2009) at 6 km resolution to demonstrate that the algorithm consistently intensifies TCs via physically plausible pathways. We show an approximately ten-fold computational savings using action minimization to study the tail of the TC intensification distribution. Further, for Hurricanes Danny and Fred, action minimization produces perturbations that preferentially reduce low-level shear as compared to upper-level shear, at least above a threshold of approximately $4 \mathrm{\ m \ s^{-1}}$. We also demonstrate that asymmetric, time-dependent patterns of heating can cause significant TC intensification beyond symmetric, azimuthally-averaged heating and find a regime of non-linear response to asymmetric heating that has not been extensively studied in previous work. △ Less

Submitted 1 May, 2019; originally announced May 2019.

Modeling Corruption in Eventually-Consistent Graph Databases

Authors: Jim Webber, Paul Ezhilchelvan, Isi Mitrani

Abstract: We present a model and analysis of an eventually consistent graph database where loosely cooperating servers accept concurrent updates to a partitioned, distributed graph. The model is high-fidelity and preserves design choices from contemporary graph database management systems. To explore the problem space, we use two common graph topologies as data models for realistic experimentation. The anal… ▽ More We present a model and analysis of an eventually consistent graph database where loosely cooperating servers accept concurrent updates to a partitioned, distributed graph. The model is high-fidelity and preserves design choices from contemporary graph database management systems. To explore the problem space, we use two common graph topologies as data models for realistic experimentation. The analysis reveals, even assuming completely fault-free hardware and bug-free software, that if it is possible for updates to interfere with one-another, corruption will occur and spread significantly through the graph within the production database lifetime. Using our model, database designers and operators can compute the rate of corruption for their systems and determine whether they are sufficiently dependable for their intended use. △ Less

Submitted 9 April, 2019; originally announced April 2019.

Comments: 6 pages, 4 Figures

Practical rare event sampling for extreme mesoscale weather

Authors: Robert J. Webber, David A. Plotkin, Morgan E O'Neill, Dorian S. Abbot, Jonathan Weare

Abstract: Extreme mesoscale weather, including tropical cyclones, squall lines, and floods, can be enormously damaging and yet challenging to simulate; hence, there is a pressing need for more efficient simulation strategies. Here we present a new rare event sampling algorithm called Quantile Diffusion Monte Carlo (Quantile DMC). Quantile DMC is a simple-to-use algorithm that can sample extreme tail behavio… ▽ More Extreme mesoscale weather, including tropical cyclones, squall lines, and floods, can be enormously damaging and yet challenging to simulate; hence, there is a pressing need for more efficient simulation strategies. Here we present a new rare event sampling algorithm called Quantile Diffusion Monte Carlo (Quantile DMC). Quantile DMC is a simple-to-use algorithm that can sample extreme tail behavior for a wide class of processes. We demonstrate the advantages of Quantile DMC compared to other sampling methods and discuss practical aspects of implementing Quantile DMC. To test the feasibility of Quantile DMC for extreme mesoscale weather, we sample extremely intense realizations of two historical tropical cyclones, 2010 Hurricane Earl and 2015 Hurricane Joaquin. Our results demonstrate Quantile DMC's potential to provide low-variance extreme weather statistics while highlighting the work that is necessary for Quantile DMC to attain greater efficiency in future applications. △ Less

Submitted 6 April, 2019; originally announced April 2019.

Comments: 18 pages, 9 figures

Unifying Sequential Monte Carlo with Resampling Matrices

Authors: Robert J. Webber

Abstract: Sequential Monte Carlo (SMC) is a class of algorithms that approximate high-dimensional expectations of a Markov chain. SMC algorithms typically include a resampling step. There are many possible ways to resample, but the relative advantages of different resampling schemes remains poorly understood. Here, a theoretical framework for comparing resampling schemes is presented. The framework uses res… ▽ More Sequential Monte Carlo (SMC) is a class of algorithms that approximate high-dimensional expectations of a Markov chain. SMC algorithms typically include a resampling step. There are many possible ways to resample, but the relative advantages of different resampling schemes remains poorly understood. Here, a theoretical framework for comparing resampling schemes is presented. The framework uses resampling matrices to provide a simple description for the SMC resampling step. The framework identifies the matrix resampling scheme that gives the lowest possible error. The framework leads to new asymptotic error formulas that can be used to compare different resampling schemes. △ Less

Submitted 29 March, 2019; originally announced March 2019.

Comments: 36 pages, 3 figures

MSC Class: 65C05 (Primary); 60J05 (Secondary)

A Statistical Comparative Planetology Approach to Maximize the Scientific Return of Future Exoplanet Characterization Efforts

Authors: Jade H. Checlair, Dorian S. Abbot, Robert J. Webber, Y. Katherina Feng, Jacob L. Bean, Edward W. Schwieterman, Christopher C. Stark, Tyler D. Robinson, Eliza Kempton, Olivia D. N. Alcabes, Daniel Apai, Giada Arney, Nicolas Cowan, Shawn Domagal-Goldman, Chuanfei Dong, David P. Fleming, Yuka Fujii, R. J. Graham, Scott D. Guzewich, Yasuhiro Hasegawa, Benjamin P. C. Hayworth, Stephen R. Kane, Edwin S. Kite, Thaddeus D. Komacek, Ravi K. Kopparapu , et al. (15 additional authors not shown)

Abstract: Provided that sufficient resources are deployed, we can look forward to an extraordinary future in which we will characterize potentially habitable planets. Until now, we have had to base interpretations of observations on habitability hypotheses that have remained untested. To test these theories observationally, we propose a statistical comparative planetology approach to questions of planetary… ▽ More Provided that sufficient resources are deployed, we can look forward to an extraordinary future in which we will characterize potentially habitable planets. Until now, we have had to base interpretations of observations on habitability hypotheses that have remained untested. To test these theories observationally, we propose a statistical comparative planetology approach to questions of planetary habitability. The key objective of this approach will be to make quick and cheap measurements of critical planetary characteristics on a large sample of exoplanets, exploiting statistical marginalization to answer broad habitability questions. This relaxes the requirement of obtaining multiple types of data for a given planet, as it allows us to test a given hypothesis from only one type of measurement using the power of an ensemble. This approach contrasts with a "systems science" approach, where a few planets would be extensively studied with many types of measurements. A systems science approach is associated with a number of difficulties which may limit overall scientific return, including: the limited spectral coverage and noise of instruments, the diversity of exoplanets, and the extensive list of potential false negatives and false positives. A statistical approach could also be complementary to a systems science framework by providing context to interpret extensive measurements on planets of particular interest. We strongly recommend future missions with a focus on exoplanet characterization, and with the capability to study large numbers of planets in a homogenous way, rather than exclusively small, intense studies directed at a small sample of planets. △ Less

Submitted 12 March, 2019; originally announced March 2019.

Comments: White paper submitted in response to the solicitation of feedback for the "2020 Decadal Survey" by the National Academy of Sciences

Microlocal analysis of a Compton tomography problem

Authors: James Webber, Eric Todd Quinto

Abstract: Here we present a novel microlocal analysis of a new toric section transform which describes a two dimensional image reconstruction problem in Compton scattering tomography and airport baggage screening. By an analysis of two separate limited data problems for the circle transform and using microlocal analysis, we show that the canonical relation of the toric section transform is 2--1. This implie… ▽ More Here we present a novel microlocal analysis of a new toric section transform which describes a two dimensional image reconstruction problem in Compton scattering tomography and airport baggage screening. By an analysis of two separate limited data problems for the circle transform and using microlocal analysis, we show that the canonical relation of the toric section transform is 2--1. This implies that there are image artefacts in the filtered backprojection reconstruction. We provide explicit expressions for the expected artefacts and demonstrate these by simulations. In addition, we prove injectivity of the forward operator for $L^\infty$ functions supported inside the open unit ball. We present reconstructions from simulated data using a discrete approach and several regularizers with varying levels of added pseudo-random noise. △ Less

Submitted 23 January, 2020; v1 submitted 25 February, 2019; originally announced February 2019.

Comments: 31 pages, 18 figures

On non-parametric density estimation on linear and non-linear manifolds using generalized Radon transforms

Authors: James Webber, Erika Hussey, Eric Miller, Shuchin Aeron

Abstract: Here we present a new non-parametric approach to density estimation and classification derived from theory in Radon transforms and image reconstruction. We start by constructing a "forward problem" in which the unknown density is mapped to a set of one dimensional empirical distribution functions computed from the raw input data. Interpreting this mapping in terms of Radon-type projections provide… ▽ More Here we present a new non-parametric approach to density estimation and classification derived from theory in Radon transforms and image reconstruction. We start by constructing a "forward problem" in which the unknown density is mapped to a set of one dimensional empirical distribution functions computed from the raw input data. Interpreting this mapping in terms of Radon-type projections provides an analytical connection between the data and the density with many very useful properties including stable invertibility, fast computation, and significant theoretical grounding. Using results from the literature in geometric inverse problems we give uniqueness results and stability estimates for our methods. We subsequently extend the ideas to address problems in manifold learning and density estimation on manifolds. We introduce two new algorithms which can be readily applied to implement density estimation using Radon transforms in low dimensions or on low dimensional manifolds embedded in $\mathbb{R}^d$. We test our algorithms performance on a range of synthetic 2-D density estimation problems, designed with a mixture of sharp edges and smooth features. We show that our algorithm can offer a consistently competitive performance when compared to the state-of-the-art density estimation methods from the literature. △ Less

Submitted 30 March, 2019; v1 submitted 11 January, 2019; originally announced January 2019.

Comments: 30 pages, 17 figures

Microlocal analysis of a spindle transform

Authors: James Webber, Sean Holman

Abstract: An analysis of the stability of the spindle transform, introduced in ("Three dimensional Compton scattering tomography" arXiv:1704.03378 [math.FA]), is presented. We do this via a microlocal approach and show that the normal operator for the spindle transform is a type of paired Lagrangian operator with "blowdown--blowdown" singularities analogous to that of a limited data synthetic aperture radar… ▽ More An analysis of the stability of the spindle transform, introduced in ("Three dimensional Compton scattering tomography" arXiv:1704.03378 [math.FA]), is presented. We do this via a microlocal approach and show that the normal operator for the spindle transform is a type of paired Lagrangian operator with "blowdown--blowdown" singularities analogous to that of a limited data synthetic aperture radar (SAR) problem studied by Felea et. al. ("Microlocal analysis of SAR imaging of a dynamic reflectivity function" SIAM 2013). We find that the normal operator for the spindle transform belongs to a class of distibutions $I^{p,l}(Δ\cup\widetildeΔ,Λ)$ studied by Felea and Marhuenda ("Microlocal analysis of SAR imaging of a dynamic reflectivity function" SIAM 2013 and "Microlocal analysis of some isospectral deformations" Trans. Amer. Math.), where $\widetildeΔ$ is reflection through the origin, and $Λ$ is associated to a rotation artefact. Later, we derive a filter to reduce the strength of the image artefact and show that it is of convolution type. We also provide simulated reconstructions to show the artefacts produced by $Λ$ and show how the filter we derived can be applied to reduce the strength of the artefact. △ Less

Submitted 9 June, 2017; originally announced June 2017.

Comments: 25 pages, 13 figures

Three dimensional Compton scattering tomography

Authors: James Webber, William Lionheart

Abstract: We propose a new acquisition geometry for electron density reconstruction in three dimensional X-ray Compton imaging using a monochromatic source. This leads us to a new three dimensional inverse problem where we aim to reconstruct a real valued function $f$ (the electron density) from its integrals over spindle tori. We prove injectivity of a generalized spindle torus transform on the set of smoo… ▽ More We propose a new acquisition geometry for electron density reconstruction in three dimensional X-ray Compton imaging using a monochromatic source. This leads us to a new three dimensional inverse problem where we aim to reconstruct a real valued function $f$ (the electron density) from its integrals over spindle tori. We prove injectivity of a generalized spindle torus transform on the set of smooth functions compactly supported on a hollow ball. This is obtained through the explicit inversion of a class of Volterra integral operators, whose solutions give us an expression for the harmonic coefficients of $f$. The polychromatic source case is later considered, and we prove injectivity of a new spindle interior transform, apple transform and apple interior transform on the set of smooth functions compactly supported on a hollow ball. A possible physical model is suggested for both source types. We also provide simulated density reconstructions with varying levels of added pseudo random noise and model the systematic error due to the attenuation of the incoming and scattered rays in our simulation. △ Less

Submitted 11 April, 2017; originally announced April 2017.

Comments: 31 pages, 15 figures

X-ray Compton scattering tomography

Authors: James Webber

Abstract: We lay the foundations for a new fast method to reconstruct the electron density in x-ray scanning applications using measurements in the dark field. This approach is applied to a type of machine configuration with fixed energy sensitive (or resolving) detectors, and where the X-ray source is polychromatic. We consider the case where the measurements in the dark field are dominated by the Compton… ▽ More We lay the foundations for a new fast method to reconstruct the electron density in x-ray scanning applications using measurements in the dark field. This approach is applied to a type of machine configuration with fixed energy sensitive (or resolving) detectors, and where the X-ray source is polychromatic. We consider the case where the measurements in the dark field are dominated by the Compton scattering process. This leads us to a 2D inverse problem where we aim to reconstruct an electron density slice from its integrals over discs whose boundaries intersect the given source point. We show that a unique solution exists for smooth densities compactly supported on an annulus centred at the source point. Using Sobolev space estimates we determine a measure for the ill posedness of our problem based on the criterion given by Natterer ("The mathematics of computerized tomography" SIAM 2001). In addition, with a combination of our method and the more common attenuation coefficient reconstruction, we show under certain assumptions that the atomic number of the target is uniquely determined. We test our method on simulated data sets with varying levels of added pseudo random noise. △ Less

Submitted 6 May, 2015; originally announced May 2015.

Comments: 22 pages not including figures. 31 pages including figures with 29 figures in total