-
The Ensemble Epanechnikov Mixture Filter
Authors:
Andrey A. Popov,
Renato Zanetti
Abstract:
In the high-dimensional setting, Gaussian mixture kernel density estimates become increasingly suboptimal. In this work we aim to show that it is practical to instead use the optimal multivariate Epanechnikov kernel. We make use of this optimal Epanechnikov mixture kernel density estimate for the sequential filtering scenario through what we term the ensemble Epanechnikov mixture filter (EnEMF). W…
▽ More
In the high-dimensional setting, Gaussian mixture kernel density estimates become increasingly suboptimal. In this work we aim to show that it is practical to instead use the optimal multivariate Epanechnikov kernel. We make use of this optimal Epanechnikov mixture kernel density estimate for the sequential filtering scenario through what we term the ensemble Epanechnikov mixture filter (EnEMF). We provide a practical implementation of the EnEMF that is as cost efficient as the comparable ensemble Gaussian mixture filter. We show on a static example that the EnEMF is robust to growth in dimension, and also that the EnEMF has a significant reduction in error per particle on the 40-variable Lorenz '96 system.
△ Less
Submitted 20 August, 2024;
originally announced August 2024.
-
Map-Matching Queries under Fréchet Distance on Low-Density Spanners
Authors:
Kevin Buchin,
Maike Buchin,
Joachim Gudmundsson,
Aleksandr Popov,
Sampson Wong
Abstract:
Map matching is a common task when analysing GPS tracks, such as vehicle trajectories. The goal is to match a recorded noisy polygonal curve to a path on the map, usually represented as a geometric graph. The Fréchet distance is a commonly used metric for curves, making it a natural fit. The map-matching problem is well-studied, yet until recently no-one tackled the data structure question: prepro…
▽ More
Map matching is a common task when analysing GPS tracks, such as vehicle trajectories. The goal is to match a recorded noisy polygonal curve to a path on the map, usually represented as a geometric graph. The Fréchet distance is a commonly used metric for curves, making it a natural fit. The map-matching problem is well-studied, yet until recently no-one tackled the data structure question: preprocess a given graph so that one can query the minimum Fréchet distance between all graph paths and a polygonal curve. Recently, Gudmundsson, Seybold, and Wong [SODA 2023, arXiv:2211.02951] studied this problem for arbitrary query polygonal curves and $c$-packed graphs. In this paper, we instead require the graphs to be $λ$-low-density $t$-spanners, which is significantly more representative of real-world networks. We also show how to report a path that minimises the distance efficiently rather than only returning the minimal distance, which was stated as an open problem in their paper.
△ Less
Submitted 27 July, 2024;
originally announced July 2024.
-
An encryption algorithm using a generalization of the Markovski algorithm and a system of orthogonal operations based on T-quasigroups
Authors:
Nadezhda Malyutina,
Alexander Popov,
Victor Shcherbacov
Abstract:
Here is a more detailed description of the algorithm proposed in [1]. This algorithm simultaneously uses two cryptographic procedures: encryption using a generalization of the Markovski algorithm [2] and encryption using a system of orthogonal operations. In this paper, we present an implementation of this algorithm based on T-quasigroups, more precisely, based on medial quasigroups.
Here is a more detailed description of the algorithm proposed in [1]. This algorithm simultaneously uses two cryptographic procedures: encryption using a generalization of the Markovski algorithm [2] and encryption using a system of orthogonal operations. In this paper, we present an implementation of this algorithm based on T-quasigroups, more precisely, based on medial quasigroups.
△ Less
Submitted 20 July, 2024;
originally announced July 2024.
-
What are You Weighting For? Improved Weights for Gaussian Mixture Filtering With Application to Cislunar Orbit Determination
Authors:
Dalton Durant,
Andrey A. Popov,
Renato Zanetti
Abstract:
This work focuses on the critical aspect of accurate weight computation during the measurement incorporation phase of Gaussian mixture filters. The proposed novel approach computes weights by linearizing the measurement model about each component's posterior estimate rather than the the prior, as traditionally done. This work proves equivalence with traditional methods for linear models, provides…
▽ More
This work focuses on the critical aspect of accurate weight computation during the measurement incorporation phase of Gaussian mixture filters. The proposed novel approach computes weights by linearizing the measurement model about each component's posterior estimate rather than the the prior, as traditionally done. This work proves equivalence with traditional methods for linear models, provides novel sigma-point extensions to the traditional and proposed methods, and empirically demonstrates improved performance in nonlinear cases. Two illustrative examples, the Avocado and a cislunar single target tracking scenario, serve to highlight the advantages of the new weight computation technique by analyzing filter accuracy and consistency through varying the number of Gaussian mixture components.
△ Less
Submitted 17 May, 2024;
originally announced May 2024.
-
Preserving Nonlinear Constraints in Variational Flow Filtering Data Assimilation
Authors:
Amit N. Subrahmanya,
Andrey A. Popov,
Reid J. Gomillion,
Adrian Sandu
Abstract:
Data assimilation aims to estimate the states of a dynamical system by optimally combining sparse and noisy observations of the physical system with uncertain forecasts produced by a computational model. The states of many dynamical systems of interest obey nonlinear physical constraints, and the corresponding dynamics is confined to a certain sub-manifold of the state space. Standard data assimil…
▽ More
Data assimilation aims to estimate the states of a dynamical system by optimally combining sparse and noisy observations of the physical system with uncertain forecasts produced by a computational model. The states of many dynamical systems of interest obey nonlinear physical constraints, and the corresponding dynamics is confined to a certain sub-manifold of the state space. Standard data assimilation techniques applied to such systems yield posterior states lying outside the manifold, violating the physical constraints. This work focuses on particle flow filters which use stochastic differential equations to evolve state samples from a prior distribution to samples from an observation-informed posterior distribution. The variational Fokker-Planck (VFP) -- a generic particle flow filtering framework -- is extended to incorporate non-linear, equality state constraints in the analysis. To this end, two algorithmic approaches that modify the VFP stochastic differential equation are discussed: (i) VFPSTAB, to inexactly preserve constraints with the addition of a stabilizing drift term, and (ii) VFPDAE, to exactly preserve constraints by treating the VFP dynamics as a stochastic differential-algebraic equation (SDAE). Additionally, an implicit-explicit time integrator is developed to evolve the VFPDAE dynamics. The strength of the proposed approach for constraint preservation in data assimilation is demonstrated on three test problems: the double pendulum, Korteweg-de-Vries, and the incompressible Navier-Stokes equations.
△ Less
Submitted 7 May, 2024;
originally announced May 2024.
-
Warm-Start Variational Quantum Policy Iteration
Authors:
Nico Meyer,
Jakob Murauer,
Alexander Popov,
Christian Ufrecht,
Axel Plinge,
Christopher Mutschler,
Daniel D. Scherer
Abstract:
Reinforcement learning is a powerful framework aiming to determine optimal behavior in highly complex decision-making scenarios. This objective can be achieved using policy iteration, which requires to solve a typically large linear system of equations. We propose the variational quantum policy iteration (VarQPI) algorithm, realizing this step with a NISQ-compatible quantum-enhanced subroutine. It…
▽ More
Reinforcement learning is a powerful framework aiming to determine optimal behavior in highly complex decision-making scenarios. This objective can be achieved using policy iteration, which requires to solve a typically large linear system of equations. We propose the variational quantum policy iteration (VarQPI) algorithm, realizing this step with a NISQ-compatible quantum-enhanced subroutine. Its scalability is supported by an analysis of the structure of generic reinforcement learning environments, laying the foundation for potential quantum advantage with utility-scale quantum computers. Furthermore, we introduce the warm-start initialization variant (WS-VarQPI) that significantly reduces resource overhead. The algorithm solves a large FrozenLake environment with an underlying 256x256-dimensional linear system, indicating its practical robustness.
△ Less
Submitted 17 July, 2024; v1 submitted 16 April, 2024;
originally announced April 2024.
-
Improving the Adaptive Moment Estimation (ADAM) stochastic optimizer through an Implicit-Explicit (IMEX) time-stepping approach
Authors:
Abhinab Bhattacharjee,
Andrey A. Popov,
Arash Sarshar,
Adrian Sandu
Abstract:
The Adam optimizer, often used in Machine Learning for neural network training, corresponds to an underlying ordinary differential equation (ODE) in the limit of very small learning rates. This work shows that the classical Adam algorithm is a first order implicit-explicit (IMEX) Euler discretization of the underlying ODE. Employing the time discretization point of view, we propose new extensions…
▽ More
The Adam optimizer, often used in Machine Learning for neural network training, corresponds to an underlying ordinary differential equation (ODE) in the limit of very small learning rates. This work shows that the classical Adam algorithm is a first order implicit-explicit (IMEX) Euler discretization of the underlying ODE. Employing the time discretization point of view, we propose new extensions of the Adam scheme obtained by using higher order IMEX methods to solve the ODE. Based on this approach, we derive a new optimization algorithm for neural network training that performs better than classical Adam on several regression and classification problems.
△ Less
Submitted 20 March, 2024;
originally announced March 2024.
-
Precision Mars Entry Navigation with Atmospheric Density Adaptation via Neural Networks
Authors:
Felipe Giraldo-Grueso,
Andrey A. Popov,
Renato Zanetti
Abstract:
Spacecraft entering Mars require precise navigation algorithms capable of accurately estimating the vehicle's position and velocity in dynamic and uncertain atmospheric environments. Discrepancies between the true Martian atmospheric density and the onboard density model can significantly impair the performance of spacecraft entry navigation filters. This work introduces a new approach to online f…
▽ More
Spacecraft entering Mars require precise navigation algorithms capable of accurately estimating the vehicle's position and velocity in dynamic and uncertain atmospheric environments. Discrepancies between the true Martian atmospheric density and the onboard density model can significantly impair the performance of spacecraft entry navigation filters. This work introduces a new approach to online filtering for Martian entry using a neural network to estimate atmospheric density and employing a consider analysis to account for the uncertainty in the estimate. The network is trained on an exponential atmospheric density model, and its parameters are dynamically adapted in real time to account for any mismatch between the true and estimated densities. The adaptation of the network is formulated as a maximum likelihood problem by leveraging the measurement innovations of the filter to identify optimal network parameters. Within the context of the maximum likelihood approach, incorporating a neural network enables the use of stochastic optimizers known for their efficiency in the machine learning domain. Performance comparisons are conducted against two online adaptive approaches, covariance matching and state augmentation and correction, in various realistic Martian entry navigation scenarios. The results show superior estimation accuracy compared to other approaches, and precise alignment of the estimated density with a broad selection of realistic Martian atmospheres sampled from perturbed Mars-GRAM data.
△ Less
Submitted 20 May, 2024; v1 submitted 17 January, 2024;
originally announced January 2024.
-
Bayesian Recursive Update for Ensemble Kalman Filters
Authors:
Kristen Michaelson,
Andrey A. Popov,
Renato Zanetti
Abstract:
Few real-world systems are amenable to truly Bayesian filtering; nonlinearities and non-Gaussian noises can wreak havoc on filters that rely on linearization and Gaussian uncertainty approximations. This article presents the Bayesian Recursive Update Filter (BRUF), a Kalman filter that uses a recursive approach to incorporate information from nonlinear measurements. The BRUF relaxes the measuremen…
▽ More
Few real-world systems are amenable to truly Bayesian filtering; nonlinearities and non-Gaussian noises can wreak havoc on filters that rely on linearization and Gaussian uncertainty approximations. This article presents the Bayesian Recursive Update Filter (BRUF), a Kalman filter that uses a recursive approach to incorporate information from nonlinear measurements. The BRUF relaxes the measurement linearity assumption of the Extended Kalman Filter (EKF) by dividing the measurement update into a user-defined number of steps. The proposed technique is extended for ensemble filters in the Bayesian Recursive Update Ensemble Kalman Filter (BRUEnKF). The performance of both filters is demonstrated in numerical examples, and new filters are introduced which exploit the theoretical foundation of the BRUF in different ways. A comparison between the BRUEnKF and Gromov flow, a popular particle flow algorithm, is presented in detail. Finally, the BRUEnKF is shown to outperform the EnKF for a very high-dimensional system.
△ Less
Submitted 27 October, 2023;
originally announced October 2023.
-
Oriented Spanners
Authors:
Kevin Buchin,
Joachim Gudmundsson,
Antonia Kalb,
Aleksandr Popov,
Carolin Rehs,
André van Renssen,
Sampson Wong
Abstract:
Given a point set $P$ in the Euclidean plane and a parameter $t$, we define an \emph{oriented $t$-spanner} $G$ as an oriented subgraph of the complete bi-directed graph such that for every pair of points, the shortest closed walk in $G$ through those points is at most a factor $t$ longer than the shortest cycle in the complete graph on $P$. We investigate the problem of computing sparse graphs wit…
▽ More
Given a point set $P$ in the Euclidean plane and a parameter $t$, we define an \emph{oriented $t$-spanner} $G$ as an oriented subgraph of the complete bi-directed graph such that for every pair of points, the shortest closed walk in $G$ through those points is at most a factor $t$ longer than the shortest cycle in the complete graph on $P$. We investigate the problem of computing sparse graphs with small oriented dilation. As we can show that minimising oriented dilation for a given number of edges is NP-hard in the plane, we first consider one-dimensional point sets. While obtaining a $1$-spanner in this setting is straightforward, already for five points such a spanner has no plane embedding with the leftmost and rightmost point on the outer face. This leads to restricting to oriented graphs with a one-page book embedding on the one-dimensional point set. For this case we present a dynamic program to compute the graph of minimum oriented dilation that runs in $O(n^7)$ time for $n$ points, and a greedy algorithm that computes a $5$-spanner in $O(n\log n)$ time. Expanding these results finally gives us a result for two-dimensional point sets: we prove that for convex point sets the greedy triangulation results in a plane oriented $t$-spanner with $t=19 \cdot t_g$, where $t_g$ is a upper bound on the dilation of the greedy triangulation.
△ Less
Submitted 12 June, 2024; v1 submitted 29 June, 2023;
originally announced June 2023.
-
Small-data Reduced Order Modeling of Chaotic Dynamics through SyCo-AE: Synthetically Constrained Autoencoders
Authors:
Andrey A. Popov,
Renato Zanetti
Abstract:
Data-driven reduced order modeling of chaotic dynamics can result in systems that either dissipate or diverge catastrophically. Leveraging non-linear dimensionality reduction of autoencoders and the freedom of non-linear operator inference with neural-networks, we aim to solve this problem by imposing a synthetic constraint in the reduced order space. The synthetic constraint allows our reduced or…
▽ More
Data-driven reduced order modeling of chaotic dynamics can result in systems that either dissipate or diverge catastrophically. Leveraging non-linear dimensionality reduction of autoencoders and the freedom of non-linear operator inference with neural-networks, we aim to solve this problem by imposing a synthetic constraint in the reduced order space. The synthetic constraint allows our reduced order model both the freedom to remain fully non-linear and highly unstable while preventing divergence. We illustrate the methodology with the classical 40-variable Lorenz '96 equations, showing that our methodology is capable of producing medium-to-long range forecasts with lower error using less data.
△ Less
Submitted 13 May, 2023;
originally announced May 2023.
-
The Model Forest Ensemble Kalman Filter
Authors:
Andrey A Popov,
Adrian Sandu
Abstract:
Traditional data assimilation uses information obtained from the propagation of one physics-driven model and combines it with information derived from real-world observations in order to obtain a better estimate of the truth of some natural process. However, in many situations multiple simulation models that describe the same physical phenomenon are available. Such models can have different source…
▽ More
Traditional data assimilation uses information obtained from the propagation of one physics-driven model and combines it with information derived from real-world observations in order to obtain a better estimate of the truth of some natural process. However, in many situations multiple simulation models that describe the same physical phenomenon are available. Such models can have different sources. On one hand there are theory-guided models are constructed from first physical principles, while on the other there are data-driven models that are constructed from snapshots of high fidelity information. In this work we provide a possible way to make use of this collection of models in data assimilation by generalizing the idea of model hierarchies into model forests -- collections of high fidelity and low fidelity models organized in a groping of model trees such as to capture various relationships between different models. We generalize the multifidelity ensemble Kalman filter that previously operated on model hierarchies into the model forest ensemble Kalman filter through a generalized theory of linear control variates. This new filter allows for much more freedom when treading the line between accuracy and speed. Numerical experiments with a high fidelity quasi-geostrophic model and two of its low fidelity reduced order models validate the accuracy of our approach.
△ Less
Submitted 21 October, 2022;
originally announced October 2022.
-
NVRadarNet: Real-Time Radar Obstacle and Free Space Detection for Autonomous Driving
Authors:
Alexander Popov,
Patrik Gebhardt,
Ke Chen,
Ryan Oldja,
Heeseok Lee,
Shane Murray,
Ruchi Bhargava,
Nikolai Smolyanskiy
Abstract:
Detecting obstacles is crucial for safe and efficient autonomous driving. To this end, we present NVRadarNet, a deep neural network (DNN) that detects dynamic obstacles and drivable free space using automotive RADAR sensors. The network utilizes temporally accumulated data from multiple RADAR sensors to detect dynamic obstacles and compute their orientation in a top-down bird's-eye view (BEV). The…
▽ More
Detecting obstacles is crucial for safe and efficient autonomous driving. To this end, we present NVRadarNet, a deep neural network (DNN) that detects dynamic obstacles and drivable free space using automotive RADAR sensors. The network utilizes temporally accumulated data from multiple RADAR sensors to detect dynamic obstacles and compute their orientation in a top-down bird's-eye view (BEV). The network also regresses drivable free space to detect unclassified obstacles. Our DNN is the first of its kind to utilize sparse RADAR signals in order to perform obstacle and free space detection in real time from RADAR data only. The network has been successfully used for perception on our autonomous vehicles in real self-driving scenarios. The network runs faster than real time on an embedded GPU and shows good generalization across geographic regions.
△ Less
Submitted 1 March, 2023; v1 submitted 28 September, 2022;
originally announced September 2022.
-
A Sensor-Based Simulation Method for Spatiotemporal Event Detection
Authors:
Yuqin Jiang,
Andrey A. Popov,
Zhenlong Li,
Michael E. Hodgson,
Binghu Huang
Abstract:
Human movements in urban areas are essential to understand human-environment interactions. However, activities and associated movements are full of uncertainties due to the complexity of a city. In this paper, we propose a novel sensor-based approach for spatiotemporal event detection based on the Discrete Empirical Interpolation Method. Specifically, we first identify the key locations, defined a…
▽ More
Human movements in urban areas are essential to understand human-environment interactions. However, activities and associated movements are full of uncertainties due to the complexity of a city. In this paper, we propose a novel sensor-based approach for spatiotemporal event detection based on the Discrete Empirical Interpolation Method. Specifically, we first identify the key locations, defined as 'sensors' , which have the strongest correlation with the whole dataset. We then simulate a regular uneventful scenario with the observation data points from those key lo-cations. By comparing the simulated and observation scenarios, events are extracted both spatially and temporally. We apply this method in New York City with taxi trip record data. Results show that this method is effective in detecting when and where events occur.
△ Less
Submitted 23 April, 2024; v1 submitted 16 August, 2022;
originally announced August 2022.
-
A Meta-learning Formulation of the Autoencoder Problem for Non-linear Dimensionality Reduction
Authors:
Andrey A. Popov,
Arash Sarshar,
Austin Chennault,
Adrian Sandu
Abstract:
A rapidly growing area of research is the use of machine learning approaches such as autoencoders for dimensionality reduction of data and models in scientific applications. We show that the canonical formulation of autoencoders suffers from several deficiencies that can hinder their performance. Using a meta-learning approach, we reformulate the autoencoder problem as a bi-level optimization proc…
▽ More
A rapidly growing area of research is the use of machine learning approaches such as autoencoders for dimensionality reduction of data and models in scientific applications. We show that the canonical formulation of autoencoders suffers from several deficiencies that can hinder their performance. Using a meta-learning approach, we reformulate the autoencoder problem as a bi-level optimization procedure that explicitly solves the dimensionality reduction task. We prove that the new formulation corrects the identified deficiencies with canonical autoencoders, provide a practical way to solve it, and showcase the strength of this formulation with a simple numerical illustration.
△ Less
Submitted 27 July, 2022; v1 submitted 14 July, 2022;
originally announced July 2022.
-
Physics-informed neural networks for PDE-constrained optimization and control
Authors:
Jostein Barry-Straume,
Arash Sarshar,
Andrey A. Popov,
Adrian Sandu
Abstract:
A fundamental problem in science and engineering is designing optimal control policies that steer a given system towards a desired outcome. This work proposes Control Physics-Informed Neural Networks (Control PINNs) that simultaneously solve for a given system state, and for the optimal control signal, in a one-stage framework that conforms to the underlying physical laws. Prior approaches use a t…
▽ More
A fundamental problem in science and engineering is designing optimal control policies that steer a given system towards a desired outcome. This work proposes Control Physics-Informed Neural Networks (Control PINNs) that simultaneously solve for a given system state, and for the optimal control signal, in a one-stage framework that conforms to the underlying physical laws. Prior approaches use a two-stage framework that first models and then controls a system in sequential order. In contrast, a Control PINN incorporates the required optimality conditions in its architecture and in its loss function. The success of Control PINNs is demonstrated by solving the following open-loop optimal control problems: (i) an analytical problem, (ii) a one-dimensional heat equation, and (iii) a two-dimensional predator-prey problem.
△ Less
Submitted 18 August, 2022; v1 submitted 6 May, 2022;
originally announced May 2022.
-
Segment Visibility Counting Queries in Polygons
Authors:
Kevin Buchin,
Bram Custers,
Ivor van der Hoog,
Maarten Löffler,
Aleksandr Popov,
Marcel Roeloffzen,
Frank Staals
Abstract:
Let $P$ be a simple polygon with $n$ vertices, and let $A$ be a set of $m$ points or line segments inside $P$. We develop data structures that can efficiently count the number of objects from $A$ that are visible to a query point or a query segment. Our main aim is to obtain fast, $O(\mathop{\textrm{polylog}} nm$), query times, while using as little space as possible. In case the query is a single…
▽ More
Let $P$ be a simple polygon with $n$ vertices, and let $A$ be a set of $m$ points or line segments inside $P$. We develop data structures that can efficiently count the number of objects from $A$ that are visible to a query point or a query segment. Our main aim is to obtain fast, $O(\mathop{\textrm{polylog}} nm$), query times, while using as little space as possible. In case the query is a single point, a simple visibility-polygon-based solution achieves $O(\log nm)$ query time using $O(nm^2)$ space. In case $A$ also contains only points, we present a smaller, $O(n + m^{2 + \varepsilon}\log n)$-space, data structure based on a hierarchical decomposition of the polygon. Building on these results, we tackle the case where the query is a line segment and $A$ contains only points. The main complication here is that the segment may intersect multiple regions of the polygon decomposition, and that a point may see multiple such pieces. Despite these issues, we show how to achieve $O(\log n\log nm)$ query time using only $O(nm^{2 + \varepsilon} + n^2)$ space. Finally, we show that we can even handle the case where the objects in $A$ are segments with the same bounds.
△ Less
Submitted 10 January, 2022;
originally announced January 2022.
-
Ensemble Variational Fokker-Planck Methods for Data Assimilation
Authors:
Amit N Subrahmanya,
Andrey A Popov,
Adrian Sandu
Abstract:
Particle flow filters solve Bayesian inference problems by smoothly transforming a set of particles into samples from the posterior distribution. Particles move in state space under the flow of an McKean-Vlasov-Ito process. This work introduces the Variational Fokker-Planck (VFP) framework for data assimilation, a general approach that includes previously known particle flow filters as special cas…
▽ More
Particle flow filters solve Bayesian inference problems by smoothly transforming a set of particles into samples from the posterior distribution. Particles move in state space under the flow of an McKean-Vlasov-Ito process. This work introduces the Variational Fokker-Planck (VFP) framework for data assimilation, a general approach that includes previously known particle flow filters as special cases. The McKean-Vlasov-Ito process that transforms particles is defined via an optimal drift that depends on the selected diffusion term. It is established that the underlying probability density - sampled by the ensemble of particles - converges to the Bayesian posterior probability density. For a finite number of particles the optimal drift contains a regularization term that nudges particles toward becoming independent random variables. Based on this analysis, we derive computationally-feasible approximate regularization approaches that penalize the mutual information between pairs of particles, and avoid particle collapse. Moreover, the diffusion plays a role akin to a particle rejuvenation approach that aims to alleviate particle collapse. The VFP framework is very flexible. Different assumptions on prior and intermediate probability distributions can be used to implement the optimal drift, and localization and covariance shrinkage can be applied to alleviate the curse of dimensionality. A robust implicit-explicit method is discussed for the efficient integration of stiff McKean-Vlasov-Ito processes. The effectiveness of the VFP framework is demonstrated on three progressively more challenging test problems, namely the Lorenz '63, Lorenz '96 and the quasi-geostrophic equations.
△ Less
Submitted 19 January, 2024; v1 submitted 27 November, 2021;
originally announced November 2021.
-
Adjoint-Matching Neural Network Surrogates for Fast 4D-Var Data Assimilation
Authors:
Austin Chennault,
Andrey A. Popov,
Amit N. Subrahmanya,
Rachel Cooper,
Ali Haisam Muhammad Rafid,
Anuj Karpatne,
Adrian Sandu
Abstract:
Data assimilation is the process of fusing information from imperfect computer simulations with noisy, sparse measurements of reality to obtain improved estimates of the state or parameters of a dynamical system of interest. The data assimilation procedures used in many geoscience applications, such as numerical weather forecasting, are variants of the our-dimensional variational (4D-Var) algorith…
▽ More
Data assimilation is the process of fusing information from imperfect computer simulations with noisy, sparse measurements of reality to obtain improved estimates of the state or parameters of a dynamical system of interest. The data assimilation procedures used in many geoscience applications, such as numerical weather forecasting, are variants of the our-dimensional variational (4D-Var) algorithm. The cost of solving the underlying 4D-Var optimization problem is dominated by the cost of repeated forward and adjoint model runs. This motivates substituting the evaluations of the physical model and its adjoint by fast, approximate surrogate models. Neural networks offer a promising approach for the data-driven creation of surrogate models. The accuracy of the surrogate 4D-Var solution depends on the accuracy with each the surrogate captures both the forward and the adjoint model dynamics. We formulate and analyze several approaches to incorporate adjoint information into the construction of neural network surrogates. The resulting networks are tested on unseen data and in a sequential data assimilation problem using the Lorenz-63 system. Surrogates constructed using adjoint information demonstrate superior performance on the 4D-Var data assimilation problem compared to a standard neural network surrogate that uses only forward dynamics information.
△ Less
Submitted 20 December, 2022; v1 submitted 16 November, 2021;
originally announced November 2021.
-
Investigation of Nonlinear Model Order Reduction of the Quasigeostrophic Equations through a Physics-Informed Convolutional Autoencoder
Authors:
Rachel Cooper,
Andrey A. Popov,
Adrian Sandu
Abstract:
Reduced order modeling (ROM) is a field of techniques that approximates complex physics-based models of real-world processes by inexpensive surrogates that capture important dynamical characteristics with a smaller number of degrees of freedom. Traditional ROM techniques such as proper orthogonal decomposition (POD) focus on linear projections of the dynamics onto a set of spectral features. In th…
▽ More
Reduced order modeling (ROM) is a field of techniques that approximates complex physics-based models of real-world processes by inexpensive surrogates that capture important dynamical characteristics with a smaller number of degrees of freedom. Traditional ROM techniques such as proper orthogonal decomposition (POD) focus on linear projections of the dynamics onto a set of spectral features. In this paper we explore the construction of ROM using autoencoders (AE) that perform nonlinear projections of the system dynamics onto a low dimensional manifold learned from data. The approach uses convolutional neural networks (CNN) to learn spatial features as opposed to spectral, and utilize a physics informed (PI) cost function in order to capture temporal features as well. Our investigation using the quasi-geostrophic equations reveals that while the PI cost function helps with spatial reconstruction, spatial features are less powerful than spectral features, and that construction of ROMs through machine learning-based methods requires significant investigation into novel non-standard methodologies.
△ Less
Submitted 27 August, 2021;
originally announced August 2021.
-
Computing the Fréchet Distance Between Uncertain Curves in One Dimension
Authors:
Kevin Buchin,
Maarten Löffler,
Tim Ophelders,
Aleksandr Popov,
Jérôme Urhausen,
Kevin Verbeek
Abstract:
We consider the problem of computing the Fréchet distance between two curves for which the exact locations of the vertices are unknown. Each vertex may be placed in a given uncertainty region for that vertex, and the objective is to place vertices so as to minimise the Fréchet distance. This problem was recently shown to be NP-hard in 2D, and it is unclear how to compute an optimal vertex placemen…
▽ More
We consider the problem of computing the Fréchet distance between two curves for which the exact locations of the vertices are unknown. Each vertex may be placed in a given uncertainty region for that vertex, and the objective is to place vertices so as to minimise the Fréchet distance. This problem was recently shown to be NP-hard in 2D, and it is unclear how to compute an optimal vertex placement at all.
We present the first general algorithmic framework for this problem. We prove that it results in a polynomial-time algorithm for curves in 1D with intervals as uncertainty regions. In contrast, we show that the problem is NP-hard in 1D in the case that vertices are placed to maximise the Fréchet distance.
We also study the weak Fréchet distance between uncertain curves. While finding the optimal placement of vertices seems more difficult than the regular Fréchet distance -- and indeed we can easily prove that the problem is NP-hard in 2D -- the optimal placement of vertices in 1D can be computed in polynomial time. Finally, we investigate the discrete weak Fréchet distance, for which, somewhat surprisingly, the problem is NP-hard already in 1D.
△ Less
Submitted 20 May, 2021;
originally announced May 2021.
-
Uncertain Curve Simplification
Authors:
Kevin Buchin,
Maarten Löffler,
Aleksandr Popov,
Marcel Roeloffzen
Abstract:
We study the problem of polygonal curve simplification under uncertainty, where instead of a sequence of exact points, each uncertain point is represented by a region, which contains the (unknown) true location of the vertex. The regions we consider are disks, line segments, convex polygons, and discrete sets of points. We are interested in finding the shortest subsequence of uncertain points such…
▽ More
We study the problem of polygonal curve simplification under uncertainty, where instead of a sequence of exact points, each uncertain point is represented by a region, which contains the (unknown) true location of the vertex. The regions we consider are disks, line segments, convex polygons, and discrete sets of points. We are interested in finding the shortest subsequence of uncertain points such that no matter what the true location of each uncertain point is, the resulting polygonal curve is a valid simplification of the original polygonal curve under the Hausdorff or the Fréchet distance. For both these distance measures, we present polynomial-time algorithms for this problem.
△ Less
Submitted 16 March, 2021;
originally announced March 2021.
-
Multifidelity Ensemble Kalman Filtering Using Surrogate Models Defined by Physics-Informed Autoencoders
Authors:
Andrey A Popov,
Adrian Sandu
Abstract:
Data assimilation is a Bayesian inference process that obtains an enhanced understanding of a physical system of interest by fusing information from an inexact physics-based model, and from noisy sparse observations of reality. The multifidelity ensemble Kalman filter (MFEnKF) recently developed by the authors combines a full-order physical model and a hierarchy of reduced order surrogate models i…
▽ More
Data assimilation is a Bayesian inference process that obtains an enhanced understanding of a physical system of interest by fusing information from an inexact physics-based model, and from noisy sparse observations of reality. The multifidelity ensemble Kalman filter (MFEnKF) recently developed by the authors combines a full-order physical model and a hierarchy of reduced order surrogate models in order to increase the computational efficiency of data assimilation. The standard MFEnKF uses linear couplings between models, and is statistically optimal in case of Gaussian probability densities. This work extends MFEnKF to work with non-linear couplings between the models. Optimal nonlinear projection and interpolation operators are obtained by appropriately trained physics-informed autoencoders, and this approach allows to construct reduced order surrogate models with less error than conventional linear methods. Numerical experiments with the canonical Lorenz '96 model illustrate that nonlinear surrogates perform better than linear projection-based ones in the context of multifidelity filtering.
△ Less
Submitted 10 March, 2021; v1 submitted 25 February, 2021;
originally announced February 2021.
-
(k, l)-Medians Clustering of Trajectories Using Continuous Dynamic Time Warping
Authors:
Milutin Brankovic,
Kevin Buchin,
Koen Klaren,
André Nusser,
Aleksandr Popov,
Sampson Wong
Abstract:
Due to the massively increasing amount of available geospatial data and the need to present it in an understandable way, clustering this data is more important than ever. As clusters might contain a large number of objects, having a representative for each cluster significantly facilitates understanding a clustering. Clustering methods relying on such representatives are called center-based. In th…
▽ More
Due to the massively increasing amount of available geospatial data and the need to present it in an understandable way, clustering this data is more important than ever. As clusters might contain a large number of objects, having a representative for each cluster significantly facilitates understanding a clustering. Clustering methods relying on such representatives are called center-based. In this work we consider the problem of center-based clustering of trajectories.
In this setting, the representative of a cluster is again a trajectory. To obtain a compact representation of the clusters and to avoid overfitting, we restrict the complexity of the representative trajectories by a parameter l. This restriction, however, makes discrete distance measures like dynamic time warping (DTW) less suited.
There is recent work on center-based clustering of trajectories with a continuous distance measure, namely, the Fréchet distance. While the Fréchet distance allows for restriction of the center complexity, it can also be sensitive to outliers, whereas averaging-type distance measures, like DTW, are less so. To obtain a trajectory clustering algorithm that allows restricting center complexity and is more robust to outliers, we propose the usage of a continuous version of DTW as distance measure, which we call continuous dynamic time warping (CDTW). Our contribution is twofold:
1. To combat the lack of practical algorithms for CDTW, we develop an approximation algorithm that computes it.
2. We develop the first clustering algorithm under this distance measure and show a practical way to compute a center from a set of trajectories and subsequently iteratively improve it.
To obtain insights into the results of clustering under CDTW on practical data, we conduct extensive experiments.
△ Less
Submitted 1 December, 2020;
originally announced December 2020.
-
MVLidarNet: Real-Time Multi-Class Scene Understanding for Autonomous Driving Using Multiple Views
Authors:
Ke Chen,
Ryan Oldja,
Nikolai Smolyanskiy,
Stan Birchfield,
Alexander Popov,
David Wehr,
Ibrahim Eden,
Joachim Pehserl
Abstract:
Autonomous driving requires the inference of actionable information such as detecting and classifying objects, and determining the drivable space. To this end, we present Multi-View LidarNet (MVLidarNet), a two-stage deep neural network for multi-class object detection and drivable space segmentation using multiple views of a single LiDAR point cloud. The first stage processes the point cloud proj…
▽ More
Autonomous driving requires the inference of actionable information such as detecting and classifying objects, and determining the drivable space. To this end, we present Multi-View LidarNet (MVLidarNet), a two-stage deep neural network for multi-class object detection and drivable space segmentation using multiple views of a single LiDAR point cloud. The first stage processes the point cloud projected onto a perspective view in order to semantically segment the scene. The second stage then processes the point cloud (along with semantic labels from the first stage) projected onto a bird's eye view, to detect and classify objects. Both stages use an encoder-decoder architecture. We show that our multi-view, multi-stage, multi-class approach is able to detect and classify objects while simultaneously determining the drivable space using a single LiDAR scan as input, in challenging scenes with more than one hundred vehicles and pedestrians at a time. The system operates efficiently at 150 fps on an embedded GPU designed for a self-driving car, including a postprocessing step to maintain identities over time. We show results on both KITTI and a much larger internal dataset, thus demonstrating the method's ability to scale by an order of magnitude.
△ Less
Submitted 17 August, 2020; v1 submitted 9 June, 2020;
originally announced June 2020.
-
Fréchet Distance for Uncertain Curves
Authors:
Kevin Buchin,
Chenglin Fan,
Maarten Löffler,
Aleksandr Popov,
Benjamin Raichel,
Marcel Roeloffzen
Abstract:
In this paper we study a wide range of variants for computing the (discrete and continuous) Fréchet distance between uncertain curves. We define an uncertain curve as a sequence of uncertainty regions, where each region is a disk, a line segment, or a set of points. A realisation of a curve is a polyline connecting one point from each region. Given an uncertain curve and a second (certain or uncer…
▽ More
In this paper we study a wide range of variants for computing the (discrete and continuous) Fréchet distance between uncertain curves. We define an uncertain curve as a sequence of uncertainty regions, where each region is a disk, a line segment, or a set of points. A realisation of a curve is a polyline connecting one point from each region. Given an uncertain curve and a second (certain or uncertain) curve, we seek to compute the lower and upper bound Fréchet distance, which are the minimum and maximum Fréchet distance for any realisations of the curves.
We prove that both the upper and lower bound problems are NP-hard for the continuous Fréchet distance in several uncertainty models, and that the upper bound problem remains hard for the discrete Fréchet distance. In contrast, the lower bound (discrete and continuous) Fréchet distance can be computed in polynomial time. Furthermore, we show that computing the expected discrete Fréchet distance is #P-hard when the uncertainty regions are modelled as point sets or line segments. The construction also extends to show #P-hardness for computing the continuous Fréchet distance when regions are modelled as point sets.
On the positive side, we argue that in any constant dimension there is a FPTAS for the lower bound problem when $Δ/ δ$ is polynomially bounded, where $δ$ is the Fréchet distance and $Δ$ bounds the diameter of the regions. We then argue there is a near-linear-time 3-approximation for the decision problem when the regions are convex and roughly $δ$-separated. Finally, we also study the setting with Sakoe--Chiba time bands, where we restrict the alignment between the two curves, and give polynomial-time algorithms for upper bound and expected discrete and continuous Fréchet distance for uncertainty regions modelled as point sets.
△ Less
Submitted 24 April, 2020;
originally announced April 2020.
-
ODE Test Problems: a MATLAB suite of initial value problems
Authors:
Steven Roberts,
Andrey A. Popov,
Adrian Sandu
Abstract:
ODE Test Problems (OTP) is an object-oriented MATLAB package offering a broad range of initial value problems which can be used to test numerical methods such as time integration methods and data assimilation (DA) methods. It includes problems that are linear and nonlinear, homogeneous and nonhomogeneous, autonomous and nonautonomous, scalar and high-dimensional, stiff and nonstiff, and chaotic an…
▽ More
ODE Test Problems (OTP) is an object-oriented MATLAB package offering a broad range of initial value problems which can be used to test numerical methods such as time integration methods and data assimilation (DA) methods. It includes problems that are linear and nonlinear, homogeneous and nonhomogeneous, autonomous and nonautonomous, scalar and high-dimensional, stiff and nonstiff, and chaotic and nonchaotic. Many are real-world problems from fields such as chemistry, astrophysics, meteorology, and electrical engineering. OTP also supports partitioned ODEs for testing IMEX methods, multirate methods, and other multimethods. Functions for plotting solutions and creating movies are available for all problems, and exact solutions are provided when available. OTP is desgined for ease of use-meaning that working with and modifying problems is simple and intuitive.
△ Less
Submitted 13 January, 2019;
originally announced January 2019.
-
A Bayesian Approach to Multivariate Adaptive Localization in Ensemble-Based Data Assimilation with Time-Dependent Extensions
Authors:
Andrey A Popov,
Adrian Sandu
Abstract:
Ever since its inception, the Ensemble Kalman Filter has elicited many heuristic methods that sought to correct it. One such method is localization---the thought that `nearby' variables should be highly correlated with `far away' variable not. Recognizing that correlation is a time-dependent property, adaptive localization is a natural extension to these heuristics. We propose a Bayesian approach…
▽ More
Ever since its inception, the Ensemble Kalman Filter has elicited many heuristic methods that sought to correct it. One such method is localization---the thought that `nearby' variables should be highly correlated with `far away' variable not. Recognizing that correlation is a time-dependent property, adaptive localization is a natural extension to these heuristics. We propose a Bayesian approach to adaptive Schur-product localization for the DEnKF, and extend it to support multiple radii of influence. We test both the empirical validity of (multivariate) adaptive localization, and of our approach. We test a simple toy problem (Lorenz'96), extending it to a multivariate model, and a more realistic geophysical problem (1.5 Layer Quasi-Geostrophic). We show that the multivariate approach has great promise on the toy problem, and that the univariate approach leads to improved filter performance for the realistic geophysical problem.
△ Less
Submitted 24 September, 2018;
originally announced September 2018.
-
Interpretable probabilistic embeddings: bridging the gap between topic models and neural networks
Authors:
Anna Potapenko,
Artem Popov,
Konstantin Vorontsov
Abstract:
We consider probabilistic topic models and more recent word embedding techniques from a perspective of learning hidden semantic representations. Inspired by a striking similarity of the two approaches, we merge them and learn probabilistic embeddings with online EM-algorithm on word co-occurrence data. The resulting embeddings perform on par with Skip-Gram Negative Sampling (SGNS) on word similari…
▽ More
We consider probabilistic topic models and more recent word embedding techniques from a perspective of learning hidden semantic representations. Inspired by a striking similarity of the two approaches, we merge them and learn probabilistic embeddings with online EM-algorithm on word co-occurrence data. The resulting embeddings perform on par with Skip-Gram Negative Sampling (SGNS) on word similarity tasks and benefit in the interpretability of the components. Next, we learn probabilistic document embeddings that outperform paragraph2vec on a document similarity task and require less memory and time for training. Finally, we employ multimodal Additive Regularization of Topic Models (ARTM) to obtain a high sparsity and learn embeddings for other modalities, such as timestamps and categories. We observe further improvement of word similarity performance and meaningful inter-modality similarities.
△ Less
Submitted 11 November, 2017;
originally announced November 2017.
-
Autonomous 3D Reconstruction Using a MAV
Authors:
Alexander Popov,
Dimitrios Zermas,
Nikolaos Papanikolopoulos
Abstract:
An approach is proposed for high resolution 3D reconstruction of an object using a Micro Air Vehicle (MAV). A system is described which autonomously captures images and performs a dense 3D reconstruction via structure from motion with no prior knowledge of the environment. Only the MAVs own sensors, the front facing camera and the Inertial Measurement Unit (IMU) are utilized. Precision agriculture…
▽ More
An approach is proposed for high resolution 3D reconstruction of an object using a Micro Air Vehicle (MAV). A system is described which autonomously captures images and performs a dense 3D reconstruction via structure from motion with no prior knowledge of the environment. Only the MAVs own sensors, the front facing camera and the Inertial Measurement Unit (IMU) are utilized. Precision agriculture is considered as an example application for the system.
△ Less
Submitted 23 June, 2015;
originally announced June 2015.
-
Strategies for Development of a Distributed Framework for Computational Sciences
Authors:
Vladimir Berezovsky,
Alexander Popov
Abstract:
This paper discusses some generic approach for developing grid-based framework for enabling establishment of workflows comprising existing software in computational sciences areas. We highlight the main requirements addressed the developing of such framework. Some strategies for enabling interoperability between convenient computation software in the grid environment has been shown. The UML based…
▽ More
This paper discusses some generic approach for developing grid-based framework for enabling establishment of workflows comprising existing software in computational sciences areas. We highlight the main requirements addressed the developing of such framework. Some strategies for enabling interoperability between convenient computation software in the grid environment has been shown. The UML based instruments of graphical description of workflows for the developing system has been suggested.
△ Less
Submitted 18 June, 2011;
originally announced June 2011.
-
P vs NP Problem in the field anthropology
Authors:
Michael A. Popov
Abstract:
An attempt of a new kind of complexity anthropology is considered.
An attempt of a new kind of complexity anthropology is considered.
△ Less
Submitted 20 April, 2009;
originally announced April 2009.