-
Exact and Approximate Moment Derivation for Probabilistic Loops With Non-Polynomial Assignments
Authors:
Andrey Kofnov,
Marcel Moosbrugger,
Miroslav Stankovič,
Ezio Bartocci,
Efstathia Bura
Abstract:
Many stochastic continuous-state dynamical systems can be modeled as probabilistic programs with nonlinear non-polynomial updates in non-nested loops. We present two methods, one approximate and one exact, to automatically compute, without sampling, moment-based invariants for such probabilistic programs as closed-form solutions parameterized by the loop iteration. The exact method applies to prob…
▽ More
Many stochastic continuous-state dynamical systems can be modeled as probabilistic programs with nonlinear non-polynomial updates in non-nested loops. We present two methods, one approximate and one exact, to automatically compute, without sampling, moment-based invariants for such probabilistic programs as closed-form solutions parameterized by the loop iteration. The exact method applies to probabilistic programs with trigonometric and exponential updates and is embedded in the Polar tool. The approximate method for moment computation applies to any nonlinear random function as it exploits the theory of polynomial chaos expansion to approximate non-polynomial updates as the sum of orthogonal polynomials. This translates the dynamical system to a non-nested loop with polynomial updates, and thus renders it conformable with the Polar tool that computes the moments of any order of the state variables. We evaluate our methods on an extensive number of examples ranging from modeling monetary policy to several physical motion systems in uncertain environments. The experimental results demonstrate the advantages of our approach with respect to the current state-of-the-art.
△ Less
Submitted 25 January, 2024; v1 submitted 12 June, 2023;
originally announced June 2023.
-
Density Elicitation with applications in Probabilistic Loops
Authors:
Andrey Kofnov,
Ezio Bartocci,
Efstathia Bura
Abstract:
Probabilistic loops can be employed to implement and to model different processes ranging from software to cyber-physical systems. One main challenge is how to automatically estimate the distribution of the underlying continuous random variables symbolically and without sampling. We develop an approach, which we call K-series estimation, to approximate statically the joint and marginal distributio…
▽ More
Probabilistic loops can be employed to implement and to model different processes ranging from software to cyber-physical systems. One main challenge is how to automatically estimate the distribution of the underlying continuous random variables symbolically and without sampling. We develop an approach, which we call K-series estimation, to approximate statically the joint and marginal distributions of a vector of random variables updated in a probabilistic non-nested loop with polynomial and non-polynomial assignments. Our approach is a general estimation method for an unknown probability density function with bounded support. It naturally complements algorithms for automatic derivation of moments in probabilistic loops such as~\cite{BartocciKS19,Moosbruggeretal2022}. Its only requirement is a finite number of moments of the unknown density. We show that Gram-Charlier (GC) series, a widely used estimation method, is a special case of K-series when the normal probability density function is used as reference distribution. We provide also a formulation suitable for estimating both univariate and multivariate distributions. We demonstrate the feasibility of our approach using multiple examples from the literature.
△ Less
Submitted 17 April, 2023;
originally announced April 2023.
-
Distribution Estimation for Probabilistic Loops
Authors:
Ahmad Karimi,
Marcel Moosbrugger,
Miroslav Stankovič,
Laura Kovács,
Ezio Bartocci,
Efstathia Bura
Abstract:
We present an algorithmic approach to estimate the value distributions of random variables of probabilistic loops whose statistical moments are (partially) known. Based on these moments, we apply two statistical methods, Maximum Entropy and Gram-Charlier series, to estimate the distributions of the loop's random variables. We measure the accuracy of our distribution estimation by comparing the res…
▽ More
We present an algorithmic approach to estimate the value distributions of random variables of probabilistic loops whose statistical moments are (partially) known. Based on these moments, we apply two statistical methods, Maximum Entropy and Gram-Charlier series, to estimate the distributions of the loop's random variables. We measure the accuracy of our distribution estimation by comparing the resulting distributions using exact and estimated moments of the probabilistic loop, and performing statistical tests. We evaluate our method on several probabilistic loops with polynomial updates over random variables drawing from common probability distributions, including examples implementing financial and biological models. For this, we leverage symbolic approaches to compute exact higher-order moments of loops as well as use sampling-based techniques to estimate moments from loop executions. Our experimental results provide practical evidence of the accuracy of our method for estimating distributions of probabilistic loop outputs.
△ Less
Submitted 13 May, 2022;
originally announced May 2022.
-
Moment-based Invariants for Probabilistic Loops with Non-polynomial Assignments
Authors:
Andrey Kofnov,
Marcel Moosbrugger,
Miroslav Stankovič,
Ezio Bartocci,
Efstathia Bura
Abstract:
We present a method to automatically approximate moment-based invariants of probabilistic programs with non-polynomial updates of continuous state variables to accommodate more complex dynamics. Our approach leverages polynomial chaos expansion to approximate non-linear functional updates as sums of orthogonal polynomials. We exploit this result to automatically estimate state-variable moments of…
▽ More
We present a method to automatically approximate moment-based invariants of probabilistic programs with non-polynomial updates of continuous state variables to accommodate more complex dynamics. Our approach leverages polynomial chaos expansion to approximate non-linear functional updates as sums of orthogonal polynomials. We exploit this result to automatically estimate state-variable moments of all orders in Prob-solvable loops with non-polynomial updates. We showcase the accuracy of our estimation approach in several examples, such as the turning vehicle model and the Taylor rule in monetary policy.
△ Less
Submitted 1 July, 2022; v1 submitted 5 May, 2022;
originally announced May 2022.
-
Fusing Sufficient Dimension Reduction with Neural Networks
Authors:
Daniel Kapla,
Lukas Fertl,
Efstathia Bura
Abstract:
We consider the regression problem where the dependence of the response Y on a set of predictors X is fully captured by the regression function E(Y | X)=g(B'X), for an unknown function g and low rank parameter B matrix. We combine neural networks with sufficient dimension reduction in order to remove the limitation of small p and n of the latter. We show in simulations that the proposed estimator…
▽ More
We consider the regression problem where the dependence of the response Y on a set of predictors X is fully captured by the regression function E(Y | X)=g(B'X), for an unknown function g and low rank parameter B matrix. We combine neural networks with sufficient dimension reduction in order to remove the limitation of small p and n of the latter. We show in simulations that the proposed estimator is on par with competing sufficient dimension reduction methods in small p and n settings, such as minimum average variance estimation and conditional variance estimation. Among those, it is the only computationally applicable in large p and n problems.
△ Less
Submitted 20 April, 2021;
originally announced April 2021.
-
Ensemble Conditional Variance Estimator for Sufficient Dimension Reduction
Authors:
Lukas Fertl,
Efstathia Bura
Abstract:
Ensemble Conditional Variance Estimation (ECVE) is a novel sufficient dimension reduction (SDR) method in regressions with continuous response and predictors. ECVE applies to general non-additive error regression models. It operates under the assumption that the predictors can be replaced by a lower dimensional projection without loss of information. It is a semiparametric forward regression model…
▽ More
Ensemble Conditional Variance Estimation (ECVE) is a novel sufficient dimension reduction (SDR) method in regressions with continuous response and predictors. ECVE applies to general non-additive error regression models. It operates under the assumption that the predictors can be replaced by a lower dimensional projection without loss of information. It is a semiparametric forward regression model based exhaustive sufficient dimension reduction estimation method that is shown to be consistent under mild assumptions. It is shown to outperform central subspace mean average variance estimation (csMAVE), its main competitor, under several simulation settings and in a benchmark data set analysis.
△ Less
Submitted 26 February, 2021;
originally announced February 2021.
-
Conditional Variance Estimator for Sufficient Dimension Reduction
Authors:
Lukas Fertl,
Efstathia Bura
Abstract:
Conditional Variance Estimation (CVE) is a novel sufficient dimension reduction (SDR) method for additive error regressions with continuous predictors and link function. It operates under the assumption that the predictors can be replaced by a lower dimensional projection without loss of information. In contrast to the majority of moment based sufficient dimension reduction methods, Conditional Va…
▽ More
Conditional Variance Estimation (CVE) is a novel sufficient dimension reduction (SDR) method for additive error regressions with continuous predictors and link function. It operates under the assumption that the predictors can be replaced by a lower dimensional projection without loss of information. In contrast to the majority of moment based sufficient dimension reduction methods, Conditional Variance Estimation is fully data driven, does not require the restrictive linearity and constant variance conditions, and is not based on inverse regression. CVE is shown to be consistent and its objective function to be uniformly convergent. CVE outperforms the mean average variance estimation, (MAVE), its main competitor, in several simulation settings, remains on par under others, while it always outperforms the usual inverse regression based linear SDR methods, such as Sliced Inverse Regression.
△ Less
Submitted 17 February, 2021;
originally announced February 2021.
