-
Conformal Approach To Gaussian Process Surrogate Evaluation With Coverage Guarantees
Authors:
Edgar Jaber,
Vincent Blot,
Nicolas Brunel,
Vincent Chabridon,
Emmanuel Remy,
Bertrand Iooss,
Didier Lucor,
Mathilde Mougeot,
Alessandro Leite
Abstract:
Gaussian processes (GPs) are a Bayesian machine learning approach widely used to construct surrogate models for the uncertainty quantification of computer simulation codes in industrial applications. It provides both a mean predictor and an estimate of the posterior prediction variance, the latter being used to produce Bayesian credibility intervals. Interpreting these intervals relies on the Gaus…
▽ More
Gaussian processes (GPs) are a Bayesian machine learning approach widely used to construct surrogate models for the uncertainty quantification of computer simulation codes in industrial applications. It provides both a mean predictor and an estimate of the posterior prediction variance, the latter being used to produce Bayesian credibility intervals. Interpreting these intervals relies on the Gaussianity of the simulation model as well as the well-specification of the priors which are not always appropriate. We propose to address this issue with the help of conformal prediction. In the present work, a method for building adaptive cross-conformal prediction intervals is proposed by weighting the non-conformity score with the posterior standard deviation of the GP. The resulting conformal prediction intervals exhibit a level of adaptivity akin to Bayesian credibility sets and display a significant correlation with the surrogate model local approximation error, while being free from the underlying model assumptions and having frequentist coverage guarantees. These estimators can thus be used for evaluating the quality of a GP surrogate model and can assist a decision-maker in the choice of the best prior for the specific application of the GP. The performance of the method is illustrated through a panel of numerical examples based on various reference databases. Moreover, the potential applicability of the method is demonstrated in the context of surrogate modeling of an expensive-to-evaluate simulator of the clogging phenomenon in steam generators of nuclear reactors.
△ Less
Submitted 15 January, 2024;
originally announced January 2024.
-
Machine learning with data assimilation and uncertainty quantification for dynamical systems: a review
Authors:
Sibo Cheng,
Cesar Quilodran-Casas,
Said Ouala,
Alban Farchi,
Che Liu,
Pierre Tandeo,
Ronan Fablet,
Didier Lucor,
Bertrand Iooss,
Julien Brajard,
Dunhui Xiao,
Tijana Janjic,
Weiping Ding,
Yike Guo,
Alberto Carrassi,
Marc Bocquet,
Rossella Arcucci
Abstract:
Data Assimilation (DA) and Uncertainty quantification (UQ) are extensively used in analysing and reducing error propagation in high-dimensional spatial-temporal dynamics. Typical applications span from computational fluid dynamics (CFD) to geoscience and climate systems. Recently, much effort has been given in combining DA, UQ and machine learning (ML) techniques. These research efforts seek to ad…
▽ More
Data Assimilation (DA) and Uncertainty quantification (UQ) are extensively used in analysing and reducing error propagation in high-dimensional spatial-temporal dynamics. Typical applications span from computational fluid dynamics (CFD) to geoscience and climate systems. Recently, much effort has been given in combining DA, UQ and machine learning (ML) techniques. These research efforts seek to address some critical challenges in high-dimensional dynamical systems, including but not limited to dynamical system identification, reduced order surrogate modelling, error covariance specification and model error correction. A large number of developed techniques and methodologies exhibit a broad applicability across numerous domains, resulting in the necessity for a comprehensive guide. This paper provides the first overview of the state-of-the-art researches in this interdisciplinary field, covering a wide range of applications. This review aims at ML scientists who attempt to apply DA and UQ techniques to improve the accuracy and the interpretability of their models, but also at DA and UQ experts who intend to integrate cutting-edge ML approaches to their systems. Therefore, this article has a special focus on how ML methods can overcome the existing limits of DA and UQ, and vice versa. Some exciting perspectives of this rapidly developing research field are also discussed.
△ Less
Submitted 18 March, 2023;
originally announced March 2023.
-
The ICSCREAM methodology: Identification of penalizing configurations in computer experiments using screening and metamodel -- Applications in thermal-hydraulics
Authors:
A. Marrel,
Bertrand Iooss,
V Chabridon
Abstract:
In the framework of risk assessment in nuclear accident analysis, best-estimatecomputer codes, associated to a probabilistic modeling of the uncertain input variables,are used to estimate safety margins. A first step in such uncertainty quantificationstudies is often to identify the critical configurations (or penalizing, in thesense of a prescribed safety margin) of several input parameters (call…
▽ More
In the framework of risk assessment in nuclear accident analysis, best-estimatecomputer codes, associated to a probabilistic modeling of the uncertain input variables,are used to estimate safety margins. A first step in such uncertainty quantificationstudies is often to identify the critical configurations (or penalizing, in thesense of a prescribed safety margin) of several input parameters (called ``scenarioinputs''), under the uncertainty on the other input parameters. However, the largeCPU-time cost of most of the computer codes used in nuclear engineering, as theones related to thermal-hydraulic accident scenario simulations, involve to develophighly efficient strategies. This work focuses on machine learning algorithms bythe way of the metamodel-based approach (i.e., a mathematical model which is fittedon a small-size sample of simulations). To achieve it with a very large numberof inputs, a specific and original methodology, called ICSCREAM (Identificationof penalizing Configurations using SCREening And Metamodel), is proposed. Thescreening of influential inputs is based on an advanced global sensitivity analysistool (HSIC importance measures). A Gaussian process metamodel is then sequentiallybuilt and used to estimate, within a Bayesian framework, the conditionalprobabilities of exceeding a high-level threshold, according to the scenario inputs.The efficiency of this methodology is illustrated on two high-dimensional (arounda hundred inputs) thermal-hydraulic industrial cases simulating an accident of primarycoolant loss in a pressurized water reactor. For both use cases, the studyfocuses on the peak cladding temperature (PCT) and critical configurations aredefined by exceeding the 90%-quantile of PCT. In both cases, the ICSCREAMmethodology allows to estimate, by using only around one thousand of code simulations,the impact of the scenario inputs and their critical areas of values.
△ Less
Submitted 27 August, 2021; v1 submitted 8 April, 2020;
originally announced April 2020.
