Showing 1–7 of 7 results for author: Thodoroff, P

Multi-fidelity experimental design for ice-sheet simulation

Authors: Pierre Thodoroff, Markus Kaiser, Rosie Williams, Robert Arthern, Scott Hosking, Neil Lawrence, James Byrne, Ieva Kazlauskaite

Abstract: Computer simulations are becoming an essential tool in many scientific fields from molecular dynamics to aeronautics. In glaciology, future predictions of sea level change require input from ice sheet models. Due to uncertainties in the forcings and the parameter choices for such models, many different realisations of the model are needed in order to produce probabilistic forecasts of sea level ch… ▽ More Computer simulations are becoming an essential tool in many scientific fields from molecular dynamics to aeronautics. In glaciology, future predictions of sea level change require input from ice sheet models. Due to uncertainties in the forcings and the parameter choices for such models, many different realisations of the model are needed in order to produce probabilistic forecasts of sea level change. For these reasons, producing robust probabilistic forecasts from an ensemble of model simulations over regions of interest can be extremely expensive for many ice sheet models. Multi-fidelity experimental design (MFED) is a strategy that models the high-fidelity output of the simulator by combining information from various resolutions in an attempt to minimize the computational costs of the process and maximize the accuracy of the posterior. In this paper, we present an application of MFED to an ice-sheet simulatorand demonstrate potential computational savings by modelling the relationship between spatial resolutions. We also analyze the behavior of MFED strategies using theoretical results from sub-modular maximization. △ Less

Submitted 17 July, 2023; originally announced July 2023.

Real-world Machine Learning Systems: A survey from a Data-Oriented Architecture Perspective

Authors: Christian Cabrera, Andrei Paleyes, Pierre Thodoroff, Neil D. Lawrence

Abstract: Machine Learning models are being deployed as parts of real-world systems with the upsurge of interest in artificial intelligence. The design, implementation, and maintenance of such systems are challenged by real-world environments that produce larger amounts of heterogeneous data and users requiring increasingly faster responses with efficient resource consumption. These requirements push preval… ▽ More Machine Learning models are being deployed as parts of real-world systems with the upsurge of interest in artificial intelligence. The design, implementation, and maintenance of such systems are challenged by real-world environments that produce larger amounts of heterogeneous data and users requiring increasingly faster responses with efficient resource consumption. These requirements push prevalent software architectures to the limit when deploying ML-based systems. Data-oriented Architecture (DOA) is an emerging concept that equips systems better for integrating ML models. DOA extends current architectures to create data-driven, loosely coupled, decentralised, open systems. Even though papers on deployed ML-based systems do not mention DOA, their authors made design decisions that implicitly follow DOA. The reasons why, how, and the extent to which DOA is adopted in these systems are unclear. Implicit design decisions limit the practitioners' knowledge of DOA to design ML-based systems in the real world. This paper answers these questions by surveying real-world deployments of ML-based systems. The survey shows the design decisions of the systems and the requirements these satisfy. Based on the survey findings, we also formulate practical advice to facilitate the deployment of ML-based systems. Finally, we outline open challenges to deploying DOA-based systems that integrate ML models. △ Less

Submitted 9 October, 2023; v1 submitted 9 February, 2023; originally announced February 2023.

Comments: Under review

Solving Schrödinger Bridges via Maximum Likelihood

Authors: Francisco Vargas, Pierre Thodoroff, Neil D. Lawrence, Austen Lamacraft

Abstract: The Schrödinger bridge problem (SBP) finds the most likely stochastic evolution between two probability distributions given a prior stochastic evolution. As well as applications in the natural sciences, problems of this kind have important applications in machine learning such as dataset alignment and hypothesis testing. Whilst the theory behind this problem is relatively mature, scalable numerica… ▽ More The Schrödinger bridge problem (SBP) finds the most likely stochastic evolution between two probability distributions given a prior stochastic evolution. As well as applications in the natural sciences, problems of this kind have important applications in machine learning such as dataset alignment and hypothesis testing. Whilst the theory behind this problem is relatively mature, scalable numerical recipes to estimate the Schrödinger bridge remain an active area of research. We prove an equivalence between the SBP and maximum likelihood estimation enabling direct application of successful machine learning techniques. We propose a numerical procedure to estimate SBPs using Gaussian process and demonstrate the practical usage of our approach in numerical simulations and experiments. △ Less

Submitted 28 May, 2022; v1 submitted 3 June, 2021; originally announced June 2021.

Comments: 9 pages + appendix (total 28 pages)

Journal ref: Entropy. 2021; 23(9):1134

Recurrent Value Functions

Authors: Pierre Thodoroff, Nishanth Anand, Lucas Caccia, Doina Precup, Joelle Pineau

Abstract: Despite recent successes in Reinforcement Learning, value-based methods often suffer from high variance hindering performance. In this paper, we illustrate this in a continuous control setting where state of the art methods perform poorly whenever sensor noise is introduced. To overcome this issue, we introduce Recurrent Value Functions (RVFs) as an alternative to estimate the value function of a… ▽ More Despite recent successes in Reinforcement Learning, value-based methods often suffer from high variance hindering performance. In this paper, we illustrate this in a continuous control setting where state of the art methods perform poorly whenever sensor noise is introduced. To overcome this issue, we introduce Recurrent Value Functions (RVFs) as an alternative to estimate the value function of a state. We propose to estimate the value function of the current state using the value function of past states visited along the trajectory. Due to the nature of their formulation, RVFs have a natural way of learning an emphasis function that selectively emphasizes important states. First, we establish RVF's asymptotic convergence properties in tabular settings. We then demonstrate their robustness on a partially observable domain and continuous control tasks. Finally, we provide a qualitative interpretation of the learned emphasis function. △ Less

Submitted 23 May, 2019; originally announced May 2019.

Temporal Regularization in Markov Decision Process

Authors: Pierre Thodoroff, Audrey Durand, Joelle Pineau, Doina Precup

Abstract: Several applications of Reinforcement Learning suffer from instability due to high variance. This is especially prevalent in high dimensional domains. Regularization is a commonly used technique in machine learning to reduce variance, at the cost of introducing some bias. Most existing regularization techniques focus on spatial (perceptual) regularization. Yet in reinforcement learning, due to the… ▽ More Several applications of Reinforcement Learning suffer from instability due to high variance. This is especially prevalent in high dimensional domains. Regularization is a commonly used technique in machine learning to reduce variance, at the cost of introducing some bias. Most existing regularization techniques focus on spatial (perceptual) regularization. Yet in reinforcement learning, due to the nature of the Bellman equation, there is an opportunity to also exploit temporal regularization based on smoothness in value estimates over trajectories. This paper explores a class of methods for temporal regularization. We formally characterize the bias induced by this technique using Markov chain concepts. We illustrate the various characteristics of temporal regularization via a sequence of simple discrete and continuous MDPs, and show that the technique provides improvement even in high-dimensional Atari games. △ Less

Submitted 10 April, 2019; v1 submitted 1 November, 2018; originally announced November 2018.

Comments: Published as a conference paper at NIPS 2018

Adversarial Balancing for Causal Inference

Authors: Michal Ozery-Flato, Pierre Thodoroff, Matan Ninio, Michal Rosen-Zvi, Tal El-Hay

Abstract: Biases in observational data of treatments pose a major challenge to estimating expected treatment outcomes in different populations. An important technique that accounts for these biases is reweighting samples to minimize the discrepancy between treatment groups. We present a novel reweighting approach that uses bi-level optimization to alternately train a discriminator to minimize classification… ▽ More Biases in observational data of treatments pose a major challenge to estimating expected treatment outcomes in different populations. An important technique that accounts for these biases is reweighting samples to minimize the discrepancy between treatment groups. We present a novel reweighting approach that uses bi-level optimization to alternately train a discriminator to minimize classification error, and a balancing weights generator that uses exponentiated gradient descent to maximize this error. This approach borrows principles from generative adversarial networks (GANs) to exploit the power of classifiers for measuring two-sample divergence. We provide theoretical results for conditions in which the estimation error is bounded by two factors: (i) the discrepancy measure induced by the discriminator; and (ii) the weights variability. Experimental results on several benchmarks comparing to previous state-of-the-art reweighting methods demonstrate the effectiveness of this approach in estimating causal effects. △ Less

Submitted 11 September, 2020; v1 submitted 17 October, 2018; originally announced October 2018.

Learning Robust Features using Deep Learning for Automatic Seizure Detection

Authors: Pierre Thodoroff, Joelle Pineau, Andrew Lim

Abstract: We present and evaluate the capacity of a deep neural network to learn robust features from EEG to automatically detect seizures. This is a challenging problem because seizure manifestations on EEG are extremely variable both inter- and intra-patient. By simultaneously capturing spectral, temporal and spatial information our recurrent convolutional neural network learns a general spatially invaria… ▽ More We present and evaluate the capacity of a deep neural network to learn robust features from EEG to automatically detect seizures. This is a challenging problem because seizure manifestations on EEG are extremely variable both inter- and intra-patient. By simultaneously capturing spectral, temporal and spatial information our recurrent convolutional neural network learns a general spatially invariant representation of a seizure. The proposed approach exceeds significantly previous results obtained on cross-patient classifiers both in terms of sensitivity and false positive rate. Furthermore, our model proves to be robust to missing channel and variable electrode montage. △ Less

Submitted 31 July, 2016; originally announced August 2016.

Comments: Presented at 2016 Machine Learning and Healthcare Conference (MLHC 2016), Los Angeles, CA