Showing 1–10 of 10 results for author: Herrera, C

Fast Iterative Solver For Neural Network Method: II. 1D Diffusion-Reaction Problems And Data Fitting

Authors: Zhiqiang Cai, Anastassia Doktorova, Robert D. Falgout, César Herrera

Abstract: This paper expands the damped block Newton (dBN) method introduced recently in [4] for 1D diffusion-reaction equations and least-squares data fitting problems. To determine the linear parameters (the weights and bias of the output layer) of the neural network (NN), the dBN method requires solving systems of linear equations involving the mass matrix. While the mass matrix for local hat basis funct… ▽ More This paper expands the damped block Newton (dBN) method introduced recently in [4] for 1D diffusion-reaction equations and least-squares data fitting problems. To determine the linear parameters (the weights and bias of the output layer) of the neural network (NN), the dBN method requires solving systems of linear equations involving the mass matrix. While the mass matrix for local hat basis functions is tri-diagonal and well-conditioned, the mass matrix for NNs is dense and ill-conditioned. For example, the condition number of the NN mass matrix for quasi-uniform meshes is at least ${\cal O}(n^4)$. We present a factorization of the mass matrix that enables solving the systems of linear equations in ${\cal O}(n)$ operations. To determine the non-linear parameters (the weights and bias of the hidden layer), one step of a damped Newton method is employed at each iteration. A Gauss-Newton method is used in place of Newton for the instances in which the Hessian matrices are singular. This modified dBN is referred to as dBGN. For both methods, the computational cost per iteration is ${\cal O}(n)$. Numerical results demonstrate the ability dBN and dBGN to efficiently achieve accurate results and outperform BFGS for select examples. △ Less

Submitted 1 July, 2024; originally announced July 2024.

MSC Class: 65K10; 65F05

Physics-informed neural networks to learn cardiac fiber orientation from multiple electroanatomical maps

Authors: Carlos Ruiz Herrera, Thomas Grandits, Gernot Plank, Paris Perdikaris, Francisco Sahli Costabal, Simone Pezzuto

Abstract: We propose FiberNet, a method to estimate \emph{in-vivo} the cardiac fiber architecture of the human atria from multiple catheter recordings of the electrical activation. Cardiac fibers play a central role in the electro-mechanical function of the heart, yet they are difficult to determine in-vivo, and hence rarely truly patient-specific in existing cardiac models. FiberNet learns the fiber arrang… ▽ More We propose FiberNet, a method to estimate \emph{in-vivo} the cardiac fiber architecture of the human atria from multiple catheter recordings of the electrical activation. Cardiac fibers play a central role in the electro-mechanical function of the heart, yet they are difficult to determine in-vivo, and hence rarely truly patient-specific in existing cardiac models. FiberNet learns the fiber arrangement by solving an inverse problem with physics-informed neural networks. The inverse problem amounts to identifying the conduction velocity tensor of a cardiac propagation model from a set of sparse activation maps. The use of multiple maps enables the simultaneous identification of all the components of the conduction velocity tensor, including the local fiber angle. We extensively test FiberNet on synthetic 2-D and 3-D examples, diffusion tensor fibers, and a patient-specific case. We show that 3 maps are sufficient to accurately capture the fibers, also in the presence of noise. With fewer maps, the role of regularization becomes prominent. Moreover, we show that the fitted model can robustly reproduce unseen activation maps. We envision that FiberNet will help the creation of patient-specific models for personalized medicine. The full code is available at http://github.com/fsahli/FiberNet. △ Less

Submitted 22 July, 2022; v1 submitted 28 January, 2022; originally announced January 2022.

Comments: 29 pages, 11 figures

Optimal Stopping via Randomized Neural Networks

Authors: Calypso Herrera, Florian Krach, Pierre Ruyssen, Josef Teichmann

Abstract: This paper presents the benefits of using randomized neural networks instead of standard basis functions or deep neural networks to approximate the solutions of optimal stopping problems. The key idea is to use neural networks, where the parameters of the hidden layers are generated randomly and only the last layer is trained, in order to approximate the continuation value. Our approaches are appl… ▽ More This paper presents the benefits of using randomized neural networks instead of standard basis functions or deep neural networks to approximate the solutions of optimal stopping problems. The key idea is to use neural networks, where the parameters of the hidden layers are generated randomly and only the last layer is trained, in order to approximate the continuation value. Our approaches are applicable to high dimensional problems where the existing approaches become increasingly impractical. In addition, since our approaches can be optimized using simple linear regression, they are easy to implement and theoretical guarantees can be provided. We test our approaches for American option pricing on Black--Scholes, Heston and rough Heston models and for optimally stopping a fractional Brownian motion. In all cases, our algorithms outperform the state-of-the-art and other relevant machine learning approaches in terms of computation time while achieving comparable results. Moreover, we show that they can also be used to efficiently compute Greeks of American options. △ Less

Submitted 1 December, 2023; v1 submitted 28 April, 2021; originally announced April 2021.

MSC Class: 60G40 (Primary); 68T07 (Secondary)

Neural Jump Ordinary Differential Equations: Consistent Continuous-Time Prediction and Filtering

Authors: Calypso Herrera, Florian Krach, Josef Teichmann

Abstract: Combinations of neural ODEs with recurrent neural networks (RNN), like GRU-ODE-Bayes or ODE-RNN are well suited to model irregularly observed time series. While those models outperform existing discrete-time approaches, no theoretical guarantees for their predictive capabilities are available. Assuming that the irregularly-sampled time series data originates from a continuous stochastic process, t… ▽ More Combinations of neural ODEs with recurrent neural networks (RNN), like GRU-ODE-Bayes or ODE-RNN are well suited to model irregularly observed time series. While those models outperform existing discrete-time approaches, no theoretical guarantees for their predictive capabilities are available. Assuming that the irregularly-sampled time series data originates from a continuous stochastic process, the $L^2$-optimal online prediction is the conditional expectation given the currently available information. We introduce the Neural Jump ODE (NJ-ODE) that provides a data-driven approach to learn, continuously in time, the conditional expectation of a stochastic process. Our approach models the conditional expectation between two observations with a neural ODE and jumps whenever a new observation is made. We define a novel training framework, which allows us to prove theoretical guarantees for the first time. In particular, we show that the output of our model converges to the $L^2$-optimal prediction. This can be interpreted as solution to a special filtering problem. We provide experiments showing that the theoretical results also hold empirically. Moreover, we experimentally show that our model outperforms the baselines in more complex learning tasks and give comparisons on real-world datasets. △ Less

Submitted 16 April, 2021; v1 submitted 8 June, 2020; originally announced June 2020.

Journal ref: International Conference on Learning Representations (2021)

Denise: Deep Robust Principal Component Analysis for Positive Semidefinite Matrices

Authors: Calypso Herrera, Florian Krach, Anastasis Kratsios, Pierre Ruyssen, Josef Teichmann

Abstract: The robust PCA of covariance matrices plays an essential role when isolating key explanatory features. The currently available methods for performing such a low-rank plus sparse decomposition are matrix specific, meaning, those algorithms must re-run for every new matrix. Since these algorithms are computationally expensive, it is preferable to learn and store a function that nearly instantaneousl… ▽ More The robust PCA of covariance matrices plays an essential role when isolating key explanatory features. The currently available methods for performing such a low-rank plus sparse decomposition are matrix specific, meaning, those algorithms must re-run for every new matrix. Since these algorithms are computationally expensive, it is preferable to learn and store a function that nearly instantaneously performs this decomposition when evaluated. Therefore, we introduce Denise, a deep learning-based algorithm for robust PCA of covariance matrices, or more generally, of symmetric positive semidefinite matrices, which learns precisely such a function. Theoretical guarantees for Denise are provided. These include a novel universal approximation theorem adapted to our geometric deep learning problem and convergence to an optimal solution to the learning problem. Our experiments show that Denise matches state-of-the-art performance in terms of decomposition quality, while being approximately $2000\times$ faster than the state-of-the-art, principal component pursuit (PCP), and $200 \times$ faster than the current speed-optimized method, fast PCP. △ Less

Submitted 6 June, 2023; v1 submitted 28 April, 2020; originally announced April 2020.

Journal ref: Transactions on Machine Learning Research (2023)

Local Lipschitz Bounds of Deep Neural Networks

Authors: Calypso Herrera, Florian Krach, Josef Teichmann

Abstract: The Lipschitz constant is an important quantity that arises in analysing the convergence of gradient-based optimization methods. It is generally unclear how to estimate the Lipschitz constant of a complex model. Thus, this paper studies an important problem that may be useful to the broader area of non-convex optimization. The main result provides a local upper bound on the Lipschitz constants of… ▽ More The Lipschitz constant is an important quantity that arises in analysing the convergence of gradient-based optimization methods. It is generally unclear how to estimate the Lipschitz constant of a complex model. Thus, this paper studies an important problem that may be useful to the broader area of non-convex optimization. The main result provides a local upper bound on the Lipschitz constants of a multi-layer feed-forward neural network and its gradient. Moreover, lower bounds are established as well, which are used to show that it is impossible to derive global upper bounds for the Lipschitz constants. In contrast to previous works, we compute the Lipschitz constants with respect to the network parameters and not with respect to the inputs. These constants are needed for the theoretical description of many step size schedulers of gradient based optimization schemes and their convergence analysis. The idea is both simple and effective. The results are extended to a generalization of neural networks, continuously deep neural networks, which are described by controlled ODEs. △ Less

Submitted 9 February, 2023; v1 submitted 27 April, 2020; originally announced April 2020.

Sustainable Recipes. A Food Recipe Sourcing and Recommendation System to Minimize Food Miles

Authors: Juan C. S. Herrera

Abstract: Sustainable Recipes is a tool that (1) connects food recipes ingredient lists with the closest organic providers to minimize the distance that food travels from farm to food preparation site and (2) recommends recipes given a GPS coordinate to minimize food miles. Sustainable Recipes provides consumers, entrepreneurs, cooking enthusiasts, and restauranteurs in the United States and elsewhere with… ▽ More Sustainable Recipes is a tool that (1) connects food recipes ingredient lists with the closest organic providers to minimize the distance that food travels from farm to food preparation site and (2) recommends recipes given a GPS coordinate to minimize food miles. Sustainable Recipes provides consumers, entrepreneurs, cooking enthusiasts, and restauranteurs in the United States and elsewhere with an easy to use interface to help them (1) connect with organic ingredient producers to source ingredients to produce food recipes minimizing food miles and (2) recommend recipes using locally grown food. The main academic contribution of Sustainable Recipes is to bridge the gap between two streams of literature in data science of food recipes: studies of food recipes and studies of food supply chains. The outcomes of the interphase are (1) a map visualization that highlights the location of the producers that can supply the ingredients for a food recipe along with a ticket consisting of their contact addresses and the food miles used to produce a recipe and (2) a list of recipes that minimize food miles for a given GPS coordinate in which the recipe is going to be produced. △ Less

Submitted 16 April, 2020; originally announced April 2020.

Comments: 9 pages, 3 figures, 1 table, 2 algorithms

ACM Class: J.0; J.4

From Stateless to Stateful Priorities: Technical Report

Authors: Christian Herrera

Abstract: We present the notion of stateful priorities for imposing precise restrictions on system actions, in order to meet safety constraints. By using stateful priorities we are able to exclusively restrict erroneous system behavior as specified by the constraint, whereas safe system behavior remains unrestricted. Given a system modeled as a network of discrete automata and an error constraint, we presen… ▽ More We present the notion of stateful priorities for imposing precise restrictions on system actions, in order to meet safety constraints. By using stateful priorities we are able to exclusively restrict erroneous system behavior as specified by the constraint, whereas safe system behavior remains unrestricted. Given a system modeled as a network of discrete automata and an error constraint, we present algorithms which use those inputs to synthesize stateful priorities. We present as well a network transformation which uses synthesized priorities for blocking all system actions leading to the input error. Our experiments with three real-world examples demonstrate the applicability of our approach. △ Less

Submitted 20 February, 2020; originally announced February 2020.

Limits of Predictability in Commuting Flows in the Absence of Data for Calibration

Authors: Yingxiang Yang, Carlos Herrera, Nathan Eagle, Marta C. Gonzalez

Abstract: The estimation of commuting flows at different spatial scales is a fundamental problem for different areas of study. Many current methods rely on parameters requiring calibration from empirical trip volumes. Their values are often not generalizable to cases without calibration data. To solve this problem we develop a statistical expression to calculate commuting trips with a quantitative functiona… ▽ More The estimation of commuting flows at different spatial scales is a fundamental problem for different areas of study. Many current methods rely on parameters requiring calibration from empirical trip volumes. Their values are often not generalizable to cases without calibration data. To solve this problem we develop a statistical expression to calculate commuting trips with a quantitative functional form to estimate the model parameter when empirical trip data is not available. We calculate commuting trip volumes at scales from within a city to an entire country, introducing a scaling parameter alpha to the recently proposed parameter free radiation model. The model requires only widely available population and facility density distributions. The parameter can be interpreted as the influence of the region scale and the degree of heterogeneity in the facility distribution. We explore in detail the scaling limitations of this problem, namely under which conditions the proposed model can be applied without trip data for calibration. On the other hand, when empirical trip data is available, we show that the proposed model's estimation accuracy is as good as other existing models. We validated the model in different regions in the U.S., then successfully applied it in three different countries. △ Less

Submitted 13 July, 2014; originally announced July 2014.

Comments: 25 pages, 5 figures, Scientific Reports, 4 (2014)

Generating Scale-free Networks with Adjustable Clustering Coefficient Via Random Walks

Authors: Carlos Herrera, Pedro J. Zufiria

Abstract: This paper presents an algorithm for generating scale-free networks with adjustable clustering coefficient. The algorithm is based on a random walk procedure combined with a triangle generation scheme which takes into account genetic factors; this way, preferential attachment and clustering control are implemented using only local information. Simulations are presented which support the validity o… ▽ More This paper presents an algorithm for generating scale-free networks with adjustable clustering coefficient. The algorithm is based on a random walk procedure combined with a triangle generation scheme which takes into account genetic factors; this way, preferential attachment and clustering control are implemented using only local information. Simulations are presented which support the validity of the scheme, characterizing its tuning capabilities. △ Less

Submitted 17 May, 2011; originally announced May 2011.