Showing 1–41 of 41 results for author: Beck, A

NoX: a Compact Open-Source RISC-V Processor for Multi-Processor Systems-on-Chip

Authors: Anderson I. Silva, Altamiro Susin, Fernanda L. Kastensmidt, Antonio Carlos S. Beck, Jose Rodrigo Azambuja

Abstract: IoT applications are one of the driving forces in making systems energy and power-efficient, given their resource constraints. However, because of security, latency, and transmission, we advocate for local computing through multi-processor systems-on-chip (MPSoCs) for edge computing. The RISC-V ISA has grown in academia and industry due to its flexibility. Still, available open-source cores cannot… ▽ More IoT applications are one of the driving forces in making systems energy and power-efficient, given their resource constraints. However, because of security, latency, and transmission, we advocate for local computing through multi-processor systems-on-chip (MPSoCs) for edge computing. The RISC-V ISA has grown in academia and industry due to its flexibility. Still, available open-source cores cannot be seamlessly integrated into MPSoCs for a fast time to market. This paper presents NoX, a compact open-source plug-and-play 32-bit RISC-V core designed in System Verilog for efficient data processing in MPSoCs. NoX has a 4-stage single-issue in-order pipeline with full bypass, providing an efficient resource-constrained architecture. Compared to industry and academia resource-constrained RISC-V cores, NoX offers a better resource usage and performance trade-off. △ Less

Submitted 25 June, 2024; originally announced June 2024.

PLUTO: Pathology-Universal Transformer

Authors: Dinkar Juyal, Harshith Padigela, Chintan Shah, Daniel Shenker, Natalia Harguindeguy, Yi Liu, Blake Martin, Yibo Zhang, Michael Nercessian, Miles Markey, Isaac Finberg, Kelsey Luu, Daniel Borders, Syed Ashar Javed, Emma Krause, Raymond Biju, Aashish Sood, Allen Ma, Jackson Nyman, John Shamshoian, Guillaume Chhor, Darpan Sanghavi, Marc Thibault, Limin Yu, Fedaa Najdawi , et al. (8 additional authors not shown)

Abstract: Pathology is the study of microscopic inspection of tissue, and a pathology diagnosis is often the medical gold standard to diagnose disease. Pathology images provide a unique challenge for computer-vision-based analysis: a single pathology Whole Slide Image (WSI) is gigapixel-sized and often contains hundreds of thousands to millions of objects of interest across multiple resolutions. In this wor… ▽ More Pathology is the study of microscopic inspection of tissue, and a pathology diagnosis is often the medical gold standard to diagnose disease. Pathology images provide a unique challenge for computer-vision-based analysis: a single pathology Whole Slide Image (WSI) is gigapixel-sized and often contains hundreds of thousands to millions of objects of interest across multiple resolutions. In this work, we propose PathoLogy Universal TransfOrmer (PLUTO): a light-weight pathology FM that is pre-trained on a diverse dataset of 195 million image tiles collected from multiple sites and extracts meaningful representations across multiple WSI scales that enable a large variety of downstream pathology tasks. In particular, we design task-specific adaptation heads that utilize PLUTO's output embeddings for tasks which span pathology scales ranging from subcellular to slide-scale, including instance segmentation, tile classification, and slide-level prediction. We compare PLUTO's performance to other state-of-the-art methods on a diverse set of external and internal benchmarks covering multiple biologically relevant tasks, tissue types, resolutions, stains, and scanners. We find that PLUTO matches or outperforms existing task-specific baselines and pathology-specific foundation models, some of which use orders-of-magnitude larger datasets and model sizes when compared to PLUTO. Our findings present a path towards a universal embedding to power pathology image analysis, and motivate further exploration around pathology foundation models in terms of data diversity, architectural improvements, sample efficiency, and practical deployability in real-world applications. △ Less

Submitted 13 May, 2024; originally announced May 2024.

GALÆXI: Solving complex compressible flows with high-order discontinuous Galerkin methods on accelerator-based systems

Authors: Daniel Kempf, Marius Kurz, Marcel Blind, Patrick Kopper, Philipp Offenhäuser, Anna Schwarz, Spencer Starr, Jens Keim, Andrea Beck

Abstract: This work presents GALÆXI as a novel, energy-efficient flow solver for the simulation of compressible flows on unstructured meshes leveraging the parallel computing power of modern Graphics Processing Units (GPUs). GALÆXI implements the high-order Discontinuous Galerkin Spectral Element Method (DGSEM) using shock capturing with a finite-volume subcell approach to ensure the stability of the high-o… ▽ More This work presents GALÆXI as a novel, energy-efficient flow solver for the simulation of compressible flows on unstructured meshes leveraging the parallel computing power of modern Graphics Processing Units (GPUs). GALÆXI implements the high-order Discontinuous Galerkin Spectral Element Method (DGSEM) using shock capturing with a finite-volume subcell approach to ensure the stability of the high-order scheme near shocks. This work provides details on the general code design, the parallelization strategy, and the implementation approach for the compute kernels with a focus on the element local mappings between volume and surface data due to the unstructured mesh. GALÆXI exhibits excellent strong scaling properties up to 1024 GPUs if each GPU is assigned a minimum of one million degrees of freedom degrees of freedom. To verify its implementation, a convergence study is performed that recovers the theoretical order of convergence of the implemented numerical schemes. Moreover, the solver is validated using both the incompressible and compressible formulation of the Taylor-Green-Vortex at a Mach number of 0.1 and 1.25, respectively. A mesh convergence study shows that the results converge to the high-fidelity reference solution and that the results match the original CPU implementation. Finally, GALÆXI is applied to a large-scale wall-resolved large eddy simulation of a linear cascade of the NASA Rotor 37. Here, the supersonic region and shocks at the leading edge are captured accurately and robustly by the implemented shock-capturing approach. It is demonstrated that GALÆXI requires less than half of the energy to carry out this simulation in comparison to the reference CPU implementation. This renders GALÆXI as a potent tool for accurate and efficient simulations of compressible flows in the realm of exascale computing and the associated new HPC architectures. △ Less

Submitted 19 April, 2024; originally announced April 2024.

Comments: 19 pages, 12 figures, 3 tables. Code available at: https://github.com/flexi-framework/galaexi

Beyond LLMs: Advancing the Landscape of Complex Reasoning

Authors: Jennifer Chu-Carroll, Andrew Beck, Greg Burnham, David OS Melville, David Nachman, A. Erdem Özcan, David Ferrucci

Abstract: Since the advent of Large Language Models a few years ago, they have often been considered the de facto solution for many AI problems. However, in addition to the many deficiencies of LLMs that prevent them from broad industry adoption, such as reliability, cost, and speed, there is a whole class of common real world problems that Large Language Models perform poorly on, namely, constraint satisfa… ▽ More Since the advent of Large Language Models a few years ago, they have often been considered the de facto solution for many AI problems. However, in addition to the many deficiencies of LLMs that prevent them from broad industry adoption, such as reliability, cost, and speed, there is a whole class of common real world problems that Large Language Models perform poorly on, namely, constraint satisfaction and optimization problems. These problems are ubiquitous and current solutions are highly specialized and expensive to implement. At Elemental Cognition, we developed our EC AI platform which takes a neuro-symbolic approach to solving constraint satisfaction and optimization problems. The platform employs, at its core, a precise and high performance logical reasoning engine, and leverages LLMs for knowledge acquisition and user interaction. This platform supports developers in specifying application logic in natural and concise language while generating application user interfaces to interact with users effectively. We evaluated LLMs against systems built on the EC AI platform in three domains and found the EC AI systems to significantly outperform LLMs on constructing valid and optimal solutions, on validating proposed solutions, and on repairing invalid solutions. △ Less

Submitted 12 February, 2024; originally announced February 2024.

Grokking in Linear Estimators -- A Solvable Model that Groks without Understanding

Authors: Noam Levi, Alon Beck, Yohai Bar-Sinai

Abstract: Grokking is the intriguing phenomenon where a model learns to generalize long after it has fit the training data. We show both analytically and numerically that grokking can surprisingly occur in linear networks performing linear tasks in a simple teacher-student setup with Gaussian inputs. In this setting, the full training dynamics is derived in terms of the training and generalization data cova… ▽ More Grokking is the intriguing phenomenon where a model learns to generalize long after it has fit the training data. We show both analytically and numerically that grokking can surprisingly occur in linear networks performing linear tasks in a simple teacher-student setup with Gaussian inputs. In this setting, the full training dynamics is derived in terms of the training and generalization data covariance matrix. We present exact predictions on how the grokking time depends on input and output dimensionality, train sample size, regularization, and network initialization. We demonstrate that the sharp increase in generalization accuracy may not imply a transition from "memorization" to "understanding", but can simply be an artifact of the accuracy measure. We provide empirical verification for our calculations, along with preliminary results indicating that some predictions also hold for deeper networks, with non-linear activations. △ Less

Submitted 25 October, 2023; originally announced October 2023.

Comments: 17 pages, 6 figures

Journal ref: Proceedings of the International Conference on Learning Representations (ICLR), 2024

Strategies and impact of learning curve estimation for CNN-based image classification

Authors: Laura Didyk, Brayden Yarish, Michael A. Beck, Christopher P. Bidinosti, Christopher J. Henry

Abstract: Learning curves are a measure for how the performance of machine learning models improves given a certain volume of training data. Over a wide variety of applications and models it was observed that learning curves follow -- to a large extent -- a power law behavior. This makes the performance of different models for a given task somewhat predictable and opens the opportunity to reduce the trainin… ▽ More Learning curves are a measure for how the performance of machine learning models improves given a certain volume of training data. Over a wide variety of applications and models it was observed that learning curves follow -- to a large extent -- a power law behavior. This makes the performance of different models for a given task somewhat predictable and opens the opportunity to reduce the training time for practitioners, who are exploring the space of possible models and hyperparameters for the problem at hand. By estimating the learning curve of a model from training on small subsets of data only the best models need to be considered for training on the full dataset. How to choose subset sizes and how often to sample models on these to obtain estimates is however not researched. Given that the goal is to reduce overall training time strategies are needed that sample the performance in a time-efficient way and yet leads to accurate learning curve estimates. In this paper we formulate the framework for these strategies and propose several strategies. Further we evaluate the strategies for simulated learning curves and in experiments with popular datasets and models for image classification tasks. △ Less

Submitted 12 October, 2023; originally announced October 2023.

Testable Likelihoods for Beyond-the-Standard Model Fits

Authors: Anja Beck, Méril Reboud, Danny van Dyk

Abstract: Studying potential BSM effects at the precision frontier requires accurate transfer of information from low-energy measurements to high-energy BSM models. We propose to use normalising flows to construct likelihood functions that achieve this transfer. Likelihood functions constructed in this way provide the means to generate additional samples and admit a ``trivial'' goodness-of-fit test in form… ▽ More Studying potential BSM effects at the precision frontier requires accurate transfer of information from low-energy measurements to high-energy BSM models. We propose to use normalising flows to construct likelihood functions that achieve this transfer. Likelihood functions constructed in this way provide the means to generate additional samples and admit a ``trivial'' goodness-of-fit test in form of a $χ^2$ test statistic. Here, we study a particular form of normalising flow, apply it to a multi-modal and non-Gaussian example, and quantify the accuracy of the likelihood function and its test statistic. △ Less

Submitted 19 September, 2023; originally announced September 2023.

Comments: 11 pages, 7 figures

Report number: IPPP/23/49

Toward Discretization-Consistent Closure Schemes for Large Eddy Simulation Using Reinforcement Learning

Authors: Andrea Beck, Marius Kurz

Abstract: This study proposes a novel method for developing discretization-consistent closure schemes for implicitly filtered Large Eddy Simulation (LES). Here, the induced filter kernel, and thus the closure terms, are determined by the properties of the grid and the discretization operator, leading to additional computational subgrid terms that are generally unknown in a priori analysis. In this work, the… ▽ More This study proposes a novel method for developing discretization-consistent closure schemes for implicitly filtered Large Eddy Simulation (LES). Here, the induced filter kernel, and thus the closure terms, are determined by the properties of the grid and the discretization operator, leading to additional computational subgrid terms that are generally unknown in a priori analysis. In this work, the task of adapting the coefficients of LES closure models is thus framed as a Markov decision process and solved in an a posteriori manner with Reinforcement Learning (RL). This optimization framework is applied to both explicit and implicit closure models. The explicit model is based on an element-local eddy viscosity model. The optimized model is found to adapt its induced viscosity within discontinuous Galerkin (DG) methods to homogenize the dissipation within an element by adding more viscosity near its center. For the implicit modeling, RL is applied to identify an optimal blending strategy for a hybrid DG and Finite Volume (FV) scheme. The resulting optimized discretization yields more accurate results in LES than either the pure DG or FV method and renders itself as a viable modeling ansatz that could initiate a novel class of high-order schemes for compressible turbulence by combining turbulence modeling with shock capturing in a single framework. All newly derived models achieve accurate results that either match or outperform traditional models for different discretizations and resolutions. Overall, the results demonstrate that the proposed RL optimization can provide discretization-consistent closures that could reduce the uncertainty in implicitly filtered LES. △ Less

Submitted 13 December, 2023; v1 submitted 12 September, 2023; originally announced September 2023.

Comments: 24 pages, 14 figures. Accepted Manuscript. This article may be downloaded for personal use only. Any other use requires prior permission of the author and AIP Publishing. This article appeared in Physics of Fluids 35 (2023) and may be found at https://doi.org/10.1063/5.0176223

Journal ref: Physics of Fluids 35 (2023) 125122

Towards Exascale CFD Simulations Using the Discontinuous Galerkin Solver FLEXI

Authors: Marcel Blind, Min Gao, Daniel Kempf, Patrick Kopper, Marius Kurz, Anna Schwarz, Andrea Beck

Abstract: Modern high-order discretizations bear considerable potential for the exascale era due to their high fidelity and the high, local computational load that allows for computational efficiency in massively parallel simulations. To this end, the discontinuous Galerkin (DG) framework FLEXI was selected to demonstrate exascale readiness within the Center of Excellence for Exascale CFD (CEEC) by simulati… ▽ More Modern high-order discretizations bear considerable potential for the exascale era due to their high fidelity and the high, local computational load that allows for computational efficiency in massively parallel simulations. To this end, the discontinuous Galerkin (DG) framework FLEXI was selected to demonstrate exascale readiness within the Center of Excellence for Exascale CFD (CEEC) by simulating shock buffet on a three-dimensional wing segment at transsonic flight conditions. This paper summarizes the recent progress made to enable the simulation of this challenging exascale problem. For this, it is first demonstrated that FLEXI scales excellently to over 500 000 CPU cores on HAWK at the HLRS. To tackle the considerable resolution requirements near the wall, a novel wall model is proposed that takes compressibility effects into account and yields decent results for the simulation of a NACA 64A-110 airfoil. To address the shocks in the domain, a finite-volume-based shock capturing method was implemented in FLEXI, which is validated here using the simulation of a linear compressor cascade at supersonic flow conditions, where the method is demonstrated to yield efficient, robust and accurate results. Lastly, we present the TensorFlow-Fortran-Binding (TFFB) as an easy-to-use library to deploy trained machine learning models in Fortran solvers such as FLEXI. △ Less

Submitted 22 June, 2023; originally announced June 2023.

Comments: 15 pages, 5 figures

A comprehensive review of 3D convolutional neural network-based classification techniques of diseased and defective crops using non-UAV-based hyperspectral images

Authors: Nooshin Noshiri, Michael A. Beck, Christopher P. Bidinosti, Christopher J. Henry

Abstract: Hyperspectral imaging (HSI) is a non-destructive and contactless technology that provides valuable information about the structure and composition of an object. It can capture detailed information about the chemical and physical properties of agricultural crops. Due to its wide spectral range, compared with multispectral- or RGB-based imaging methods, HSI can be a more effective tool for monitorin… ▽ More Hyperspectral imaging (HSI) is a non-destructive and contactless technology that provides valuable information about the structure and composition of an object. It can capture detailed information about the chemical and physical properties of agricultural crops. Due to its wide spectral range, compared with multispectral- or RGB-based imaging methods, HSI can be a more effective tool for monitoring crop health and productivity. With the advent of this imaging tool in agrotechnology, researchers can more accurately address issues related to the detection of diseased and defective crops in the agriculture industry. This allows to implement the most suitable and accurate farming solutions, such as irrigation and fertilization before crops enter a damaged and difficult-to-recover phase of growth in the field. While HSI provides valuable insights into the object under investigation, the limited number of HSI datasets for crop evaluation presently poses a bottleneck. Dealing with the curse of dimensionality presents another challenge due to the abundance of spectral and spatial information in each hyperspectral cube. State-of-the-art methods based on 1D- and 2D-CNNs struggle to efficiently extract spectral and spatial information. On the other hand, 3D-CNN-based models have shown significant promise in achieving better classification and detection results by leveraging spectral and spatial features simultaneously. Despite the apparent benefits of 3D-CNN-based models, their usage for classification purposes in this area of research has remained limited. This paper seeks to address this gap by reviewing 3D-CNN-based architectures and the typical deep learning pipeline, including preprocessing and visualization of results, for the classification of hyperspectral images of diseased and defective crops. Furthermore, we discuss open research areas and challenges when utilizing 3D-CNNs with HSI data. △ Less

Submitted 15 June, 2023; originally announced June 2023.

Journal ref: Smart Agricultural Technology 5 (2023) 100316

Inside Out: Transforming Images of Lab-Grown Plants for Machine Learning Applications in Agriculture

Authors: A. E. Krosney, P. Sotoodeh, C. J. Henry, M. A. Beck, C. P. Bidinosti

Abstract: Machine learning tasks often require a significant amount of training data for the resultant network to perform suitably for a given problem in any domain. In agriculture, dataset sizes are further limited by phenotypical differences between two plants of the same genotype, often as a result of differing growing conditions. Synthetically-augmented datasets have shown promise in improving existing… ▽ More Machine learning tasks often require a significant amount of training data for the resultant network to perform suitably for a given problem in any domain. In agriculture, dataset sizes are further limited by phenotypical differences between two plants of the same genotype, often as a result of differing growing conditions. Synthetically-augmented datasets have shown promise in improving existing models when real data is not available. In this paper, we employ a contrastive unpaired translation (CUT) generative adversarial network (GAN) and simple image processing techniques to translate indoor plant images to appear as field images. While we train our network to translate an image containing only a single plant, we show that our method is easily extendable to produce multiple-plant field images. Furthermore, we use our synthetic multi-plant images to train several YoloV5 nano object detection models to perform the task of plant detection and measure the accuracy of the model on real field data images. Including training data generated by the CUT-GAN leads to better plant detection performance compared to a network trained solely on real data. △ Less

Submitted 5 November, 2022; originally announced November 2022.

Comments: 35 pages, 23 figures

On the ergodicity assumption in Performance-Based Engineering

Authors: Andre T. Beck, Rubia M. Bosse, Isabela D. Rodrigues

Abstract: In the Performance-Based Engineering (PBE) framework, uncertainties in system parameters, or modelling uncertainties, have been shown to have significant effects on capacity fragilities and annual collapse rates of buildings. Yet, since modelling uncertainties are non-ergodic variables, their consideration in failure rate calculations offends the Poisson assumption of independent crossings. This p… ▽ More In the Performance-Based Engineering (PBE) framework, uncertainties in system parameters, or modelling uncertainties, have been shown to have significant effects on capacity fragilities and annual collapse rates of buildings. Yet, since modelling uncertainties are non-ergodic variables, their consideration in failure rate calculations offends the Poisson assumption of independent crossings. This problem has been addressed in the literature, and errors found negligible for small annual collapse failure rates. However, the errors could be significant for serviceability limit states, and when failure rates are integrated in time, to provide lifetime failure probabilities. Herein, we present a novel formulation to fully avoid the error in integration of non-ergodic variables. The proposed product-of-lognormals formulation is fully compatible with popular fragility modelling approaches in PBE context. Moreover, we address collapse limit states of realistic reinforced concrete buildings, and find errors of the order of 5 to 8% for 50-year lifetimes, up to 14% for 100 years. Computation of accurate lifetime failure probabilities in a PBE context is clearly important, as it allows comparison with lifetime target reliability values for other structural analysis formulations. △ Less

Submitted 26 September, 2022; originally announced October 2022.

Journal ref: Structural Safety 97 (2022) 102218

Deep Reinforcement Learning for Turbulence Modeling in Large Eddy Simulations

Authors: Marius Kurz, Philipp Offenhäuser, Andrea Beck

Abstract: Over the last years, supervised learning (SL) has established itself as the state-of-the-art for data-driven turbulence modeling. In the SL paradigm, models are trained based on a dataset, which is typically computed a priori from a high-fidelity solution by applying the respective filter function, which separates the resolved and the unresolved flow scales. For implicitly filtered large eddy simu… ▽ More Over the last years, supervised learning (SL) has established itself as the state-of-the-art for data-driven turbulence modeling. In the SL paradigm, models are trained based on a dataset, which is typically computed a priori from a high-fidelity solution by applying the respective filter function, which separates the resolved and the unresolved flow scales. For implicitly filtered large eddy simulation (LES), this approach is infeasible, since here, the employed discretization itself acts as an implicit filter function. As a consequence, the exact filter form is generally not known and thus, the corresponding closure terms cannot be computed even if the full solution is available. The reinforcement learning (RL) paradigm can be used to avoid this inconsistency by training not on a previously obtained training dataset, but instead by interacting directly with the dynamical LES environment itself. This allows to incorporate the potentially complex implicit LES filter into the training process by design. In this work, we apply a reinforcement learning framework to find an optimal eddy-viscosity for implicitly filtered large eddy simulations of forced homogeneous isotropic turbulence. For this, we formulate the task of turbulence modeling as an RL task with a policy network based on convolutional neural networks that adapts the eddy-viscosity in LES dynamically in space and time based on the local flow state only. We demonstrate that the trained models can provide long-term stable simulations and that they outperform established analytical models in terms of accuracy. In addition, the models generalize well to other resolutions and discretizations. We thus demonstrate that RL can provide a framework for consistent, accurate and stable turbulence modeling especially for implicitly filtered LES. △ Less

Submitted 20 December, 2022; v1 submitted 21 June, 2022; originally announced June 2022.

Comments: 17 pages, 9 figures. Accepted Manuscript

Journal ref: International Journal of Heat and Fluid Flow 99 (2023) 109094

Investigating classification learning curves for automatically generated and labelled plant images

Authors: Michael A. Beck, Christopher P. Bidinosti, Christopher J. Henry, Manisha Ajmani

Abstract: In the context of supervised machine learning a learning curve describes how a model's performance on unseen data relates to the amount of samples used to train the model. In this paper we present a dataset of plant images with representatives of crops and weeds common to the Manitoba prairies at different growth stages. We determine the learning curve for a classification task on this data with t… ▽ More In the context of supervised machine learning a learning curve describes how a model's performance on unseen data relates to the amount of samples used to train the model. In this paper we present a dataset of plant images with representatives of crops and weeds common to the Manitoba prairies at different growth stages. We determine the learning curve for a classification task on this data with the ResNet architecture. Our results are in accordance with previous studies and add to the evidence that learning curves are governed by power-law relationships over large scales, applications, and models. We further investigate how label noise and the reduction of trainable parameters impacts the learning curve on this dataset. Both effects lead to the model requiring disproportionally larger training sets to achieve the same classification performance as observed without these effects. △ Less

Submitted 30 June, 2022; v1 submitted 22 May, 2022; originally announced May 2022.

Deep Reinforcement Learning for Computational Fluid Dynamics on HPC Systems

Authors: Marius Kurz, Philipp Offenhäuser, Dominic Viola, Oleksandr Shcherbakov, Michael Resch, Andrea Beck

Abstract: Reinforcement learning (RL) is highly suitable for devising control strategies in the context of dynamical systems. A prominent instance of such a dynamical system is the system of equations governing fluid dynamics. Recent research results indicate that RL-augmented computational fluid dynamics (CFD) solvers can exceed the current state of the art, for example in the field of turbulence modeling.… ▽ More Reinforcement learning (RL) is highly suitable for devising control strategies in the context of dynamical systems. A prominent instance of such a dynamical system is the system of equations governing fluid dynamics. Recent research results indicate that RL-augmented computational fluid dynamics (CFD) solvers can exceed the current state of the art, for example in the field of turbulence modeling. However, while in supervised learning, the training data can be generated a priori in an offline manner, RL requires constant run-time interaction and data exchange with the CFD solver during training. In order to leverage the potential of RL-enhanced CFD, the interaction between the CFD solver and the RL algorithm thus have to be implemented efficiently on high-performance computing (HPC) hardware. To this end, we present Relexi as a scalable RL framework that bridges the gap between machine learning workflows and modern CFD solvers on HPC systems providing both components with its specialized hardware. Relexi is built with modularity in mind and allows easy integration of various HPC solvers by means of the in-memory data transfer provided by the SmartSim library. Here, we demonstrate that the Relexi framework can scale up to hundreds of parallel environment on thousands of cores. This allows to leverage modern HPC resources to either enable larger problems or faster turnaround times. Finally, we demonstrate the potential of an RL-augmented CFD solver by finding a control strategy for optimal eddy viscosity selection in large eddy simulations. △ Less

Submitted 13 May, 2022; originally announced May 2022.

Comments: 11 pages, 5 figures, the code can be accessed at https://github.com/flexi-framework/relexi

Journal ref: Journal of Computational Science 65 (2022) 101884

A Task Programming Implementation for the Particle in Cell Code Smilei

Authors: Francesco Massimo, Mathieu Lobet, Julien Derouillat, Arnaud Beck, Guillaume Bouchard, Mickael Grech, Frédéric Pérez, Tommaso Vinci

Abstract: An implementation of the electromagnetic Particle in Cell loop in the code Smilei using task programming is presented. Through OpenMP, the macro-particles operations are formulated in terms of tasks. This formulation allows asynchronous execution respecting the data dependencies of the macro-particle operations, the most time-consuming part of the code in simulations of interest for plasma physics… ▽ More An implementation of the electromagnetic Particle in Cell loop in the code Smilei using task programming is presented. Through OpenMP, the macro-particles operations are formulated in terms of tasks. This formulation allows asynchronous execution respecting the data dependencies of the macro-particle operations, the most time-consuming part of the code in simulations of interest for plasma physics. Through some benchmarks it is shown that this formulation can help mitigating the load imbalance of these operations at the OpenMP thread level. The improvements in strong scaling for load-imbalanced physical cases are discussed. △ Less

Submitted 27 April, 2022; originally announced April 2022.

Comments: 14 pages, 15 figures, conference: Platform for Advanced Scientific Computing 2022 (PASC22)

The TerraByte Client: providing access to terabytes of plant data

Authors: Michael A. Beck, Christopher P. Bidinosti, Christopher J. Henry, Manisha Ajmani

Abstract: In this paper we demonstrate the TerraByte Client, a software to download user-defined plant datasets from a data portal hosted at Compute Canada. To that end the client offers two key functionalities: (1) It allows the user to get an overview on what data is available and a quick way to visually check samples of that data. For this the client receives the results of queries to a database and disp… ▽ More In this paper we demonstrate the TerraByte Client, a software to download user-defined plant datasets from a data portal hosted at Compute Canada. To that end the client offers two key functionalities: (1) It allows the user to get an overview on what data is available and a quick way to visually check samples of that data. For this the client receives the results of queries to a database and displays the number of images that fulfill the search criteria. Furthermore, a sample can be downloaded within seconds to confirm that the data suits the user's needs. (2) The user can then download the specified data to their own drive. This data is prepared into chunks server-side and sent to the user's end-system, where it is automatically extracted into individual files. The first chunks of data are available for inspection after a brief waiting period of a minute or less depending on available bandwidth and type of data. The TerraByte Client has a full graphical user interface for easy usage and uses end-to-end encryption. The user interface is built on top of a low-level client. This architecture in combination of offering the client program open-source makes it possible for the user to develop their own user interface or use the client's functionality directly. An example for direct usage could be to download specific data on demand within a larger application, such as training machine learning models. △ Less

Submitted 25 March, 2022; originally announced March 2022.

IMPACT: Integrated Bottom-Up Greenhouse Gas Emission Pathways for Cities

Authors: Juliana Felkner, Zoltan Nagy, Ariane L. Beck, D. Cale Reeves, Steven Richter, Vivek Shastry, Eli Ramthun, Edward Mbata, Stephen Zigmund, Benjamin Marshall, Linnea Marks, Vianey Rueda, Jasmine Triplett, Sarah Domedead, Jose R Vazquez-Canteli, Varun Rai

Abstract: Increasing urbanization puts pressure on cities to prioritize sustainable growth and avoid carbon lock-in. Available modeling frameworks fall acutely of guiding such pivotal decision-making at the local level. Financial incentives, behavioral interventions, and mandates drive sustainable technology adoption, while land-use zoning plays a critical role in carbon emissions from the built environment… ▽ More Increasing urbanization puts pressure on cities to prioritize sustainable growth and avoid carbon lock-in. Available modeling frameworks fall acutely of guiding such pivotal decision-making at the local level. Financial incentives, behavioral interventions, and mandates drive sustainable technology adoption, while land-use zoning plays a critical role in carbon emissions from the built environment. Researchers typically evaluate impacts of policies top down, on a national scale, or else post-hoc on developments vis-à-vis different polices in the past. Such analyses cannot forecast emission pathways for specific cities, and hence cannot serve as input to local policymakers. Here, we present IMPACT pathways, from a bottom-up model with residence level granularity, that integrate technology adoption policies with zoning policies, climate change, and grid decarbonization scenarios. With the city at the heart of our analysis, we identify an emission premium for sprawling and show that adverse policy combinations exist that can exhibit rebounding emissions over time. △ Less

Submitted 13 June, 2024; v1 submitted 30 January, 2022; originally announced February 2022.

Comments: Under review

Comparison of robust, reliability-based and non-probabilistic topology optimization under uncertain loads and stress constraints

Authors: Gustavo Assis da Silva, Eduardo Lenz Cardoso, Andre T. Beck

Abstract: It is nowadays widely acknowledged that optimal structural design should be robust with respect to the uncertainties in loads and material parameters. However, there are several alternatives to consider such uncertainties in structural optimization problems. This paper presents a comprehensive comparison between the results of three different approaches to topology optimization under uncertain loa… ▽ More It is nowadays widely acknowledged that optimal structural design should be robust with respect to the uncertainties in loads and material parameters. However, there are several alternatives to consider such uncertainties in structural optimization problems. This paper presents a comprehensive comparison between the results of three different approaches to topology optimization under uncertain loading, considering stress constraints: 1) the robust formulation, which requires only the mean and standard deviation of stresses at each element; 2) the reliability-based formulation, which imposes a reliability constraint on computed stresses; 3) the non-probabilistic formulation, which considers a worst-case scenario for the stresses caused by uncertain loads. The information required by each method, regarding the uncertain loads, and the uncertainty propagation approach used in each case is quite different. The robust formulation requires only mean and standard deviation of uncertain loads; stresses are computed via a first-order perturbation approach. The reliability-based formulation requires full probability distributions of random loads, reliability constraints are computed via a first-order performance measure approach. The non-probabilistic formulation is applicable for bounded uncertain loads; only lower and upper bounds are used, and worst-case stresses are computed via a nested optimization with anti-optimization. The three approaches are quite different in the handling of uncertainties; however, the basic topology optimization framework is the same: the traditional density approach is employed for material parameterization, while the augmented Lagrangian method is employed to solve the resulting problem, in order to handle the large number of stress constraints. △ Less

Submitted 25 January, 2022; originally announced January 2022.

Comments: 16 pages, 5 figures

Journal ref: Probabilistic Engineering Mechanics 59 (2020) 103039

Optimal design of redundant structural systems: fundamentals

Authors: Andre T. Beck

Abstract: In spite of extended recent interest in System Reliability-Based Design Optimization (System RBDO) and life-cycle cost or Risk Optimization (RO), there is a lack of published studies on optimal design of redundant hyperstatic systems with objective consideration of (a) progressive collapse and (b) the impact of epistemic uncertainties. This paper investigates the fundamental aspects of the problem… ▽ More In spite of extended recent interest in System Reliability-Based Design Optimization (System RBDO) and life-cycle cost or Risk Optimization (RO), there is a lack of published studies on optimal design of redundant hyperstatic systems with objective consideration of (a) progressive collapse and (b) the impact of epistemic uncertainties. This paper investigates the fundamental aspects of the problem, by addressing the optimal design of simple two-bar active and passive redundant systems. Progressive collapse is objectively addressed, differentiating consequences of direct collapse of statically determinate structures, and progressive collapse of redundant, statically indeterminate structures. It is well known that reliability of a structural system also depends on nonstructural factors, or factors beyond structural design, such as unanticipated loading, manufacturing quality, quality of workmanship and human errors. These factors can be taken into account in an encompassing risk analysis, which accounts for physical and epistemic uncertainties, and which contributes a fixed latent failure probability to the structural optimization problem. Results presented herein show that the latent failure probability is the single most important parameter in determining optimal solutions, in System RBDO and in RO solutions. When the latent failure probability is smaller than target (System RBDO) or optimal (RO) failure probabilities, there is an equivalence between redundant and non-redundant (hyperstatic and isostatic) designs. However, when the latent reliability is smaller than target or optimal reliabilities, optimal designs become necessarily redundant (hyperstatic); as the only way to make system reliability larger than the latent reliability is by making structural systems redundant. △ Less

Submitted 25 January, 2022; originally announced January 2022.

Comments: 47 pages, 14 figures

Journal ref: Engineering Structures 219, 110542, 2020

Presenting an extensive lab- and field-image dataset of crops and weeds for computer vision tasks in agriculture

Authors: Michael A. Beck, Chen-Yi Liu, Christopher P. Bidinosti, Christopher J. Henry, Cara M. Godee, Manisha Ajmani

Abstract: We present two large datasets of labelled plant-images that are suited towards the training of machine learning and computer vision models. The first dataset encompasses as the day of writing over 1.2 million images of indoor-grown crops and weeds common to the Canadian Prairies and many US states. The second dataset consists of over 540,000 images of plants imaged in farmland. All indoor plant im… ▽ More We present two large datasets of labelled plant-images that are suited towards the training of machine learning and computer vision models. The first dataset encompasses as the day of writing over 1.2 million images of indoor-grown crops and weeds common to the Canadian Prairies and many US states. The second dataset consists of over 540,000 images of plants imaged in farmland. All indoor plant images are labelled by species and we provide rich etadata on the level of individual images. This comprehensive database allows to filter the datasets under user-defined specifications such as for example the crop-type or the age of the plant. Furthermore, the indoor dataset contains images of plants taken from a wide variety of angles, including profile shots, top-down shots, and angled perspectives. The images taken from plants in fields are all from a top-down perspective and contain usually multiple plants per image. For these images metadata is also available. In this paper we describe both datasets' characteristics with respect to plant variety, plant age, and number of images. We further introduce an open-access sample of the indoor-dataset that contains 1,000 images of each species covered in our dataset. These, in total 14,000 images, had been selected, such that they form a representative sample with respect to plant age and ndividual plants per species. This sample serves as a quick entry point for new users to the dataset, allowing them to explore the data on a small scale and find the parameters of data most useful for their application without having to deal with hundreds of thousands of individual images. △ Less

Submitted 12 August, 2021; originally announced August 2021.

Structured Deep Kernel Networks for Data-Driven Closure Terms of Turbulent Flows

Authors: Tizian Wenzel, Marius Kurz, Andrea Beck, Gabriele Santin, Bernard Haasdonk

Abstract: Standard kernel methods for machine learning usually struggle when dealing with large datasets. We review a recently introduced Structured Deep Kernel Network (SDKN) approach that is capable of dealing with high-dimensional and huge datasets - and enjoys typical standard machine learning approximation properties. We extend the SDKN to combine it with standard machine learning modules and compare i… ▽ More Standard kernel methods for machine learning usually struggle when dealing with large datasets. We review a recently introduced Structured Deep Kernel Network (SDKN) approach that is capable of dealing with high-dimensional and huge datasets - and enjoys typical standard machine learning approximation properties. We extend the SDKN to combine it with standard machine learning modules and compare it with Neural Networks on the scientific challenge of data-driven prediction of closure terms of turbulent flows. We show experimentally that the SDKNs are capable of dealing with large datasets and achieve near-perfect accuracy on the given application. △ Less

Submitted 25 March, 2021; originally announced March 2021.

A Perspective on Machine Learning Methods in Turbulence Modelling

Authors: Andrea Beck, Marius Kurz

Abstract: This work presents a review of the current state of research in data-driven turbulence closure modeling. It offers a perspective on the challenges and open issues, but also on the advantages and promises of machine learning methods applied to parameter estimation, model identification, closure term reconstruction and beyond, mostly from the perspective of Large Eddy Simulation and related techniqu… ▽ More This work presents a review of the current state of research in data-driven turbulence closure modeling. It offers a perspective on the challenges and open issues, but also on the advantages and promises of machine learning methods applied to parameter estimation, model identification, closure term reconstruction and beyond, mostly from the perspective of Large Eddy Simulation and related techniques. We stress that consistency of the training data, the model, the underlying physics and the discretization is a key issue that needs to be considered for a successful ML-augmented modeling strategy. In order to make the discussion useful for non-experts in either field, we introduce both the modeling problem in turbulence as well as the prominent ML paradigms and methods in a concise and self-consistent manner. Following, we present a survey of the current data-driven model concepts and methods, highlight important developments and put them into the context of the discussed challenges. △ Less

Submitted 23 October, 2020; originally announced October 2020.

Comments: 37 pages, 12 figures

Journal ref: GAMM-Mitteilungen 44 (2021) e202100002

A machine learning framework for LES closure terms

Authors: Marius Kurz, Andrea Beck

Abstract: In the present work, we explore the capability of artificial neural networks (ANN) to predict the closure terms for large eddy simulations (LES) solely from coarse-scale data. To this end, we derive a consistent framework for LES closure models, with special emphasis laid upon the incorporation of implicit discretization-based filters and numerical approximation errors. We investigate implicit fil… ▽ More In the present work, we explore the capability of artificial neural networks (ANN) to predict the closure terms for large eddy simulations (LES) solely from coarse-scale data. To this end, we derive a consistent framework for LES closure models, with special emphasis laid upon the incorporation of implicit discretization-based filters and numerical approximation errors. We investigate implicit filter types, which are inspired by the solution representation of discontinuous Galerkin and finite volume schemes and mimic the behaviour of the discretization operator, and a global Fourier cutoff filter as a representative of a typical explicit LES filter. Within the perfect LES framework, we compute the exact closure terms for the different LES filter functions from direct numerical simulation results of decaying homogeneous isotropic turbulence. Multiple ANN with a multilayer perceptron (MLP) or a gated recurrent unit (GRU) architecture are trained to predict the computed closure terms solely from coarse-scale input data. For the given application, the GRU architecture clearly outperforms the MLP networks in terms of accuracy, whilst reaching up to 99.9% cross-correlation between the networks' predictions and the exact closure terms for all considered filter functions. The GRU networks are also shown to generalize well across different LES filters and resolutions. The present study can thus be seen as a starting point for the investigation of data-based modeling approaches for LES, which not only include the physical closure terms, but account for the discretization effects in implicitly filtered LES as well. △ Less

Submitted 1 October, 2020; originally announced October 2020.

Journal ref: Electronic transactions on numerical analysis ETNA 56 (2022) 117-137

An Efficient Sliding Mesh Interface Method for High-Order Discontinuous Galerkin Schemes

Authors: Jakob Dürrwächter, Marius Kurz, Patrick Kopper, Daniel Kempf, Claus-Dieter Munz, Andrea Beck

Abstract: Sliding meshes are a powerful method to treat deformed domains in computational fluid dynamics, where different parts of the domain are in relative motion. In this paper, we present an efficient implementation of a sliding mesh method into a discontinuous Galerkin compressible Navier-Stokes solver and its application to a large eddy simulation of a 1-1/2 stage turbine. The method is based on the m… ▽ More Sliding meshes are a powerful method to treat deformed domains in computational fluid dynamics, where different parts of the domain are in relative motion. In this paper, we present an efficient implementation of a sliding mesh method into a discontinuous Galerkin compressible Navier-Stokes solver and its application to a large eddy simulation of a 1-1/2 stage turbine. The method is based on the mortar method and is high-order accurate. It can handle three-dimensional sliding mesh interfaces with various interface shapes. For plane interfaces, which are the most common case, conservativity and free-stream preservation are ensured. We put an emphasis on efficient parallel implementation. Our implementation generates little computational and storage overhead. Inter-node communication via MPI in a dynamically changing mesh topology is reduced to a bare minimum by ensuring a priori information about communication partners and data sorting. We provide performance and scaling results showing the capability of the implementation strategy. Apart from analytical validation computations and convergence results, we present a wall-resolved implicit LES of the 1-1/2 stage Aachen turbine test case as a large scale practical application example. △ Less

Submitted 10 August, 2020; originally announced August 2020.

Journal ref: Computers & Fluids 217 (2021) 104825

Large scale three-dimensional manufacturing tolerant stress-constrained topology optimization

Authors: Gustavo Assis da Silva, Niels Aage, André Teófilo Beck, Ole Sigmund

Abstract: In topology optimization, the treatment of stress constraints for very large scale problems has so far not been tractable due to the failure of robust agglomeration methods, i.e. their inability to accurately handle the locality of the stress constraints. This paper presents a three-dimensional design methodology that alleviates this shortcoming using both deterministic and robust problem formulat… ▽ More In topology optimization, the treatment of stress constraints for very large scale problems has so far not been tractable due to the failure of robust agglomeration methods, i.e. their inability to accurately handle the locality of the stress constraints. This paper presents a three-dimensional design methodology that alleviates this shortcoming using both deterministic and robust problem formulations. The robust formulation, based on the three-field density projection approach, is extended to handle manufacturing uncertainty in three-dimensional stress-constrained problems. Several numerical examples are solved and further post-processed with body-fitted meshes using commercial software. The numerical investigations demonstrate that: (1) the employed solution approach based on the augmented Lagrangian method is able to handle large problems, with hundreds of millions of stress constraints; (2) if appropriate interpolation parameters are adopted, voxel-based (fixed grid) models can be used to compute von Mises stresses with excellent accuracy; and (3) in order to ensure manufacturing tolerance in three-dimensional stress-constrained topology optimization, a combination of double filtering and more than three realizations may be required. △ Less

Submitted 23 June, 2020; originally announced June 2020.

Journal ref: International Journal for Numerical Methods in Engineering, 122 (2021) 548-578

An embedded system for the automated generation of labeled plant images to enable machine learning applications in agriculture

Authors: Michael A. Beck, Chen-Yi Liu, Christopher P. Bidinosti, Christopher J. Henry, Cara M. Godee, Manisha Ajmani

Abstract: A lack of sufficient training data, both in terms of variety and quantity, is often the bottleneck in the development of machine learning (ML) applications in any domain. For agricultural applications, ML-based models designed to perform tasks such as autonomous plant classification will typically be coupled to just one or perhaps a few plant species. As a consequence, each crop-specific task is v… ▽ More A lack of sufficient training data, both in terms of variety and quantity, is often the bottleneck in the development of machine learning (ML) applications in any domain. For agricultural applications, ML-based models designed to perform tasks such as autonomous plant classification will typically be coupled to just one or perhaps a few plant species. As a consequence, each crop-specific task is very likely to require its own specialized training data, and the question of how to serve this need for data now often overshadows the more routine exercise of actually training such models. To tackle this problem, we have developed an embedded robotic system to automatically generate and label large datasets of plant images for ML applications in agriculture. The system can image plants from virtually any angle, thereby ensuring a wide variety of data; and with an imaging rate of up to one image per second, it can produce lableled datasets on the scale of thousands to tens of thousands of images per day. As such, this system offers an important alternative to time- and cost-intensive methods of manual generation and labeling. Furthermore, the use of a uniform background made of blue keying fabric enables additional image processing techniques such as background replacement and plant segmentation. It also helps in the training process, essentially forcing the model to focus on the plant features and eliminating random correlations. To demonstrate the capabilities of our system, we generated a dataset of over 34,000 labeled images, with which we trained an ML-model to distinguish grasses from non-grasses in test data from a variety of sources. We now plan to generate much larger datasets of Canadian crop plants and weeds that will be made publicly available in the hope of further enabling ML applications in the agriculture sector. △ Less

Submitted 1 April, 2021; v1 submitted 1 June, 2020; originally announced June 2020.

Comments: 35 pages, 8 figures, Preprint submitted to PLoS One

Proactive Aging Mitigation in CGRAs through Utilization-Aware Allocation

Authors: Marcelo Brandalero, Bernardo Neuhaus Lignati, Antonio Carlos Schneider Beck, Muhammad Shafique, Michael Hübner

Abstract: Resource balancing has been effectively used to mitigate the long-term aging effects of Negative Bias Temperature Instability (NBTI) in multi-core and Graphics Processing Unit (GPU) architectures. In this work, we investigate this strategy in Coarse-Grained Reconfigurable Arrays (CGRAs) with a novel application-to-CGRA allocation approach. By introducing important extensions to the reconfiguration… ▽ More Resource balancing has been effectively used to mitigate the long-term aging effects of Negative Bias Temperature Instability (NBTI) in multi-core and Graphics Processing Unit (GPU) architectures. In this work, we investigate this strategy in Coarse-Grained Reconfigurable Arrays (CGRAs) with a novel application-to-CGRA allocation approach. By introducing important extensions to the reconfiguration logic and the datapath, we enable the dynamic movement of configurations throughout the fabric and allow overutilized Functional Units (FUs) to recover from stress-induced NBTI aging. Implementing the approach in a resource-constrained state-of-the-art CGRA reveals $2.2\times$ lifetime improvement with negligible performance overheads and less than $10\%$ increase in area. △ Less

Submitted 5 November, 2020; v1 submitted 22 April, 2020; originally announced April 2020.

Comments: Please cite this as: M. Brandalero, B. N. Lignati, A. Carlos Schneider Beck, M. Shafique and M. Hübner, "Proactive Aging Mitigation in CGRAs through Utilization-Aware Allocation," 2020 57th ACM/IEEE Design Automation Conference (DAC), San Francisco, CA, USA, 2020, pp. 1-6, doi: 10.1109/DAC18072.2020.9218586

ACM Class: C.1; B.8

A Neural Network based Shock Detection and Localization Approach for Discontinuous Galerkin Methods

Authors: Andrea D. Beck, Jonas Zeifang, Anna Schwarz, David G. Flad

Abstract: The stable and accurate approximation of discontinuities such as shocks on a finite computational mesh is a challenging task. Detection of shocks or strong discontinuities in the flow solution is typically achieved through a priori troubled cell indicators, which guide the subsequent action of an appropriate shock capturing mechanism. Arriving at a stable and accurate solution often requires empir… ▽ More The stable and accurate approximation of discontinuities such as shocks on a finite computational mesh is a challenging task. Detection of shocks or strong discontinuities in the flow solution is typically achieved through a priori troubled cell indicators, which guide the subsequent action of an appropriate shock capturing mechanism. Arriving at a stable and accurate solution often requires empirically based parameter tuning and adjustments of the indicator settings to the discretization and solution at hand. In this work, we propose to separate the task of shock detection and shock capturing more strongly and aim to develop a shock indicator that is robust, accurate, requires minimal user input and is suitable for high order element-based methods like discontinuous Galerkin and flux reconstruction methods. The novel indicator is learned from analytical data through a supervised learning strategy; its input is given by the high order solution field, its output is an element-local map of the shock position. We use state of the art methods from edge detection in image analysis based on deep convolutional multiscale networks and deep supervision to train the indicators. The resulting networks are then used as black box indicators, showing their robustness and accuracy on well established canonical testcases. All simulations are run ab initio using the developed indicators, showing that they provide also stability during the strongly transient phases. In particular for high order schemes with large cells and considerable inner-cell resolution capabilities, we demonstrate how the additional accurate prediction of the position of the shock front can be exploited to guide inner-element shock capturing strategies. △ Less

Submitted 20 January, 2020; originally announced January 2020.

FLEXI: A high order discontinuous Galerkin framework for hyperbolic-parabolic conservation laws

Authors: Nico Krais, Andrea Beck, Thomas Bolemann, Hannes Frank, David Flad, Gregor Gassner, Florian Hindenlang, Malte Hoffmann, Thomas Kuhn, Matthias Sonntag, Claus-Dieter Munz

Abstract: High order (HO) schemes are attractive candidates for the numerical solution of multiscale problems occurring in fluid dynamics and related disciplines. Among the HO discretization variants, discontinuous Galerkin schemes offer a collection of advantageous features which have lead to a strong increase in interest in them and related formulations in the last decade. The methods have matured suffici… ▽ More High order (HO) schemes are attractive candidates for the numerical solution of multiscale problems occurring in fluid dynamics and related disciplines. Among the HO discretization variants, discontinuous Galerkin schemes offer a collection of advantageous features which have lead to a strong increase in interest in them and related formulations in the last decade. The methods have matured sufficiently to be of practical use for a range of problems, for example in direct numerical and large eddy simulation of turbulence. However, in order to take full advantage of the potential benefits of these methods, all steps in the simulation chain must be designed and executed with HO in mind. Especially in this area, many commercially available closed-source solutions fall short. In this work, we therefor present the FLEXI framework, a HO consistent, open-source simulation tool chain for solving the compressible Navier-Stokes equations in a high performance computing setting. We describe the numerical algorithms and implementation details and give an overview of the features and capabilities of all parts of the framework. Beyond these technical details, we also discuss the important, but often overlooked issues of code stability, reproducibility and user-friendliness. The benefits gained by developing an open-source framework are discussed, with a particular focus on usability for the open-source community. We close with sample applications that demonstrate the wide range of use cases and the expandability of FLEXI and an overview of current and future developments. △ Less

Submitted 7 October, 2019; originally announced October 2019.

A large eddy simulation method for DGSEM using non-linearly optimized relaxation filters

Authors: David G Flad, Andrea D Beck, Philipp Guthke

Abstract: In this paper, we apply a specifically designed dissipative spatial filter as sub-grid scale model within the increasingly popular discontinuous Galerkin methods and the closely related flux reconstruction high order methods for large eddy simulation. The parameters of the filter kernel are optimized with data obtained from direct numerical simulation, that is filtered and used as a ground truth t… ▽ More In this paper, we apply a specifically designed dissipative spatial filter as sub-grid scale model within the increasingly popular discontinuous Galerkin methods and the closely related flux reconstruction high order methods for large eddy simulation. The parameters of the filter kernel are optimized with data obtained from direct numerical simulation, that is filtered and used as a ground truth to fit the overall kinetic energy and dissipation rate over time. The optimization is carried out for polynomial degree 3 to 10. The optimal kernels are rigorously tested in the limit of infinite Reynolds number flows (HIT and Taylor Green Vortex flow). Additionally, a brief extension to plane turbulent channel flow is given. Besides the overall good performance, the method is especially attractive in combination with wall modeled LES, because it avoids the computation of second order derivatives for very high Reynolds number flows. △ Less

Submitted 28 January, 2020; v1 submitted 31 May, 2019; originally announced May 2019.

Predicting breast tumor proliferation from whole-slide images: the TUPAC16 challenge

Authors: Mitko Veta, Yujing J. Heng, Nikolas Stathonikos, Babak Ehteshami Bejnordi, Francisco Beca, Thomas Wollmann, Karl Rohr, Manan A. Shah, Dayong Wang, Mikael Rousson, Martin Hedlund, David Tellez, Francesco Ciompi, Erwan Zerhouni, David Lanyi, Matheus Viana, Vassili Kovalev, Vitali Liauchuk, Hady Ahmady Phoulady, Talha Qaiser, Simon Graham, Nasir Rajpoot, Erik Sjöblom, Jesper Molin, Kyunghyun Paeng , et al. (8 additional authors not shown)

Abstract: Tumor proliferation is an important biomarker indicative of the prognosis of breast cancer patients. Assessment of tumor proliferation in a clinical setting is highly subjective and labor-intensive task. Previous efforts to automate tumor proliferation assessment by image analysis only focused on mitosis detection in predefined tumor regions. However, in a real-world scenario, automatic mitosis de… ▽ More Tumor proliferation is an important biomarker indicative of the prognosis of breast cancer patients. Assessment of tumor proliferation in a clinical setting is highly subjective and labor-intensive task. Previous efforts to automate tumor proliferation assessment by image analysis only focused on mitosis detection in predefined tumor regions. However, in a real-world scenario, automatic mitosis detection should be performed in whole-slide images (WSIs) and an automatic method should be able to produce a tumor proliferation score given a WSI as input. To address this, we organized the TUmor Proliferation Assessment Challenge 2016 (TUPAC16) on prediction of tumor proliferation scores from WSIs. The challenge dataset consisted of 500 training and 321 testing breast cancer histopathology WSIs. In order to ensure fair and independent evaluation, only the ground truth for the training dataset was provided to the challenge participants. The first task of the challenge was to predict mitotic scores, i.e., to reproduce the manual method of assessing tumor proliferation by a pathologist. The second task was to predict the gene expression based PAM50 proliferation scores from the WSI. The best performing automatic method for the first task achieved a quadratic-weighted Cohen's kappa score of $κ$ = 0.567, 95% CI [0.464, 0.671] between the predicted scores and the ground truth. For the second task, the predictions of the top method had a Spearman's correlation coefficient of r = 0.617, 95% CI [0.581 0.651] with the ground truth. This was the first study that investigated tumor proliferation assessment from WSIs. The achieved results are promising given the difficulty of the tasks and weakly-labelled nature of the ground truth. However, further research is needed to improve the practical utility of image analysis methods for this task. △ Less

Submitted 29 March, 2019; v1 submitted 22 July, 2018; originally announced July 2018.

Comments: Overview paper of the TUPAC16 challenge: http://tupac.tue-image.nl/

Deep Neural Networks for Data-Driven Turbulence Models

Authors: Andrea D. Beck, David G. Flad, Claus-Dieter Munz

Abstract: In this work, we present a novel data-based approach to turbulence modelling for Large Eddy Simulation (LES) by artificial neural networks. We define the exact closure terms including the discretization operators and generate training data from direct numerical simulations of decaying homogeneous isotropic turbulence. We design and train artificial neural networks based on local convolution filter… ▽ More In this work, we present a novel data-based approach to turbulence modelling for Large Eddy Simulation (LES) by artificial neural networks. We define the exact closure terms including the discretization operators and generate training data from direct numerical simulations of decaying homogeneous isotropic turbulence. We design and train artificial neural networks based on local convolution filters to predict the underlying unknown non-linear mapping from the coarse grid quantities to the closure terms without a priori assumptions. All investigated networks are able to generalize from the data and learn approximations with a cross correlation of up to 47% and even 73% for the inner elements, leading to the conclusion that the current training success is data-bound. We further show that selecting both the coarse grid primitive variables as well as the coarse grid LES operator as input features significantly improves training results. Finally, we construct a stable and accurate LES model from the learned closure terms. Therefore, we translate the model predictions into a data-adaptive, pointwise eddy viscosity closure and show that the resulting LES scheme performs well compared to current state of the art approaches. This work represents the starting point for further research into data-driven, universal turbulence models. △ Less

Submitted 15 June, 2018; v1 submitted 10 June, 2018; originally announced June 2018.

On Multi-Layer Basis Pursuit, Efficient Algorithms and Convolutional Neural Networks

Authors: Jeremias Sulam, Aviad Aberdam, Amir Beck, Michael Elad

Abstract: Parsimonious representations are ubiquitous in modeling and processing information. Motivated by the recent Multi-Layer Convolutional Sparse Coding (ML-CSC) model, we herein generalize the traditional Basis Pursuit problem to a multi-layer setting, introducing similar sparse enforcing penalties at different representation layers in a symbiotic relation between synthesis and analysis sparse priors.… ▽ More Parsimonious representations are ubiquitous in modeling and processing information. Motivated by the recent Multi-Layer Convolutional Sparse Coding (ML-CSC) model, we herein generalize the traditional Basis Pursuit problem to a multi-layer setting, introducing similar sparse enforcing penalties at different representation layers in a symbiotic relation between synthesis and analysis sparse priors. We explore different iterative methods to solve this new problem in practice, and we propose a new Multi-Layer Iterative Soft Thresholding Algorithm (ML-ISTA), as well as a fast version (ML-FISTA). We show that these nested first order algorithms converge, in the sense that the function value of near-fixed points can get arbitrarily close to the solution of the original problem. We further show how these algorithms effectively implement particular recurrent convolutional neural networks (CNNs) that generalize feed-forward ones without introducing any parameters. We present and analyze different architectures resulting unfolding the iterations of the proposed pursuit algorithms, including a new Learned ML-ISTA, providing a principled way to construct deep recurrent CNNs. Unlike other similar constructions, these architectures unfold a global pursuit holistically for the entire network. We demonstrate the emerging constructions in a supervised learning setting, consistently improving the performance of classical CNNs while maintaining the number of parameters constant. △ Less

Submitted 21 November, 2018; v1 submitted 2 June, 2018; originally announced June 2018.

On Fienup Methods for Regularized Phase Retrieval

Authors: Edouard Pauwels, Amir Beck, Yonina C. Eldar, Shoham Sabach

Abstract: Alternating minimization, or Fienup methods, have a long history in phase retrieval. We provide new insights related to the empirical and theoretical analysis of these algorithms when used with Fourier measurements and combined with convex priors. In particular, we show that Fienup methods can be viewed as performing alternating minimization on a regularized nonconvex least-squares problem with re… ▽ More Alternating minimization, or Fienup methods, have a long history in phase retrieval. We provide new insights related to the empirical and theoretical analysis of these algorithms when used with Fourier measurements and combined with convex priors. In particular, we show that Fienup methods can be viewed as performing alternating minimization on a regularized nonconvex least-squares problem with respect to amplitude measurements. We then prove that under mild additional structural assumptions on the prior (semi-algebraicity), the sequence of signal estimates has a smooth convergent behaviour towards a critical point of the nonconvex regularized least-squares objective. Finally, we propose an extension to Fienup techniques, based on a projected gradient descent interpretation and acceleration using inertial terms. We demonstrate experimentally that this modification combined with an $\ell_1$ prior constitutes a competitive approach for sparse phase retrieval. △ Less

Submitted 27 February, 2017; originally announced February 2017.

Deep learning-based assessment of tumor-associated stroma for diagnosing breast cancer in histopathology images

Authors: Babak Ehteshami Bejnordi, Jimmy Linz, Ben Glass, Maeve Mullooly, Gretchen L Gierach, Mark E Sherman, Nico Karssemeijer, Jeroen van der Laak, Andrew H Beck

Abstract: Diagnosis of breast carcinomas has so far been limited to the morphological interpretation of epithelial cells and the assessment of epithelial tissue architecture. Consequently, most of the automated systems have focused on characterizing the epithelial regions of the breast to detect cancer. In this paper, we propose a system for classification of hematoxylin and eosin (H&E) stained breast speci… ▽ More Diagnosis of breast carcinomas has so far been limited to the morphological interpretation of epithelial cells and the assessment of epithelial tissue architecture. Consequently, most of the automated systems have focused on characterizing the epithelial regions of the breast to detect cancer. In this paper, we propose a system for classification of hematoxylin and eosin (H&E) stained breast specimens based on convolutional neural networks that primarily targets the assessment of tumor-associated stroma to diagnose breast cancer patients. We evaluate the performance of our proposed system using a large cohort containing 646 breast tissue biopsies. Our evaluations show that the proposed system achieves an area under ROC of 0.92, demonstrating the discriminative power of previously neglected tumor-associated stroma as a diagnostic biomarker. △ Less

Submitted 19 February, 2017; originally announced February 2017.

Comments: 5 pages, 2 figures, ISBI 2017 Submission

Crowdsourcing scoring of immunohistochemistry images: Evaluating Performance of the Crowd and an Automated Computational Method

Authors: Humayun Irshad, Eun-Yeong Oh, Daniel Schmolze, Liza M Quintana, Laura Collins, Rulla M. Tamimi, Andrew H. Beck

Abstract: The assessment of protein expression in immunohistochemistry (IHC) images provides important diagnostic, prognostic and predictive information for guiding cancer diagnosis and therapy. Manual scoring of IHC images represents a logistical challenge, as the process is labor intensive and time consuming. Since the last decade, computational methods have been developed to enable the application of qua… ▽ More The assessment of protein expression in immunohistochemistry (IHC) images provides important diagnostic, prognostic and predictive information for guiding cancer diagnosis and therapy. Manual scoring of IHC images represents a logistical challenge, as the process is labor intensive and time consuming. Since the last decade, computational methods have been developed to enable the application of quantitative methods for the analysis and interpretation of protein expression in IHC images. These methods have not yet replaced manual scoring for the assessment of IHC in the majority of diagnostic laboratories and in many large-scale research studies. An alternative approach is crowdsourcing the quantification of IHC images to an undefined crowd. The aim of this study is to quantify IHC images for labeling of ER status with two different crowdsourcing approaches, image labeling and nuclei labeling, and compare their performance with automated methods. Crowdsourcing-derived scores obtained greater concordance with the pathologist interpretations for both image labeling and nuclei labeling tasks (83% and 87%), as compared to the pathologist concordance achieved by the automated method (81%) on 5,483 TMA images from 1,909 breast cancer patients. This analysis shows that crowdsourcing the scoring of protein expression in IHC images is a promising new approach for large scale cancer molecular pathology studies. △ Less

Submitted 23 June, 2016; v1 submitted 21 June, 2016; originally announced June 2016.

Deep Learning for Identifying Metastatic Breast Cancer

Authors: Dayong Wang, Aditya Khosla, Rishab Gargeya, Humayun Irshad, Andrew H. Beck

Abstract: The International Symposium on Biomedical Imaging (ISBI) held a grand challenge to evaluate computational systems for the automated detection of metastatic breast cancer in whole slide images of sentinel lymph node biopsies. Our team won both competitions in the grand challenge, obtaining an area under the receiver operating curve (AUC) of 0.925 for the task of whole slide image classification and… ▽ More The International Symposium on Biomedical Imaging (ISBI) held a grand challenge to evaluate computational systems for the automated detection of metastatic breast cancer in whole slide images of sentinel lymph node biopsies. Our team won both competitions in the grand challenge, obtaining an area under the receiver operating curve (AUC) of 0.925 for the task of whole slide image classification and a score of 0.7051 for the tumor localization task. A pathologist independently reviewed the same images, obtaining a whole slide image classification AUC of 0.966 and a tumor localization score of 0.733. Combining our deep learning system's predictions with the human pathologist's diagnoses increased the pathologist's AUC to 0.995, representing an approximately 85 percent reduction in human error rate. These results demonstrate the power of using deep learning to produce significant improvements in the accuracy of pathological diagnoses. △ Less

Submitted 18 June, 2016; originally announced June 2016.

Smoothing and Decomposition for Analysis Sparse Recovery

Authors: Zhao Tan, Yonina C. Eldar, Amir Beck, Arye Nehorai

Abstract: We consider algorithms and recovery guarantees for the analysis sparse model in which the signal is sparse with respect to a highly coherent frame. We consider the use of a monotone version of the fast iterative shrinkage- thresholding algorithm (MFISTA) to solve the analysis sparse recovery problem. Since the proximal operator in MFISTA does not have a closed-form solution for the analysis model,… ▽ More We consider algorithms and recovery guarantees for the analysis sparse model in which the signal is sparse with respect to a highly coherent frame. We consider the use of a monotone version of the fast iterative shrinkage- thresholding algorithm (MFISTA) to solve the analysis sparse recovery problem. Since the proximal operator in MFISTA does not have a closed-form solution for the analysis model, it cannot be applied directly. Instead, we examine two alternatives based on smoothing and decomposition transformations that relax the original sparse recovery problem, and then implement MFISTA on the relaxed formulation. We refer to these two methods as smoothing-based and decomposition-based MFISTA. We analyze the convergence of both algorithms, and establish that smoothing- based MFISTA converges more rapidly when applied to general nonsmooth optimization problems. We then derive a performance bound on the reconstruction error using these techniques. The bound proves that our methods can recover a signal sparse in a redundant tight frame when the measurement matrix satisfies a properly adapted restricted isometry property. Numerical examples demonstrate the performance of our methods and show that smoothing-based MFISTA converges faster than the decomposition-based alternative in real applications, such as MRI image reconstruction. △ Less

Submitted 14 January, 2014; v1 submitted 22 August, 2013; originally announced August 2013.

Comments: Submitted on Aug 22th, 2013; Updated on Dec 1st, 2013

GESPAR: Efficient Phase Retrieval of Sparse Signals

Authors: Yoav Shechtman, Amir Beck, Yonina C. Eldar

Abstract: We consider the problem of phase retrieval, namely, recovery of a signal from the magnitude of its Fourier transform, or of any other linear transform. Due to the loss of the Fourier phase information, this problem is ill-posed. Therefore, prior information on the signal is needed in order to enable its recovery. In this work we consider the case in which the signal is known to be sparse, i.e., it… ▽ More We consider the problem of phase retrieval, namely, recovery of a signal from the magnitude of its Fourier transform, or of any other linear transform. Due to the loss of the Fourier phase information, this problem is ill-posed. Therefore, prior information on the signal is needed in order to enable its recovery. In this work we consider the case in which the signal is known to be sparse, i.e., it consists of a small number of nonzero elements in an appropriate basis. We propose a fast local search method for recovering a sparse signal from measurements of its Fourier transform (or other linear transform) magnitude which we refer to as GESPAR: GrEedy Sparse PhAse Retrieval. Our algorithm does not require matrix lifting, unlike previous approaches, and therefore is potentially suitable for large scale problems such as images. Simulation results indicate that GESPAR is fast and more accurate than existing techniques in a variety of settings. △ Less

Submitted 21 September, 2013; v1 submitted 6 January, 2013; originally announced January 2013.

Comments: Generalized to non-Fourier measurements, added 2D simulations, and a theorem for convergence to stationary point

Sparsity Constrained Nonlinear Optimization: Optimality Conditions and Algorithms

Authors: Amir Beck, Yonina C. Eldar

Abstract: This paper treats the problem of minimizing a general continuously differentiable function subject to sparsity constraints. We present and analyze several different optimality criteria which are based on the notions of stationarity and coordinate-wise optimality. These conditions are then used to derive three numerical algorithms aimed at finding points satisfying the resulting optimality criteria… ▽ More This paper treats the problem of minimizing a general continuously differentiable function subject to sparsity constraints. We present and analyze several different optimality criteria which are based on the notions of stationarity and coordinate-wise optimality. These conditions are then used to derive three numerical algorithms aimed at finding points satisfying the resulting optimality criteria: the iterative hard thresholding method and the greedy and partial sparse-simplex methods. The first algorithm is essentially a gradient projection method while the remaining two algorithms are of coordinate descent type. The theoretical convergence of these methods and their relations to the derived optimality conditions are studied. The algorithms and results are illustrated by several numerical examples. △ Less

Submitted 20 March, 2012; originally announced March 2012.

Comments: submitted to SIAM Optimization