Showing 1–40 of 40 results for author: Mayr, A

Ranking evaluation metrics from a group-theoretic perspective

Authors: Chiara Balestra, Andreas Mayr, Emmanuel Müller

Abstract: Confronted with the challenge of identifying the most suitable metric to validate the merits of newly proposed models, the decision-making process is anything but straightforward. Given that comparing rankings introduces its own set of formidable challenges and the likely absence of a universal metric applicable to all scenarios, the scenario does not get any better. Furthermore, metrics designed… ▽ More Confronted with the challenge of identifying the most suitable metric to validate the merits of newly proposed models, the decision-making process is anything but straightforward. Given that comparing rankings introduces its own set of formidable challenges and the likely absence of a universal metric applicable to all scenarios, the scenario does not get any better. Furthermore, metrics designed for specific contexts, such as for Recommender Systems, sometimes extend to other domains without a comprehensive grasp of their underlying mechanisms, resulting in unforeseen outcomes and potential misuses. Complicating matters further, distinct metrics may emphasize different aspects of rankings, frequently leading to seemingly contradictory comparisons of model results and hindering the trustworthiness of evaluations. We unveil these aspects in the domain of ranking evaluation metrics. Firstly, we show instances resulting in inconsistent evaluations, sources of potential mistrust in commonly used metrics; by quantifying the frequency of such disagreements, we prove that these are common in rankings. Afterward, we conceptualize rankings using the mathematical formalism of symmetric groups detaching from possible domains where the metrics have been created; through this approach, we can rigorously and formally establish essential mathematical properties for ranking evaluation metrics, essential for a deeper comprehension of the source of inconsistent evaluations. We conclude with a discussion, connecting our theoretical analysis to the practical applications, highlighting which properties are important in each domain where rankings are commonly evaluated. In conclusion, our analysis sheds light on ranking evaluation metrics, highlighting that inconsistent evaluations should not be seen as a source of mistrust but as the need to carefully choose how to evaluate our models in the future. △ Less

Submitted 14 August, 2024; originally announced August 2024.

Spectroscopic Performance of Detectors for Athena's WFI: Measurements and Simulation

Authors: Johannes Müller-Seidlitz, Robert Andritschke, Valentin Emberger, Michael Bonholzer, Günter Hauser, Peter Lechner, Astrid Mayr, Jonas Reiffers, Anna Schweingruber, Wolfgang Treberspurg

Abstract: The depleted p-channel field effect transistor is the chosen sensor type for the Wide Field Imager of the Athena mission. It will be used in two types of cameras. One will enable observations of a field of view of 40' x 40' by using an array of four 512 x 512 pixel sensors in a 2 x 2 configuration. A second, small one is designed to investigate bright, point-like sources with a time resolution of… ▽ More The depleted p-channel field effect transistor is the chosen sensor type for the Wide Field Imager of the Athena mission. It will be used in two types of cameras. One will enable observations of a field of view of 40' x 40' by using an array of four 512 x 512 pixel sensors in a 2 x 2 configuration. A second, small one is designed to investigate bright, point-like sources with a time resolution of up to 40 microseconds. Sensors of final size, layout, and technology were fabricated, assembled and characterised. Also, first results from the flight production are available and confirm the excellent performance. In order to be able to estimate the future performance of degraded detectors, a simulation was developed that takes into account the non-analytical threshold effects on the basis of measurement results. We present the measurement analysis and the comparison of simulated and measured values as well as first attempts to use the Monte Carlo simulation to predict performance results based on noise measurements. △ Less

Submitted 3 July, 2024; originally announced July 2024.

Comments: SPIE Space Telescopes and Instrumentation 2024: Ultraviolet to Gamma Ray

Low temperature proton irradiation with DEPFETs for Athena's Wide Field Imager

Authors: Valentin Emberger, Robert Andritschke, Parviz Azhdarzadeh, Günter Hauser, Astrid Mayr, Johannes Müller-Seidlitz, Abbas Rezaei, Wolfgang Treberer-Treberspurg

Abstract: The Wide Field Imager (WFI), one of two instruments on ESA's next large X-ray mission Athena, is designed for imaging spectroscopy of X-rays in the range of 0.2 to 15 keV with a large field of view and high count rate capability. The focal plane consists of back-illuminated DEPFET (Depleted p-channel field effect transistor) sensors that have a high radiation tolerance and provide a near Fano-limi… ▽ More The Wide Field Imager (WFI), one of two instruments on ESA's next large X-ray mission Athena, is designed for imaging spectroscopy of X-rays in the range of 0.2 to 15 keV with a large field of view and high count rate capability. The focal plane consists of back-illuminated DEPFET (Depleted p-channel field effect transistor) sensors that have a high radiation tolerance and provide a near Fano-limited energy resolution. To achieve this, a very low noise readout is required, about 3 electrons ENC at beginning of life is foreseen. This makes the device very susceptible to any radiation induced worsening of the readout noise. The main mechanism of degradation will be the increase of dark current due to displacement damage caused primarily by high energy protons. To study the expected performance degradation, a prototype detector module with fully representative pixel layout and fabrication technology was irradiated with 62.4 MeV protons at the accelerator facility MedAustron in Wiener Neustadt. A total dose equivalent to 3.3 $\textstyle{10^{9}}$ 10-MeV protons/$\mathrm{cm^{2}}$ was applied in two steps. During, in-between and after the irradiations the detector remained at the operating temperature of 213 K and was fully biased and operated. Data was recorded to analyze the signal of all incident particles. We report on the increase of dark current after the irradiation and present the current related damage rate at 213 K. The effect of low temperature annealing at 213 K , 236 K, 253 K, 273 K, and 289 K is presented. △ Less

Submitted 2 July, 2024; v1 submitted 28 June, 2024; originally announced June 2024.

Comments: to be published in SPIE Proceeding of Astronomical Telescopes + Instrumentation, 2024

Enhanced variable selection for boosting sparser and less complex models in distributional copula regression

Authors: Annika Strömer, Nadja Klein, Christian Staerk, Florian Faschingbauer, Hannah Klinkhammer, Andreas Mayr

Abstract: Structured additive distributional copula regression allows to model the joint distribution of multivariate outcomes by relating all distribution parameters to covariates. Estimation via statistical boosting enables accounting for high-dimensional data and incorporating data-driven variable selection, both of which are useful given the complexity of the model class. However, as known from univaria… ▽ More Structured additive distributional copula regression allows to model the joint distribution of multivariate outcomes by relating all distribution parameters to covariates. Estimation via statistical boosting enables accounting for high-dimensional data and incorporating data-driven variable selection, both of which are useful given the complexity of the model class. However, as known from univariate (distributional) regression, the standard boosting algorithm tends to select too many variables with minor importance, particularly in settings with large sample sizes, leading to complex models with difficult interpretation. To counteract this behavior and to avoid selecting base-learners with only a negligible impact, we combined the ideas of probing, stability selection and a new deselection approach with statistical boosting for distributional copula regression. In a simulation study and an application to the joint modelling of weight and length of newborns, we found that all proposed methods enhance variable selection by reducing the number of false positives. However, only stability selection and the deselection approach yielded similar predictive performance to classical boosting. Finally, the deselection approach is better scalable to larger datasets and led to a competitive predictive performance, which we further illustrated in a genomic cohort study from the UK Biobank by modelling the joint genetic predisposition for two phenotypes. △ Less

Submitted 6 June, 2024; originally announced June 2024.

Deep Gaussian mixture model for unsupervised image segmentation

Authors: Matthias Schwab, Agnes Mayr, Markus Haltmeier

Abstract: The recent emergence of deep learning has led to a great deal of work on designing supervised deep semantic segmentation algorithms. As in many tasks sufficient pixel-level labels are very difficult to obtain, we propose a method which combines a Gaussian mixture model (GMM) with unsupervised deep learning techniques. In the standard GMM the pixel values with each sub-region are modelled by a Gaus… ▽ More The recent emergence of deep learning has led to a great deal of work on designing supervised deep semantic segmentation algorithms. As in many tasks sufficient pixel-level labels are very difficult to obtain, we propose a method which combines a Gaussian mixture model (GMM) with unsupervised deep learning techniques. In the standard GMM the pixel values with each sub-region are modelled by a Gaussian distribution. In order to identify the different regions, the parameter vector that minimizes the negative log-likelihood (NLL) function regarding the GMM has to be approximated. For this task, usually iterative optimization methods such as the expectation-maximization (EM) algorithm are used. In this paper, we propose to estimate these parameters directly from the image using a convolutional neural network (CNN). We thus change the iterative procedure in the EM algorithm replacing the expectation-step by a gradient-step with regard to the networks parameters. This means that the network is trained to minimize the NLL function of the GMM which comes with at least two advantages. As once trained, the network is able to predict label probabilities very quickly compared with time consuming iterative optimization methods. Secondly, due to the deep image prior our method is able to partially overcome one of the main disadvantages of GMM, which is not taking into account correlation between neighboring pixels, as it assumes independence between them. We demonstrate the advantages of our method in various experiments on the example of myocardial infarct segmentation on multi-sequence MRI images. △ Less

Submitted 18 April, 2024; originally announced April 2024.

A Balanced Statistical Boosting Approach for GAMLSS via New Step Lengths

Authors: Alexandra Daub, Andreas Mayr, Boyao Zhang, Elisabeth Bergherr

Abstract: Component-wise gradient boosting algorithms are popular for their intrinsic variable selection and implicit regularization, which can be especially beneficial for very flexible model classes. When estimating generalized additive models for location, scale and shape (GAMLSS) by means of a component-wise gradient boosting algorithm, an important part of the estimation procedure is to determine the r… ▽ More Component-wise gradient boosting algorithms are popular for their intrinsic variable selection and implicit regularization, which can be especially beneficial for very flexible model classes. When estimating generalized additive models for location, scale and shape (GAMLSS) by means of a component-wise gradient boosting algorithm, an important part of the estimation procedure is to determine the relative complexity of the submodels corresponding to the different distribution parameters. Existing methods either suffer from a computationally expensive tuning procedure or can be biased by structural differences in the negative gradients' sizes, which, if encountered, lead to imbalances between the different submodels. Shrunk optimal step lengths have been suggested to replace the typical small fixed step lengths for a non-cyclical boosting algorithm limited to a Gaussian response variable in order to address this issue. In this article, we propose a new adaptive step length approach that accounts for the relative size of the fitted base-learners to ensure a natural balance between the different submodels. The new balanced boosting approach thus represents a computationally efficient and easily generalizable alternative to shrunk optimal step lengths. We implemented the balanced non-cyclical boosting algorithm for a Gaussian, a negative binomial as well as a Weibull distributed response variable and demonstrate the competitive performance of the new adaptive step length approach by means of a simulation study, in the analysis of count data modeling the number of doctor's visits as well as for survival data in an oncological trial. △ Less

Submitted 12 April, 2024; originally announced April 2024.

Comments: 34 pages, 26 figures

VN-EGNN: E(3)-Equivariant Graph Neural Networks with Virtual Nodes Enhance Protein Binding Site Identification

Authors: Florian Sestak, Lisa Schneckenreiter, Johannes Brandstetter, Sepp Hochreiter, Andreas Mayr, Günter Klambauer

Abstract: Being able to identify regions within or around proteins, to which ligands can potentially bind, is an essential step to develop new drugs. Binding site identification methods can now profit from the availability of large amounts of 3D structures in protein structure databases or from AlphaFold predictions. Current binding site identification methods heavily rely on graph neural networks (GNNs), u… ▽ More Being able to identify regions within or around proteins, to which ligands can potentially bind, is an essential step to develop new drugs. Binding site identification methods can now profit from the availability of large amounts of 3D structures in protein structure databases or from AlphaFold predictions. Current binding site identification methods heavily rely on graph neural networks (GNNs), usually designed to output E(3)-equivariant predictions. Such methods turned out to be very beneficial for physics-related tasks like binding energy or motion trajectory prediction. However, the performance of GNNs at binding site identification is still limited potentially due to the lack of dedicated nodes that model hidden geometric entities, such as binding pockets. In this work, we extend E(n)-Equivariant Graph Neural Networks (EGNNs) by adding virtual nodes and applying an extended message passing scheme. The virtual nodes in these graphs are dedicated quantities to learn representations of binding sites, which leads to improved predictive performance. In our experiments, we show that our proposed method VN-EGNN sets a new state-of-the-art at locating binding site centers on COACH420, HOLO4K and PDBbind2020. △ Less

Submitted 10 April, 2024; originally announced April 2024.

GNN-VPA: A Variance-Preserving Aggregation Strategy for Graph Neural Networks

Authors: Lisa Schneckenreiter, Richard Freinschlag, Florian Sestak, Johannes Brandstetter, Günter Klambauer, Andreas Mayr

Abstract: Graph neural networks (GNNs), and especially message-passing neural networks, excel in various domains such as physics, drug discovery, and molecular modeling. The expressivity of GNNs with respect to their ability to discriminate non-isomorphic graphs critically depends on the functions employed for message aggregation and graph-level readout. By applying signal propagation theory, we propose a v… ▽ More Graph neural networks (GNNs), and especially message-passing neural networks, excel in various domains such as physics, drug discovery, and molecular modeling. The expressivity of GNNs with respect to their ability to discriminate non-isomorphic graphs critically depends on the functions employed for message aggregation and graph-level readout. By applying signal propagation theory, we propose a variance-preserving aggregation function (VPA) that maintains expressivity, but yields improved forward and backward dynamics. Experiments demonstrate that VPA leads to increased predictive performance for popular GNN architectures as well as improved learning dynamics. Our results could pave the way towards normalizer-free or self-normalizing GNNs. △ Less

Submitted 7 March, 2024; originally announced March 2024.

Comments: Accepted at ICLR 2024 (Tiny Papers Track)

Boosting Distributional Copula Regression for Bivariate Binary, Discrete and Mixed Responses

Authors: Guillermo Briseño Sanchez, Nadja Klein, Hannah Klinkhammer, Andreas Mayr

Abstract: Motivated by challenges in the analysis of biomedical data and observational studies, we develop statistical boosting for the general class of bivariate distributional copula regression with arbitrary marginal distributions, which is suited to model binary, count, continuous or mixed outcomes. In our framework, the joint distribution of arbitrary, bivariate responses is modelled through a parametr… ▽ More Motivated by challenges in the analysis of biomedical data and observational studies, we develop statistical boosting for the general class of bivariate distributional copula regression with arbitrary marginal distributions, which is suited to model binary, count, continuous or mixed outcomes. In our framework, the joint distribution of arbitrary, bivariate responses is modelled through a parametric copula. To arrive at a model for the entire conditional distribution, not only the marginal distribution parameters but also the copula parameters are related to covariates through additive predictors. We suggest efficient and scalable estimation by means of an adapted component-wise gradient boosting algorithm with statistical models as base-learners. A key benefit of boosting as opposed to classical likelihood or Bayesian estimation is the implicit data-driven variable selection mechanism as well as shrinkage without additional input or assumptions from the analyst. To the best of our knowledge, our implementation is the only one that combines a wide range of covariate effects, marginal distributions, copula functions, and implicit data-driven variable selection. We showcase the versatility of our approach on data from genetic epidemiology, healthcare utilization and childhood undernutrition. Our developments are implemented in the R package gamboostLSS, fostering transparent and reproducible research. △ Less

Submitted 4 March, 2024; originally announced March 2024.

Redundancy-aware unsupervised rankings for collections of gene sets

Authors: Chiara Balestra, Carlo Maj, Emmanuel Müller, Andreas Mayr

Abstract: The biological roles of gene sets are used to group them into collections. These collections are often characterized by being high-dimensional, overlapping, and redundant families of sets, thus precluding a straightforward interpretation and study of their content. Bioinformatics looked for solutions to reduce their dimension or increase their intepretability. One possibility lies in aggregating o… ▽ More The biological roles of gene sets are used to group them into collections. These collections are often characterized by being high-dimensional, overlapping, and redundant families of sets, thus precluding a straightforward interpretation and study of their content. Bioinformatics looked for solutions to reduce their dimension or increase their intepretability. One possibility lies in aggregating overlapping gene sets to create larger pathways, but the modified biological pathways are hardly biologically justifiable. We propose to use importance scores to rank the pathways in the collections studying the context from a set covering perspective. The proposed Shapley values-based scores consider the distribution of the singletons and the size of the sets in the families; Furthermore, a trick allows us to circumvent the usual exponential complexity of Shapley values' computation. Finally, we address the challenge of including a redundancy awareness in the obtained rankings where, in our case, sets are redundant if they show prominent intersections. The rankings can be used to reduce the dimension of collections of gene sets, such that they show lower redundancy and still a high coverage of the genes. We further investigate the impact of our selection on Gene Sets Enrichment Analysis. The proposed method shows a practical utility in bioinformatics to increase the interpretability of the collections of gene sets and a step forward to include redundancy into Shapley values computations. △ Less

Submitted 30 July, 2023; originally announced July 2023.

Comments: arXiv admin note: substantial text overlap with arXiv:2207.12184

Error correcting 2D-3D cascaded network for myocardial infarct scar segmentation on late gadolinium enhancement cardiac magnetic resonance images

Authors: Matthias Schwab, Mathias Pamminger, Christian Kremser, Daniel Obmann, Markus Haltmeier, Agnes Mayr

Abstract: Late gadolinium enhancement (LGE) cardiac magnetic resonance (CMR) imaging is considered the in vivo reference standard for assessing infarct size (IS) and microvascular obstruction (MVO) in ST-elevation myocardial infarction (STEMI) patients. However, the exact quantification of those markers of myocardial infarct severity remains challenging and very time-consuming. As LGE distribution patterns… ▽ More Late gadolinium enhancement (LGE) cardiac magnetic resonance (CMR) imaging is considered the in vivo reference standard for assessing infarct size (IS) and microvascular obstruction (MVO) in ST-elevation myocardial infarction (STEMI) patients. However, the exact quantification of those markers of myocardial infarct severity remains challenging and very time-consuming. As LGE distribution patterns can be quite complex and hard to delineate from the blood pool or epicardial fat, automatic segmentation of LGE CMR images is challenging. In this work, we propose a cascaded framework of two-dimensional and three-dimensional convolutional neural networks (CNNs) which enables to calculate the extent of myocardial infarction in a fully automated way. By artificially generating segmentation errors which are characteristic for 2D CNNs during training of the cascaded framework we are enforcing the detection and correction of 2D segmentation errors and hence improve the segmentation accuracy of the entire method. The proposed method was trained and evaluated in a five-fold cross validation using the training dataset from the EMIDEC challenge. We perform comparative experiments where our framework outperforms state-of-the-art methods of the EMIDEC challenge, as well as 2D and 3D nnU-Net. Furthermore, in extensive ablation studies we show the advantages that come with the proposed error correcting cascaded method. △ Less

Submitted 26 June, 2023; originally announced June 2023.

Spectroscopic performance of flight-like DEPFET sensors for Athena's WFI

Authors: Johannes Müller-Seidlitz, Robert Andritschke, Michael Bonholzer, Valentin Emberger, Günter Hauser, Maximilian Herrmann, Peter Lechner, Astrid Mayr, Julian Oser

Abstract: The Wide Field Imager for the Athena X-ray telescope is composed of two back side illuminated detectors using DEPFET sensors operated in rolling shutter readout mode: A large detector array featuring four sensors with 512x512 pixels each and a small detector that facilitates the high count rate capability of the WFI for the investigation of bright, point-like sources. Both sensors were fabricated… ▽ More The Wide Field Imager for the Athena X-ray telescope is composed of two back side illuminated detectors using DEPFET sensors operated in rolling shutter readout mode: A large detector array featuring four sensors with 512x512 pixels each and a small detector that facilitates the high count rate capability of the WFI for the investigation of bright, point-like sources. Both sensors were fabricated in full size featuring the pixel layout, fabrication technology and readout mode chosen in a preceding prototyping phase. We present the spectroscopic performance of these flight-like detectors for different photon energies in the relevant part of the targeted energy range from 0.2 keV to 15 keV with respect to the timing requirements of the instrument. For 5.9 keV photons generated by an iron-55 source the spectral performance expressed as Full Width at Half Maximum of the emission peak in the spectrum is 126.0 eV for the Large Detector and 129.1 eV for the Fast Detector. A preliminary analysis of the camera's signal chain also allows for a first prediction of the performance in space at the end of the nominal operation phase. △ Less

Submitted 8 August, 2022; originally announced August 2022.

Comments: SPIE Space Telescopes and Instrumentation 2022: Ultraviolet to Gamma Ray

Journal ref: Proc. SPIE 12181, Space Telescopes and Instrumentation 2022: Ultraviolet to Gamma Ray, 121813W (31 August 2022)

Redundancy-aware unsupervised ranking based on game theory -- application to gene enrichment analysis

Authors: Chiara Balestra, Carlo Maj, Emmanuel Mueller, Andreas Mayr

Abstract: Gene set collections are a common ground to study the enrichment of genes for specific phenotypic traits. Gene set enrichment analysis aims to identify genes that are over-represented in gene sets collections and might be associated with a specific phenotypic trait. However, as this involves a massive number of hypothesis testing, it is often questionable whether a pre-processing step to reduce ge… ▽ More Gene set collections are a common ground to study the enrichment of genes for specific phenotypic traits. Gene set enrichment analysis aims to identify genes that are over-represented in gene sets collections and might be associated with a specific phenotypic trait. However, as this involves a massive number of hypothesis testing, it is often questionable whether a pre-processing step to reduce gene sets collections' sizes is helpful. Moreover, the often highly overlapping gene sets and the consequent low interpretability of gene sets' collections demand for a reduction of the included gene sets. Inspired by this bioinformatics context, we propose a method to rank sets within a family of sets based on the distribution of the singletons and their size. We obtain sets' importance scores by computing Shapley values without incurring into the usual exponential number of evaluations of the value function. Moreover, we address the challenge of including a redundancy awareness in the rankings obtained where, in our case, sets are redundant if they show prominent intersections. We finally evaluate our approach for gene sets collections; the rankings obtained show low redundancy and high coverage of the genes. The unsupervised nature of the proposed ranking does not allow for an evident increase in the number of significant gene sets for specific phenotypic traits when reducing the size of the collections. However, we believe that the rankings proposed are of use in bioinformatics to increase interpretability of the gene sets collections and a step forward to include redundancy into Shapley values computations. △ Less

Submitted 22 July, 2022; originally announced July 2022.

Boosting Multivariate Structured Additive Distributional Regression Models

Authors: Annika Strömer, Nadja Klein, Christian Staerk, Hannah Klinkhammer, Andreas Mayr

Abstract: We develop a model-based boosting approach for multivariate distributional regression within the framework of generalized additive models for location, scale, and shape. Our approach enables the simultaneous modeling of all distribution parameters of an arbitrary parametric distribution of a multivariate response conditional on explanatory variables, while being applicable to potentially high-dime… ▽ More We develop a model-based boosting approach for multivariate distributional regression within the framework of generalized additive models for location, scale, and shape. Our approach enables the simultaneous modeling of all distribution parameters of an arbitrary parametric distribution of a multivariate response conditional on explanatory variables, while being applicable to potentially high-dimensional data. Moreover, the boosting algorithm incorporates data-driven variable selection, taking various different types of effects into account. As a special merit of our approach, it allows for modelling the association between multiple continuous or discrete outcomes through the relevant covariates. After a detailed simulation study investigating estimation and prediction performance, we demonstrate the full flexibility of our approach in three diverse biomedical applications. The first is based on high-dimensional genomic cohort data from the UK Biobank, considering a bivariate binary response (chronic ischemic heart disease and high cholesterol). Here, we are able to identify genetic variants that are informative for the association between cholesterol and heart disease. The second application considers the demand for health care in Australia with the number of consultations and the number of prescribed medications as a bivariate count response. The third application analyses two dimensions of childhood undernutrition in Nigeria as a bivariate response and we find that the correlation between the two undernutrition scores is considerably different depending on the child's age and the region the child lives in. △ Less

Submitted 18 July, 2022; originally announced July 2022.

Unsupervised Features Ranking via Coalitional Game Theory for Categorical Data

Authors: Chiara Balestra, Florian Huber, Andreas Mayr, Emmanuel Müller

Abstract: Not all real-world data are labeled, and when labels are not available, it is often costly to obtain them. Moreover, as many algorithms suffer from the curse of dimensionality, reducing the features in the data to a smaller set is often of great utility. Unsupervised feature selection aims to reduce the number of features, often using feature importance scores to quantify the relevancy of single f… ▽ More Not all real-world data are labeled, and when labels are not available, it is often costly to obtain them. Moreover, as many algorithms suffer from the curse of dimensionality, reducing the features in the data to a smaller set is often of great utility. Unsupervised feature selection aims to reduce the number of features, often using feature importance scores to quantify the relevancy of single features to the task at hand. These scores can be based only on the distribution of variables and the quantification of their interactions. The previous literature, mainly investigating anomaly detection and clusters, fails to address the redundancy-elimination issue. We propose an evaluation of correlations among features to compute feature importance scores representing the contribution of single features in explaining the dataset's structure. Based on Coalitional Game Theory, our feature importance scores include a notion of redundancy awareness making them a tool to achieve redundancy-free feature selection. We show that the deriving features' selection outperforms competing methods in lowering the redundancy rate while maximizing the information contained in the data. We also introduce an approximated version of the algorithm to reduce the complexity of Shapley values' computations. △ Less

Submitted 17 May, 2022; originally announced May 2022.

Comments: To be published in DaWaK 2022 (accepted). Code available at https://github.com/chiarabales/unsupervised_sv @inproceedings{balestraUSV, author = {Chiara Balestra and Florian Huber and Andreas Mayr and Emmanuel Müller}, title = {Unsupervised Features Ranking via Coalitional Game Theory for Categorical Data}, booktitle = {DaWaK 2022}, year = {2022}}

Boosting Distributional Copula Regression

Authors: Nicolai Hans, Nadja Klein, Florian Faschingbauer, Michael Schneider, Andreas Mayr

Abstract: Capturing complex dependence structures between outcome variables (e.g., study endpoints) is of high relevance in contemporary biomedical data problems and medical research. Distributional copula regression provides a flexible tool to model the joint distribution of multiple outcome variables by disentangling the marginal response distributions and their dependence structure. In a regression setup… ▽ More Capturing complex dependence structures between outcome variables (e.g., study endpoints) is of high relevance in contemporary biomedical data problems and medical research. Distributional copula regression provides a flexible tool to model the joint distribution of multiple outcome variables by disentangling the marginal response distributions and their dependence structure. In a regression setup each parameter of the copula model, i.e. the marginal distribution parameters and the copula dependence parameters, can be related to covariates via structured additive predictors. We propose a framework to fit distributional copula regression models via a model-based boosting algorithm. Model-based boosting is a modern estimation technique that incorporates useful features like an intrinsic variable selection mechanism, parameter shrinkage and the capability to fit regression models in high dimensional data setting, i.e. situations with more covariates than observations. Thus, model-based boosting does not only complement existing Bayesian and maximum-likelihood based estimation frameworks for this model class but rather enables unique intrinsic mechanisms that can be helpful in many applied problems. The performance of our boosting algorithm in the context of copula regression models with continuous margins is evaluated in simulation studies that cover low- and high-dimensional data settings and situations with and without dependence between the responses. Moreover, distributional copula boosting is used to jointly analyze and predict the length and the weight of newborns conditional on sonographic measurements of the fetus before delivery together with other clinical variables. △ Less

Submitted 25 February, 2022; originally announced February 2022.

Deselection of Base-Learners for Statistical Boosting -- with an Application to Distributional Regression

Authors: Annika Strömer, Christian Staerk, Nadja Klein, Leonie Weinhold, Stephanie Titze, Andreas Mayr

Abstract: We present a new procedure for enhanced variable selection for component-wise gradient boosting. Statistical boosting is a computational approach that emerged from machine learning, which allows to fit regression models in the presence of high-dimensional data. Furthermore, the algorithm can lead to data-driven variable selection. In practice, however, the final models typically tend to include to… ▽ More We present a new procedure for enhanced variable selection for component-wise gradient boosting. Statistical boosting is a computational approach that emerged from machine learning, which allows to fit regression models in the presence of high-dimensional data. Furthermore, the algorithm can lead to data-driven variable selection. In practice, however, the final models typically tend to include too many variables in some situations. This occurs particularly for low-dimensional data (p<n), where we observe a slow overfitting behavior of boosting. As a result, more variables get included into the final model without altering the prediction accuracy. Many of these false positives are incorporated with a small coefficient and therefore have a small impact, but lead to a larger model. We try to overcome this issue by giving the algorithm the chance to deselect base-learners with minor importance. We analyze the impact of the new approach on variable selection and prediction performance in comparison to alternative methods including boosting with earlier stopping as well as twin boosting. We illustrate our approach with data of an ongoing cohort study for chronic kidney disease patients, where the most influential predictors for the health-related quality of life measure are selected in a distributional regression approach based on beta regression. △ Less

Submitted 3 February, 2022; originally announced February 2022.

Estimating the course of the COVID-19 pandemic in Germany via spline-based hierarchical modelling of death counts

Authors: Tobias Wistuba, Andreas Mayr, Christian Staerk

Abstract: The effective reproduction number is a key figure to monitor the course of the COVID-19 pandemic. In this study we consider a retrospective modelling approach for estimating the effective reproduction number based on death counts during the first year of the pandemic in Germany. The proposed Bayesian hierarchical model incorporates splines to estimate reproduction numbers flexibly over time while… ▽ More The effective reproduction number is a key figure to monitor the course of the COVID-19 pandemic. In this study we consider a retrospective modelling approach for estimating the effective reproduction number based on death counts during the first year of the pandemic in Germany. The proposed Bayesian hierarchical model incorporates splines to estimate reproduction numbers flexibly over time while adjusting for varying effective infection fatality rates. The approach also provides estimates of dark figures regarding undetected infections over time. Results for Germany illustrate that estimated reproduction numbers based on death counts are often similar to classical estimates based on confirmed cases. However, considering death counts proves to be more robust against shifts in testing policies: during the second wave of infections, classical estimation of the reproduction number suggests a flattening/ decreasing trend of infections following the "lockdown light" in November 2020, while our results indicate that true numbers of infections continued to rise until the "second lockdown" in December 2020. This observation is associated with more stringent testing criteria introduced concurrently with the "lockdown light", which is reflected in subsequently increasing dark figures of infections estimated by our model. These findings illustrate that the retrospective viewpoint can provide additional insights regarding the course of the pandemic. In light of progressive vaccinations, shifting the focus from modelling confirmed cases to reported deaths with the possibility to incorporate effective infection fatality rates might be of increasing relevance for the future surveillance of the pandemic. △ Less

Submitted 6 September, 2021; originally announced September 2021.

Boundary Graph Neural Networks for 3D Simulations

Authors: Andreas Mayr, Sebastian Lehner, Arno Mayrhofer, Christoph Kloss, Sepp Hochreiter, Johannes Brandstetter

Abstract: The abundance of data has given machine learning considerable momentum in natural sciences and engineering, though modeling of physical processes is often difficult. A particularly tough problem is the efficient representation of geometric boundaries. Triangularized geometric boundaries are well understood and ubiquitous in engineering applications. However, it is notoriously difficult to integrat… ▽ More The abundance of data has given machine learning considerable momentum in natural sciences and engineering, though modeling of physical processes is often difficult. A particularly tough problem is the efficient representation of geometric boundaries. Triangularized geometric boundaries are well understood and ubiquitous in engineering applications. However, it is notoriously difficult to integrate them into machine learning approaches due to their heterogeneity with respect to size and orientation. In this work, we introduce an effective theory to model particle-boundary interactions, which leads to our new Boundary Graph Neural Networks (BGNNs) that dynamically modify graph structures to obey boundary conditions. The new BGNNs are tested on complex 3D granular flow processes of hoppers, rotating drums and mixers, which are all standard components of modern industrial machinery but still have complicated geometry. BGNNs are evaluated in terms of computational efficiency as well as prediction accuracy of particle flows and mixing entropies. BGNNs are able to accurately reproduce 3D granular flows within simulation uncertainties over hundreds of thousands of simulation timesteps. Most notably, in our experiments, particles stay within the geometric objects without using handcrafted conditions or restrictions. △ Less

Submitted 20 April, 2023; v1 submitted 21 June, 2021; originally announced June 2021.

Comments: accepted for presentation at the Thirty-Seventh AAAI Conference on Artificial Intelligence (AAAI-23)

Learning 3D Granular Flow Simulations

Authors: Andreas Mayr, Sebastian Lehner, Arno Mayrhofer, Christoph Kloss, Sepp Hochreiter, Johannes Brandstetter

Abstract: Recently, the application of machine learning models has gained momentum in natural sciences and engineering, which is a natural fit due to the abundance of data in these fields. However, the modeling of physical processes from simulation data without first principle solutions remains difficult. Here, we present a Graph Neural Networks approach towards accurate modeling of complex 3D granular flow… ▽ More Recently, the application of machine learning models has gained momentum in natural sciences and engineering, which is a natural fit due to the abundance of data in these fields. However, the modeling of physical processes from simulation data without first principle solutions remains difficult. Here, we present a Graph Neural Networks approach towards accurate modeling of complex 3D granular flow simulation processes created by the discrete element method LIGGGHTS and concentrate on simulations of physical systems found in real world applications like rotating drums and hoppers. We discuss how to implement Graph Neural Networks that deal with 3D objects, boundary conditions, particle - particle, and particle - boundary interactions such that an accurate modeling of relevant physical quantities is made possible. Finally, we compare the machine learning based trajectories to LIGGGHTS trajectories in terms of particle flows and mixing entropies. △ Less

Submitted 4 May, 2021; originally announced May 2021.

Estimating effective infection fatality rates during the course of the COVID-19 pandemic in Germany

Authors: Christian Staerk, Tobias Wistuba, Andreas Mayr

Abstract: The infection fatality rate (IFR) of the Coronavirus Disease 2019 (COVID-19) is one of the most discussed figures in the context of this pandemic. Using German COVID-19 surveillance data and age-group specific IFR estimates from multiple international studies, this work investigates time-dependent variations in effective IFR over the course of the pandemic. Three different methods for estimating (… ▽ More The infection fatality rate (IFR) of the Coronavirus Disease 2019 (COVID-19) is one of the most discussed figures in the context of this pandemic. Using German COVID-19 surveillance data and age-group specific IFR estimates from multiple international studies, this work investigates time-dependent variations in effective IFR over the course of the pandemic. Three different methods for estimating (effective) IFRs are presented: (a) population-averaged IFRs based on the assumption that the infection risk is independent of age and time, (b) effective IFRs based on the assumption that the age distribution of confirmed cases approximately reflects the age distribution of infected individuals, and (c) effective IFRs accounting for age- and time-dependent dark figures of infections. Results show that effective IFRs in Germany are estimated to vary over time, as the age distributions of confirmed cases and estimated infections are changing during the course of the pandemic. In particular during the first and second waves of infections in spring and autumn/winter 2020, there has been a pronounced shift in the age distribution of confirmed cases towards older age groups, resulting in larger effective IFR estimates. The temporary increase in effective IFR during the first wave is estimated to be smaller but still remains when adjusting for age- and time-dependent dark figures. A comparison of effective IFRs with observed CFRs indicates that a substantial fraction of the time-dependent variability in observed mortality can be explained by changes in the age distribution of infections. Furthermore, a vanishing gap between effective IFRs and observed CFRs is apparent after the first infection wave, while a moderately increasing gap can be observed during the second wave. Further research is warranted to obtain timely age-stratified IFR estimates. △ Less

Submitted 21 January, 2021; v1 submitted 4 November, 2020; originally announced November 2020.

Cross-Domain Few-Shot Learning by Representation Fusion

Authors: Thomas Adler, Johannes Brandstetter, Michael Widrich, Andreas Mayr, David Kreil, Michael Kopp, Günter Klambauer, Sepp Hochreiter

Abstract: In order to quickly adapt to new data, few-shot learning aims at learning from few examples, often by using already acquired knowledge. The new data often differs from the previously seen data due to a domain shift, that is, a change of the input-target distribution. While several methods perform well on small domain shifts like new target classes with similar inputs, larger domain shifts are stil… ▽ More In order to quickly adapt to new data, few-shot learning aims at learning from few examples, often by using already acquired knowledge. The new data often differs from the previously seen data due to a domain shift, that is, a change of the input-target distribution. While several methods perform well on small domain shifts like new target classes with similar inputs, larger domain shifts are still challenging. Large domain shifts may result in high-level concepts that are not shared between the original and the new domain, whereas low-level concepts like edges in images might still be shared and useful. For cross-domain few-shot learning, we suggest representation fusion to unify different abstraction levels of a deep neural network into one representation. We propose Cross-domain Hebbian Ensemble Few-shot learning (CHEF), which achieves representation fusion by an ensemble of Hebbian learners acting on different layers of a deep neural network. Ablation studies show that representation fusion is a decisive factor to boost cross-domain few-shot learning. On the few-shot datasets miniImagenet and tieredImagenet with small domain shifts, CHEF is competitive with state-of-the-art methods. On cross-domain few-shot benchmark challenges with larger domain shifts, CHEF establishes novel state-of-the-art results in all categories. We further apply CHEF on a real-world cross-domain application in drug discovery. We consider a domain shift from bioactive molecules to environmental chemicals and drugs with twelve associated toxicity prediction tasks. On these tasks, that are highly relevant for computational drug discovery, CHEF significantly outperforms all its competitors. Github: https://github.com/ml-jku/chef △ Less

Submitted 16 February, 2021; v1 submitted 13 October, 2020; originally announced October 2020.

Large-scale ligand-based virtual screening for SARS-CoV-2 inhibitors using deep neural networks

Authors: Markus Hofmarcher, Andreas Mayr, Elisabeth Rumetshofer, Peter Ruch, Philipp Renz, Johannes Schimunek, Philipp Seidl, Andreu Vall, Michael Widrich, Sepp Hochreiter, Günter Klambauer

Abstract: Due to the current severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic, there is an urgent need for novel therapies and drugs. We conducted a large-scale virtual screening for small molecules that are potential CoV-2 inhibitors. To this end, we utilized "ChemAI", a deep neural network trained on more than 220M data points across 3.6M molecules from three public drug-discovery dat… ▽ More Due to the current severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic, there is an urgent need for novel therapies and drugs. We conducted a large-scale virtual screening for small molecules that are potential CoV-2 inhibitors. To this end, we utilized "ChemAI", a deep neural network trained on more than 220M data points across 3.6M molecules from three public drug-discovery databases. With ChemAI, we screened and ranked one billion molecules from the ZINC database for favourable effects against CoV-2. We then reduced the result to the 30,000 top-ranked compounds, which are readily accessible and purchasable via the ZINC database. Additionally, we screened the DrugBank using ChemAI to allow for drug repurposing, which would be a fast way towards a therapy. We provide these top-ranked compounds of ZINC and DrugBank as a library for further screening with bioassays at https://github.com/ml-jku/sars-cov-inhibitors-chemai. △ Less

Submitted 17 August, 2020; v1 submitted 25 March, 2020; originally announced April 2020.

Comments: Additional results added. Various corrections to formulations and typos

RefCurv: A Software for the Construction of Pediatric Reference Curves

Authors: Christian Winkler, Katharina Linden, Andreas Mayr, Thomas Schultz, Thomas Welchowski, Johannes Breuer, Ulrike Herberg

Abstract: In medicine, reference curves serve as an important tool for everyday clinical practice. Pediatricians assess the growth process of children with the help of percentile curves serving as norm references. The mathematical methods for the construction of these reference curves are sophisticated and often require technical knowledge beyond the scope of physicians. An easy-to-use software for life sci… ▽ More In medicine, reference curves serve as an important tool for everyday clinical practice. Pediatricians assess the growth process of children with the help of percentile curves serving as norm references. The mathematical methods for the construction of these reference curves are sophisticated and often require technical knowledge beyond the scope of physicians. An easy-to-use software for life scientists and physicians is missing. As a consequence, most medical publications do not document the construction properly. This project aims to develop a software that enables non-technical users to apply modern statistical methods to create and analyze reference curves. In this paper, we present RefCurv, a software that facilitates the construction of reference curves. The software comprises functionalities to select and visualize data. Users can fit models to the data and graphically present them as percentile curves. Furthermore, the software provides features to highlight possible outliers, perform model selection, and analyze the sensitivity. RefCurv is an open-source software with a graphical user interface (GUI) written in Python. It uses R and the gamlss add-on package (Rigby and Stasinopoulos (2005)) as the underlying statistical engine. In summary, RefCurv is the first software based on the gamlss package, which enables practitioners to construct and analyze reference curves in a user-friendly GUI. In broader terms, the software brings together the fields of statistical learning and medical application. Consequently, RefCurv can help to establish the construction of reference curves in other medical fields. △ Less

Submitted 28 January, 2019; originally announced January 2019.

Comments: You can find more information about the software (tutorials, link to the source code, etc.) on https://refcurv.com

Extension of the Gradient Boosting Algorithm for Joint Modeling of Longitudinal and Time-to-Event data

Authors: Colin Griesbach, Andreas Mayr, Elisabeth Waldmann

Abstract: In various data situations joint models are an efficient tool to analyze relationships between time dependent covariates and event times or to correct for event-dependent dropout occurring in regression analysis. Joint modeling connects a longitudinal and a survival submodel within a single joint likelihood which then can be maximized by standard optimization methods. Main burdens of these convent… ▽ More In various data situations joint models are an efficient tool to analyze relationships between time dependent covariates and event times or to correct for event-dependent dropout occurring in regression analysis. Joint modeling connects a longitudinal and a survival submodel within a single joint likelihood which then can be maximized by standard optimization methods. Main burdens of these conventional methods are that the computational effort increases rapidly in higher dimensions and they do not offer special tools for proper variable selection. Gradient boosting techniques are well known among statisticians for addressing exactly these problems, hence an initial boosting algorithm to fit a basic joint model based on functional gradient descent methods has been proposed. Aim of this work is to extend this algorithm in order to fit a model incorporating baseline covariates affecting solely the survival part of the model. The extended algorithm is evaluated based on low and high dimensional simulation runs as well as a data set on AIDS patients, where the longitudinal submodel models the underlying profile of the CD4 cell count which then gets included alongside several baseline covariates in the survival submodel. △ Less

Submitted 24 October, 2018; originally announced October 2018.

Gradient boosting in Markov-switching generalized additive models for location, scale and shape

Authors: Timo Adam, Andreas Mayr, Thomas Kneib

Abstract: We propose a novel class of flexible latent-state time series regression models which we call Markov-switching generalized additive models for location, scale and shape. In contrast to conventional Markov-switching regression models, the presented methodology allows us to model different state-dependent parameters of the response distribution - not only the mean, but also variance, skewness and ku… ▽ More We propose a novel class of flexible latent-state time series regression models which we call Markov-switching generalized additive models for location, scale and shape. In contrast to conventional Markov-switching regression models, the presented methodology allows us to model different state-dependent parameters of the response distribution - not only the mean, but also variance, skewness and kurtosis parameters - as potentially smooth functions of a given set of explanatory variables. In addition, the set of possible distributions that can be specified for the response is not limited to the exponential family but additionally includes, for instance, a variety of Box-Cox-transformed, zero-inflated and mixture distributions. We propose an estimation approach based on the EM algorithm, where we use the gradient boosting framework to prevent overfitting while simultaneously performing variable selection. The feasibility of the suggested approach is assessed in simulation experiments and illustrated in a real-data setting, where we model the conditional distribution of the daily average price of energy in Spain over time. △ Less

Submitted 17 May, 2018; v1 submitted 6 October, 2017; originally announced October 2017.

Self-Normalizing Neural Networks

Authors: Günter Klambauer, Thomas Unterthiner, Andreas Mayr, Sepp Hochreiter

Abstract: Deep Learning has revolutionized vision via convolutional neural networks (CNNs) and natural language processing via recurrent neural networks (RNNs). However, success stories of Deep Learning with standard feed-forward neural networks (FNNs) are rare. FNNs that perform well are typically shallow and, therefore cannot exploit many levels of abstract representations. We introduce self-normalizing n… ▽ More Deep Learning has revolutionized vision via convolutional neural networks (CNNs) and natural language processing via recurrent neural networks (RNNs). However, success stories of Deep Learning with standard feed-forward neural networks (FNNs) are rare. FNNs that perform well are typically shallow and, therefore cannot exploit many levels of abstract representations. We introduce self-normalizing neural networks (SNNs) to enable high-level abstract representations. While batch normalization requires explicit normalization, neuron activations of SNNs automatically converge towards zero mean and unit variance. The activation function of SNNs are "scaled exponential linear units" (SELUs), which induce self-normalizing properties. Using the Banach fixed-point theorem, we prove that activations close to zero mean and unit variance that are propagated through many network layers will converge towards zero mean and unit variance -- even under the presence of noise and perturbations. This convergence property of SNNs allows to (1) train deep networks with many layers, (2) employ strong regularization, and (3) to make learning highly robust. Furthermore, for activations not close to unit variance, we prove an upper and lower bound on the variance, thus, vanishing and exploding gradients are impossible. We compared SNNs on (a) 121 tasks from the UCI machine learning repository, on (b) drug discovery benchmarks, and on (c) astronomy tasks with standard FNNs and other machine learning methods such as random forests and support vector machines. SNNs significantly outperformed all competing FNN methods at 121 UCI tasks, outperformed all competing methods at the Tox21 dataset, and set a new record at an astronomy data set. The winning SNN architectures are often very deep. Implementations are available at: github.com/bioinf-jku/SNNs. △ Less

Submitted 7 September, 2017; v1 submitted 8 June, 2017; originally announced June 2017.

Comments: 9 pages (+ 93 pages appendix)

Journal ref: Advances in Neural Information Processing Systems 30 (NIPS 2017)

An update on statistical boosting in biomedicine

Authors: Andreas Mayr, Benjamin Hofner, Elisabeth Waldmann, Tobias Hepp, Olaf Gefeller, Matthias Schmid

Abstract: Statistical boosting algorithms have triggered a lot of research during the last decade. They combine a powerful machine-learning approach with classical statistical modelling, offering various practical advantages like automated variable selection and implicit regularization of effect estimates. They are extremely flexible, as the underlying base-learners (regression functions defining the type o… ▽ More Statistical boosting algorithms have triggered a lot of research during the last decade. They combine a powerful machine-learning approach with classical statistical modelling, offering various practical advantages like automated variable selection and implicit regularization of effect estimates. They are extremely flexible, as the underlying base-learners (regression functions defining the type of effect for the explanatory variables) can be combined with any kind of loss function (target function to be optimized, defining the type of regression setting). In this review article, we highlight the most recent methodological developments on statistical boosting regarding variable selection, functional regression and advanced time-to-event modelling. Additionally, we provide a short overview on relevant applications of statistical boosting in biomedicine. △ Less

Submitted 27 February, 2017; originally announced February 2017.

Probing for sparse and fast variable selection with model-based boosting

Authors: Janek Thomas, Tobias Hepp, Andreas Mayr, Bernd Bischl

Abstract: We present a new variable selection method based on model-based gradient boosting and randomly permuted variables. Model-based boosting is a tool to fit a statistical model while performing variable selection at the same time. A drawback of the fitting lies in the need of multiple model fits on slightly altered data (e.g. cross-validation or bootstrap) to find the optimal number of boosting iterat… ▽ More We present a new variable selection method based on model-based gradient boosting and randomly permuted variables. Model-based boosting is a tool to fit a statistical model while performing variable selection at the same time. A drawback of the fitting lies in the need of multiple model fits on slightly altered data (e.g. cross-validation or bootstrap) to find the optimal number of boosting iterations and prevent overfitting. In our proposed approach, we augment the data set with randomly permuted versions of the true variables, so called shadow variables, and stop the step-wise fitting as soon as such a variable would be added to the model. This allows variable selection in a single fit of the model without requiring further parameter tuning. We show that our probing approach can compete with state-of-the-art selection methods like stability selection in a high-dimensional classification benchmark and apply it on gene expression data for the estimation of riboflavin production of Bacillus subtilis. △ Less

Submitted 15 February, 2017; originally announced February 2017.

Comments: 14 pages, 2 figures

Stability selection for component-wise gradient boosting in multiple dimensions

Authors: Janek Thomas, Andreas Mayr, Bernd Bischl, Matthias Schmid, Adam Smith, Benjamin Hofner

Abstract: We present a new algorithm for boosting generalized additive models for location, scale and shape (GAMLSS) that allows to incorporate stability selection, an increasingly popular way to obtain stable sets of covariates while controlling the per-family error rate (PFER). The model is fitted repeatedly to subsampled data and variables with high selection frequencies are extracted. To apply stability… ▽ More We present a new algorithm for boosting generalized additive models for location, scale and shape (GAMLSS) that allows to incorporate stability selection, an increasingly popular way to obtain stable sets of covariates while controlling the per-family error rate (PFER). The model is fitted repeatedly to subsampled data and variables with high selection frequencies are extracted. To apply stability selection to boosted GAMLSS, we develop a new "noncyclical" fitting algorithm that incorporates an additional selection step of the best-fitting distribution parameter in each iteration. This new algorithms has the additional advantage that optimizing the tuning parameters of boosting is reduced from a multi-dimensional to a one-dimensional problem with vastly decreased complexity. The performance of the novel algorithm is evaluated in an extensive simulation study. We apply this new algorithm to a study to estimate abundance of common eider in Massachusetts, USA, featuring excess zeros, overdispersion, non-linearity and spatio-temporal structures. Eider abundance is estimated via boosted GAMLSS, allowing both mean and overdispersion to be regressed on covariates. Stability selection is used to obtain a sparse set of stable predictors. △ Less

Submitted 30 November, 2016; originally announced November 2016.

Comments: 16 pages

Operationalised product quality models and assessment: The Quamoco approach

Authors: Stefan Wagner, Andreas Goeb, Lars Heinemann, Michael Kläs, Constanza Lampasona, Klaus Lochmann, Alois Mayr, Reinhold Plösch, Andreas Seidl, Jonathan Streit, Adam Trendowicz

Abstract: Software quality models provide either abstract quality characteristics or concrete quality measurements; there is no seamless integration of these two aspects. Reasons for this include the complexity of quality and the various quality profiles in different domains which make it difficult to build operationalised quality models. In the project Quamoco, we developed a comprehensive approach for clo… ▽ More Software quality models provide either abstract quality characteristics or concrete quality measurements; there is no seamless integration of these two aspects. Reasons for this include the complexity of quality and the various quality profiles in different domains which make it difficult to build operationalised quality models. In the project Quamoco, we developed a comprehensive approach for closing this gap. It combined constructive research, which involved quality experts from academia and industry in workshops, sprint work and reviews, with empirical studies. All deliverables within the project were peer-reviewed by two project members from a different area. Most deliverables were developed in two or three iterations and underwent an evaluation. We contribute a comprehensive quality modelling and assessment approach: (1) A meta quality model defines the structure of operationalised quality models. It includes the concept of a product factor, which bridges the gap between concrete measurements and abstract quality aspects, and allows modularisation to create modules for specific domains. (2) A largely technology-independent base quality model reduces the effort and complexity of building quality models for specific domains. For Java and C# systems, we refined it with about 300 concrete product factors and 500 measures. (3) A concrete and comprehensive quality assessment approach makes use of the concepts in the meta-model. (4) An empirical evaluation of the above results using real-world software systems. (5) The extensive, open-source tool support is in a mature state. (6) The model for embedded software systems is a proof-of-concept for domain-specific quality models. We provide a broad basis for the development and application of quality models in industrial practice as well as a basis for further extension, validation and comparison with other approaches in research. △ Less

Submitted 28 November, 2016; originally announced November 2016.

Comments: 41 pages, 14 figures

Journal ref: Information and Software Technology, Volume 62, June 2015, pages 101-123

Boosting Joint Models for Longitudinal and Time-to-Event Data

Authors: Elisabeth Waldmann, David Taylor-Robinson, Nadja Klein, Thomas Kneib, Tania Pressler, Matthias Schmid, Andreas Mayr

Abstract: Joint Models for longitudinal and time-to-event data have gained a lot of attention in the last few years as they are a helpful technique to approach common a data structure in clinical studies where longitudinal outcomes are recorded alongside event times. Those two processes are often linked and the two outcomes should thus be modeled jointly in order to prevent the potential bias introduced by… ▽ More Joint Models for longitudinal and time-to-event data have gained a lot of attention in the last few years as they are a helpful technique to approach common a data structure in clinical studies where longitudinal outcomes are recorded alongside event times. Those two processes are often linked and the two outcomes should thus be modeled jointly in order to prevent the potential bias introduced by independent modelling. Commonly, joint models are estimated in likelihood based expectation maximization or Bayesian approaches using frameworks where variable selection is problematic and which do not immediately work for high-dimensional data. In this paper, we propose a boosting algorithm tackling these challenges by being able to simultaneously estimate predictors for joint models and automatically select the most influential variables even in high-dimensional data situations. We analyse the performance of the new algorithm in a simulation study and apply it to the Danish cystic fibrosis registry which collects longitudinal lung function data on patients with cystic fibrosis together with data regarding the onset of pulmonary infections. This is the first approach to combine state-of-the art algorithms from the field of machine-learning with the model class of joint models, providing a fully data-driven mechanism to select variables and predictor effects in a unified framework of boosting joint models. △ Less

Submitted 22 December, 2016; v1 submitted 9 September, 2016; originally announced September 2016.

Signal Regression Models for Location, Scale and Shape with an Application to Stock Returns

Authors: Sarah Brockhaus, Andreas Fuest, Andreas Mayr, Sonja Greven

Abstract: We discuss scalar-on-function regression models where all parameters of the assumed response distribution can be modeled depending on covariates. We thus combine signal regression models with generalized additive models for location, scale and shape (GAMLSS). We compare two fundamentally different methods for estimation, a gradient boosting and a penalized likelihood based approach, and address pr… ▽ More We discuss scalar-on-function regression models where all parameters of the assumed response distribution can be modeled depending on covariates. We thus combine signal regression models with generalized additive models for location, scale and shape (GAMLSS). We compare two fundamentally different methods for estimation, a gradient boosting and a penalized likelihood based approach, and address practically important points like identifiability and model choice. Estimation by a component-wise gradient boosting algorithm allows for high dimensional data settings and variable selection. Estimation by a penalized likelihood based approach has the advantage of directly provided statistical inference. The motivating application is a time series of stock returns where it is of interest to model both the expectation and the variance depending on lagged response values and functional liquidity curves. △ Less

Submitted 13 May, 2016; originally announced May 2016.

Toxicity Prediction using Deep Learning

Authors: Thomas Unterthiner, Andreas Mayr, Günter Klambauer, Sepp Hochreiter

Abstract: Everyday we are exposed to various chemicals via food additives, cleaning and cosmetic products and medicines -- and some of them might be toxic. However testing the toxicity of all existing compounds by biological experiments is neither financially nor logistically feasible. Therefore the government agencies NIH, EPA and FDA launched the Tox21 Data Challenge within the "Toxicology in the 21st Cen… ▽ More Everyday we are exposed to various chemicals via food additives, cleaning and cosmetic products and medicines -- and some of them might be toxic. However testing the toxicity of all existing compounds by biological experiments is neither financially nor logistically feasible. Therefore the government agencies NIH, EPA and FDA launched the Tox21 Data Challenge within the "Toxicology in the 21st Century" (Tox21) initiative. The goal of this challenge was to assess the performance of computational methods in predicting the toxicity of chemical compounds. State of the art toxicity prediction methods build upon specifically-designed chemical descriptors developed over decades. Though Deep Learning is new to the field and was never applied to toxicity prediction before, it clearly outperformed all other participating methods. In this application paper we show that deep nets automatically learn features resembling well-established toxicophores. In total, our Deep Learning approach won both of the panel-challenges (nuclear receptors and stress response) as well as the overall Grand Challenge, and thereby sets a new standard in tox prediction. △ Less

Submitted 4 March, 2015; originally announced March 2015.

Rectified Factor Networks

Authors: Djork-Arné Clevert, Andreas Mayr, Thomas Unterthiner, Sepp Hochreiter

Abstract: We propose rectified factor networks (RFNs) to efficiently construct very sparse, non-linear, high-dimensional representations of the input. RFN models identify rare and small events in the input, have a low interference between code units, have a small reconstruction error, and explain the data covariance structure. RFN learning is a generalized alternating minimization algorithm derived from the… ▽ More We propose rectified factor networks (RFNs) to efficiently construct very sparse, non-linear, high-dimensional representations of the input. RFN models identify rare and small events in the input, have a low interference between code units, have a small reconstruction error, and explain the data covariance structure. RFN learning is a generalized alternating minimization algorithm derived from the posterior regularization method which enforces non-negative and normalized posterior means. We proof convergence and correctness of the RFN learning algorithm. On benchmarks, RFNs are compared to other unsupervised methods like autoencoders, RBMs, factor analysis, ICA, and PCA. In contrast to previous sparse coding methods, RFNs yield sparser codes, capture the data's covariance structure more precisely, and have a significantly smaller reconstruction error. We test RFNs as pretraining technique for deep networks on different vision datasets, where RFNs were superior to RBMs and autoencoders. On gene expression data from two pharmaceutical drug discovery studies, RFNs detected small and rare gene modules that revealed highly relevant new biological insights which were so far missed by other unsupervised methods. △ Less

Submitted 11 June, 2015; v1 submitted 23 February, 2015; originally announced February 2015.

Comments: 9 pages + 49 pages supplement

Journal ref: Advances in Neural Information Processing Systems 28 (NIPS 2015)

gamboostLSS: An R Package for Model Building and Variable Selection in the GAMLSS Framework

Authors: Benjamin Hofner, Andreas Mayr, Matthias Schmid

Abstract: Generalized additive models for location, scale and shape (GAMLSS) are a flexible class of regression models that allow to model multiple parameters of a distribution function, such as the mean and the standard deviation, simultaneously. With the R package gamboostLSS, we provide a boosting method to fit these models. Variable selection and model choice are naturally available within this regulari… ▽ More Generalized additive models for location, scale and shape (GAMLSS) are a flexible class of regression models that allow to model multiple parameters of a distribution function, such as the mean and the standard deviation, simultaneously. With the R package gamboostLSS, we provide a boosting method to fit these models. Variable selection and model choice are naturally available within this regularized regression framework. To introduce and illustrate the R package gamboostLSS and its infrastructure, we use a data set on stunted growth in India. In addition to the specification and application of the model itself, we present a variety of convenience functions, including methods for tuning parameter selection, prediction and visualization of results. The package gamboostLSS is available from CRAN (http://cran.r-project.org/package=gamboostLSS). △ Less

Submitted 7 July, 2014; originally announced July 2014.

Extending Statistical Boosting - An Overview of Recent Methodological Developments

Authors: Andreas Mayr, Harald Binder, Olaf Gefeller, Matthias Schmid

Abstract: Boosting algorithms to simultaneously estimate and select predictor effects in statistical models have gained substantial interest during the last decade. This review article aims to highlight recent methodological developments regarding boosting algorithms for statistical modelling especially focusing on topics relevant for biomedical research. We suggest a unified framework for gradient boosting… ▽ More Boosting algorithms to simultaneously estimate and select predictor effects in statistical models have gained substantial interest during the last decade. This review article aims to highlight recent methodological developments regarding boosting algorithms for statistical modelling especially focusing on topics relevant for biomedical research. We suggest a unified framework for gradient boosting and likelihood-based boosting (statistical boosting) which have been addressed strictly separated in the literature up to now. Statistical boosting algorithms have been adapted to carry out unbiased variable selection and automated model choice during the fitting process and can nowadays be applied in almost any possible type of regression setting in combination with a large amount of different types of predictor effects. The methodological developments on statistical boosting during the last ten years can be grouped into three different lines of research: (i) efforts to ensure variable selection leading to sparser models, (ii) developments regarding different types of predictor effects and their selection (model choice), (iii) approaches to extend the statistical boosting framework to new regression settings. △ Less

Submitted 18 November, 2014; v1 submitted 7 March, 2014; originally announced March 2014.

Journal ref: Methods Inf Med 2014; 53(6): 428-435

The Evolution of Boosting Algorithms - From Machine Learning to Statistical Modelling

Authors: Andreas Mayr, Harald Binder, Olaf Gefeller, Matthias Schmid

Abstract: The concept of boosting emerged from the field of machine learning. The basic idea is to boost the accuracy of a weak classifying tool by combining various instances into a more accurate prediction. This general concept was later adapted to the field of statistical modelling. This review article attempts to highlight this evolution of boosting algorithms from machine learning to statistical modell… ▽ More The concept of boosting emerged from the field of machine learning. The basic idea is to boost the accuracy of a weak classifying tool by combining various instances into a more accurate prediction. This general concept was later adapted to the field of statistical modelling. This review article attempts to highlight this evolution of boosting algorithms from machine learning to statistical modelling. We describe the AdaBoost algorithm for classification as well as the two most prominent statistical boosting approaches, gradient boosting and likelihood-based boosting. Although both appraoches are typically treated separately in the literature, they share the same methodological roots and follow the same fundamental concepts. Compared to the initial machine learning algorithms, which must be seen as black-box prediction schemes, statistical boosting result in statistical models which offer a straight-forward interpretation. We highlight the methodological background and present the most common software implementations. Worked out examples and corresponding R code can be found in the Appendix. △ Less

Submitted 18 November, 2014; v1 submitted 6 March, 2014; originally announced March 2014.

Journal ref: Methods Inf Med 2014; 53(6): 419-427

Boosting the concordance index for survival data - a unified framework to derive and evaluate biomarker combinations

Authors: Andreas Mayr, Matthias Schmid

Abstract: The development of molecular signatures for the prediction of time-to-event outcomes is a methodologically challenging task in bioinformatics and biostatistics. Although there are numerous approaches for the derivation of marker combinations and their evaluation, the underlying methodology often suffers from the problem that different optimization criteria are mixed during the feature selection, e… ▽ More The development of molecular signatures for the prediction of time-to-event outcomes is a methodologically challenging task in bioinformatics and biostatistics. Although there are numerous approaches for the derivation of marker combinations and their evaluation, the underlying methodology often suffers from the problem that different optimization criteria are mixed during the feature selection, estimation and evaluation steps. This might result in marker combinations that are only suboptimal regarding the evaluation criterion of interest. To address this issue, we propose a unified framework to derive and evaluate biomarker combinations. Our approach is based on the concordance index for time-to-event data, which is a non-parametric measure to quantify the discrimatory power of a prediction rule. Specifically, we propose a component-wise boosting algorithm that results in linear biomarker combinations that are optimal with respect to a smoothed version of the concordance index. We investigate the performance of our algorithm in a large-scale simulation study and in two molecular data sets for the prediction of survival in breast cancer patients. Our numerical results show that the new approach is not only methodologically sound but can also lead to a higher discriminatory power than traditional approaches for the derivation of gene signatures. △ Less

Submitted 25 October, 2013; v1 submitted 24 July, 2013; originally announced July 2013.

Comments: revised manuscript - added simulation study, additional results

Journal ref: PloS ONE 2014, 9(1): e84483

Design and Simulation of Molecular Nonvolatile Single-Electron Resistive Switches

Authors: Nikita Simonian, Andreas Mayr, Konstantin K. Likharev

Abstract: We have carried out a preliminary design and simulation of a single-electron resistive switch based on a system of two linear, parallel, electrostatically-coupled molecules: one implementing a single-electron transistor and another serving as a single-electron trap. To verify our design, we have performed a theoretical analysis of this "memristive" device, based on a combination of ab-initio calcu… ▽ More We have carried out a preliminary design and simulation of a single-electron resistive switch based on a system of two linear, parallel, electrostatically-coupled molecules: one implementing a single-electron transistor and another serving as a single-electron trap. To verify our design, we have performed a theoretical analysis of this "memristive" device, based on a combination of ab-initio calculations of the electronic structures of the molecules and the general theory of single-electron tunneling in systems with discrete energy spectra. Our results show that such molecular assemblies, with a length below 10 nm and a footprint area of about 5 nm$^2$, may combine sub-second switching times with multi-year retention times and high ($> 10^3$) ON/OFF current ratios, at room temperature. Moreover, Monte Carlo simulations of self-assembled monolayers (SAM) based on such molecular assemblies have shown that such monolayers may also be used as resistive switches, with comparable characteristics and, in addition, be highly tolerant to defects and stray offset charges. △ Less

Submitted 18 October, 2012; originally announced October 2012.

Comments: 17 pages, 22 figures