Showing 1–50 of 50 results for author: Wagner, J

Distribution Aggregation via Continuous Thiele's Rules

Authors: Jonathan Wagner, Reshef Meir

Abstract: We introduce the class of \textit{Continuous Thiele's Rules} that generalize the familiar \textbf{Thiele's rules} \cite{janson2018phragmens} of multi-winner voting to distribution aggregation problems. Each rule in that class maximizes $\sum_if(π^i)$ where $π^i$ is an agent $i$'s satisfaction and $f$ could be any twice differentiable, increasing and concave real function. Based on a single quantit… ▽ More We introduce the class of \textit{Continuous Thiele's Rules} that generalize the familiar \textbf{Thiele's rules} \cite{janson2018phragmens} of multi-winner voting to distribution aggregation problems. Each rule in that class maximizes $\sum_if(π^i)$ where $π^i$ is an agent $i$'s satisfaction and $f$ could be any twice differentiable, increasing and concave real function. Based on a single quantity we call the \textit{'Inequality Aversion'} of $f$ (elsewhere known as "Relative Risk Aversion"), we derive bounds on the Egalitarian loss, welfare loss and the approximation of \textit{Average Fair Share}, leading to a quantifiable, continuous presentation of their inevitable trade-offs. In particular, we show that the Nash Product Rule satisfies\textit{ Average Fair Share} in our setting. △ Less

Submitted 2 August, 2024; originally announced August 2024.

An Evaluation of Immersive Infographics for News Reporting: Quantifying the Effect of Mobile AR Concrete Scales Infographics on Volume Understanding

Authors: Mariane Giambastiani, Jorge Wagner, Carla M. Dal Sasso Freitas, Luciana Nedel

Abstract: Augmented Reality (AR) allows us to represent information in the user's own environment and, therefore, convey a visceral feeling of its true physical scale. Journalists increasingly leverage this opportunity through immersive infographics, an extension of conventional infographics reliant on familiar references to convey volumes, heights, weights, and sizes. Our goal is to measure the contributio… ▽ More Augmented Reality (AR) allows us to represent information in the user's own environment and, therefore, convey a visceral feeling of its true physical scale. Journalists increasingly leverage this opportunity through immersive infographics, an extension of conventional infographics reliant on familiar references to convey volumes, heights, weights, and sizes. Our goal is to measure the contribution of immersive mobile AR concrete scales infographics to the user's understanding of the information scale. We focus on infographics powered by tablet-based mobile AR, given its current much more widespread use for news consumption compared to headset-based AR. We designed and implemented a study apparatus containing three alternative representation methods (textual analogies, image infographic, and AR infographic) for three different pieces of news with different characteristics and scales. In a controlled user study, we asked 26 participants to represent the expected volume of the information in the real world with the help of an AR mobile application. We also compared their subjective feelings when interacting with the different representations. While both image and AR infographics led to significantly better comprehension than textual analogies alone across different kinds of news, AR infographics led, on average, to a 31.8% smaller volume estimation error than static ones. Our findings indicate that mobile AR concrete scales infographics can contribute to news reporting by increasing readers' abilities to comprehend volume information. △ Less

Submitted 10 July, 2024; originally announced July 2024.

ACM Class: H.5.1

HEST-1k: A Dataset for Spatial Transcriptomics and Histology Image Analysis

Authors: Guillaume Jaume, Paul Doucet, Andrew H. Song, Ming Y. Lu, Cristina Almagro-Pérez, Sophia J. Wagner, Anurag J. Vaidya, Richard J. Chen, Drew F. K. Williamson, Ahrong Kim, Faisal Mahmood

Abstract: Spatial transcriptomics (ST) enables interrogating the molecular composition of tissue with ever-increasing resolution, depth, and sensitivity. However, costs, rapidly evolving technology, and lack of standards have constrained computational methods in ST to narrow tasks and small cohorts. In addition, the underlying tissue morphology as reflected by H&E-stained whole slide images (WSIs) encodes r… ▽ More Spatial transcriptomics (ST) enables interrogating the molecular composition of tissue with ever-increasing resolution, depth, and sensitivity. However, costs, rapidly evolving technology, and lack of standards have constrained computational methods in ST to narrow tasks and small cohorts. In addition, the underlying tissue morphology as reflected by H&E-stained whole slide images (WSIs) encodes rich information often overlooked in ST studies. Here, we introduce HEST-1k, a collection of 1,108 spatial transcriptomic profiles, each linked to a WSI and metadata. HEST-1k was assembled using HEST-Library from 131 public and internal cohorts encompassing 25 organs, two species (Homo Sapiens and Mus Musculus), and 320 cancer samples from 25 cancer types. HEST-1k processing enabled the identification of 1.5 million expression--morphology pairs and 60 million nuclei. HEST-1k is tested on three use cases: (1) benchmarking foundation models for histopathology (HEST-Benchmark), (2) biomarker identification, and (3) multimodal representation learning. HEST-1k, HEST-Library, and HEST-Benchmark can be freely accessed via https://github.com/mahmoodlab/hest. △ Less

Submitted 23 June, 2024; originally announced June 2024.

Comments: Under review

Engineering software 2.0 by interpolating neural networks: unifying training, solving, and calibration

Authors: Chanwook Park, Sourav Saha, Jiachen Guo, Xiaoyu Xie, Satyajit Mojumder, Miguel A. Bessa, Dong Qian, Wei Chen, Gregory J. Wagner, Jian Cao, Wing Kam Liu

Abstract: The evolution of artificial intelligence (AI) and neural network theories has revolutionized the way software is programmed, shifting from a hard-coded series of codes to a vast neural network. However, this transition in engineering software has faced challenges such as data scarcity, multi-modality of data, low model accuracy, and slow inference. Here, we propose a new network based on interpola… ▽ More The evolution of artificial intelligence (AI) and neural network theories has revolutionized the way software is programmed, shifting from a hard-coded series of codes to a vast neural network. However, this transition in engineering software has faced challenges such as data scarcity, multi-modality of data, low model accuracy, and slow inference. Here, we propose a new network based on interpolation theories and tensor decomposition, the interpolating neural network (INN). Instead of interpolating training data, a common notion in computer science, INN interpolates interpolation points in the physical space whose coordinates and values are trainable. It can also extrapolate if the interpolation points reside outside of the range of training data and the interpolation functions have a larger support domain. INN features orders of magnitude fewer trainable parameters, faster training, a smaller memory footprint, and higher model accuracy compared to feed-forward neural networks (FFNN) or physics-informed neural networks (PINN). INN is poised to usher in Engineering Software 2.0, a unified neural network that spans various domains of space, time, parameters, and initial/boundary conditions. This has previously been computationally prohibitive due to the exponentially growing number of trainable parameters, easily exceeding the parameter size of ChatGPT, which is over 1 trillion. INN addresses this challenge by leveraging tensor decomposition and tensor product, with adaptable network architecture. △ Less

Submitted 22 April, 2024; v1 submitted 16 April, 2024; originally announced April 2024.

Comments: 9 pages, 3 figures

DinoBloom: A Foundation Model for Generalizable Cell Embeddings in Hematology

Authors: Valentin Koch, Sophia J. Wagner, Salome Kazeminia, Ece Sancar, Matthias Hehr, Julia Schnabel, Tingying Peng, Carsten Marr

Abstract: In hematology, computational models offer significant potential to improve diagnostic accuracy, streamline workflows, and reduce the tedious work of analyzing single cells in peripheral blood or bone marrow smears. However, clinical adoption of computational models has been hampered by the lack of generalization due to large batch effects, small dataset sizes, and poor performance in transfer lear… ▽ More In hematology, computational models offer significant potential to improve diagnostic accuracy, streamline workflows, and reduce the tedious work of analyzing single cells in peripheral blood or bone marrow smears. However, clinical adoption of computational models has been hampered by the lack of generalization due to large batch effects, small dataset sizes, and poor performance in transfer learning from natural images. To address these challenges, we introduce DinoBloom, the first foundation model for single cell images in hematology, utilizing a tailored DINOv2 pipeline. Our model is built upon an extensive collection of 13 diverse, publicly available datasets of peripheral blood and bone marrow smears, the most substantial open-source cohort in hematology so far, comprising over 380,000 white blood cell images. To assess its generalization capability, we evaluate it on an external dataset with a challenging domain shift. We show that our model outperforms existing medical and non-medical vision models in (i) linear probing and k-nearest neighbor evaluations for cell-type classification on blood and bone marrow smears and (ii) weakly supervised multiple instance learning for acute myeloid leukemia subtyping by a large margin. A family of four DinoBloom models (small, base, large, and giant) can be adapted for a wide range of downstream applications, be a strong baseline for classification problems, and facilitate the assessment of batch effects in new datasets. All models are available at github.com/marrlab/DinoBloom. △ Less

Submitted 7 April, 2024; originally announced April 2024.

Reimagining TaxiVis through an Immersive Space-Time Cube metaphor and reflecting on potential benefits of Immersive Analytics for urban data exploration

Authors: Jorge Wagner, Claudio T. Silva, Wolfgang Stuerzlinger, Luciana Nedel

Abstract: Current visualization research has identified the potential of more immersive settings for data exploration, leveraging VR and AR technologies. To explore how a traditional visualization system could be adapted into an immersive framework, and how it could benefit from this, we decided to revisit a landmark paper presented ten years ago at IEEE VIS. TaxiVis, by Ferreira et al., enabled interactive… ▽ More Current visualization research has identified the potential of more immersive settings for data exploration, leveraging VR and AR technologies. To explore how a traditional visualization system could be adapted into an immersive framework, and how it could benefit from this, we decided to revisit a landmark paper presented ten years ago at IEEE VIS. TaxiVis, by Ferreira et al., enabled interactive spatio-temporal querying of a large dataset of taxi trips in New York City. Here, we reimagine how TaxiVis' functionalities could be implemented and extended in a 3D immersive environment. Among the unique features we identify as being enabled by the Immersive TaxiVis prototype are alternative uses of the additional visual dimension, a fully visual 3D spatio-temporal query framework, and the opportunity to explore the data at different scales and frames of reference. By revisiting the case studies from the original paper, we demonstrate workflows that can benefit from this immersive perspective. Through reporting on our experience, and on the vision and reasoning behind our design decisions, we hope to contribute to the debate on how conventional and immersive visualization paradigms can complement each other and on how the exploration of urban datasets can be facilitated in the coming years. △ Less

Submitted 23 May, 2024; v1 submitted 1 February, 2024; originally announced February 2024.

Comments: Published in the proceedings of the IEEE VR 2024 conference

ACM Class: H.5.1

Journal ref: 2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR), Orlando, FL, USA, 2024, pp. 827-838

B-Cos Aligned Transformers Learn Human-Interpretable Features

Authors: Manuel Tran, Amal Lahiani, Yashin Dicente Cid, Melanie Boxberg, Peter Lienemann, Christian Matek, Sophia J. Wagner, Fabian J. Theis, Eldad Klaiman, Tingying Peng

Abstract: Vision Transformers (ViTs) and Swin Transformers (Swin) are currently state-of-the-art in computational pathology. However, domain experts are still reluctant to use these models due to their lack of interpretability. This is not surprising, as critical decisions need to be transparent and understandable. The most common approach to understanding transformers is to visualize their attention. Howev… ▽ More Vision Transformers (ViTs) and Swin Transformers (Swin) are currently state-of-the-art in computational pathology. However, domain experts are still reluctant to use these models due to their lack of interpretability. This is not surprising, as critical decisions need to be transparent and understandable. The most common approach to understanding transformers is to visualize their attention. However, attention maps of ViTs are often fragmented, leading to unsatisfactory explanations. Here, we introduce a novel architecture called the B-cos Vision Transformer (BvT) that is designed to be more interpretable. It replaces all linear transformations with the B-cos transform to promote weight-input alignment. In a blinded study, medical experts clearly ranked BvTs above ViTs, suggesting that our network is better at capturing biomedically relevant structures. This is also true for the B-cos Swin Transformer (Bwin). Compared to the Swin Transformer, it even improves the F1-score by up to 4.7% on two public datasets. △ Less

Submitted 18 January, 2024; v1 submitted 16 January, 2024; originally announced January 2024.

Comments: Accepted at MICCAI 2023 (oral). Camera-ready available at https://doi.org/10.1007/978-3-031-43993-3_50

Low-resource finetuning of foundation models beats state-of-the-art in histopathology

Authors: Benedikt Roth, Valentin Koch, Sophia J. Wagner, Julia A. Schnabel, Carsten Marr, Tingying Peng

Abstract: To handle the large scale of whole slide images in computational pathology, most approaches first tessellate the images into smaller patches, extract features from these patches, and finally aggregate the feature vectors with weakly-supervised learning. The performance of this workflow strongly depends on the quality of the extracted features. Recently, foundation models in computer vision showed… ▽ More To handle the large scale of whole slide images in computational pathology, most approaches first tessellate the images into smaller patches, extract features from these patches, and finally aggregate the feature vectors with weakly-supervised learning. The performance of this workflow strongly depends on the quality of the extracted features. Recently, foundation models in computer vision showed that leveraging huge amounts of data through supervised or self-supervised learning improves feature quality and generalizability for a variety of tasks. In this study, we benchmark the most popular vision foundation models as feature extractors for histopathology data. We evaluate the models in two settings: slide-level classification and patch-level classification. We show that foundation models are a strong baseline. Our experiments demonstrate that by finetuning a foundation model on a single GPU for only two hours or three days depending on the dataset, we can match or outperform state-of-the-art feature extractors for computational pathology. These findings imply that even with little resources one can finetune a feature extractor tailored towards a specific downstream task and dataset. This is a considerable shift from the current state, where only few institutions with large amounts of resources and datasets are able to train a feature extractor. We publish all code used for training and evaluation as well as the finetuned models. △ Less

Submitted 9 January, 2024; originally announced January 2024.

From Whole-slide Image to Biomarker Prediction: A Protocol for End-to-End Deep Learning in Computational Pathology

Authors: Omar S. M. El Nahhas, Marko van Treeck, Georg Wölflein, Michaela Unger, Marta Ligero, Tim Lenz, Sophia J. Wagner, Katherine J. Hewitt, Firas Khader, Sebastian Foersch, Daniel Truhn, Jakob Nikolas Kather

Abstract: Hematoxylin- and eosin (H&E) stained whole-slide images (WSIs) are the foundation of diagnosis of cancer. In recent years, development of deep learning-based methods in computational pathology enabled the prediction of biomarkers directly from WSIs. However, accurately linking tissue phenotype to biomarkers at scale remains a crucial challenge for democratizing complex biomarkers in precision onco… ▽ More Hematoxylin- and eosin (H&E) stained whole-slide images (WSIs) are the foundation of diagnosis of cancer. In recent years, development of deep learning-based methods in computational pathology enabled the prediction of biomarkers directly from WSIs. However, accurately linking tissue phenotype to biomarkers at scale remains a crucial challenge for democratizing complex biomarkers in precision oncology. This protocol describes a practical workflow for solid tumor associative modeling in pathology (STAMP), enabling prediction of biomarkers directly from WSIs using deep learning. The STAMP workflow is biomarker agnostic and allows for genetic- and clinicopathologic tabular data to be included as an additional input, together with histopathology images. The protocol consists of five main stages which have been successfully applied to various research problems: formal problem definition, data preprocessing, modeling, evaluation and clinical translation. The STAMP workflow differentiates itself through its focus on serving as a collaborative framework that can be used by clinicians and engineers alike for setting up research projects in the field of computational pathology. As an example task, we applied STAMP to the prediction of microsatellite instability (MSI) status in colorectal cancer, showing accurate performance for the identification of MSI-high tumors. Moreover, we provide an open-source codebase which has been deployed at several hospitals across the globe to set up computational pathology workflows. The STAMP workflow requires one workday of hands-on computational execution and basic command line knowledge. △ Less

Submitted 18 December, 2023; originally announced December 2023.

Statistical Parameterized Physics-Based Machine Learning Digital Twin Models for Laser Powder Bed Fusion Process

Authors: Yangfan Li, Satyajit Mojumder, Ye Lu, Abdullah Al Amin, Jiachen Guo, Xiaoyu Xie, Wei Chen, Gregory J. Wagner, Jian Cao, Wing Kam Liu

Abstract: A digital twin (DT) is a virtual representation of physical process, products and/or systems that requires a high-fidelity computational model for continuous update through the integration of sensor data and user input. In the context of laser powder bed fusion (LPBF) additive manufacturing, a digital twin of the manufacturing process can offer predictions for the produced parts, diagnostics for m… ▽ More A digital twin (DT) is a virtual representation of physical process, products and/or systems that requires a high-fidelity computational model for continuous update through the integration of sensor data and user input. In the context of laser powder bed fusion (LPBF) additive manufacturing, a digital twin of the manufacturing process can offer predictions for the produced parts, diagnostics for manufacturing defects, as well as control capabilities. This paper introduces a parameterized physics-based digital twin (PPB-DT) for the statistical predictions of LPBF metal additive manufacturing process. We accomplish this by creating a high-fidelity computational model that accurately represents the melt pool phenomena and subsequently calibrating and validating it through controlled experiments. In PPB-DT, a mechanistic reduced-order method-driven stochastic calibration process is introduced, which enables the statistical predictions of the melt pool geometries and the identification of defects such as lack-of-fusion porosity and surface roughness, specifically for diagnostic applications. Leveraging data derived from this physics-based model and experiments, we have trained a machine learning-based digital twin (PPB-ML-DT) model for predicting, monitoring, and controlling melt pool geometries. These proposed digital twin models can be employed for predictions, control, optimization, and quality assurance within the LPBF process, ultimately expediting product development and certification in LPBF-based metal additive manufacturing. △ Less

Submitted 13 November, 2023; originally announced November 2023.

Comments: arXiv admin note: text overlap with arXiv:2208.02907

Can you text what is happening? Integrating pre-trained language encoders into trajectory prediction models for autonomous driving

Authors: Ali Keysan, Andreas Look, Eitan Kosman, Gonca Gürsun, Jörg Wagner, Yu Yao, Barbara Rakitsch

Abstract: In autonomous driving tasks, scene understanding is the first step towards predicting the future behavior of the surrounding traffic participants. Yet, how to represent a given scene and extract its features are still open research questions. In this study, we propose a novel text-based representation of traffic scenes and process it with a pre-trained language encoder. First, we show that text-… ▽ More In autonomous driving tasks, scene understanding is the first step towards predicting the future behavior of the surrounding traffic participants. Yet, how to represent a given scene and extract its features are still open research questions. In this study, we propose a novel text-based representation of traffic scenes and process it with a pre-trained language encoder. First, we show that text-based representations, combined with classical rasterized image representations, lead to descriptive scene embeddings. Second, we benchmark our predictions on the nuScenes dataset and show significant improvements compared to baselines. Third, we show in an ablation study that a joint encoder of text and rasterized images outperforms the individual encoders confirming that both representations have their complementary strengths. △ Less

Submitted 13 September, 2023; v1 submitted 11 September, 2023; originally announced September 2023.

In Situ Soil Property Estimation for Autonomous Earthmoving Using Physics-Infused Neural Networks

Authors: W. Jacob Wagner, Ahmet Soylemezoglu, Dustin Nottage, Katherine Driggs-Campbell

Abstract: A novel, learning-based method for in situ estimation of soil properties using a physics-infused neural network (PINN) is presented. The network is trained to produce estimates of soil cohesion, angle of internal friction, soil-tool friction, soil failure angle, and residual depth of cut which are then passed through an earthmoving model based on the fundamental equation of earthmoving (FEE) to pr… ▽ More A novel, learning-based method for in situ estimation of soil properties using a physics-infused neural network (PINN) is presented. The network is trained to produce estimates of soil cohesion, angle of internal friction, soil-tool friction, soil failure angle, and residual depth of cut which are then passed through an earthmoving model based on the fundamental equation of earthmoving (FEE) to produce an estimated force. The network ingests a short history of kinematic observations along with past control commands and predicts interaction forces accurately with average error of less than 2kN, 13% of the measured force. To validate the approach, an earthmoving simulation of a bladed vehicle is developed using Vortex Studio, enabling comparison of the estimated parameters to pseudo-ground-truth values which is challenging in real-world experiments. The proposed approach is shown to enable accurate estimation of interaction forces and produces meaningful parameter estimates even when the model and the environmental physics deviate substantially. △ Less

Submitted 5 September, 2023; originally announced September 2023.

Comments: 10 pages, 6 figures, to be published in proceedings of 16th European-African Regional Conference of the International Society for Terrain-Vehicle Systems (ISTVS)

ACM Class: I.2.9

A Robust Localization Solution for an Uncrewed Ground Vehicle in Unstructured Outdoor GNSS-Denied Environments

Authors: W. Jacob Wagner, Isaac Blankenau, Maribel DeLaTorre, Amartya Purushottam, Ahmet Soylemezoglu

Abstract: This work addresses the challenge of developing a localization system for an uncrewed ground vehicle (UGV) operating autonomously in unstructured outdoor Global Navigation Satellite System (GNSS)-denied environments. The goal is to enable accurate mapping and long-range navigation with practical applications in domains such as autonomous construction, military engineering missions, and exploration… ▽ More This work addresses the challenge of developing a localization system for an uncrewed ground vehicle (UGV) operating autonomously in unstructured outdoor Global Navigation Satellite System (GNSS)-denied environments. The goal is to enable accurate mapping and long-range navigation with practical applications in domains such as autonomous construction, military engineering missions, and exploration of non-Earth planets. The proposed system - Terrain-Referenced Assured Engineer Localization System (TRAELS) - integrates pose estimates produced by two complementary terrain referenced navigation (TRN) methods with wheel odometry and inertial measurement unit (IMU) measurements using an Extended Kalman Filter (EKF). Unlike simultaneous localization and mapping (SLAM) systems that require loop closures, the described approach maintains accuracy over long distances and one-way missions without the need to revisit previous positions. Evaluation of TRAELS is performed across a range of environments. In regions where a combination of distinctive geometric and ground surface features are present, the developed TRN methods are leveraged by TRAELS to consistently achieve an absolute trajectory error of less than 3.0 m. The approach is also shown to be capable of recovering from large accumulated drift when traversing feature-sparse areas, which is essential in ensuring robust performance of the system across a wide variety of challenging GNSS-denied environments. Overall, the effectiveness of the system in providing precise localization and mapping capabilities in challenging GNSS-denied environments is demonstrated and an analysis is performed leading to insights for improving TRN approaches for UGVs. △ Less

Submitted 5 September, 2023; originally announced September 2023.

Comments: 15 pages, 9 figures, 2 tables, to be published in The Proceedings of the Institute of Navigation GNSS+ 2023 conference (ION GNSS+ 23)

ACM Class: I.2.9

An Initial Exploration: Learning to Generate Realistic Audio for Silent Video

Authors: Matthew Martel, Jackson Wagner

Abstract: Generating realistic audio effects for movies and other media is a challenging task that is accomplished today primarily through physical techniques known as Foley art. Foley artists create sounds with common objects (e.g., boxing gloves, broken glass) in time with video as it is playing to generate captivating audio tracks. In this work, we aim to develop a deep-learning based framework that does… ▽ More Generating realistic audio effects for movies and other media is a challenging task that is accomplished today primarily through physical techniques known as Foley art. Foley artists create sounds with common objects (e.g., boxing gloves, broken glass) in time with video as it is playing to generate captivating audio tracks. In this work, we aim to develop a deep-learning based framework that does much the same - observes video in it's natural sequence and generates realistic audio to accompany it. Notably, we have reason to believe this is achievable due to advancements in realistic audio generation techniques conditioned on other inputs (e.g., Wavenet conditioned on text). We explore several different model architectures to accomplish this task that process both previously-generated audio and video context. These include deep-fusion CNN, dilated Wavenet CNN with visual context, and transformer-based architectures. We find that the transformer-based architecture yields the most promising results, matching low-frequencies to visual patterns effectively, but failing to generate more nuanced waveforms. △ Less

Submitted 23 August, 2023; originally announced August 2023.

Speech-based Age and Gender Prediction with Transformers

Authors: Felix Burkhardt, Johannes Wagner, Hagen Wierstorf, Florian Eyben, Björn Schuller

Abstract: We report on the curation of several publicly available datasets for age and gender prediction. Furthermore, we present experiments to predict age and gender with models based on a pre-trained wav2vec 2.0. Depending on the dataset, we achieve an MAE between 7.1 years and 10.8 years for age, and at least 91.1% ACC for gender (female, male, child). Compared to a modelling approach built on handcraft… ▽ More We report on the curation of several publicly available datasets for age and gender prediction. Furthermore, we present experiments to predict age and gender with models based on a pre-trained wav2vec 2.0. Depending on the dataset, we achieve an MAE between 7.1 years and 10.8 years for age, and at least 91.1% ACC for gender (female, male, child). Compared to a modelling approach built on handcrafted features, our proposed system shows an improvement of 9% UAR for age and 4% UAR for gender. To make our findings reproducible, we release the best performing model to the community as well as the sample lists of the data splits. △ Less

Submitted 29 June, 2023; originally announced June 2023.

Comments: 5 pages, submitted to 15th ITG Conference on Speech Communication

Broadening the perspective for sustainable AI: Comprehensive sustainability criteria and indicators for AI systems

Authors: Friederike Rohde, Josephin Wagner, Andreas Meyer, Philipp Reinhard, Marcus Voss, Ulrich Petschow, Anne Mollen

Abstract: The increased use of AI systems is associated with multi-faceted societal, environmental, and economic consequences. These include non-transparent decision-making processes, discrimination, increasing inequalities, rising energy consumption and greenhouse gas emissions in AI model development and application, and an increasing concentration of economic power. By considering the multi-dimensionalit… ▽ More The increased use of AI systems is associated with multi-faceted societal, environmental, and economic consequences. These include non-transparent decision-making processes, discrimination, increasing inequalities, rising energy consumption and greenhouse gas emissions in AI model development and application, and an increasing concentration of economic power. By considering the multi-dimensionality of sustainability, this paper takes steps towards substantiating the call for an overarching perspective on "sustainable AI". It presents the SCAIS Framework (Sustainability Criteria and Indicators for Artificial Intelligence Systems) which contains a set 19 sustainability criteria for sustainable AI and 67 indicators that is based on the results of a critical review and expert workshops. This interdisciplinary approach contributes a unique holistic perspective to facilitate and structure the discourse on sustainable AI. Further, it provides a concrete framework that lays the foundation for developing standards and tools to support the conscious development and application of AI systems. △ Less

Submitted 22 November, 2023; v1 submitted 22 June, 2023; originally announced June 2023.

Improving accuracy of GPT-3/4 results on biomedical data using a retrieval-augmented language model

Authors: David Soong, Sriram Sridhar, Han Si, Jan-Samuel Wagner, Ana Caroline Costa Sá, Christina Y Yu, Kubra Karagoz, Meijian Guan, Hisham Hamadeh, Brandon W Higgs

Abstract: Large language models (LLMs) have made significant advancements in natural language processing (NLP). Broad corpora capture diverse patterns but can introduce irrelevance, while focused corpora enhance reliability by reducing misleading information. Training LLMs on focused corpora poses computational challenges. An alternative approach is to use a retrieval-augmentation (RetA) method tested in a… ▽ More Large language models (LLMs) have made significant advancements in natural language processing (NLP). Broad corpora capture diverse patterns but can introduce irrelevance, while focused corpora enhance reliability by reducing misleading information. Training LLMs on focused corpora poses computational challenges. An alternative approach is to use a retrieval-augmentation (RetA) method tested in a specific domain. To evaluate LLM performance, OpenAI's GPT-3, GPT-4, Bing's Prometheus, and a custom RetA model were compared using 19 questions on diffuse large B-cell lymphoma (DLBCL) disease. Eight independent reviewers assessed responses based on accuracy, relevance, and readability (rated 1-3). The RetA model performed best in accuracy (12/19 3-point scores, total=47) and relevance (13/19, 50), followed by GPT-4 (8/19, 43; 11/19, 49). GPT-4 received the highest readability scores (17/19, 55), followed by GPT-3 (15/19, 53) and the RetA model (11/19, 47). Prometheus underperformed in accuracy (34), relevance (32), and readability (38). Both GPT-3.5 and GPT-4 had more hallucinations in all 19 responses compared to the RetA model and Prometheus. Hallucinations were mostly associated with non-existent references or fabricated efficacy data. These findings suggest that RetA models, supplemented with domain-specific corpora, may outperform general-purpose LLMs in accuracy and relevance within specific domains. However, this evaluation was limited to specific questions and metrics and may not capture challenges in semantic search and other NLP tasks. Further research will explore different LLM architectures, RetA methodologies, and evaluation methods to assess strengths and limitations more comprehensively. △ Less

Submitted 30 May, 2023; v1 submitted 26 May, 2023; originally announced May 2023.

Report number: 2305.17116

Journal ref: PLOS Digit Health, 3(8) , 2024

Vision and Control for Grasping Clear Plastic Bags

Authors: Joohwan Seo, Jackson Wagner, Anuj Raicura, Jake Kim

Abstract: We develop two novel vision methods for planning effective grasps for clear plastic bags, as well as a control method to enable a Sawyer arm with a parallel gripper to execute the grasps. The first vision method is based on classical image processing and heuristics (e.g., Canny edge detection) to select a grasp target and angle. The second uses a deep-learning model trained on a human-labeled data… ▽ More We develop two novel vision methods for planning effective grasps for clear plastic bags, as well as a control method to enable a Sawyer arm with a parallel gripper to execute the grasps. The first vision method is based on classical image processing and heuristics (e.g., Canny edge detection) to select a grasp target and angle. The second uses a deep-learning model trained on a human-labeled data set to mimic human grasp decisions. A clustering algorithm is used to de-noise the outputs of each vision method. Subsequently, a workspace PD control method is used to execute each grasp. Of the two vision methods, we find the deep-learning based method to be more effective. △ Less

Submitted 12 May, 2023; originally announced May 2023.

Comments: 5 pages, 6 figures

Strategy-proof Budgeting via a VCG-like Mechanism

Authors: Jonathan Wagner, Reshef Meir

Abstract: We present a strategy-proof public goods budgeting mechanism where agents determine both the total volume of expanses and the specific allocation. It is constructed as a modification of VCG to a less typical environment, namely where we do not assume quasi-linear utilities nor direct revelation. We further show that under plausible assumptions it satisfies strategy-proofness in strictly dominant s… ▽ More We present a strategy-proof public goods budgeting mechanism where agents determine both the total volume of expanses and the specific allocation. It is constructed as a modification of VCG to a less typical environment, namely where we do not assume quasi-linear utilities nor direct revelation. We further show that under plausible assumptions it satisfies strategy-proofness in strictly dominant strategies, and consequently implements the social optimum as a Coalition Proof Nash Equilibrium. A primary (albeit not an exclusive) motivation of our model is Participatory Budgeting, where members of a community collectively decide the spending policy of public tax dollars. While incentives alignment in our mechanism, as in classic VCG, is achieved via individual payments we charge from agents, in a PB context that seems unreasonable. Our second main result thus provides that, under further specifications relevant in that context, these payments will vanish in large populations. In the last section we expand the mechanism's definition to a class of mechanisms in which the designer can prioritize certain outcomes she sees as desirable. In particular we give the example of favoring equitable (egalitarian) allocations. △ Less

Submitted 13 March, 2023; originally announced March 2023.

Comments: 27 pages, Manuscript submitted for review to the 24nd ACM Conference on Economics & Computation (EC'23)

audb -- Sharing and Versioning of Audio and Annotation Data in Python

Authors: Hagen Wierstorf, Johannes Wagner, Florian Eyben, Felix Burkhardt, Björn W. Schuller

Abstract: Driven by the need for larger and more diverse datasets to pre-train and fine-tune increasingly complex machine learning models, the number of datasets is rapidly growing. audb is an open-source Python library that supports versioning and documentation of audio datasets. It aims to provide a standardized and simple user-interface to publish, maintain, and access the annotations and audio files of… ▽ More Driven by the need for larger and more diverse datasets to pre-train and fine-tune increasingly complex machine learning models, the number of datasets is rapidly growing. audb is an open-source Python library that supports versioning and documentation of audio datasets. It aims to provide a standardized and simple user-interface to publish, maintain, and access the annotations and audio files of a dataset. To efficiently store the data on a server, audb automatically resolves dependencies between versions of a dataset and only uploads newly added or altered files when a new version is published. The library supports partial loading of a dataset and local caching for fast access. audb is a lightweight library and can be interfaced from any machine learning library. It supports the management of datasets on a single PC, within a university or company, or within a whole research community. △ Less

Submitted 10 May, 2023; v1 submitted 1 March, 2023; originally announced March 2023.

Fully transformer-based biomarker prediction from colorectal cancer histology: a large-scale multicentric study

Authors: Sophia J. Wagner, Daniel Reisenbüchler, Nicholas P. West, Jan Moritz Niehues, Gregory Patrick Veldhuizen, Philip Quirke, Heike I. Grabsch, Piet A. van den Brandt, Gordon G. A. Hutchins, Susan D. Richman, Tanwei Yuan, Rupert Langer, Josien Christina Anna Jenniskens, Kelly Offermans, Wolfram Mueller, Richard Gray, Stephen B. Gruber, Joel K. Greenson, Gad Rennert, Joseph D. Bonner, Daniel Schmolze, Jacqueline A. James, Maurice B. Loughrey, Manuel Salto-Tellez, Hermann Brenner , et al. (6 additional authors not shown)

Abstract: Background: Deep learning (DL) can extract predictive and prognostic biomarkers from routine pathology slides in colorectal cancer. For example, a DL test for the diagnosis of microsatellite instability (MSI) in CRC has been approved in 2022. Current approaches rely on convolutional neural networks (CNNs). Transformer networks are outperforming CNNs and are replacing them in many applications, but… ▽ More Background: Deep learning (DL) can extract predictive and prognostic biomarkers from routine pathology slides in colorectal cancer. For example, a DL test for the diagnosis of microsatellite instability (MSI) in CRC has been approved in 2022. Current approaches rely on convolutional neural networks (CNNs). Transformer networks are outperforming CNNs and are replacing them in many applications, but have not been used for biomarker prediction in cancer at a large scale. In addition, most DL approaches have been trained on small patient cohorts, which limits their clinical utility. Methods: In this study, we developed a new fully transformer-based pipeline for end-to-end biomarker prediction from pathology slides. We combine a pre-trained transformer encoder and a transformer network for patch aggregation, capable of yielding single and multi-target prediction at patient level. We train our pipeline on over 9,000 patients from 10 colorectal cancer cohorts. Results: A fully transformer-based approach massively improves the performance, generalizability, data efficiency, and interpretability as compared with current state-of-the-art algorithms. After training on a large multicenter cohort, we achieve a sensitivity of 0.97 with a negative predictive value of 0.99 for MSI prediction on surgical resection specimens. We demonstrate for the first time that resection specimen-only training reaches clinical-grade performance on endoscopic biopsy tissue, solving a long-standing diagnostic problem. Interpretation: A fully transformer-based end-to-end pipeline trained on thousands of pathology slides yields clinical-grade performance for biomarker prediction on surgical resections and biopsies. Our new methods are freely available under an open source license. △ Less

Submitted 1 March, 2023; v1 submitted 23 January, 2023; originally announced January 2023.

Comments: Updated Figure 2 and Table A.5

MABe22: A Multi-Species Multi-Task Benchmark for Learned Representations of Behavior

Authors: Jennifer J. Sun, Markus Marks, Andrew Ulmer, Dipam Chakraborty, Brian Geuther, Edward Hayes, Heng Jia, Vivek Kumar, Sebastian Oleszko, Zachary Partridge, Milan Peelman, Alice Robie, Catherine E. Schretter, Keith Sheppard, Chao Sun, Param Uttarwar, Julian M. Wagner, Eric Werner, Joseph Parker, Pietro Perona, Yisong Yue, Kristin Branson, Ann Kennedy

Abstract: We introduce MABe22, a large-scale, multi-agent video and trajectory benchmark to assess the quality of learned behavior representations. This dataset is collected from a variety of biology experiments, and includes triplets of interacting mice (4.7 million frames video+pose tracking data, 10 million frames pose only), symbiotic beetle-ant interactions (10 million frames video data), and groups of… ▽ More We introduce MABe22, a large-scale, multi-agent video and trajectory benchmark to assess the quality of learned behavior representations. This dataset is collected from a variety of biology experiments, and includes triplets of interacting mice (4.7 million frames video+pose tracking data, 10 million frames pose only), symbiotic beetle-ant interactions (10 million frames video data), and groups of interacting flies (4.4 million frames of pose tracking data). Accompanying these data, we introduce a panel of real-life downstream analysis tasks to assess the quality of learned representations by evaluating how well they preserve information about the experimental conditions (e.g. strain, time of day, optogenetic stimulation) and animal behavior. We test multiple state-of-the-art self-supervised video and trajectory representation learning methods to demonstrate the use of our benchmark, revealing that methods developed using human action datasets do not fully translate to animal datasets. We hope that our benchmark and dataset encourage a broader exploration of behavior representation learning methods across species and settings. △ Less

Submitted 30 June, 2023; v1 submitted 21 July, 2022; originally announced July 2022.

Comments: To appear in ICML 2023, Project website: https://sites.google.com/view/computational-behavior/our-datasets/mabe2022-dataset

Quantum Circuit Compiler for a Shuttling-Based Trapped-Ion Quantum Computer

Authors: Fabian Kreppel, Christian Melzer, Diego Olvera Millán, Janis Wagner, Janine Hilder, Ulrich Poschinger, Ferdinand Schmidt-Kaler, André Brinkmann

Abstract: The increasing capabilities of quantum computing hardware and the challenge of realizing deep quantum circuits require fully automated and efficient tools for compiling quantum circuits. To express arbitrary circuits in a sequence of native gates specific to the quantum computer architecture, it is necessary to make algorithms portable across the landscape of quantum hardware providers. In this wo… ▽ More The increasing capabilities of quantum computing hardware and the challenge of realizing deep quantum circuits require fully automated and efficient tools for compiling quantum circuits. To express arbitrary circuits in a sequence of native gates specific to the quantum computer architecture, it is necessary to make algorithms portable across the landscape of quantum hardware providers. In this work, we present a compiler capable of transforming and optimizing a quantum circuit targeting a shuttling-based trapped-ion quantum processor. It consists of custom algorithms set on top of the quantum circuit framework Pytket. The performance was evaluated for a wide range of quantum circuits and the results show that the gate counts can be reduced by factors up to 5.1 compared to standard Pytket and up to 2.2 compared to standard Qiskit compilation. △ Less

Submitted 2 November, 2023; v1 submitted 5 July, 2022; originally announced July 2022.

Comments: 35 pages, 25 figures, 4 tables, accepted in Quantum

MSC Class: 81P65; 81P68; 68Q09 ACM Class: D.3.4

Journal ref: Quantum 7, 1176 (2023)

Local Attention Graph-based Transformer for Multi-target Genetic Alteration Prediction

Authors: Daniel Reisenbüchler, Sophia J. Wagner, Melanie Boxberg, Tingying Peng

Abstract: Classical multiple instance learning (MIL) methods are often based on the identical and independent distributed assumption between instances, hence neglecting the potentially rich contextual information beyond individual entities. On the other hand, Transformers with global self-attention modules have been proposed to model the interdependencies among all instances. However, in this paper we quest… ▽ More Classical multiple instance learning (MIL) methods are often based on the identical and independent distributed assumption between instances, hence neglecting the potentially rich contextual information beyond individual entities. On the other hand, Transformers with global self-attention modules have been proposed to model the interdependencies among all instances. However, in this paper we question: Is global relation modeling using self-attention necessary, or can we appropriately restrict self-attention calculations to local regimes in large-scale whole slide images (WSIs)? We propose a general-purpose local attention graph-based Transformer for MIL (LA-MIL), introducing an inductive bias by explicitly contextualizing instances in adaptive local regimes of arbitrary size. Additionally, an efficiently adapted loss function enables our approach to learn expressive WSI embeddings for the joint analysis of multiple biomarkers. We demonstrate that LA-MIL achieves state-of-the-art results in mutation prediction for gastrointestinal cancer, outperforming existing models on important biomarkers such as microsatellite instability for colorectal cancer. Our findings suggest that local self-attention sufficiently models dependencies on par with global modules. Our LA-MIL implementation is available at https://github.com/agentdr1/LA_MIL. △ Less

Submitted 17 June, 2022; v1 submitted 13 May, 2022; originally announced May 2022.

Deep Vehicle Detection in Satellite Video

Authors: Roman Pflugfelder, Axel Weissenfeld, Julian Wagner

Abstract: This work presents a deep learning approach for vehicle detection in satellite video. Vehicle detection is perhaps impossible in single EO satellite images due to the tininess of vehicles (4-10 pixel) and their similarity to the background. Instead, we consider satellite video which overcomes the lack of spatial information by temporal consistency of vehicle movement. A new spatiotemporal model of… ▽ More This work presents a deep learning approach for vehicle detection in satellite video. Vehicle detection is perhaps impossible in single EO satellite images due to the tininess of vehicles (4-10 pixel) and their similarity to the background. Instead, we consider satellite video which overcomes the lack of spatial information by temporal consistency of vehicle movement. A new spatiotemporal model of a compact $3 \times 3$ convolutional, neural network is proposed which neglects pooling layers and uses leaky ReLUs. Then we use a reformulation of the output heatmap including Non-Maximum-Suppression (NMS) for the final segmentation. Empirical results on two new annotated satellite videos reconfirm the applicability of this approach for vehicle detection. They more importantly indicate that pre-training on WAMI data and then fine-tuning on few annotated video frames for a new video is sufficient. In our experiment only five annotated images yield a $F_1$ score of 0.81 on a new video showing more complex traffic patterns than the Las Vegas video. Our best result on Las Vegas is a $F_1$ score of 0.87 which makes the proposed approach a leading method for this benchmark. △ Less

Submitted 7 June, 2022; v1 submitted 14 April, 2022; originally announced April 2022.

Probing Speech Emotion Recognition Transformers for Linguistic Knowledge

Authors: Andreas Triantafyllopoulos, Johannes Wagner, Hagen Wierstorf, Maximilian Schmitt, Uwe Reichel, Florian Eyben, Felix Burkhardt, Björn W. Schuller

Abstract: Large, pre-trained neural networks consisting of self-attention layers (transformers) have recently achieved state-of-the-art results on several speech emotion recognition (SER) datasets. These models are typically pre-trained in self-supervised manner with the goal to improve automatic speech recognition performance -- and thus, to understand linguistic information. In this work, we investigate t… ▽ More Large, pre-trained neural networks consisting of self-attention layers (transformers) have recently achieved state-of-the-art results on several speech emotion recognition (SER) datasets. These models are typically pre-trained in self-supervised manner with the goal to improve automatic speech recognition performance -- and thus, to understand linguistic information. In this work, we investigate the extent in which this information is exploited during SER fine-tuning. Using a reproducible methodology based on open-source tools, we synthesise prosodically neutral speech utterances while varying the sentiment of the text. Valence predictions of the transformer model are very reactive to positive and negative sentiment content, as well as negations, but not to intensifiers or reducers, while none of those linguistic features impact arousal or dominance. These findings show that transformers can successfully leverage linguistic information to improve their valence predictions, and that linguistic analysis should be included in their testing. △ Less

Submitted 26 July, 2022; v1 submitted 1 April, 2022; originally announced April 2022.

Comments: Accepted in INTERSPEECH 2022

Journal ref: Proc. Interspeech 2022, 146-150

Dawn of the transformer era in speech emotion recognition: closing the valence gap

Authors: Johannes Wagner, Andreas Triantafyllopoulos, Hagen Wierstorf, Maximilian Schmitt, Felix Burkhardt, Florian Eyben, Björn W. Schuller

Abstract: Recent advances in transformer-based architectures which are pre-trained in self-supervised manner have shown great promise in several machine learning tasks. In the audio domain, such architectures have also been successfully utilised in the field of speech emotion recognition (SER). However, existing works have not evaluated the influence of model size and pre-training data on downstream perform… ▽ More Recent advances in transformer-based architectures which are pre-trained in self-supervised manner have shown great promise in several machine learning tasks. In the audio domain, such architectures have also been successfully utilised in the field of speech emotion recognition (SER). However, existing works have not evaluated the influence of model size and pre-training data on downstream performance, and have shown limited attention to generalisation, robustness, fairness, and efficiency. The present contribution conducts a thorough analysis of these aspects on several pre-trained variants of wav2vec 2.0 and HuBERT that we fine-tuned on the dimensions arousal, dominance, and valence of MSP-Podcast, while additionally using IEMOCAP and MOSI to test cross-corpus generalisation. To the best of our knowledge, we obtain the top performance for valence prediction without use of explicit linguistic information, with a concordance correlation coefficient (CCC) of .638 on MSP-Podcast. Furthermore, our investigations reveal that transformer-based architectures are more robust to small perturbations compared to a CNN-based baseline and fair with respect to biological sex groups, but not towards individual speakers. Finally, we are the first to show that their extraordinary success on valence is based on implicit linguistic information learnt during fine-tuning of the transformer layers, which explains why they perform on-par with recent multimodal approaches that explicitly utilise textual information. Our findings collectively paint the following picture: transformer-based architectures constitute the new state-of-the-art in SER, but further advances are needed to mitigate remaining robustness and individual speaker issues. To make our findings reproducible, we release the best performing model to the community. △ Less

Submitted 7 September, 2023; v1 submitted 14 March, 2022; originally announced March 2022.

Journal ref: in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 9, pp. 10745-10759, 1 Sept. 2023

S5CL: Unifying Fully-Supervised, Self-Supervised, and Semi-Supervised Learning Through Hierarchical Contrastive Learning

Authors: Manuel Tran, Sophia J. Wagner, Melanie Boxberg, Tingying Peng

Abstract: In computational pathology, we often face a scarcity of annotations and a large amount of unlabeled data. One method for dealing with this is semi-supervised learning which is commonly split into a self-supervised pretext task and a subsequent model fine-tuning. Here, we compress this two-stage training into one by introducing S5CL, a unified framework for fully-supervised, self-supervised, and se… ▽ More In computational pathology, we often face a scarcity of annotations and a large amount of unlabeled data. One method for dealing with this is semi-supervised learning which is commonly split into a self-supervised pretext task and a subsequent model fine-tuning. Here, we compress this two-stage training into one by introducing S5CL, a unified framework for fully-supervised, self-supervised, and semi-supervised learning. With three contrastive losses defined for labeled, unlabeled, and pseudo-labeled images, S5CL can learn feature representations that reflect the hierarchy of distance relationships: similar images and augmentations are embedded the closest, followed by different looking images of the same class, while images from separate classes have the largest distance. Moreover, S5CL allows us to flexibly combine these losses to adapt to different scenarios. Evaluations of our framework on two public histopathological datasets show strong improvements in the case of sparse labels: for a H&E-stained colorectal cancer dataset, the accuracy increases by up to 9% compared to supervised cross-entropy loss; for a highly imbalanced dataset of single white blood cells from leukemia patient blood smears, the F1-score increases by up to 6%. △ Less

Submitted 14 March, 2022; originally announced March 2022.

Multi-Modality Microscopy Image Style Transfer for Nuclei Segmentation

Authors: Ye Liu, Sophia J. Wagner, Tingying Peng

Abstract: Annotating microscopy images for nuclei segmentation is laborious and time-consuming. To leverage the few existing annotations, also across multiple modalities, we propose a novel microscopy-style augmentation technique based on a generative adversarial network (GAN). Unlike other style transfer methods, it can not only deal with different cell assay types and lighting conditions, but also with di… ▽ More Annotating microscopy images for nuclei segmentation is laborious and time-consuming. To leverage the few existing annotations, also across multiple modalities, we propose a novel microscopy-style augmentation technique based on a generative adversarial network (GAN). Unlike other style transfer methods, it can not only deal with different cell assay types and lighting conditions, but also with different imaging modalities, such as bright-field and fluorescence microscopy. Using disentangled representations for content and style, we can preserve the structure of the original image while altering its style during augmentation. We evaluate our data augmentation on the 2018 Data Science Bowl dataset consisting of various cell assays, lighting conditions, and imaging modalities. With our style augmentation, the segmentation accuracy of the two top-ranked Mask R-CNN-based nuclei segmentation algorithms in the competition increases significantly. Thus, our augmentation technique renders the downstream task more robust to the test data heterogeneity and helps counteract class imbalance without resampling of minority classes. △ Less

Submitted 23 November, 2021; originally announced November 2021.

Comments: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

Revisiting Tri-training of Dependency Parsers

Authors: Joachim Wagner, Jennifer Foster

Abstract: We compare two orthogonal semi-supervised learning techniques, namely tri-training and pretrained word embeddings, in the task of dependency parsing. We explore language-specific FastText and ELMo embeddings and multilingual BERT embeddings. We focus on a low resource scenario as semi-supervised learning can be expected to have the most impact here. Based on treebank size and available ELMo models… ▽ More We compare two orthogonal semi-supervised learning techniques, namely tri-training and pretrained word embeddings, in the task of dependency parsing. We explore language-specific FastText and ELMo embeddings and multilingual BERT embeddings. We focus on a low resource scenario as semi-supervised learning can be expected to have the most impact here. Based on treebank size and available ELMo models, we select Hungarian, Uyghur (a zero-shot language for mBERT) and Vietnamese. Furthermore, we include English in a simulated low-resource setting. We find that pretrained word embeddings make more effective use of unlabelled data than tri-training but that the two approaches can be successfully combined. △ Less

Submitted 16 September, 2021; originally announced September 2021.

Comments: 17 pages, 1 figure, to be published at EMNLP 2021

gaBERT -- an Irish Language Model

Authors: James Barry, Joachim Wagner, Lauren Cassidy, Alan Cowap, Teresa Lynn, Abigail Walsh, Mícheál J. Ó Meachair, Jennifer Foster

Abstract: The BERT family of neural language models have become highly popular due to their ability to provide sequences of text with rich context-sensitive token encodings which are able to generalise well to many NLP tasks. We introduce gaBERT, a monolingual BERT model for the Irish language. We compare our gaBERT model to multilingual BERT and the monolingual Irish WikiBERT, and we show that gaBERT provi… ▽ More The BERT family of neural language models have become highly popular due to their ability to provide sequences of text with rich context-sensitive token encodings which are able to generalise well to many NLP tasks. We introduce gaBERT, a monolingual BERT model for the Irish language. We compare our gaBERT model to multilingual BERT and the monolingual Irish WikiBERT, and we show that gaBERT provides better representations for a downstream parsing task. We also show how different filtering criteria, vocabulary size and the choice of subword tokenisation model affect downstream performance. We compare the results of fine-tuning a gaBERT model with an mBERT model for the task of identifying verbal multiword expressions, and show that the fine-tuned gaBERT model also performs better at this task. We release gaBERT and related code to the community. △ Less

Submitted 28 June, 2022; v1 submitted 27 July, 2021; originally announced July 2021.

Comments: Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 4774-4788, Marseille, France, 20-25 June 2022, European Language Resources Association (ELRA)

Structure-Preserving Multi-Domain Stain Color Augmentation using Style-Transfer with Disentangled Representations

Authors: Sophia J. Wagner, Nadieh Khalili, Raghav Sharma, Melanie Boxberg, Carsten Marr, Walter de Back, Tingying Peng

Abstract: In digital pathology, different staining procedures and scanners cause substantial color variations in whole-slide images (WSIs), especially across different laboratories. These color shifts result in a poor generalization of deep learning-based methods from the training domain to external pathology data. To increase test performance, stain normalization techniques are used to reduce the variance… ▽ More In digital pathology, different staining procedures and scanners cause substantial color variations in whole-slide images (WSIs), especially across different laboratories. These color shifts result in a poor generalization of deep learning-based methods from the training domain to external pathology data. To increase test performance, stain normalization techniques are used to reduce the variance between training and test domain. Alternatively, color augmentation can be applied during training leading to a more robust model without the extra step of color normalization at test time. We propose a novel color augmentation technique, HistAuGAN, that can simulate a wide variety of realistic histology stain colors, thus making neural networks stain-invariant when applied during training. Based on a generative adversarial network (GAN) for image-to-image translation, our model disentangles the content of the image, i.e., the morphological tissue structure, from the stain color attributes. It can be trained on multiple domains and, therefore, learns to cover different stain colors as well as other domain-specific variations introduced in the slide preparation and imaging process. We demonstrate that HistAuGAN outperforms conventional color augmentation techniques on a classification task on the publicly available dataset Camelyon17 and show that it is able to mitigate present batch effects. △ Less

Submitted 26 July, 2021; originally announced July 2021.

Comments: accepted at MICCAI 2021, code and model weights are available at http://github.com/sophiajw/HistAuGAN

The DCU-EPFL Enhanced Dependency Parser at the IWPT 2021 Shared Task

Authors: James Barry, Alireza Mohammadshahi, Joachim Wagner, Jennifer Foster, James Henderson

Abstract: We describe the DCU-EPFL submission to the IWPT 2021 Shared Task on Parsing into Enhanced Universal Dependencies. The task involves parsing Enhanced UD graphs, which are an extension of the basic dependency trees designed to be more facilitative towards representing semantic structure. Evaluation is carried out on 29 treebanks in 17 languages and participants are required to parse the data from ea… ▽ More We describe the DCU-EPFL submission to the IWPT 2021 Shared Task on Parsing into Enhanced Universal Dependencies. The task involves parsing Enhanced UD graphs, which are an extension of the basic dependency trees designed to be more facilitative towards representing semantic structure. Evaluation is carried out on 29 treebanks in 17 languages and participants are required to parse the data from each language starting from raw strings. Our approach uses the Stanza pipeline to preprocess the text files, XLMRoBERTa to obtain contextualized token representations, and an edge-scoring and labeling model to predict the enhanced graph. Finally, we run a post-processing script to ensure all of our outputs are valid Enhanced UD graphs. Our system places 6th out of 9 participants with a coarse Enhanced Labeled Attachment Score (ELAS) of 83.57. We carry out additional post-deadline experiments which include using Trankit for pre-processing, XLM-RoBERTa-LARGE, treebank concatenation, and multitask learning between a basic and an enhanced dependency parser. All of these modifications improve our initial score and our final system has a coarse ELAS of 88.04. △ Less

Submitted 5 July, 2021; originally announced July 2021.

Comments: Submitted to the IWPT 2021 Shared Task: From Raw Text to Enhanced Universal Dependencies: the Parsing Shared Task at IWPT 2021

Pedestrian Behavior Prediction for Automated Driving: Requirements, Metrics, and Relevant Features

Authors: Michael Herman, Jörg Wagner, Vishnu Prabhakaran, Nicolas Möser, Hanna Ziesche, Waleed Ahmed, Lutz Bürkle, Ernst Kloppenburg, Claudius Gläser

Abstract: Automated vehicles require a comprehensive understanding of traffic situations to ensure safe and anticipatory driving. In this context, the prediction of pedestrians is particularly challenging as pedestrian behavior can be influenced by multiple factors. In this paper, we thoroughly analyze the requirements on pedestrian behavior prediction for automated driving via a system-level approach. To t… ▽ More Automated vehicles require a comprehensive understanding of traffic situations to ensure safe and anticipatory driving. In this context, the prediction of pedestrians is particularly challenging as pedestrian behavior can be influenced by multiple factors. In this paper, we thoroughly analyze the requirements on pedestrian behavior prediction for automated driving via a system-level approach. To this end we investigate real-world pedestrian-vehicle interactions with human drivers. Based on human driving behavior we then derive appropriate reaction patterns of an automated vehicle and determine requirements for the prediction of pedestrians. This includes a novel metric tailored to measure prediction performance from a system-level perspective. The proposed metric is evaluated on a large-scale dataset comprising thousands of real-world pedestrian-vehicle interactions. We furthermore conduct an ablation study to evaluate the importance of different contextual cues and compare these results to ones obtained using established performance metrics for pedestrian prediction. Our results highlight the importance of a system-level approach to pedestrian behavior prediction. △ Less

Submitted 16 October, 2021; v1 submitted 15 December, 2020; originally announced December 2020.

Comments: This work has been submitted to the IEEE Transactions on Intelligent Transportation Systems for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible. Revision: Extended requirement analysis and evaluation. 16 pages

papaya2: 2D Irreducible Minkowski Tensor computation

Authors: Fabian M. Schaller, Jenny Wagner, Sebastian C. Kapfer

Abstract: A common challenge in scientific and technical domains is the quantitative description of geometries and shapes, e.g. in the analysis of microscope imagery or astronomical observation data. Frequently, it is desirable to go beyond scalar shape metrics such as porosity and surface to volume ratios because the samples are anisotropic or because direction-dependent quantities such as conductances or… ▽ More A common challenge in scientific and technical domains is the quantitative description of geometries and shapes, e.g. in the analysis of microscope imagery or astronomical observation data. Frequently, it is desirable to go beyond scalar shape metrics such as porosity and surface to volume ratios because the samples are anisotropic or because direction-dependent quantities such as conductances or elasticity are of interest. Minkowski Tensors are a systematic family of versatile and robust higher-order shape descriptors that allow for shape characterization of arbitrary order and promise a path to systematic structure-function relationships for direction-dependent properties. Papaya2 is a software to calculate 2D higher-order shape metrics with a library interface, support for Irreducible Minkowski Tensors and interpolated marching squares. Extensions to Matlab, JavaScript and Python are provided as well. While the tensor of inertia is computed by many tools, we are not aware of other open-source software which provides higher-rank shape characterization in 2D. △ Less

Submitted 28 October, 2020; originally announced October 2020.

Comments: 5 pages, 3 figures, published in the Journal of Open Source Software, code available at https://morphometry.org/software/papaya2/

Journal ref: Journal of Open Source Software, 5(54) (2020)

The ADAPT Enhanced Dependency Parser at the IWPT 2020 Shared Task

Authors: James Barry, Joachim Wagner, Jennifer Foster

Abstract: We describe the ADAPT system for the 2020 IWPT Shared Task on parsing enhanced Universal Dependencies in 17 languages. We implement a pipeline approach using UDPipe and UDPipe-future to provide initial levels of annotation. The enhanced dependency graph is either produced by a graph-based semantic dependency parser or is built from the basic tree using a small set of heuristics. Our results show t… ▽ More We describe the ADAPT system for the 2020 IWPT Shared Task on parsing enhanced Universal Dependencies in 17 languages. We implement a pipeline approach using UDPipe and UDPipe-future to provide initial levels of annotation. The enhanced dependency graph is either produced by a graph-based semantic dependency parser or is built from the basic tree using a small set of heuristics. Our results show that, for the majority of languages, a semantic dependency parser can be successfully applied to the task of parsing enhanced dependencies. Unfortunately, we did not ensure a connected graph as part of our pipeline approach and our competition submission relied on a last-minute fix to pass the validation script which harmed our official evaluation scores significantly. Our submission ranked eighth in the official evaluation with a macro-averaged coarse ELAS F1 of 67.23 and a treebank average of 67.49. We later implemented our own graph-connecting fix which resulted in a score of 79.53 (language average) or 79.76 (treebank average), which would have placed fourth in the competition evaluation. △ Less

Submitted 3 September, 2020; originally announced September 2020.

Comments: Submitted to the 2020 IWPT shared task on parsing Enhanced Universal Dependencies

Journal ref: Proceedings of the 16th International Conference on Parsing Technologies and the IWPT 2020 Shared Task (2020) 227-235

Treebank Embedding Vectors for Out-of-domain Dependency Parsing

Authors: Joachim Wagner, James Barry, Jennifer Foster

Abstract: A recent advance in monolingual dependency parsing is the idea of a treebank embedding vector, which allows all treebanks for a particular language to be used as training data while at the same time allowing the model to prefer training data from one treebank over others and to select the preferred treebank at test time. We build on this idea by 1) introducing a method to predict a treebank vector… ▽ More A recent advance in monolingual dependency parsing is the idea of a treebank embedding vector, which allows all treebanks for a particular language to be used as training data while at the same time allowing the model to prefer training data from one treebank over others and to select the preferred treebank at test time. We build on this idea by 1) introducing a method to predict a treebank vector for sentences that do not come from a treebank used in training, and 2) exploring what happens when we move away from predefined treebank embedding vectors during test time and instead devise tailored interpolations. We show that 1) there are interpolated vectors that are superior to the predefined ones, and 2) treebank vectors can be predicted with sufficient accuracy, for nine out of ten test languages, to match the performance of an oracle approach that knows the most suitable predefined treebank embedding for the test set. △ Less

Submitted 2 May, 2020; originally announced May 2020.

Comments: Camera ready for ACL 2020

On Learning Vehicle Detection in Satellite Video

Authors: Roman Pflugfelder, Axel Weissenfeld, Julian Wagner

Abstract: Vehicle detection in aerial and satellite images is still challenging due to their tiny appearance in pixels compared to the overall size of remote sensing imagery. Classical methods of object detection very often fail in this scenario due to violation of implicit assumptions made such as rich texture, small to moderate ratios between image size and object size. Satellite video is a very new modal… ▽ More Vehicle detection in aerial and satellite images is still challenging due to their tiny appearance in pixels compared to the overall size of remote sensing imagery. Classical methods of object detection very often fail in this scenario due to violation of implicit assumptions made such as rich texture, small to moderate ratios between image size and object size. Satellite video is a very new modality which introduces temporal consistency as inductive bias. Approaches for vehicle detection in satellite video use either background subtraction, frame differencing or subspace methods showing moderate performance (0.26 - 0.82 $F_1$ score). This work proposes to apply recent work on deep learning for wide-area motion imagery (WAMI) on satellite video. We show in a first approach comparable results (0.84 $F_1$) on Planet's SkySat-1 LasVegas video with room for further improvement. △ Less

Submitted 29 January, 2020; originally announced January 2020.

Comments: accepted by Computer Vision Winter Workshop (https://cvww2020.vicos.si)

Towards a Predictive Patent Analytics and Evaluation Platform

Authors: Nebula Alam, Khoi-Nguyen Tran, Sue Ann Chen, John Wagner, Josh Andres, Mukesh Mohania

Abstract: The importance of patents is well recognised across many regions of the world. Many patent mining systems have been proposed, but with limited predictive capabilities. In this demo, we showcase how predictive algorithms leveraging the state-of-the-art machine learning and deep learning techniques can be used to improve understanding of patents for inventors, patent evaluators, and business analyst… ▽ More The importance of patents is well recognised across many regions of the world. Many patent mining systems have been proposed, but with limited predictive capabilities. In this demo, we showcase how predictive algorithms leveraging the state-of-the-art machine learning and deep learning techniques can be used to improve understanding of patents for inventors, patent evaluators, and business analysts alike. Our demo video is available at http://ibm.biz/ecml2019-demo-patent-analytics △ Less

Submitted 31 October, 2019; originally announced October 2019.

Comments: ECML-PKDD 2019 - Demo Track

Cross-lingual Parsing with Polyglot Training and Multi-treebank Learning: A Faroese Case Study

Authors: James Barry, Joachim Wagner, Jennifer Foster

Abstract: Cross-lingual dependency parsing involves transferring syntactic knowledge from one language to another. It is a crucial component for inducing dependency parsers in low-resource scenarios where no training data for a language exists. Using Faroese as the target language, we compare two approaches using annotation projection: first, projecting from multiple monolingual source models; second, proje… ▽ More Cross-lingual dependency parsing involves transferring syntactic knowledge from one language to another. It is a crucial component for inducing dependency parsers in low-resource scenarios where no training data for a language exists. Using Faroese as the target language, we compare two approaches using annotation projection: first, projecting from multiple monolingual source models; second, projecting from a single polyglot model which is trained on the combination of all source languages. Furthermore, we reproduce multi-source projection (Tyers et al., 2018), in which dependency trees of multiple sources are combined. Finally, we apply multi-treebank modelling to the projected treebanks, in addition to or alternatively to polyglot modelling on the source side. We find that polyglot training on the source languages produces an overall trend of better results on the target language but the single best result for the target language is obtained by projecting from monolingual source parsing models and then training multi-treebank POS tagging and parsing models on the target side. △ Less

Submitted 17 October, 2019; originally announced October 2019.

Comments: Submitted to the DeepLo workshop at EMNLP

Interpretable and Fine-Grained Visual Explanations for Convolutional Neural Networks

Authors: Jörg Wagner, Jan Mathias Köhler, Tobias Gindele, Leon Hetzel, Jakob Thaddäus Wiedemer, Sven Behnke

Abstract: To verify and validate networks, it is essential to gain insight into their decisions, limitations as well as possible shortcomings of training data. In this work, we propose a post-hoc, optimization based visual explanation method, which highlights the evidence in the input image for a specific prediction. Our approach is based on a novel technique to defend against adversarial evidence (i.e. fau… ▽ More To verify and validate networks, it is essential to gain insight into their decisions, limitations as well as possible shortcomings of training data. In this work, we propose a post-hoc, optimization based visual explanation method, which highlights the evidence in the input image for a specific prediction. Our approach is based on a novel technique to defend against adversarial evidence (i.e. faulty evidence due to artefacts) by filtering gradients during optimization. The defense does not depend on human-tuned parameters. It enables explanations which are both fine-grained and preserve the characteristics of images, such as edges and colors. The explanations are interpretable, suited for visualizing detailed evidence and can be tested as they are valid model inputs. We qualitatively and quantitatively evaluate our approach on a multitude of models and datasets. △ Less

Submitted 7 August, 2019; originally announced August 2019.

Comments: In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA, June 2019

Wasserstein Adversarial Imitation Learning

Authors: Huang Xiao, Michael Herman, Joerg Wagner, Sebastian Ziesche, Jalal Etesami, Thai Hong Linh

Abstract: Imitation Learning describes the problem of recovering an expert policy from demonstrations. While inverse reinforcement learning approaches are known to be very sample-efficient in terms of expert demonstrations, they usually require problem-dependent reward functions or a (task-)specific reward-function regularization. In this paper, we show a natural connection between inverse reinforcement lea… ▽ More Imitation Learning describes the problem of recovering an expert policy from demonstrations. While inverse reinforcement learning approaches are known to be very sample-efficient in terms of expert demonstrations, they usually require problem-dependent reward functions or a (task-)specific reward-function regularization. In this paper, we show a natural connection between inverse reinforcement learning approaches and Optimal Transport, that enables more general reward functions with desirable properties (e.g., smoothness). Based on our observation, we propose a novel approach called Wasserstein Adversarial Imitation Learning. Our approach considers the Kantorovich potentials as a reward function and further leverages regularized optimal transport to enable large-scale applications. In several robotic experiments, our approach outperforms the baselines in terms of average cumulative rewards and shows a significant improvement in sample-efficiency, by requiring just one expert demonstration. △ Less

Submitted 19 June, 2019; originally announced June 2019.

Functionally Modular and Interpretable Temporal Filtering for Robust Segmentation

Authors: Jörg Wagner, Volker Fischer, Michael Herman, Sven Behnke

Abstract: The performance of autonomous systems heavily relies on their ability to generate a robust representation of the environment. Deep neural networks have greatly improved vision-based perception systems but still fail in challenging situations, e.g. sensor outages or heavy weather. These failures are often introduced by data-inherent perturbations, which significantly reduce the information provided… ▽ More The performance of autonomous systems heavily relies on their ability to generate a robust representation of the environment. Deep neural networks have greatly improved vision-based perception systems but still fail in challenging situations, e.g. sensor outages or heavy weather. These failures are often introduced by data-inherent perturbations, which significantly reduce the information provided to the perception system. We propose a functionally modularized temporal filter, which stabilizes an abstract feature representation of a single-frame segmentation model using information of previous time steps. Our filter module splits the filter task into multiple less complex and more interpretable subtasks. The basic structure of the filter is inspired by a Bayes estimator consisting of a prediction and an update step. To make the prediction more transparent, we implement it using a geometric projection and estimate its parameters. This additionally enables the decomposition of the filter task into static representation filtering and low-dimensional motion filtering. Our model can cope with missing frames and is trainable in an end-to-end fashion. Using photorealistic, synthetic video data, we show the ability of the proposed architecture to overcome data-inherent perturbations. The experiments especially highlight advantages introduced by an interpretable and explicit filter module. △ Less

Submitted 15 October, 2018; v1 submitted 9 October, 2018; originally announced October 2018.

Comments: In Proceedings of 29th British Machine Vision Conference (BMVC), Newcastle upon Tyne, UK, 2018

Hierarchical Recurrent Filtering for Fully Convolutional DenseNets

Authors: Jörg Wagner, Volker Fischer, Michael Herman, Sven Behnke

Abstract: Generating a robust representation of the environment is a crucial ability of learning agents. Deep learning based methods have greatly improved perception systems but still fail in challenging situations. These failures are often not solvable on the basis of a single image. In this work, we present a parameter-efficient temporal filtering concept which extends an existing single-frame segmentatio… ▽ More Generating a robust representation of the environment is a crucial ability of learning agents. Deep learning based methods have greatly improved perception systems but still fail in challenging situations. These failures are often not solvable on the basis of a single image. In this work, we present a parameter-efficient temporal filtering concept which extends an existing single-frame segmentation model to work with multiple frames. The resulting recurrent architecture temporally filters representations on all abstraction levels in a hierarchical manner, while decoupling temporal dependencies from scene representation. Using a synthetic dataset, we show the ability of our model to cope with data perturbations and highlight the importance of recurrent and hierarchical filtering. △ Less

Submitted 15 October, 2018; v1 submitted 5 October, 2018; originally announced October 2018.

Comments: In Proceedings of 26th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN), Bruges, Belgium, 2018

Dynamic simulations in SixTrack

Authors: K. Sjobak, V. K. Berglyd Olsen, R. De Maria, M. Fitterer, A. Santamaría García, H. Garcia-Morales, A. Mereghetti, J. F. Wagner, S. J. Wretborn

Abstract: The DYNK module allows element settings in SixTrack to be changed on a turn-by-turn basis. This document contains a technical description of the DYNK module in SixTrack. It is mainly intended for a developer or advanced user who wants to modify the DYNK module, for example by adding more functions that can be used to calculate new element settings, or to add support for new elements that can be us… ▽ More The DYNK module allows element settings in SixTrack to be changed on a turn-by-turn basis. This document contains a technical description of the DYNK module in SixTrack. It is mainly intended for a developer or advanced user who wants to modify the DYNK module, for example by adding more functions that can be used to calculate new element settings, or to add support for new elements that can be used with DYNK. △ Less

Submitted 19 August, 2018; originally announced August 2018.

Comments: Submission to CERN yellow report / conference proceeding, the 2015 collimation tracking code workshop

Applying Cooperative Machine Learning to Speed Up the Annotation of Social Signals in Large Multi-modal Corpora

Authors: Johannes Wagner, Tobias Baur, Yue Zhang, Michel F. Valstar, Björn Schuller, Elisabeth André

Abstract: Scientific disciplines, such as Behavioural Psychology, Anthropology and recently Social Signal Processing are concerned with the systematic exploration of human behaviour. A typical work-flow includes the manual annotation (also called coding) of social signals in multi-modal corpora of considerable size. For the involved annotators this defines an exhausting and time-consuming task. In the artic… ▽ More Scientific disciplines, such as Behavioural Psychology, Anthropology and recently Social Signal Processing are concerned with the systematic exploration of human behaviour. A typical work-flow includes the manual annotation (also called coding) of social signals in multi-modal corpora of considerable size. For the involved annotators this defines an exhausting and time-consuming task. In the article at hand we present a novel method and also provide the tools to speed up the coding procedure. To this end, we suggest and evaluate the use of Cooperative Machine Learning (CML) techniques to reduce manual labelling efforts by combining the power of computational capabilities and human intelligence. The proposed CML strategy starts with a small number of labelled instances and concentrates on predicting local parts first. Afterwards, a session-independent classification model is created to finish the remaining parts of the database. Confidence values are computed to guide the manual inspection and correction of the predictions. To bring the proposed approach into application we introduce NOVA - an open-source tool for collaborative and machine-aided annotations. In particular, it gives labellers immediate access to CML strategies and directly provides visual feedback on the results. Our experiments show that the proposed method has the potential to significantly reduce human labelling efforts. △ Less

Submitted 7 February, 2018; originally announced February 2018.

Inverse Reinforcement Learning with Simultaneous Estimation of Rewards and Dynamics

Authors: Michael Herman, Tobias Gindele, Jörg Wagner, Felix Schmitt, Wolfram Burgard

Abstract: Inverse Reinforcement Learning (IRL) describes the problem of learning an unknown reward function of a Markov Decision Process (MDP) from observed behavior of an agent. Since the agent's behavior originates in its policy and MDP policies depend on both the stochastic system dynamics as well as the reward function, the solution of the inverse problem is significantly influenced by both. Current IRL… ▽ More Inverse Reinforcement Learning (IRL) describes the problem of learning an unknown reward function of a Markov Decision Process (MDP) from observed behavior of an agent. Since the agent's behavior originates in its policy and MDP policies depend on both the stochastic system dynamics as well as the reward function, the solution of the inverse problem is significantly influenced by both. Current IRL approaches assume that if the transition model is unknown, additional samples from the system's dynamics are accessible, or the observed behavior provides enough samples of the system's dynamics to solve the inverse problem accurately. These assumptions are often not satisfied. To overcome this, we present a gradient-based IRL approach that simultaneously estimates the system's dynamics. By solving the combined optimization problem, our approach takes into account the bias of the demonstrations, which stems from the generating policy. The evaluation on a synthetic MDP and a transfer learning task shows improvements regarding the sample efficiency as well as the accuracy of the estimated reward functions and transition models. △ Less

Submitted 13 April, 2016; originally announced April 2016.

Comments: accepted to appear in AISTATS 2016

JSAI: Designing a Sound, Configurable, and Efficient Static Analyzer for JavaScript

Authors: Vineeth Kashyap, Kyle Dewey, Ethan A. Kuefner, John Wagner, Kevin Gibbons, John Sarracino, Ben Wiedermann, Ben Hardekopf

Abstract: We describe JSAI, an abstract interpreter for JavaScript. JSAI uses novel abstract domains to compute a reduced product of type inference, pointer analysis, string analysis, integer and boolean constant propagation, and control-flow analysis. In addition, JSAI allows for analysis control-flow sensitivity (i.e., context-, path-, and heap-sensitivity) to be modularly configured without requiring any… ▽ More We describe JSAI, an abstract interpreter for JavaScript. JSAI uses novel abstract domains to compute a reduced product of type inference, pointer analysis, string analysis, integer and boolean constant propagation, and control-flow analysis. In addition, JSAI allows for analysis control-flow sensitivity (i.e., context-, path-, and heap-sensitivity) to be modularly configured without requiring any changes to the analysis implementation. JSAI is designed to be provably sound with respect to a specific concrete semantics for JavaScript, which has been extensively tested against existing production-quality JavaScript implementations. We provide a comprehensive evaluation of JSAI's performance and precision using an extensive benchmark suite. This benchmark suite includes real-world JavaScript applications, machine-generated JavaScript code via Emscripten, and browser addons. We use JSAI's configurability to evaluate a large number of analysis sensitivities (some well-known, some novel) and observe some surprising results. We believe that JSAI's configurability and its formal specifications position it as a useful research platform to experiment on novel sensitivities, abstract domains, and client analyses for JavaScript. △ Less

Submitted 17 March, 2014; originally announced March 2014.

ACM Class: F.3.2; D.3.1

On the Distortion of the Eigenvalue Spectrum in MIMO Amplify-and-Forward Multi-Hop Channels

Authors: Joerg Wagner, Armin Wittneben

Abstract: Consider a wireless MIMO multi-hop channel with n_s non-cooperating source antennas and n_d fully cooperating destination antennas, as well as L clusters containing k non-cooperating relay antennas each. The source signal traverses all L clusters of relay antennas, before it reaches the destination. When relay antennas within the same cluster scale their received signals by the same constant bef… ▽ More Consider a wireless MIMO multi-hop channel with n_s non-cooperating source antennas and n_d fully cooperating destination antennas, as well as L clusters containing k non-cooperating relay antennas each. The source signal traverses all L clusters of relay antennas, before it reaches the destination. When relay antennas within the same cluster scale their received signals by the same constant before the retransmission, the equivalent channel matrix H relating the input signals at the source antennas to the output signals at the destination antennas is proportional to the product of channel matrices H_l, l=1,...,L+1, corresponding to the individual hops. We perform an asymptotic capacity analysis for this channel as follows: In a first instance we take the limits n_s->infty, n_d->infty and k->infty, but keep both n_s/n_d and k/n_d fixed. Then, we take the limits L->infty and k/n_d->infty. Requiring that the H_l's satisfy the conditions needed for the Marcenko-Pastur law, we prove that the capacity scales linearly in min{n_s,n_d}, as long as the ratio k/n_d scales at least linearly in L. Moreover, we show that up to a noise penalty and a pre-log factor the capacity of a point-to-point MIMO channel is approached, when this scaling is slightly faster than linear. Conversely, almost all spatial degrees of freedom vanish for less than linear scaling. △ Less

Submitted 27 August, 2007; originally announced August 2007.

Comments: submitted to Eurasip Journal on Wireless Communications and Networking

Large N Analysis of Amplify-and-Forward MIMO Relay Channels with Correlated Rayleigh Fading

Authors: Joerg Wagner, Boris Rankov, Armin Wittneben

Abstract: In this correspondence the cumulants of the mutual information of the flat Rayleigh fading amplify-and-forward MIMO relay channel without direct link between source and destination are derived in the large array limit. The analysis is based on the replica trick and covers both spatially independent and correlated fading in the first and the second hop, while beamforming at all terminals is restr… ▽ More In this correspondence the cumulants of the mutual information of the flat Rayleigh fading amplify-and-forward MIMO relay channel without direct link between source and destination are derived in the large array limit. The analysis is based on the replica trick and covers both spatially independent and correlated fading in the first and the second hop, while beamforming at all terminals is restricted to deterministic weight matrices. Expressions for mean and variance of the mutual information are obtained. Their parameters are determined by a nonlinear equation system. All higher cumulants are shown to vanish as the number of antennas n goes to infinity. In conclusion the distribution of the mutual information I becomes Gaussian in the large n limit and is completely characterized by the expressions obtained for mean and variance of I. Comparisons with simulation results show that the asymptotic results serve as excellent approximations for systems with only few antennas at each node. The derivation of the results follows the technique formalized by Moustakas et al. in [1]. Although the evaluations are more involved for the MIMO relay channel compared to point-to-point MIMO channels, the structure of the results is surprisingly simple again. In particular an elegant formula for the mean of the mutual information is obtained, i.e., the ergodic capacity of the two-hop amplify-and-forward MIMO relay channel without direct link. △ Less

Submitted 22 December, 2006; originally announced December 2006.

Comments: submitted for publication to IEEE Transactions on Information Theory