-
Distribution Aggregation via Continuous Thiele's Rules
Authors:
Jonathan Wagner,
Reshef Meir
Abstract:
We introduce the class of \textit{Continuous Thiele's Rules} that generalize the familiar \textbf{Thiele's rules} \cite{janson2018phragmens} of multi-winner voting to distribution aggregation problems. Each rule in that class maximizes $\sum_if(π^i)$ where $π^i$ is an agent $i$'s satisfaction and $f$ could be any twice differentiable, increasing and concave real function. Based on a single quantit…
▽ More
We introduce the class of \textit{Continuous Thiele's Rules} that generalize the familiar \textbf{Thiele's rules} \cite{janson2018phragmens} of multi-winner voting to distribution aggregation problems. Each rule in that class maximizes $\sum_if(π^i)$ where $π^i$ is an agent $i$'s satisfaction and $f$ could be any twice differentiable, increasing and concave real function. Based on a single quantity we call the \textit{'Inequality Aversion'} of $f$ (elsewhere known as "Relative Risk Aversion"), we derive bounds on the Egalitarian loss, welfare loss and the approximation of \textit{Average Fair Share}, leading to a quantifiable, continuous presentation of their inevitable trade-offs. In particular, we show that the Nash Product Rule satisfies\textit{ Average Fair Share} in our setting.
△ Less
Submitted 2 August, 2024;
originally announced August 2024.
-
An Evaluation of Immersive Infographics for News Reporting: Quantifying the Effect of Mobile AR Concrete Scales Infographics on Volume Understanding
Authors:
Mariane Giambastiani,
Jorge Wagner,
Carla M. Dal Sasso Freitas,
Luciana Nedel
Abstract:
Augmented Reality (AR) allows us to represent information in the user's own environment and, therefore, convey a visceral feeling of its true physical scale. Journalists increasingly leverage this opportunity through immersive infographics, an extension of conventional infographics reliant on familiar references to convey volumes, heights, weights, and sizes. Our goal is to measure the contributio…
▽ More
Augmented Reality (AR) allows us to represent information in the user's own environment and, therefore, convey a visceral feeling of its true physical scale. Journalists increasingly leverage this opportunity through immersive infographics, an extension of conventional infographics reliant on familiar references to convey volumes, heights, weights, and sizes. Our goal is to measure the contribution of immersive mobile AR concrete scales infographics to the user's understanding of the information scale. We focus on infographics powered by tablet-based mobile AR, given its current much more widespread use for news consumption compared to headset-based AR. We designed and implemented a study apparatus containing three alternative representation methods (textual analogies, image infographic, and AR infographic) for three different pieces of news with different characteristics and scales. In a controlled user study, we asked 26 participants to represent the expected volume of the information in the real world with the help of an AR mobile application. We also compared their subjective feelings when interacting with the different representations. While both image and AR infographics led to significantly better comprehension than textual analogies alone across different kinds of news, AR infographics led, on average, to a 31.8% smaller volume estimation error than static ones. Our findings indicate that mobile AR concrete scales infographics can contribute to news reporting by increasing readers' abilities to comprehend volume information.
△ Less
Submitted 10 July, 2024;
originally announced July 2024.
-
HEST-1k: A Dataset for Spatial Transcriptomics and Histology Image Analysis
Authors:
Guillaume Jaume,
Paul Doucet,
Andrew H. Song,
Ming Y. Lu,
Cristina Almagro-Pérez,
Sophia J. Wagner,
Anurag J. Vaidya,
Richard J. Chen,
Drew F. K. Williamson,
Ahrong Kim,
Faisal Mahmood
Abstract:
Spatial transcriptomics (ST) enables interrogating the molecular composition of tissue with ever-increasing resolution, depth, and sensitivity. However, costs, rapidly evolving technology, and lack of standards have constrained computational methods in ST to narrow tasks and small cohorts. In addition, the underlying tissue morphology as reflected by H&E-stained whole slide images (WSIs) encodes r…
▽ More
Spatial transcriptomics (ST) enables interrogating the molecular composition of tissue with ever-increasing resolution, depth, and sensitivity. However, costs, rapidly evolving technology, and lack of standards have constrained computational methods in ST to narrow tasks and small cohorts. In addition, the underlying tissue morphology as reflected by H&E-stained whole slide images (WSIs) encodes rich information often overlooked in ST studies. Here, we introduce HEST-1k, a collection of 1,108 spatial transcriptomic profiles, each linked to a WSI and metadata. HEST-1k was assembled using HEST-Library from 131 public and internal cohorts encompassing 25 organs, two species (Homo Sapiens and Mus Musculus), and 320 cancer samples from 25 cancer types. HEST-1k processing enabled the identification of 1.5 million expression--morphology pairs and 60 million nuclei. HEST-1k is tested on three use cases: (1) benchmarking foundation models for histopathology (HEST-Benchmark), (2) biomarker identification, and (3) multimodal representation learning. HEST-1k, HEST-Library, and HEST-Benchmark can be freely accessed via https://github.com/mahmoodlab/hest.
△ Less
Submitted 23 June, 2024;
originally announced June 2024.
-
Engineering software 2.0 by interpolating neural networks: unifying training, solving, and calibration
Authors:
Chanwook Park,
Sourav Saha,
Jiachen Guo,
Xiaoyu Xie,
Satyajit Mojumder,
Miguel A. Bessa,
Dong Qian,
Wei Chen,
Gregory J. Wagner,
Jian Cao,
Wing Kam Liu
Abstract:
The evolution of artificial intelligence (AI) and neural network theories has revolutionized the way software is programmed, shifting from a hard-coded series of codes to a vast neural network. However, this transition in engineering software has faced challenges such as data scarcity, multi-modality of data, low model accuracy, and slow inference. Here, we propose a new network based on interpola…
▽ More
The evolution of artificial intelligence (AI) and neural network theories has revolutionized the way software is programmed, shifting from a hard-coded series of codes to a vast neural network. However, this transition in engineering software has faced challenges such as data scarcity, multi-modality of data, low model accuracy, and slow inference. Here, we propose a new network based on interpolation theories and tensor decomposition, the interpolating neural network (INN). Instead of interpolating training data, a common notion in computer science, INN interpolates interpolation points in the physical space whose coordinates and values are trainable. It can also extrapolate if the interpolation points reside outside of the range of training data and the interpolation functions have a larger support domain. INN features orders of magnitude fewer trainable parameters, faster training, a smaller memory footprint, and higher model accuracy compared to feed-forward neural networks (FFNN) or physics-informed neural networks (PINN). INN is poised to usher in Engineering Software 2.0, a unified neural network that spans various domains of space, time, parameters, and initial/boundary conditions. This has previously been computationally prohibitive due to the exponentially growing number of trainable parameters, easily exceeding the parameter size of ChatGPT, which is over 1 trillion. INN addresses this challenge by leveraging tensor decomposition and tensor product, with adaptable network architecture.
△ Less
Submitted 22 April, 2024; v1 submitted 16 April, 2024;
originally announced April 2024.
-
DinoBloom: A Foundation Model for Generalizable Cell Embeddings in Hematology
Authors:
Valentin Koch,
Sophia J. Wagner,
Salome Kazeminia,
Ece Sancar,
Matthias Hehr,
Julia Schnabel,
Tingying Peng,
Carsten Marr
Abstract:
In hematology, computational models offer significant potential to improve diagnostic accuracy, streamline workflows, and reduce the tedious work of analyzing single cells in peripheral blood or bone marrow smears. However, clinical adoption of computational models has been hampered by the lack of generalization due to large batch effects, small dataset sizes, and poor performance in transfer lear…
▽ More
In hematology, computational models offer significant potential to improve diagnostic accuracy, streamline workflows, and reduce the tedious work of analyzing single cells in peripheral blood or bone marrow smears. However, clinical adoption of computational models has been hampered by the lack of generalization due to large batch effects, small dataset sizes, and poor performance in transfer learning from natural images. To address these challenges, we introduce DinoBloom, the first foundation model for single cell images in hematology, utilizing a tailored DINOv2 pipeline. Our model is built upon an extensive collection of 13 diverse, publicly available datasets of peripheral blood and bone marrow smears, the most substantial open-source cohort in hematology so far, comprising over 380,000 white blood cell images. To assess its generalization capability, we evaluate it on an external dataset with a challenging domain shift. We show that our model outperforms existing medical and non-medical vision models in (i) linear probing and k-nearest neighbor evaluations for cell-type classification on blood and bone marrow smears and (ii) weakly supervised multiple instance learning for acute myeloid leukemia subtyping by a large margin. A family of four DinoBloom models (small, base, large, and giant) can be adapted for a wide range of downstream applications, be a strong baseline for classification problems, and facilitate the assessment of batch effects in new datasets. All models are available at github.com/marrlab/DinoBloom.
△ Less
Submitted 7 April, 2024;
originally announced April 2024.
-
Reimagining TaxiVis through an Immersive Space-Time Cube metaphor and reflecting on potential benefits of Immersive Analytics for urban data exploration
Authors:
Jorge Wagner,
Claudio T. Silva,
Wolfgang Stuerzlinger,
Luciana Nedel
Abstract:
Current visualization research has identified the potential of more immersive settings for data exploration, leveraging VR and AR technologies. To explore how a traditional visualization system could be adapted into an immersive framework, and how it could benefit from this, we decided to revisit a landmark paper presented ten years ago at IEEE VIS. TaxiVis, by Ferreira et al., enabled interactive…
▽ More
Current visualization research has identified the potential of more immersive settings for data exploration, leveraging VR and AR technologies. To explore how a traditional visualization system could be adapted into an immersive framework, and how it could benefit from this, we decided to revisit a landmark paper presented ten years ago at IEEE VIS. TaxiVis, by Ferreira et al., enabled interactive spatio-temporal querying of a large dataset of taxi trips in New York City. Here, we reimagine how TaxiVis' functionalities could be implemented and extended in a 3D immersive environment. Among the unique features we identify as being enabled by the Immersive TaxiVis prototype are alternative uses of the additional visual dimension, a fully visual 3D spatio-temporal query framework, and the opportunity to explore the data at different scales and frames of reference. By revisiting the case studies from the original paper, we demonstrate workflows that can benefit from this immersive perspective. Through reporting on our experience, and on the vision and reasoning behind our design decisions, we hope to contribute to the debate on how conventional and immersive visualization paradigms can complement each other and on how the exploration of urban datasets can be facilitated in the coming years.
△ Less
Submitted 23 May, 2024; v1 submitted 1 February, 2024;
originally announced February 2024.
-
B-Cos Aligned Transformers Learn Human-Interpretable Features
Authors:
Manuel Tran,
Amal Lahiani,
Yashin Dicente Cid,
Melanie Boxberg,
Peter Lienemann,
Christian Matek,
Sophia J. Wagner,
Fabian J. Theis,
Eldad Klaiman,
Tingying Peng
Abstract:
Vision Transformers (ViTs) and Swin Transformers (Swin) are currently state-of-the-art in computational pathology. However, domain experts are still reluctant to use these models due to their lack of interpretability. This is not surprising, as critical decisions need to be transparent and understandable. The most common approach to understanding transformers is to visualize their attention. Howev…
▽ More
Vision Transformers (ViTs) and Swin Transformers (Swin) are currently state-of-the-art in computational pathology. However, domain experts are still reluctant to use these models due to their lack of interpretability. This is not surprising, as critical decisions need to be transparent and understandable. The most common approach to understanding transformers is to visualize their attention. However, attention maps of ViTs are often fragmented, leading to unsatisfactory explanations. Here, we introduce a novel architecture called the B-cos Vision Transformer (BvT) that is designed to be more interpretable. It replaces all linear transformations with the B-cos transform to promote weight-input alignment. In a blinded study, medical experts clearly ranked BvTs above ViTs, suggesting that our network is better at capturing biomedically relevant structures. This is also true for the B-cos Swin Transformer (Bwin). Compared to the Swin Transformer, it even improves the F1-score by up to 4.7% on two public datasets.
△ Less
Submitted 18 January, 2024; v1 submitted 16 January, 2024;
originally announced January 2024.
-
Low-resource finetuning of foundation models beats state-of-the-art in histopathology
Authors:
Benedikt Roth,
Valentin Koch,
Sophia J. Wagner,
Julia A. Schnabel,
Carsten Marr,
Tingying Peng
Abstract:
To handle the large scale of whole slide images in computational pathology, most approaches first tessellate the images into smaller patches, extract features from these patches, and finally aggregate the feature vectors with weakly-supervised learning. The performance of this workflow strongly depends on the quality of the extracted features. Recently, foundation models in computer vision showed…
▽ More
To handle the large scale of whole slide images in computational pathology, most approaches first tessellate the images into smaller patches, extract features from these patches, and finally aggregate the feature vectors with weakly-supervised learning. The performance of this workflow strongly depends on the quality of the extracted features. Recently, foundation models in computer vision showed that leveraging huge amounts of data through supervised or self-supervised learning improves feature quality and generalizability for a variety of tasks. In this study, we benchmark the most popular vision foundation models as feature extractors for histopathology data. We evaluate the models in two settings: slide-level classification and patch-level classification. We show that foundation models are a strong baseline. Our experiments demonstrate that by finetuning a foundation model on a single GPU for only two hours or three days depending on the dataset, we can match or outperform state-of-the-art feature extractors for computational pathology. These findings imply that even with little resources one can finetune a feature extractor tailored towards a specific downstream task and dataset. This is a considerable shift from the current state, where only few institutions with large amounts of resources and datasets are able to train a feature extractor. We publish all code used for training and evaluation as well as the finetuned models.
△ Less
Submitted 9 January, 2024;
originally announced January 2024.
-
From Whole-slide Image to Biomarker Prediction: A Protocol for End-to-End Deep Learning in Computational Pathology
Authors:
Omar S. M. El Nahhas,
Marko van Treeck,
Georg Wölflein,
Michaela Unger,
Marta Ligero,
Tim Lenz,
Sophia J. Wagner,
Katherine J. Hewitt,
Firas Khader,
Sebastian Foersch,
Daniel Truhn,
Jakob Nikolas Kather
Abstract:
Hematoxylin- and eosin (H&E) stained whole-slide images (WSIs) are the foundation of diagnosis of cancer. In recent years, development of deep learning-based methods in computational pathology enabled the prediction of biomarkers directly from WSIs. However, accurately linking tissue phenotype to biomarkers at scale remains a crucial challenge for democratizing complex biomarkers in precision onco…
▽ More
Hematoxylin- and eosin (H&E) stained whole-slide images (WSIs) are the foundation of diagnosis of cancer. In recent years, development of deep learning-based methods in computational pathology enabled the prediction of biomarkers directly from WSIs. However, accurately linking tissue phenotype to biomarkers at scale remains a crucial challenge for democratizing complex biomarkers in precision oncology. This protocol describes a practical workflow for solid tumor associative modeling in pathology (STAMP), enabling prediction of biomarkers directly from WSIs using deep learning. The STAMP workflow is biomarker agnostic and allows for genetic- and clinicopathologic tabular data to be included as an additional input, together with histopathology images. The protocol consists of five main stages which have been successfully applied to various research problems: formal problem definition, data preprocessing, modeling, evaluation and clinical translation. The STAMP workflow differentiates itself through its focus on serving as a collaborative framework that can be used by clinicians and engineers alike for setting up research projects in the field of computational pathology. As an example task, we applied STAMP to the prediction of microsatellite instability (MSI) status in colorectal cancer, showing accurate performance for the identification of MSI-high tumors. Moreover, we provide an open-source codebase which has been deployed at several hospitals across the globe to set up computational pathology workflows. The STAMP workflow requires one workday of hands-on computational execution and basic command line knowledge.
△ Less
Submitted 18 December, 2023;
originally announced December 2023.
-
Statistical Parameterized Physics-Based Machine Learning Digital Twin Models for Laser Powder Bed Fusion Process
Authors:
Yangfan Li,
Satyajit Mojumder,
Ye Lu,
Abdullah Al Amin,
Jiachen Guo,
Xiaoyu Xie,
Wei Chen,
Gregory J. Wagner,
Jian Cao,
Wing Kam Liu
Abstract:
A digital twin (DT) is a virtual representation of physical process, products and/or systems that requires a high-fidelity computational model for continuous update through the integration of sensor data and user input. In the context of laser powder bed fusion (LPBF) additive manufacturing, a digital twin of the manufacturing process can offer predictions for the produced parts, diagnostics for m…
▽ More
A digital twin (DT) is a virtual representation of physical process, products and/or systems that requires a high-fidelity computational model for continuous update through the integration of sensor data and user input. In the context of laser powder bed fusion (LPBF) additive manufacturing, a digital twin of the manufacturing process can offer predictions for the produced parts, diagnostics for manufacturing defects, as well as control capabilities. This paper introduces a parameterized physics-based digital twin (PPB-DT) for the statistical predictions of LPBF metal additive manufacturing process. We accomplish this by creating a high-fidelity computational model that accurately represents the melt pool phenomena and subsequently calibrating and validating it through controlled experiments. In PPB-DT, a mechanistic reduced-order method-driven stochastic calibration process is introduced, which enables the statistical predictions of the melt pool geometries and the identification of defects such as lack-of-fusion porosity and surface roughness, specifically for diagnostic applications. Leveraging data derived from this physics-based model and experiments, we have trained a machine learning-based digital twin (PPB-ML-DT) model for predicting, monitoring, and controlling melt pool geometries. These proposed digital twin models can be employed for predictions, control, optimization, and quality assurance within the LPBF process, ultimately expediting product development and certification in LPBF-based metal additive manufacturing.
△ Less
Submitted 13 November, 2023;
originally announced November 2023.
-
Can you text what is happening? Integrating pre-trained language encoders into trajectory prediction models for autonomous driving
Authors:
Ali Keysan,
Andreas Look,
Eitan Kosman,
Gonca Gürsun,
Jörg Wagner,
Yu Yao,
Barbara Rakitsch
Abstract:
In autonomous driving tasks, scene understanding is the first step towards predicting the future behavior of the surrounding traffic participants. Yet, how to represent a given scene and extract its features are still open research questions. In this study, we propose a novel text-based representation of traffic scenes and process it with a pre-trained language encoder.
First, we show that text-…
▽ More
In autonomous driving tasks, scene understanding is the first step towards predicting the future behavior of the surrounding traffic participants. Yet, how to represent a given scene and extract its features are still open research questions. In this study, we propose a novel text-based representation of traffic scenes and process it with a pre-trained language encoder.
First, we show that text-based representations, combined with classical rasterized image representations, lead to descriptive scene embeddings. Second, we benchmark our predictions on the nuScenes dataset and show significant improvements compared to baselines. Third, we show in an ablation study that a joint encoder of text and rasterized images outperforms the individual encoders confirming that both representations have their complementary strengths.
△ Less
Submitted 13 September, 2023; v1 submitted 11 September, 2023;
originally announced September 2023.
-
In Situ Soil Property Estimation for Autonomous Earthmoving Using Physics-Infused Neural Networks
Authors:
W. Jacob Wagner,
Ahmet Soylemezoglu,
Dustin Nottage,
Katherine Driggs-Campbell
Abstract:
A novel, learning-based method for in situ estimation of soil properties using a physics-infused neural network (PINN) is presented. The network is trained to produce estimates of soil cohesion, angle of internal friction, soil-tool friction, soil failure angle, and residual depth of cut which are then passed through an earthmoving model based on the fundamental equation of earthmoving (FEE) to pr…
▽ More
A novel, learning-based method for in situ estimation of soil properties using a physics-infused neural network (PINN) is presented. The network is trained to produce estimates of soil cohesion, angle of internal friction, soil-tool friction, soil failure angle, and residual depth of cut which are then passed through an earthmoving model based on the fundamental equation of earthmoving (FEE) to produce an estimated force. The network ingests a short history of kinematic observations along with past control commands and predicts interaction forces accurately with average error of less than 2kN, 13% of the measured force. To validate the approach, an earthmoving simulation of a bladed vehicle is developed using Vortex Studio, enabling comparison of the estimated parameters to pseudo-ground-truth values which is challenging in real-world experiments. The proposed approach is shown to enable accurate estimation of interaction forces and produces meaningful parameter estimates even when the model and the environmental physics deviate substantially.
△ Less
Submitted 5 September, 2023;
originally announced September 2023.
-
A Robust Localization Solution for an Uncrewed Ground Vehicle in Unstructured Outdoor GNSS-Denied Environments
Authors:
W. Jacob Wagner,
Isaac Blankenau,
Maribel DeLaTorre,
Amartya Purushottam,
Ahmet Soylemezoglu
Abstract:
This work addresses the challenge of developing a localization system for an uncrewed ground vehicle (UGV) operating autonomously in unstructured outdoor Global Navigation Satellite System (GNSS)-denied environments. The goal is to enable accurate mapping and long-range navigation with practical applications in domains such as autonomous construction, military engineering missions, and exploration…
▽ More
This work addresses the challenge of developing a localization system for an uncrewed ground vehicle (UGV) operating autonomously in unstructured outdoor Global Navigation Satellite System (GNSS)-denied environments. The goal is to enable accurate mapping and long-range navigation with practical applications in domains such as autonomous construction, military engineering missions, and exploration of non-Earth planets. The proposed system - Terrain-Referenced Assured Engineer Localization System (TRAELS) - integrates pose estimates produced by two complementary terrain referenced navigation (TRN) methods with wheel odometry and inertial measurement unit (IMU) measurements using an Extended Kalman Filter (EKF). Unlike simultaneous localization and mapping (SLAM) systems that require loop closures, the described approach maintains accuracy over long distances and one-way missions without the need to revisit previous positions. Evaluation of TRAELS is performed across a range of environments. In regions where a combination of distinctive geometric and ground surface features are present, the developed TRN methods are leveraged by TRAELS to consistently achieve an absolute trajectory error of less than 3.0 m. The approach is also shown to be capable of recovering from large accumulated drift when traversing feature-sparse areas, which is essential in ensuring robust performance of the system across a wide variety of challenging GNSS-denied environments. Overall, the effectiveness of the system in providing precise localization and mapping capabilities in challenging GNSS-denied environments is demonstrated and an analysis is performed leading to insights for improving TRN approaches for UGVs.
△ Less
Submitted 5 September, 2023;
originally announced September 2023.
-
An Initial Exploration: Learning to Generate Realistic Audio for Silent Video
Authors:
Matthew Martel,
Jackson Wagner
Abstract:
Generating realistic audio effects for movies and other media is a challenging task that is accomplished today primarily through physical techniques known as Foley art. Foley artists create sounds with common objects (e.g., boxing gloves, broken glass) in time with video as it is playing to generate captivating audio tracks. In this work, we aim to develop a deep-learning based framework that does…
▽ More
Generating realistic audio effects for movies and other media is a challenging task that is accomplished today primarily through physical techniques known as Foley art. Foley artists create sounds with common objects (e.g., boxing gloves, broken glass) in time with video as it is playing to generate captivating audio tracks. In this work, we aim to develop a deep-learning based framework that does much the same - observes video in it's natural sequence and generates realistic audio to accompany it. Notably, we have reason to believe this is achievable due to advancements in realistic audio generation techniques conditioned on other inputs (e.g., Wavenet conditioned on text). We explore several different model architectures to accomplish this task that process both previously-generated audio and video context. These include deep-fusion CNN, dilated Wavenet CNN with visual context, and transformer-based architectures. We find that the transformer-based architecture yields the most promising results, matching low-frequencies to visual patterns effectively, but failing to generate more nuanced waveforms.
△ Less
Submitted 23 August, 2023;
originally announced August 2023.
-
Speech-based Age and Gender Prediction with Transformers
Authors:
Felix Burkhardt,
Johannes Wagner,
Hagen Wierstorf,
Florian Eyben,
Björn Schuller
Abstract:
We report on the curation of several publicly available datasets for age and gender prediction. Furthermore, we present experiments to predict age and gender with models based on a pre-trained wav2vec 2.0. Depending on the dataset, we achieve an MAE between 7.1 years and 10.8 years for age, and at least 91.1% ACC for gender (female, male, child). Compared to a modelling approach built on handcraft…
▽ More
We report on the curation of several publicly available datasets for age and gender prediction. Furthermore, we present experiments to predict age and gender with models based on a pre-trained wav2vec 2.0. Depending on the dataset, we achieve an MAE between 7.1 years and 10.8 years for age, and at least 91.1% ACC for gender (female, male, child). Compared to a modelling approach built on handcrafted features, our proposed system shows an improvement of 9% UAR for age and 4% UAR for gender. To make our findings reproducible, we release the best performing model to the community as well as the sample lists of the data splits.
△ Less
Submitted 29 June, 2023;
originally announced June 2023.
-
Broadening the perspective for sustainable AI: Comprehensive sustainability criteria and indicators for AI systems
Authors:
Friederike Rohde,
Josephin Wagner,
Andreas Meyer,
Philipp Reinhard,
Marcus Voss,
Ulrich Petschow,
Anne Mollen
Abstract:
The increased use of AI systems is associated with multi-faceted societal, environmental, and economic consequences. These include non-transparent decision-making processes, discrimination, increasing inequalities, rising energy consumption and greenhouse gas emissions in AI model development and application, and an increasing concentration of economic power. By considering the multi-dimensionalit…
▽ More
The increased use of AI systems is associated with multi-faceted societal, environmental, and economic consequences. These include non-transparent decision-making processes, discrimination, increasing inequalities, rising energy consumption and greenhouse gas emissions in AI model development and application, and an increasing concentration of economic power. By considering the multi-dimensionality of sustainability, this paper takes steps towards substantiating the call for an overarching perspective on "sustainable AI". It presents the SCAIS Framework (Sustainability Criteria and Indicators for Artificial Intelligence Systems) which contains a set 19 sustainability criteria for sustainable AI and 67 indicators that is based on the results of a critical review and expert workshops. This interdisciplinary approach contributes a unique holistic perspective to facilitate and structure the discourse on sustainable AI. Further, it provides a concrete framework that lays the foundation for developing standards and tools to support the conscious development and application of AI systems.
△ Less
Submitted 22 November, 2023; v1 submitted 22 June, 2023;
originally announced June 2023.
-
Improving accuracy of GPT-3/4 results on biomedical data using a retrieval-augmented language model
Authors:
David Soong,
Sriram Sridhar,
Han Si,
Jan-Samuel Wagner,
Ana Caroline Costa Sá,
Christina Y Yu,
Kubra Karagoz,
Meijian Guan,
Hisham Hamadeh,
Brandon W Higgs
Abstract:
Large language models (LLMs) have made significant advancements in natural language processing (NLP). Broad corpora capture diverse patterns but can introduce irrelevance, while focused corpora enhance reliability by reducing misleading information. Training LLMs on focused corpora poses computational challenges. An alternative approach is to use a retrieval-augmentation (RetA) method tested in a…
▽ More
Large language models (LLMs) have made significant advancements in natural language processing (NLP). Broad corpora capture diverse patterns but can introduce irrelevance, while focused corpora enhance reliability by reducing misleading information. Training LLMs on focused corpora poses computational challenges. An alternative approach is to use a retrieval-augmentation (RetA) method tested in a specific domain.
To evaluate LLM performance, OpenAI's GPT-3, GPT-4, Bing's Prometheus, and a custom RetA model were compared using 19 questions on diffuse large B-cell lymphoma (DLBCL) disease. Eight independent reviewers assessed responses based on accuracy, relevance, and readability (rated 1-3).
The RetA model performed best in accuracy (12/19 3-point scores, total=47) and relevance (13/19, 50), followed by GPT-4 (8/19, 43; 11/19, 49). GPT-4 received the highest readability scores (17/19, 55), followed by GPT-3 (15/19, 53) and the RetA model (11/19, 47). Prometheus underperformed in accuracy (34), relevance (32), and readability (38).
Both GPT-3.5 and GPT-4 had more hallucinations in all 19 responses compared to the RetA model and Prometheus. Hallucinations were mostly associated with non-existent references or fabricated efficacy data.
These findings suggest that RetA models, supplemented with domain-specific corpora, may outperform general-purpose LLMs in accuracy and relevance within specific domains. However, this evaluation was limited to specific questions and metrics and may not capture challenges in semantic search and other NLP tasks. Further research will explore different LLM architectures, RetA methodologies, and evaluation methods to assess strengths and limitations more comprehensively.
△ Less
Submitted 30 May, 2023; v1 submitted 26 May, 2023;
originally announced May 2023.
-
Vision and Control for Grasping Clear Plastic Bags
Authors:
Joohwan Seo,
Jackson Wagner,
Anuj Raicura,
Jake Kim
Abstract:
We develop two novel vision methods for planning effective grasps for clear plastic bags, as well as a control method to enable a Sawyer arm with a parallel gripper to execute the grasps. The first vision method is based on classical image processing and heuristics (e.g., Canny edge detection) to select a grasp target and angle. The second uses a deep-learning model trained on a human-labeled data…
▽ More
We develop two novel vision methods for planning effective grasps for clear plastic bags, as well as a control method to enable a Sawyer arm with a parallel gripper to execute the grasps. The first vision method is based on classical image processing and heuristics (e.g., Canny edge detection) to select a grasp target and angle. The second uses a deep-learning model trained on a human-labeled data set to mimic human grasp decisions. A clustering algorithm is used to de-noise the outputs of each vision method. Subsequently, a workspace PD control method is used to execute each grasp. Of the two vision methods, we find the deep-learning based method to be more effective.
△ Less
Submitted 12 May, 2023;
originally announced May 2023.
-
Strategy-proof Budgeting via a VCG-like Mechanism
Authors:
Jonathan Wagner,
Reshef Meir
Abstract:
We present a strategy-proof public goods budgeting mechanism where agents determine both the total volume of expanses and the specific allocation. It is constructed as a modification of VCG to a less typical environment, namely where we do not assume quasi-linear utilities nor direct revelation. We further show that under plausible assumptions it satisfies strategy-proofness in strictly dominant s…
▽ More
We present a strategy-proof public goods budgeting mechanism where agents determine both the total volume of expanses and the specific allocation. It is constructed as a modification of VCG to a less typical environment, namely where we do not assume quasi-linear utilities nor direct revelation. We further show that under plausible assumptions it satisfies strategy-proofness in strictly dominant strategies, and consequently implements the social optimum as a Coalition Proof Nash Equilibrium. A primary (albeit not an exclusive) motivation of our model is Participatory Budgeting, where members of a community collectively decide the spending policy of public tax dollars. While incentives alignment in our mechanism, as in classic VCG, is achieved via individual payments we charge from agents, in a PB context that seems unreasonable. Our second main result thus provides that, under further specifications relevant in that context, these payments will vanish in large populations. In the last section we expand the mechanism's definition to a class of mechanisms in which the designer can prioritize certain outcomes she sees as desirable. In particular we give the example of favoring equitable (egalitarian) allocations.
△ Less
Submitted 13 March, 2023;
originally announced March 2023.
-
audb -- Sharing and Versioning of Audio and Annotation Data in Python
Authors:
Hagen Wierstorf,
Johannes Wagner,
Florian Eyben,
Felix Burkhardt,
Björn W. Schuller
Abstract:
Driven by the need for larger and more diverse datasets to pre-train and fine-tune increasingly complex machine learning models, the number of datasets is rapidly growing. audb is an open-source Python library that supports versioning and documentation of audio datasets. It aims to provide a standardized and simple user-interface to publish, maintain, and access the annotations and audio files of…
▽ More
Driven by the need for larger and more diverse datasets to pre-train and fine-tune increasingly complex machine learning models, the number of datasets is rapidly growing. audb is an open-source Python library that supports versioning and documentation of audio datasets. It aims to provide a standardized and simple user-interface to publish, maintain, and access the annotations and audio files of a dataset. To efficiently store the data on a server, audb automatically resolves dependencies between versions of a dataset and only uploads newly added or altered files when a new version is published. The library supports partial loading of a dataset and local caching for fast access. audb is a lightweight library and can be interfaced from any machine learning library. It supports the management of datasets on a single PC, within a university or company, or within a whole research community.
△ Less
Submitted 10 May, 2023; v1 submitted 1 March, 2023;
originally announced March 2023.
-
Fully transformer-based biomarker prediction from colorectal cancer histology: a large-scale multicentric study
Authors:
Sophia J. Wagner,
Daniel Reisenbüchler,
Nicholas P. West,
Jan Moritz Niehues,
Gregory Patrick Veldhuizen,
Philip Quirke,
Heike I. Grabsch,
Piet A. van den Brandt,
Gordon G. A. Hutchins,
Susan D. Richman,
Tanwei Yuan,
Rupert Langer,
Josien Christina Anna Jenniskens,
Kelly Offermans,
Wolfram Mueller,
Richard Gray,
Stephen B. Gruber,
Joel K. Greenson,
Gad Rennert,
Joseph D. Bonner,
Daniel Schmolze,
Jacqueline A. James,
Maurice B. Loughrey,
Manuel Salto-Tellez,
Hermann Brenner
, et al. (6 additional authors not shown)
Abstract:
Background: Deep learning (DL) can extract predictive and prognostic biomarkers from routine pathology slides in colorectal cancer. For example, a DL test for the diagnosis of microsatellite instability (MSI) in CRC has been approved in 2022. Current approaches rely on convolutional neural networks (CNNs). Transformer networks are outperforming CNNs and are replacing them in many applications, but…
▽ More
Background: Deep learning (DL) can extract predictive and prognostic biomarkers from routine pathology slides in colorectal cancer. For example, a DL test for the diagnosis of microsatellite instability (MSI) in CRC has been approved in 2022. Current approaches rely on convolutional neural networks (CNNs). Transformer networks are outperforming CNNs and are replacing them in many applications, but have not been used for biomarker prediction in cancer at a large scale. In addition, most DL approaches have been trained on small patient cohorts, which limits their clinical utility. Methods: In this study, we developed a new fully transformer-based pipeline for end-to-end biomarker prediction from pathology slides. We combine a pre-trained transformer encoder and a transformer network for patch aggregation, capable of yielding single and multi-target prediction at patient level. We train our pipeline on over 9,000 patients from 10 colorectal cancer cohorts. Results: A fully transformer-based approach massively improves the performance, generalizability, data efficiency, and interpretability as compared with current state-of-the-art algorithms. After training on a large multicenter cohort, we achieve a sensitivity of 0.97 with a negative predictive value of 0.99 for MSI prediction on surgical resection specimens. We demonstrate for the first time that resection specimen-only training reaches clinical-grade performance on endoscopic biopsy tissue, solving a long-standing diagnostic problem. Interpretation: A fully transformer-based end-to-end pipeline trained on thousands of pathology slides yields clinical-grade performance for biomarker prediction on surgical resections and biopsies. Our new methods are freely available under an open source license.
△ Less
Submitted 1 March, 2023; v1 submitted 23 January, 2023;
originally announced January 2023.
-
MABe22: A Multi-Species Multi-Task Benchmark for Learned Representations of Behavior
Authors:
Jennifer J. Sun,
Markus Marks,
Andrew Ulmer,
Dipam Chakraborty,
Brian Geuther,
Edward Hayes,
Heng Jia,
Vivek Kumar,
Sebastian Oleszko,
Zachary Partridge,
Milan Peelman,
Alice Robie,
Catherine E. Schretter,
Keith Sheppard,
Chao Sun,
Param Uttarwar,
Julian M. Wagner,
Eric Werner,
Joseph Parker,
Pietro Perona,
Yisong Yue,
Kristin Branson,
Ann Kennedy
Abstract:
We introduce MABe22, a large-scale, multi-agent video and trajectory benchmark to assess the quality of learned behavior representations. This dataset is collected from a variety of biology experiments, and includes triplets of interacting mice (4.7 million frames video+pose tracking data, 10 million frames pose only), symbiotic beetle-ant interactions (10 million frames video data), and groups of…
▽ More
We introduce MABe22, a large-scale, multi-agent video and trajectory benchmark to assess the quality of learned behavior representations. This dataset is collected from a variety of biology experiments, and includes triplets of interacting mice (4.7 million frames video+pose tracking data, 10 million frames pose only), symbiotic beetle-ant interactions (10 million frames video data), and groups of interacting flies (4.4 million frames of pose tracking data). Accompanying these data, we introduce a panel of real-life downstream analysis tasks to assess the quality of learned representations by evaluating how well they preserve information about the experimental conditions (e.g. strain, time of day, optogenetic stimulation) and animal behavior. We test multiple state-of-the-art self-supervised video and trajectory representation learning methods to demonstrate the use of our benchmark, revealing that methods developed using human action datasets do not fully translate to animal datasets. We hope that our benchmark and dataset encourage a broader exploration of behavior representation learning methods across species and settings.
△ Less
Submitted 30 June, 2023; v1 submitted 21 July, 2022;
originally announced July 2022.
-
Quantum Circuit Compiler for a Shuttling-Based Trapped-Ion Quantum Computer
Authors:
Fabian Kreppel,
Christian Melzer,
Diego Olvera Millán,
Janis Wagner,
Janine Hilder,
Ulrich Poschinger,
Ferdinand Schmidt-Kaler,
André Brinkmann
Abstract:
The increasing capabilities of quantum computing hardware and the challenge of realizing deep quantum circuits require fully automated and efficient tools for compiling quantum circuits. To express arbitrary circuits in a sequence of native gates specific to the quantum computer architecture, it is necessary to make algorithms portable across the landscape of quantum hardware providers. In this wo…
▽ More
The increasing capabilities of quantum computing hardware and the challenge of realizing deep quantum circuits require fully automated and efficient tools for compiling quantum circuits. To express arbitrary circuits in a sequence of native gates specific to the quantum computer architecture, it is necessary to make algorithms portable across the landscape of quantum hardware providers. In this work, we present a compiler capable of transforming and optimizing a quantum circuit targeting a shuttling-based trapped-ion quantum processor. It consists of custom algorithms set on top of the quantum circuit framework Pytket. The performance was evaluated for a wide range of quantum circuits and the results show that the gate counts can be reduced by factors up to 5.1 compared to standard Pytket and up to 2.2 compared to standard Qiskit compilation.
△ Less
Submitted 2 November, 2023; v1 submitted 5 July, 2022;
originally announced July 2022.
-
Local Attention Graph-based Transformer for Multi-target Genetic Alteration Prediction
Authors:
Daniel Reisenbüchler,
Sophia J. Wagner,
Melanie Boxberg,
Tingying Peng
Abstract:
Classical multiple instance learning (MIL) methods are often based on the identical and independent distributed assumption between instances, hence neglecting the potentially rich contextual information beyond individual entities. On the other hand, Transformers with global self-attention modules have been proposed to model the interdependencies among all instances. However, in this paper we quest…
▽ More
Classical multiple instance learning (MIL) methods are often based on the identical and independent distributed assumption between instances, hence neglecting the potentially rich contextual information beyond individual entities. On the other hand, Transformers with global self-attention modules have been proposed to model the interdependencies among all instances. However, in this paper we question: Is global relation modeling using self-attention necessary, or can we appropriately restrict self-attention calculations to local regimes in large-scale whole slide images (WSIs)? We propose a general-purpose local attention graph-based Transformer for MIL (LA-MIL), introducing an inductive bias by explicitly contextualizing instances in adaptive local regimes of arbitrary size. Additionally, an efficiently adapted loss function enables our approach to learn expressive WSI embeddings for the joint analysis of multiple biomarkers. We demonstrate that LA-MIL achieves state-of-the-art results in mutation prediction for gastrointestinal cancer, outperforming existing models on important biomarkers such as microsatellite instability for colorectal cancer. Our findings suggest that local self-attention sufficiently models dependencies on par with global modules. Our LA-MIL implementation is available at https://github.com/agentdr1/LA_MIL.
△ Less
Submitted 17 June, 2022; v1 submitted 13 May, 2022;
originally announced May 2022.
-
Deep Vehicle Detection in Satellite Video
Authors:
Roman Pflugfelder,
Axel Weissenfeld,
Julian Wagner
Abstract:
This work presents a deep learning approach for vehicle detection in satellite video. Vehicle detection is perhaps impossible in single EO satellite images due to the tininess of vehicles (4-10 pixel) and their similarity to the background. Instead, we consider satellite video which overcomes the lack of spatial information by temporal consistency of vehicle movement. A new spatiotemporal model of…
▽ More
This work presents a deep learning approach for vehicle detection in satellite video. Vehicle detection is perhaps impossible in single EO satellite images due to the tininess of vehicles (4-10 pixel) and their similarity to the background. Instead, we consider satellite video which overcomes the lack of spatial information by temporal consistency of vehicle movement. A new spatiotemporal model of a compact $3 \times 3$ convolutional, neural network is proposed which neglects pooling layers and uses leaky ReLUs. Then we use a reformulation of the output heatmap including Non-Maximum-Suppression (NMS) for the final segmentation. Empirical results on two new annotated satellite videos reconfirm the applicability of this approach for vehicle detection. They more importantly indicate that pre-training on WAMI data and then fine-tuning on few annotated video frames for a new video is sufficient. In our experiment only five annotated images yield a $F_1$ score of 0.81 on a new video showing more complex traffic patterns than the Las Vegas video. Our best result on Las Vegas is a $F_1$ score of 0.87 which makes the proposed approach a leading method for this benchmark.
△ Less
Submitted 7 June, 2022; v1 submitted 14 April, 2022;
originally announced April 2022.
-
Probing Speech Emotion Recognition Transformers for Linguistic Knowledge
Authors:
Andreas Triantafyllopoulos,
Johannes Wagner,
Hagen Wierstorf,
Maximilian Schmitt,
Uwe Reichel,
Florian Eyben,
Felix Burkhardt,
Björn W. Schuller
Abstract:
Large, pre-trained neural networks consisting of self-attention layers (transformers) have recently achieved state-of-the-art results on several speech emotion recognition (SER) datasets. These models are typically pre-trained in self-supervised manner with the goal to improve automatic speech recognition performance -- and thus, to understand linguistic information. In this work, we investigate t…
▽ More
Large, pre-trained neural networks consisting of self-attention layers (transformers) have recently achieved state-of-the-art results on several speech emotion recognition (SER) datasets. These models are typically pre-trained in self-supervised manner with the goal to improve automatic speech recognition performance -- and thus, to understand linguistic information. In this work, we investigate the extent in which this information is exploited during SER fine-tuning. Using a reproducible methodology based on open-source tools, we synthesise prosodically neutral speech utterances while varying the sentiment of the text. Valence predictions of the transformer model are very reactive to positive and negative sentiment content, as well as negations, but not to intensifiers or reducers, while none of those linguistic features impact arousal or dominance. These findings show that transformers can successfully leverage linguistic information to improve their valence predictions, and that linguistic analysis should be included in their testing.
△ Less
Submitted 26 July, 2022; v1 submitted 1 April, 2022;
originally announced April 2022.
-
Dawn of the transformer era in speech emotion recognition: closing the valence gap
Authors:
Johannes Wagner,
Andreas Triantafyllopoulos,
Hagen Wierstorf,
Maximilian Schmitt,
Felix Burkhardt,
Florian Eyben,
Björn W. Schuller
Abstract:
Recent advances in transformer-based architectures which are pre-trained in self-supervised manner have shown great promise in several machine learning tasks. In the audio domain, such architectures have also been successfully utilised in the field of speech emotion recognition (SER). However, existing works have not evaluated the influence of model size and pre-training data on downstream perform…
▽ More
Recent advances in transformer-based architectures which are pre-trained in self-supervised manner have shown great promise in several machine learning tasks. In the audio domain, such architectures have also been successfully utilised in the field of speech emotion recognition (SER). However, existing works have not evaluated the influence of model size and pre-training data on downstream performance, and have shown limited attention to generalisation, robustness, fairness, and efficiency. The present contribution conducts a thorough analysis of these aspects on several pre-trained variants of wav2vec 2.0 and HuBERT that we fine-tuned on the dimensions arousal, dominance, and valence of MSP-Podcast, while additionally using IEMOCAP and MOSI to test cross-corpus generalisation. To the best of our knowledge, we obtain the top performance for valence prediction without use of explicit linguistic information, with a concordance correlation coefficient (CCC) of .638 on MSP-Podcast. Furthermore, our investigations reveal that transformer-based architectures are more robust to small perturbations compared to a CNN-based baseline and fair with respect to biological sex groups, but not towards individual speakers. Finally, we are the first to show that their extraordinary success on valence is based on implicit linguistic information learnt during fine-tuning of the transformer layers, which explains why they perform on-par with recent multimodal approaches that explicitly utilise textual information. Our findings collectively paint the following picture: transformer-based architectures constitute the new state-of-the-art in SER, but further advances are needed to mitigate remaining robustness and individual speaker issues. To make our findings reproducible, we release the best performing model to the community.
△ Less
Submitted 7 September, 2023; v1 submitted 14 March, 2022;
originally announced March 2022.
-
S5CL: Unifying Fully-Supervised, Self-Supervised, and Semi-Supervised Learning Through Hierarchical Contrastive Learning
Authors:
Manuel Tran,
Sophia J. Wagner,
Melanie Boxberg,
Tingying Peng
Abstract:
In computational pathology, we often face a scarcity of annotations and a large amount of unlabeled data. One method for dealing with this is semi-supervised learning which is commonly split into a self-supervised pretext task and a subsequent model fine-tuning. Here, we compress this two-stage training into one by introducing S5CL, a unified framework for fully-supervised, self-supervised, and se…
▽ More
In computational pathology, we often face a scarcity of annotations and a large amount of unlabeled data. One method for dealing with this is semi-supervised learning which is commonly split into a self-supervised pretext task and a subsequent model fine-tuning. Here, we compress this two-stage training into one by introducing S5CL, a unified framework for fully-supervised, self-supervised, and semi-supervised learning. With three contrastive losses defined for labeled, unlabeled, and pseudo-labeled images, S5CL can learn feature representations that reflect the hierarchy of distance relationships: similar images and augmentations are embedded the closest, followed by different looking images of the same class, while images from separate classes have the largest distance. Moreover, S5CL allows us to flexibly combine these losses to adapt to different scenarios. Evaluations of our framework on two public histopathological datasets show strong improvements in the case of sparse labels: for a H&E-stained colorectal cancer dataset, the accuracy increases by up to 9% compared to supervised cross-entropy loss; for a highly imbalanced dataset of single white blood cells from leukemia patient blood smears, the F1-score increases by up to 6%.
△ Less
Submitted 14 March, 2022;
originally announced March 2022.
-
Multi-Modality Microscopy Image Style Transfer for Nuclei Segmentation
Authors:
Ye Liu,
Sophia J. Wagner,
Tingying Peng
Abstract:
Annotating microscopy images for nuclei segmentation is laborious and time-consuming. To leverage the few existing annotations, also across multiple modalities, we propose a novel microscopy-style augmentation technique based on a generative adversarial network (GAN). Unlike other style transfer methods, it can not only deal with different cell assay types and lighting conditions, but also with di…
▽ More
Annotating microscopy images for nuclei segmentation is laborious and time-consuming. To leverage the few existing annotations, also across multiple modalities, we propose a novel microscopy-style augmentation technique based on a generative adversarial network (GAN). Unlike other style transfer methods, it can not only deal with different cell assay types and lighting conditions, but also with different imaging modalities, such as bright-field and fluorescence microscopy. Using disentangled representations for content and style, we can preserve the structure of the original image while altering its style during augmentation. We evaluate our data augmentation on the 2018 Data Science Bowl dataset consisting of various cell assays, lighting conditions, and imaging modalities. With our style augmentation, the segmentation accuracy of the two top-ranked Mask R-CNN-based nuclei segmentation algorithms in the competition increases significantly. Thus, our augmentation technique renders the downstream task more robust to the test data heterogeneity and helps counteract class imbalance without resampling of minority classes.
△ Less
Submitted 23 November, 2021;
originally announced November 2021.
-
Revisiting Tri-training of Dependency Parsers
Authors:
Joachim Wagner,
Jennifer Foster
Abstract:
We compare two orthogonal semi-supervised learning techniques, namely tri-training and pretrained word embeddings, in the task of dependency parsing. We explore language-specific FastText and ELMo embeddings and multilingual BERT embeddings. We focus on a low resource scenario as semi-supervised learning can be expected to have the most impact here. Based on treebank size and available ELMo models…
▽ More
We compare two orthogonal semi-supervised learning techniques, namely tri-training and pretrained word embeddings, in the task of dependency parsing. We explore language-specific FastText and ELMo embeddings and multilingual BERT embeddings. We focus on a low resource scenario as semi-supervised learning can be expected to have the most impact here. Based on treebank size and available ELMo models, we select Hungarian, Uyghur (a zero-shot language for mBERT) and Vietnamese. Furthermore, we include English in a simulated low-resource setting. We find that pretrained word embeddings make more effective use of unlabelled data than tri-training but that the two approaches can be successfully combined.
△ Less
Submitted 16 September, 2021;
originally announced September 2021.
-
gaBERT -- an Irish Language Model
Authors:
James Barry,
Joachim Wagner,
Lauren Cassidy,
Alan Cowap,
Teresa Lynn,
Abigail Walsh,
Mícheál J. Ó Meachair,
Jennifer Foster
Abstract:
The BERT family of neural language models have become highly popular due to their ability to provide sequences of text with rich context-sensitive token encodings which are able to generalise well to many NLP tasks. We introduce gaBERT, a monolingual BERT model for the Irish language. We compare our gaBERT model to multilingual BERT and the monolingual Irish WikiBERT, and we show that gaBERT provi…
▽ More
The BERT family of neural language models have become highly popular due to their ability to provide sequences of text with rich context-sensitive token encodings which are able to generalise well to many NLP tasks. We introduce gaBERT, a monolingual BERT model for the Irish language. We compare our gaBERT model to multilingual BERT and the monolingual Irish WikiBERT, and we show that gaBERT provides better representations for a downstream parsing task. We also show how different filtering criteria, vocabulary size and the choice of subword tokenisation model affect downstream performance. We compare the results of fine-tuning a gaBERT model with an mBERT model for the task of identifying verbal multiword expressions, and show that the fine-tuned gaBERT model also performs better at this task. We release gaBERT and related code to the community.
△ Less
Submitted 28 June, 2022; v1 submitted 27 July, 2021;
originally announced July 2021.
-
Structure-Preserving Multi-Domain Stain Color Augmentation using Style-Transfer with Disentangled Representations
Authors:
Sophia J. Wagner,
Nadieh Khalili,
Raghav Sharma,
Melanie Boxberg,
Carsten Marr,
Walter de Back,
Tingying Peng
Abstract:
In digital pathology, different staining procedures and scanners cause substantial color variations in whole-slide images (WSIs), especially across different laboratories. These color shifts result in a poor generalization of deep learning-based methods from the training domain to external pathology data. To increase test performance, stain normalization techniques are used to reduce the variance…
▽ More
In digital pathology, different staining procedures and scanners cause substantial color variations in whole-slide images (WSIs), especially across different laboratories. These color shifts result in a poor generalization of deep learning-based methods from the training domain to external pathology data. To increase test performance, stain normalization techniques are used to reduce the variance between training and test domain. Alternatively, color augmentation can be applied during training leading to a more robust model without the extra step of color normalization at test time. We propose a novel color augmentation technique, HistAuGAN, that can simulate a wide variety of realistic histology stain colors, thus making neural networks stain-invariant when applied during training. Based on a generative adversarial network (GAN) for image-to-image translation, our model disentangles the content of the image, i.e., the morphological tissue structure, from the stain color attributes. It can be trained on multiple domains and, therefore, learns to cover different stain colors as well as other domain-specific variations introduced in the slide preparation and imaging process. We demonstrate that HistAuGAN outperforms conventional color augmentation techniques on a classification task on the publicly available dataset Camelyon17 and show that it is able to mitigate present batch effects.
△ Less
Submitted 26 July, 2021;
originally announced July 2021.
-
The DCU-EPFL Enhanced Dependency Parser at the IWPT 2021 Shared Task
Authors:
James Barry,
Alireza Mohammadshahi,
Joachim Wagner,
Jennifer Foster,
James Henderson
Abstract:
We describe the DCU-EPFL submission to the IWPT 2021 Shared Task on Parsing into Enhanced Universal Dependencies. The task involves parsing Enhanced UD graphs, which are an extension of the basic dependency trees designed to be more facilitative towards representing semantic structure. Evaluation is carried out on 29 treebanks in 17 languages and participants are required to parse the data from ea…
▽ More
We describe the DCU-EPFL submission to the IWPT 2021 Shared Task on Parsing into Enhanced Universal Dependencies. The task involves parsing Enhanced UD graphs, which are an extension of the basic dependency trees designed to be more facilitative towards representing semantic structure. Evaluation is carried out on 29 treebanks in 17 languages and participants are required to parse the data from each language starting from raw strings. Our approach uses the Stanza pipeline to preprocess the text files, XLMRoBERTa to obtain contextualized token representations, and an edge-scoring and labeling model to predict the enhanced graph. Finally, we run a post-processing script to ensure all of our outputs are valid Enhanced UD graphs. Our system places 6th out of 9 participants with a coarse Enhanced Labeled Attachment Score (ELAS) of 83.57. We carry out additional post-deadline experiments which include using Trankit for pre-processing, XLM-RoBERTa-LARGE, treebank concatenation, and multitask learning between a basic and an enhanced dependency parser. All of these modifications improve our initial score and our final system has a coarse ELAS of 88.04.
△ Less
Submitted 5 July, 2021;
originally announced July 2021.
-
Pedestrian Behavior Prediction for Automated Driving: Requirements, Metrics, and Relevant Features
Authors:
Michael Herman,
Jörg Wagner,
Vishnu Prabhakaran,
Nicolas Möser,
Hanna Ziesche,
Waleed Ahmed,
Lutz Bürkle,
Ernst Kloppenburg,
Claudius Gläser
Abstract:
Automated vehicles require a comprehensive understanding of traffic situations to ensure safe and anticipatory driving. In this context, the prediction of pedestrians is particularly challenging as pedestrian behavior can be influenced by multiple factors. In this paper, we thoroughly analyze the requirements on pedestrian behavior prediction for automated driving via a system-level approach. To t…
▽ More
Automated vehicles require a comprehensive understanding of traffic situations to ensure safe and anticipatory driving. In this context, the prediction of pedestrians is particularly challenging as pedestrian behavior can be influenced by multiple factors. In this paper, we thoroughly analyze the requirements on pedestrian behavior prediction for automated driving via a system-level approach. To this end we investigate real-world pedestrian-vehicle interactions with human drivers. Based on human driving behavior we then derive appropriate reaction patterns of an automated vehicle and determine requirements for the prediction of pedestrians. This includes a novel metric tailored to measure prediction performance from a system-level perspective. The proposed metric is evaluated on a large-scale dataset comprising thousands of real-world pedestrian-vehicle interactions. We furthermore conduct an ablation study to evaluate the importance of different contextual cues and compare these results to ones obtained using established performance metrics for pedestrian prediction. Our results highlight the importance of a system-level approach to pedestrian behavior prediction.
△ Less
Submitted 16 October, 2021; v1 submitted 15 December, 2020;
originally announced December 2020.
-
papaya2: 2D Irreducible Minkowski Tensor computation
Authors:
Fabian M. Schaller,
Jenny Wagner,
Sebastian C. Kapfer
Abstract:
A common challenge in scientific and technical domains is the quantitative description of geometries and shapes, e.g. in the analysis of microscope imagery or astronomical observation data. Frequently, it is desirable to go beyond scalar shape metrics such as porosity and surface to volume ratios because the samples are anisotropic or because direction-dependent quantities such as conductances or…
▽ More
A common challenge in scientific and technical domains is the quantitative description of geometries and shapes, e.g. in the analysis of microscope imagery or astronomical observation data. Frequently, it is desirable to go beyond scalar shape metrics such as porosity and surface to volume ratios because the samples are anisotropic or because direction-dependent quantities such as conductances or elasticity are of interest. Minkowski Tensors are a systematic family of versatile and robust higher-order shape descriptors that allow for shape characterization of arbitrary order and promise a path to systematic structure-function relationships for direction-dependent properties. Papaya2 is a software to calculate 2D higher-order shape metrics with a library interface, support for Irreducible Minkowski Tensors and interpolated marching squares. Extensions to Matlab, JavaScript and Python are provided as well. While the tensor of inertia is computed by many tools, we are not aware of other open-source software which provides higher-rank shape characterization in 2D.
△ Less
Submitted 28 October, 2020;
originally announced October 2020.
-
The ADAPT Enhanced Dependency Parser at the IWPT 2020 Shared Task
Authors:
James Barry,
Joachim Wagner,
Jennifer Foster
Abstract:
We describe the ADAPT system for the 2020 IWPT Shared Task on parsing enhanced Universal Dependencies in 17 languages. We implement a pipeline approach using UDPipe and UDPipe-future to provide initial levels of annotation. The enhanced dependency graph is either produced by a graph-based semantic dependency parser or is built from the basic tree using a small set of heuristics. Our results show t…
▽ More
We describe the ADAPT system for the 2020 IWPT Shared Task on parsing enhanced Universal Dependencies in 17 languages. We implement a pipeline approach using UDPipe and UDPipe-future to provide initial levels of annotation. The enhanced dependency graph is either produced by a graph-based semantic dependency parser or is built from the basic tree using a small set of heuristics. Our results show that, for the majority of languages, a semantic dependency parser can be successfully applied to the task of parsing enhanced dependencies.
Unfortunately, we did not ensure a connected graph as part of our pipeline approach and our competition submission relied on a last-minute fix to pass the validation script which harmed our official evaluation scores significantly. Our submission ranked eighth in the official evaluation with a macro-averaged coarse ELAS F1 of 67.23 and a treebank average of 67.49. We later implemented our own graph-connecting fix which resulted in a score of 79.53 (language average) or 79.76 (treebank average), which would have placed fourth in the competition evaluation.
△ Less
Submitted 3 September, 2020;
originally announced September 2020.
-
Treebank Embedding Vectors for Out-of-domain Dependency Parsing
Authors:
Joachim Wagner,
James Barry,
Jennifer Foster
Abstract:
A recent advance in monolingual dependency parsing is the idea of a treebank embedding vector, which allows all treebanks for a particular language to be used as training data while at the same time allowing the model to prefer training data from one treebank over others and to select the preferred treebank at test time. We build on this idea by 1) introducing a method to predict a treebank vector…
▽ More
A recent advance in monolingual dependency parsing is the idea of a treebank embedding vector, which allows all treebanks for a particular language to be used as training data while at the same time allowing the model to prefer training data from one treebank over others and to select the preferred treebank at test time. We build on this idea by 1) introducing a method to predict a treebank vector for sentences that do not come from a treebank used in training, and 2) exploring what happens when we move away from predefined treebank embedding vectors during test time and instead devise tailored interpolations. We show that 1) there are interpolated vectors that are superior to the predefined ones, and 2) treebank vectors can be predicted with sufficient accuracy, for nine out of ten test languages, to match the performance of an oracle approach that knows the most suitable predefined treebank embedding for the test set.
△ Less
Submitted 2 May, 2020;
originally announced May 2020.
-
On Learning Vehicle Detection in Satellite Video
Authors:
Roman Pflugfelder,
Axel Weissenfeld,
Julian Wagner
Abstract:
Vehicle detection in aerial and satellite images is still challenging due to their tiny appearance in pixels compared to the overall size of remote sensing imagery. Classical methods of object detection very often fail in this scenario due to violation of implicit assumptions made such as rich texture, small to moderate ratios between image size and object size. Satellite video is a very new modal…
▽ More
Vehicle detection in aerial and satellite images is still challenging due to their tiny appearance in pixels compared to the overall size of remote sensing imagery. Classical methods of object detection very often fail in this scenario due to violation of implicit assumptions made such as rich texture, small to moderate ratios between image size and object size. Satellite video is a very new modality which introduces temporal consistency as inductive bias. Approaches for vehicle detection in satellite video use either background subtraction, frame differencing or subspace methods showing moderate performance (0.26 - 0.82 $F_1$ score). This work proposes to apply recent work on deep learning for wide-area motion imagery (WAMI) on satellite video. We show in a first approach comparable results (0.84 $F_1$) on Planet's SkySat-1 LasVegas video with room for further improvement.
△ Less
Submitted 29 January, 2020;
originally announced January 2020.
-
Towards a Predictive Patent Analytics and Evaluation Platform
Authors:
Nebula Alam,
Khoi-Nguyen Tran,
Sue Ann Chen,
John Wagner,
Josh Andres,
Mukesh Mohania
Abstract:
The importance of patents is well recognised across many regions of the world. Many patent mining systems have been proposed, but with limited predictive capabilities. In this demo, we showcase how predictive algorithms leveraging the state-of-the-art machine learning and deep learning techniques can be used to improve understanding of patents for inventors, patent evaluators, and business analyst…
▽ More
The importance of patents is well recognised across many regions of the world. Many patent mining systems have been proposed, but with limited predictive capabilities. In this demo, we showcase how predictive algorithms leveraging the state-of-the-art machine learning and deep learning techniques can be used to improve understanding of patents for inventors, patent evaluators, and business analysts alike. Our demo video is available at http://ibm.biz/ecml2019-demo-patent-analytics
△ Less
Submitted 31 October, 2019;
originally announced October 2019.
-
Cross-lingual Parsing with Polyglot Training and Multi-treebank Learning: A Faroese Case Study
Authors:
James Barry,
Joachim Wagner,
Jennifer Foster
Abstract:
Cross-lingual dependency parsing involves transferring syntactic knowledge from one language to another. It is a crucial component for inducing dependency parsers in low-resource scenarios where no training data for a language exists. Using Faroese as the target language, we compare two approaches using annotation projection: first, projecting from multiple monolingual source models; second, proje…
▽ More
Cross-lingual dependency parsing involves transferring syntactic knowledge from one language to another. It is a crucial component for inducing dependency parsers in low-resource scenarios where no training data for a language exists. Using Faroese as the target language, we compare two approaches using annotation projection: first, projecting from multiple monolingual source models; second, projecting from a single polyglot model which is trained on the combination of all source languages. Furthermore, we reproduce multi-source projection (Tyers et al., 2018), in which dependency trees of multiple sources are combined. Finally, we apply multi-treebank modelling to the projected treebanks, in addition to or alternatively to polyglot modelling on the source side. We find that polyglot training on the source languages produces an overall trend of better results on the target language but the single best result for the target language is obtained by projecting from monolingual source parsing models and then training multi-treebank POS tagging and parsing models on the target side.
△ Less
Submitted 17 October, 2019;
originally announced October 2019.
-
Interpretable and Fine-Grained Visual Explanations for Convolutional Neural Networks
Authors:
Jörg Wagner,
Jan Mathias Köhler,
Tobias Gindele,
Leon Hetzel,
Jakob Thaddäus Wiedemer,
Sven Behnke
Abstract:
To verify and validate networks, it is essential to gain insight into their decisions, limitations as well as possible shortcomings of training data. In this work, we propose a post-hoc, optimization based visual explanation method, which highlights the evidence in the input image for a specific prediction. Our approach is based on a novel technique to defend against adversarial evidence (i.e. fau…
▽ More
To verify and validate networks, it is essential to gain insight into their decisions, limitations as well as possible shortcomings of training data. In this work, we propose a post-hoc, optimization based visual explanation method, which highlights the evidence in the input image for a specific prediction. Our approach is based on a novel technique to defend against adversarial evidence (i.e. faulty evidence due to artefacts) by filtering gradients during optimization. The defense does not depend on human-tuned parameters. It enables explanations which are both fine-grained and preserve the characteristics of images, such as edges and colors. The explanations are interpretable, suited for visualizing detailed evidence and can be tested as they are valid model inputs. We qualitatively and quantitatively evaluate our approach on a multitude of models and datasets.
△ Less
Submitted 7 August, 2019;
originally announced August 2019.
-
Wasserstein Adversarial Imitation Learning
Authors:
Huang Xiao,
Michael Herman,
Joerg Wagner,
Sebastian Ziesche,
Jalal Etesami,
Thai Hong Linh
Abstract:
Imitation Learning describes the problem of recovering an expert policy from demonstrations. While inverse reinforcement learning approaches are known to be very sample-efficient in terms of expert demonstrations, they usually require problem-dependent reward functions or a (task-)specific reward-function regularization. In this paper, we show a natural connection between inverse reinforcement lea…
▽ More
Imitation Learning describes the problem of recovering an expert policy from demonstrations. While inverse reinforcement learning approaches are known to be very sample-efficient in terms of expert demonstrations, they usually require problem-dependent reward functions or a (task-)specific reward-function regularization. In this paper, we show a natural connection between inverse reinforcement learning approaches and Optimal Transport, that enables more general reward functions with desirable properties (e.g., smoothness). Based on our observation, we propose a novel approach called Wasserstein Adversarial Imitation Learning. Our approach considers the Kantorovich potentials as a reward function and further leverages regularized optimal transport to enable large-scale applications. In several robotic experiments, our approach outperforms the baselines in terms of average cumulative rewards and shows a significant improvement in sample-efficiency, by requiring just one expert demonstration.
△ Less
Submitted 19 June, 2019;
originally announced June 2019.
-
Functionally Modular and Interpretable Temporal Filtering for Robust Segmentation
Authors:
Jörg Wagner,
Volker Fischer,
Michael Herman,
Sven Behnke
Abstract:
The performance of autonomous systems heavily relies on their ability to generate a robust representation of the environment. Deep neural networks have greatly improved vision-based perception systems but still fail in challenging situations, e.g. sensor outages or heavy weather. These failures are often introduced by data-inherent perturbations, which significantly reduce the information provided…
▽ More
The performance of autonomous systems heavily relies on their ability to generate a robust representation of the environment. Deep neural networks have greatly improved vision-based perception systems but still fail in challenging situations, e.g. sensor outages or heavy weather. These failures are often introduced by data-inherent perturbations, which significantly reduce the information provided to the perception system. We propose a functionally modularized temporal filter, which stabilizes an abstract feature representation of a single-frame segmentation model using information of previous time steps. Our filter module splits the filter task into multiple less complex and more interpretable subtasks. The basic structure of the filter is inspired by a Bayes estimator consisting of a prediction and an update step. To make the prediction more transparent, we implement it using a geometric projection and estimate its parameters. This additionally enables the decomposition of the filter task into static representation filtering and low-dimensional motion filtering. Our model can cope with missing frames and is trainable in an end-to-end fashion. Using photorealistic, synthetic video data, we show the ability of the proposed architecture to overcome data-inherent perturbations. The experiments especially highlight advantages introduced by an interpretable and explicit filter module.
△ Less
Submitted 15 October, 2018; v1 submitted 9 October, 2018;
originally announced October 2018.
-
Hierarchical Recurrent Filtering for Fully Convolutional DenseNets
Authors:
Jörg Wagner,
Volker Fischer,
Michael Herman,
Sven Behnke
Abstract:
Generating a robust representation of the environment is a crucial ability of learning agents. Deep learning based methods have greatly improved perception systems but still fail in challenging situations. These failures are often not solvable on the basis of a single image. In this work, we present a parameter-efficient temporal filtering concept which extends an existing single-frame segmentatio…
▽ More
Generating a robust representation of the environment is a crucial ability of learning agents. Deep learning based methods have greatly improved perception systems but still fail in challenging situations. These failures are often not solvable on the basis of a single image. In this work, we present a parameter-efficient temporal filtering concept which extends an existing single-frame segmentation model to work with multiple frames. The resulting recurrent architecture temporally filters representations on all abstraction levels in a hierarchical manner, while decoupling temporal dependencies from scene representation. Using a synthetic dataset, we show the ability of our model to cope with data perturbations and highlight the importance of recurrent and hierarchical filtering.
△ Less
Submitted 15 October, 2018; v1 submitted 5 October, 2018;
originally announced October 2018.
-
Dynamic simulations in SixTrack
Authors:
K. Sjobak,
V. K. Berglyd Olsen,
R. De Maria,
M. Fitterer,
A. Santamaría García,
H. Garcia-Morales,
A. Mereghetti,
J. F. Wagner,
S. J. Wretborn
Abstract:
The DYNK module allows element settings in SixTrack to be changed on a turn-by-turn basis. This document contains a technical description of the DYNK module in SixTrack. It is mainly intended for a developer or advanced user who wants to modify the DYNK module, for example by adding more functions that can be used to calculate new element settings, or to add support for new elements that can be us…
▽ More
The DYNK module allows element settings in SixTrack to be changed on a turn-by-turn basis. This document contains a technical description of the DYNK module in SixTrack. It is mainly intended for a developer or advanced user who wants to modify the DYNK module, for example by adding more functions that can be used to calculate new element settings, or to add support for new elements that can be used with DYNK.
△ Less
Submitted 19 August, 2018;
originally announced August 2018.
-
Applying Cooperative Machine Learning to Speed Up the Annotation of Social Signals in Large Multi-modal Corpora
Authors:
Johannes Wagner,
Tobias Baur,
Yue Zhang,
Michel F. Valstar,
Björn Schuller,
Elisabeth André
Abstract:
Scientific disciplines, such as Behavioural Psychology, Anthropology and recently Social Signal Processing are concerned with the systematic exploration of human behaviour. A typical work-flow includes the manual annotation (also called coding) of social signals in multi-modal corpora of considerable size. For the involved annotators this defines an exhausting and time-consuming task. In the artic…
▽ More
Scientific disciplines, such as Behavioural Psychology, Anthropology and recently Social Signal Processing are concerned with the systematic exploration of human behaviour. A typical work-flow includes the manual annotation (also called coding) of social signals in multi-modal corpora of considerable size. For the involved annotators this defines an exhausting and time-consuming task. In the article at hand we present a novel method and also provide the tools to speed up the coding procedure. To this end, we suggest and evaluate the use of Cooperative Machine Learning (CML) techniques to reduce manual labelling efforts by combining the power of computational capabilities and human intelligence. The proposed CML strategy starts with a small number of labelled instances and concentrates on predicting local parts first. Afterwards, a session-independent classification model is created to finish the remaining parts of the database. Confidence values are computed to guide the manual inspection and correction of the predictions. To bring the proposed approach into application we introduce NOVA - an open-source tool for collaborative and machine-aided annotations. In particular, it gives labellers immediate access to CML strategies and directly provides visual feedback on the results. Our experiments show that the proposed method has the potential to significantly reduce human labelling efforts.
△ Less
Submitted 7 February, 2018;
originally announced February 2018.
-
Inverse Reinforcement Learning with Simultaneous Estimation of Rewards and Dynamics
Authors:
Michael Herman,
Tobias Gindele,
Jörg Wagner,
Felix Schmitt,
Wolfram Burgard
Abstract:
Inverse Reinforcement Learning (IRL) describes the problem of learning an unknown reward function of a Markov Decision Process (MDP) from observed behavior of an agent. Since the agent's behavior originates in its policy and MDP policies depend on both the stochastic system dynamics as well as the reward function, the solution of the inverse problem is significantly influenced by both. Current IRL…
▽ More
Inverse Reinforcement Learning (IRL) describes the problem of learning an unknown reward function of a Markov Decision Process (MDP) from observed behavior of an agent. Since the agent's behavior originates in its policy and MDP policies depend on both the stochastic system dynamics as well as the reward function, the solution of the inverse problem is significantly influenced by both. Current IRL approaches assume that if the transition model is unknown, additional samples from the system's dynamics are accessible, or the observed behavior provides enough samples of the system's dynamics to solve the inverse problem accurately. These assumptions are often not satisfied. To overcome this, we present a gradient-based IRL approach that simultaneously estimates the system's dynamics. By solving the combined optimization problem, our approach takes into account the bias of the demonstrations, which stems from the generating policy. The evaluation on a synthetic MDP and a transfer learning task shows improvements regarding the sample efficiency as well as the accuracy of the estimated reward functions and transition models.
△ Less
Submitted 13 April, 2016;
originally announced April 2016.
-
JSAI: Designing a Sound, Configurable, and Efficient Static Analyzer for JavaScript
Authors:
Vineeth Kashyap,
Kyle Dewey,
Ethan A. Kuefner,
John Wagner,
Kevin Gibbons,
John Sarracino,
Ben Wiedermann,
Ben Hardekopf
Abstract:
We describe JSAI, an abstract interpreter for JavaScript. JSAI uses novel abstract domains to compute a reduced product of type inference, pointer analysis, string analysis, integer and boolean constant propagation, and control-flow analysis. In addition, JSAI allows for analysis control-flow sensitivity (i.e., context-, path-, and heap-sensitivity) to be modularly configured without requiring any…
▽ More
We describe JSAI, an abstract interpreter for JavaScript. JSAI uses novel abstract domains to compute a reduced product of type inference, pointer analysis, string analysis, integer and boolean constant propagation, and control-flow analysis. In addition, JSAI allows for analysis control-flow sensitivity (i.e., context-, path-, and heap-sensitivity) to be modularly configured without requiring any changes to the analysis implementation. JSAI is designed to be provably sound with respect to a specific concrete semantics for JavaScript, which has been extensively tested against existing production-quality JavaScript implementations.
We provide a comprehensive evaluation of JSAI's performance and precision using an extensive benchmark suite. This benchmark suite includes real-world JavaScript applications, machine-generated JavaScript code via Emscripten, and browser addons. We use JSAI's configurability to evaluate a large number of analysis sensitivities (some well-known, some novel) and observe some surprising results. We believe that JSAI's configurability and its formal specifications position it as a useful research platform to experiment on novel sensitivities, abstract domains, and client analyses for JavaScript.
△ Less
Submitted 17 March, 2014;
originally announced March 2014.
-
On the Distortion of the Eigenvalue Spectrum in MIMO Amplify-and-Forward Multi-Hop Channels
Authors:
Joerg Wagner,
Armin Wittneben
Abstract:
Consider a wireless MIMO multi-hop channel with n_s non-cooperating source antennas and n_d fully cooperating destination antennas, as well as L clusters containing k non-cooperating relay antennas each. The source signal traverses all L clusters of relay antennas, before it reaches the destination. When relay antennas within the same cluster scale their received signals by the same constant bef…
▽ More
Consider a wireless MIMO multi-hop channel with n_s non-cooperating source antennas and n_d fully cooperating destination antennas, as well as L clusters containing k non-cooperating relay antennas each. The source signal traverses all L clusters of relay antennas, before it reaches the destination. When relay antennas within the same cluster scale their received signals by the same constant before the retransmission, the equivalent channel matrix H relating the input signals at the source antennas to the output signals at the destination antennas is proportional to the product of channel matrices H_l, l=1,...,L+1, corresponding to the individual hops. We perform an asymptotic capacity analysis for this channel as follows: In a first instance we take the limits n_s->infty, n_d->infty and k->infty, but keep both n_s/n_d and k/n_d fixed. Then, we take the limits L->infty and k/n_d->infty. Requiring that the H_l's satisfy the conditions needed for the Marcenko-Pastur law, we prove that the capacity scales linearly in min{n_s,n_d}, as long as the ratio k/n_d scales at least linearly in L. Moreover, we show that up to a noise penalty and a pre-log factor the capacity of a point-to-point MIMO channel is approached, when this scaling is slightly faster than linear. Conversely, almost all spatial degrees of freedom vanish for less than linear scaling.
△ Less
Submitted 27 August, 2007;
originally announced August 2007.
-
Large N Analysis of Amplify-and-Forward MIMO Relay Channels with Correlated Rayleigh Fading
Authors:
Joerg Wagner,
Boris Rankov,
Armin Wittneben
Abstract:
In this correspondence the cumulants of the mutual information of the flat Rayleigh fading amplify-and-forward MIMO relay channel without direct link between source and destination are derived in the large array limit. The analysis is based on the replica trick and covers both spatially independent and correlated fading in the first and the second hop, while beamforming at all terminals is restr…
▽ More
In this correspondence the cumulants of the mutual information of the flat Rayleigh fading amplify-and-forward MIMO relay channel without direct link between source and destination are derived in the large array limit. The analysis is based on the replica trick and covers both spatially independent and correlated fading in the first and the second hop, while beamforming at all terminals is restricted to deterministic weight matrices. Expressions for mean and variance of the mutual information are obtained. Their parameters are determined by a nonlinear equation system. All higher cumulants are shown to vanish as the number of antennas n goes to infinity. In conclusion the distribution of the mutual information I becomes Gaussian in the large n limit and is completely characterized by the expressions obtained for mean and variance of I. Comparisons with simulation results show that the asymptotic results serve as excellent approximations for systems with only few antennas at each node. The derivation of the results follows the technique formalized by Moustakas et al. in [1]. Although the evaluations are more involved for the MIMO relay channel compared to point-to-point MIMO channels, the structure of the results is surprisingly simple again. In particular an elegant formula for the mean of the mutual information is obtained, i.e., the ergodic capacity of the two-hop amplify-and-forward MIMO relay channel without direct link.
△ Less
Submitted 22 December, 2006;
originally announced December 2006.