-
Retina-inspired Object Motion Segmentation
Authors:
Victoria Clerico,
Shay Snyder,
Arya Lohia,
Md Abdullah-Al Kaiser,
Gregory Schwartz,
Akhilesh Jaiswal,
Maryam Parsa
Abstract:
Dynamic Vision Sensors (DVS) have emerged as a revolutionary technology with a high temporal resolution that far surpasses RGB cameras. DVS technology draws biological inspiration from photoreceptors and the initial retinal synapse. Our research showcases the potential of additional retinal functionalities to extract visual features. We provide a domain-agnostic and efficient algorithm for ego-mot…
▽ More
Dynamic Vision Sensors (DVS) have emerged as a revolutionary technology with a high temporal resolution that far surpasses RGB cameras. DVS technology draws biological inspiration from photoreceptors and the initial retinal synapse. Our research showcases the potential of additional retinal functionalities to extract visual features. We provide a domain-agnostic and efficient algorithm for ego-motion compensation based on Object Motion Sensitivity (OMS), one of the multiple robust features computed within the mammalian retina. We develop a framework based on experimental neuroscience that translates OMS' biological circuitry to a low-overhead algorithm. OMS processes DVS data from dynamic scenes to perform pixel-wise object motion segmentation. Using a real and a synthetic dataset, we highlight OMS' ability to differentiate object motion from ego-motion, bypassing the need for deep networks. This paper introduces a bio-inspired computer vision method that dramatically reduces the number of parameters by a factor of 1000 compared to prior works. Our work paves the way for robust, high-speed, and low-bandwidth decision-making for in-sensor computations.
△ Less
Submitted 18 August, 2024;
originally announced August 2024.
-
Hardware-Algorithm Re-engineering of Retinal Circuit for Intelligent Object Motion Segmentation
Authors:
Jason Sinaga,
Victoria Clerico,
Md Abdullah-Al Kaiser,
Shay Snyder,
Arya Lohia,
Gregory Schwartz,
Maryam Parsa,
Akhilesh Jaiswal
Abstract:
Recent advances in retinal neuroscience have fueled various hardware and algorithmic efforts to develop retina-inspired solutions for computer vision tasks. In this work, we focus on a fundamental visual feature within the mammalian retina, Object Motion Sensitivity (OMS). Using DVS data from EV-IMO dataset, we analyze the performance of an algorithmic implementation of OMS circuitry for motion se…
▽ More
Recent advances in retinal neuroscience have fueled various hardware and algorithmic efforts to develop retina-inspired solutions for computer vision tasks. In this work, we focus on a fundamental visual feature within the mammalian retina, Object Motion Sensitivity (OMS). Using DVS data from EV-IMO dataset, we analyze the performance of an algorithmic implementation of OMS circuitry for motion segmentation in presence of ego-motion. This holistic analysis considers the underlying constraints arising from the hardware circuit implementation. We present novel CMOS circuits that implement OMS functionality inside image sensors, while providing run-time re-configurability for key algorithmic parameters. In-sensor technologies for dynamical environment adaptation are crucial for ensuring high system performance. Finally, we verify the functionality and re-configurability of the proposed CMOS circuit designs through Cadence simulations in 180nm technology. In summary, the presented work lays foundation for hardware-algorithm re-engineering of known biological circuits to suit application needs.
△ Less
Submitted 8 September, 2024; v1 submitted 31 July, 2024;
originally announced August 2024.
-
Optical tools for laser machining along six orders of magnitude
Authors:
Julian Hellstern,
Christoph Tillkorn,
Tim Hieronymus,
Myriam Kaiser,
Torsten Beck,
Daniel Flamm
Abstract:
We present an overview on the development and characterization of multiscale laser processing optics for versatile material modifications across more than six orders of magnitude. Starting with solutions for micromachining we present high-NA microscope objectives creating sub-wavelength material modifications on macroscopic scales with highest peak intensities. Moving on to the millimeter range, t…
▽ More
We present an overview on the development and characterization of multiscale laser processing optics for versatile material modifications across more than six orders of magnitude. Starting with solutions for micromachining we present high-NA microscope objectives creating sub-wavelength material modifications on macroscopic scales with highest peak intensities. Moving on to the millimeter range, the adaptability and scalability of scanning optics is examined for large-area machining. Finally, we explore line beam optics in the meter range, evaluating their use in uniform material processing using average powers above 100kW. This study provides an insight into the design and performance characteristics of such optics and demonstrates their potential in advanced laser processing.
△ Less
Submitted 28 March, 2024;
originally announced March 2024.
-
Selective laser etching of displays: Closing the gap between optical simulations and fabrication
Authors:
Martin Wimmer,
Myriam Kaiser,
Jonas Kleiner,
Jannis Wolff,
Max Kahmann,
Daniel Flamm
Abstract:
Simulations and measurements on selective laser etching of display glasses are reported. By means of a holographic 3D beam splitter, ultrashort laser pulses are focused inside the volume of a glass sample creating type III modifications along a specific trajectory like pearls on a string. Superimposed by a feed of the glass sample a full 3D area of modifications is achieved building the cornerston…
▽ More
Simulations and measurements on selective laser etching of display glasses are reported. By means of a holographic 3D beam splitter, ultrashort laser pulses are focused inside the volume of a glass sample creating type III modifications along a specific trajectory like pearls on a string. Superimposed by a feed of the glass sample a full 3D area of modifications is achieved building the cornerstone for subsequent etch processes. Based on KOH the modifications are selectively etched at a much higher rate compared to unmodified regions resulting in a separation of the glass along the trajectory of modifications. For gaining further insight into the etch process, we perform simulations on this wet chemical process and compare it to our experimental results.
△ Less
Submitted 25 March, 2024;
originally announced March 2024.
-
Exploring Spatial Generalized Functional Linear Models: A Comparative Simulation Study and Analysis of COVID-19
Authors:
Sooran Kim,
Mark S. Kaiser,
Xiongtao Dai
Abstract:
Implementation of spatial generalized linear models with a functional covariate can be accomplished through the use of a truncated basis expansion of the covariate process. In practice, one must select a truncation level for use. We compare five criteria for the selection of an appropriate truncation level, including AIC and BIC based on a log composite likelihood, a fraction of variance explained…
▽ More
Implementation of spatial generalized linear models with a functional covariate can be accomplished through the use of a truncated basis expansion of the covariate process. In practice, one must select a truncation level for use. We compare five criteria for the selection of an appropriate truncation level, including AIC and BIC based on a log composite likelihood, a fraction of variance explained criterion, a fitted mean squared error, and a prediction error with one standard error rule. Based on the use of extensive simulation studies, we propose that BIC constitutes a reasonable default criterion for the selection of the truncation level for use in a spatial functional generalized linear model. In addition, we demonstrate that the spatial model with a functional covariate outperforms other models when the data contain spatial structure and response variables are in fact influenced by a functional covariate process. We apply the spatial functional generalized linear model to a problem in which the objective is to relate COVID-19 vaccination rates in counties of states in the Midwestern United States to the number of new cases from previous weeks in those same geographic regions.
△ Less
Submitted 26 March, 2024; v1 submitted 5 March, 2024;
originally announced March 2024.
-
Toward High Performance, Programmable Extreme-Edge Intelligence for Neuromorphic Vision Sensors utilizing Magnetic Domain Wall Motion-based MTJ
Authors:
Md Abdullah-Al Kaiser,
Gourav Datta,
Peter A. Beerel,
Akhilesh R. Jaiswal
Abstract:
The desire to empower resource-limited edge devices with computer vision (CV) must overcome the high energy consumption of collecting and processing vast sensory data. To address the challenge, this work proposes an energy-efficient non-von-Neumann in-pixel processing solution for neuromorphic vision sensors employing emerging (X) magnetic domain wall magnetic tunnel junction (MDWMTJ) for the firs…
▽ More
The desire to empower resource-limited edge devices with computer vision (CV) must overcome the high energy consumption of collecting and processing vast sensory data. To address the challenge, this work proposes an energy-efficient non-von-Neumann in-pixel processing solution for neuromorphic vision sensors employing emerging (X) magnetic domain wall magnetic tunnel junction (MDWMTJ) for the first time, in conjunction with CMOS-based neuromorphic pixels. Our hybrid CMOS+X approach performs in-situ massively parallel asynchronous analog convolution, exhibiting low power consumption and high accuracy across various CV applications by leveraging the non-volatility and programmability of the MDWMTJ. Moreover, our developed device-circuit-algorithm co-design framework captures device constraints (low tunnel-magnetoresistance, low dynamic range) and circuit constraints (non-linearity, process variation, area consideration) based on monte-carlo simulations and device parameters utilizing GF22nm FD-SOI technology. Our experimental results suggest we can achieve an average of 45.3% reduction in backend-processor energy, maintaining similar front-end energy compared to the state-of-the-art and high accuracy of 79.17% and 95.99% on the DVS-CIFAR10 and IBM DVS128-Gesture datasets, respectively.
△ Less
Submitted 23 February, 2024;
originally announced February 2024.
-
Generalized linear models with spatial dependence and a functional covariate
Authors:
Sooran Kim,
Mark S. Kaiser,
Xiongtao Dai
Abstract:
We extend generalized functional linear models under independence to a situation in which a functional covariate is related to a scalar response variable that exhibits spatial dependence. For estimation, we apply basis expansion and truncation for dimension reduction of the covariate process followed by a composite likelihood estimating equation to handle the spatial dependency. We develop asympto…
▽ More
We extend generalized functional linear models under independence to a situation in which a functional covariate is related to a scalar response variable that exhibits spatial dependence. For estimation, we apply basis expansion and truncation for dimension reduction of the covariate process followed by a composite likelihood estimating equation to handle the spatial dependency. We develop asymptotic results for the proposed model under a repeating lattice asymptotic context, allowing us to construct a confidence interval for the spatial dependence parameter and a confidence band for the parameter function. A binary conditionals model is presented as a concrete illustration and is used in simulation studies to verify the applicability of the asymptotic inferential results.
△ Less
Submitted 20 February, 2024;
originally announced February 2024.
-
Compromise-Free Scaling of Qubit Speed and Coherence
Authors:
Miguel J. Carballido,
Simon Svab,
Rafael S. Eggli,
Taras Patlatiuk,
Pierre Chevalier Kwon,
Jonas Schuff,
Rahel M. Kaiser,
Leon C. Camenzind,
Ang Li,
Natalia Ares,
Erik P. A. M Bakkers,
Stefano Bosco,
J. Carlos Egues,
Daniel Loss,
Dominik M. Zumbühl
Abstract:
Across a broad range of qubits, a pervasive trade-off becomes obvious: increased coherence seems to be only possible at the cost of qubit speed. This is consistent with the notion that protecting a qubit from its noisy surroundings also limits the control over it. Indeed, from ions to atoms, to superconductors and spins, the leading qubits share a similar Q-factor - the product of speed and cohere…
▽ More
Across a broad range of qubits, a pervasive trade-off becomes obvious: increased coherence seems to be only possible at the cost of qubit speed. This is consistent with the notion that protecting a qubit from its noisy surroundings also limits the control over it. Indeed, from ions to atoms, to superconductors and spins, the leading qubits share a similar Q-factor - the product of speed and coherence time - even though the speed and coherence of various qubits can differ by up to 8 orders of magnitude. This is the qubit speed-coherence dilemma: qubits are either coherent but slow or fast but short-lived. Here, we demonstrate a qubit for which we can triple the speed while simultaneously quadrupling the Hahn-echo coherence time when tuning a local electric field. In this way, the qubit speed and coherence scale together without compromise on either quantity, boosting the Q-factor by over an order of magnitude. Our qubit is a hole spin in a Ge/Si core/shell nanowire providing strong 1D confinement, resulting in the direct Rashba spin-orbit interaction. Due to Heavy-hole light-hole mixing a maximum of the spin-orbit strength is reached at finite electrical field. At the local maximum, charge fluctuations are decoupled from the qubit and coherence is enhanced, yet the drive speed becomes maximal. Our proof-of-concept experiment shows that a properly engineered qubit can be made faster and simultaneously more coherent, removing an important roadblock. Further, it demonstrates that through all-electrical control a qubit can be sped up, without coupling more strongly to the electrical noise environment. As charge fluctuators are unavoidable in semiconductors and all-electrical control is highly scalable, our results improve the prospects for quantum computing in Si and Ge.
△ Less
Submitted 22 May, 2024; v1 submitted 11 February, 2024;
originally announced February 2024.
-
Properties of Test Statistics for Nonparametric Cointegrating Regression Functions Based on Subsamples
Authors:
Sepideh Mosaferi,
Mark S. Kaiser,
Daniel J. Nordman
Abstract:
Nonparametric cointegrating regression models have been extensively used in financial markets, stock prices, heavy traffic, climate data sets, and energy markets. Models with parametric regression functions can be more appealing in practice compared to non-parametric forms, but do result in potential functional misspecification. Thus, there exists a vast literature on developing a model specificat…
▽ More
Nonparametric cointegrating regression models have been extensively used in financial markets, stock prices, heavy traffic, climate data sets, and energy markets. Models with parametric regression functions can be more appealing in practice compared to non-parametric forms, but do result in potential functional misspecification. Thus, there exists a vast literature on developing a model specification test for parametric forms of regression functions. In this paper, we develop two test statistics which are applicable for the endogenous regressors driven by long memory and semi-long memory input shocks in the regression model. The limit distributions of the test statistics under these two scenarios are complicated and cannot be effectively used in practice. To overcome this difficulty, we use the subsampling method and compute the test statistics on smaller blocks of the data to construct their empirical distributions. Throughout, Monte Carlo simulation studies are used to illustrate the properties of test statistics. We also provide an empirical example of relating gross domestic product to total output of carbon dioxide in two European countries.
△ Less
Submitted 26 December, 2023;
originally announced December 2023.
-
Simultaneous Analysis of Continuously Embedded Reissner-Mindlin Shells in 3D Bulk Domains
Authors:
Michael Wolfgang Kaiser,
Thomas-Peter Fries
Abstract:
A mechanical model and numerical method for the simultaneous analysis of Reissner-Mindlin shells with geometries implied by a continuous set of level sets (isosurfaces) over some three-dimensional bulk domain is presented. A three-dimensional mesh in the bulk domain is used in a tailored FEM formulation where the elements are by no means conforming to the level sets representing the shape of the i…
▽ More
A mechanical model and numerical method for the simultaneous analysis of Reissner-Mindlin shells with geometries implied by a continuous set of level sets (isosurfaces) over some three-dimensional bulk domain is presented. A three-dimensional mesh in the bulk domain is used in a tailored FEM formulation where the elements are by no means conforming to the level sets representing the shape of the individual shells. However, the shell geometries are bounded by the intersection curves of the level sets with the boundary of the bulk domain so that the boundaries are meshed conformingly. This results in a method which was coined Bulk Trace FEM before. The simultaneously considered, continuously embedded shells may be useful in the structural design process or for the continuous reinforcement of bulk domains. Numerical results confirm higher-order convergence rates.
△ Less
Submitted 28 November, 2023;
originally announced November 2023.
-
Reinforcement Learning for Safety Testing: Lessons from A Mobile Robot Case Study
Authors:
Tom P. Huck,
Martin Kaiser,
Constantin Cronrath,
Bengt Lennartson,
Torsten Kröger,
Tamim Asfour
Abstract:
Safety-critical robot systems need thorough testing to expose design flaws and software bugs which could endanger humans. Testing in simulation is becoming increasingly popular, as it can be applied early in the development process and does not endanger any real-world operators. However, not all safety-critical flaws become immediately observable in simulation. Some may only become observable unde…
▽ More
Safety-critical robot systems need thorough testing to expose design flaws and software bugs which could endanger humans. Testing in simulation is becoming increasingly popular, as it can be applied early in the development process and does not endanger any real-world operators. However, not all safety-critical flaws become immediately observable in simulation. Some may only become observable under certain critical conditions. If these conditions are not covered, safety flaws may remain undetected. Creating critical tests is therefore crucial. In recent years, there has been a trend towards using Reinforcement Learning (RL) for this purpose. Guided by domain-specific reward functions, RL algorithms are used to learn critical test strategies. This paper presents a case study in which the collision avoidance behavior of a mobile robot is subjected to RL-based testing. The study confirms prior research which shows that RL can be an effective testing tool. However, the study also highlights certain challenges associated with RL-based testing, namely (i) a possible lack of diversity in test conditions and (ii) the phenomenon of reward hacking where the RL agent behaves in undesired ways due to a misalignment of reward and test specification. The challenges are illustrated with data and examples from the experiments, and possible mitigation strategies are discussed.
△ Less
Submitted 6 November, 2023;
originally announced November 2023.
-
Hardware-Algorithm Co-design Enabling Processing-in-Pixel-in-Memory (P2M) for Neuromorphic Vision Sensors
Authors:
Md Abdullah-Al Kaiser,
Akhilesh R. Jaiswal
Abstract:
The high volume of data transmission between the edge sensor and the cloud processor leads to energy and throughput bottlenecks for resource-constrained edge devices focused on computer vision. Hence, researchers are investigating different approaches (e.g., near-sensor processing, in-sensor processing, in-pixel processing) by executing computations closer to the sensor to reduce the transmission…
▽ More
The high volume of data transmission between the edge sensor and the cloud processor leads to energy and throughput bottlenecks for resource-constrained edge devices focused on computer vision. Hence, researchers are investigating different approaches (e.g., near-sensor processing, in-sensor processing, in-pixel processing) by executing computations closer to the sensor to reduce the transmission bandwidth. Specifically, in-pixel processing for neuromorphic vision sensors (e.g., dynamic vision sensors (DVS)) involves incorporating asynchronous multiply-accumulate (MAC) operations within the pixel array, resulting in improved energy efficiency. In a CMOS implementation, low overhead energy-efficient analog MAC accumulates charges on a passive capacitor; however, the capacitor's limited charge retention time affects the algorithmic integration time choices, impacting the algorithmic accuracy, bandwidth, energy, and training efficiency. Consequently, this results in a design trade-off on the hardware aspect-creating a need for a low-leakage compute unit while maintaining the area and energy benefits. In this work, we present a holistic analysis of the hardware-algorithm co-design trade-off based on the limited integration time posed by the hardware and techniques to improve the leakage performance of the in-pixel analog MAC operations.
△ Less
Submitted 7 October, 2023;
originally announced October 2023.
-
Robust Training for Conversational Question Answering Models with Reinforced Reformulation Generation
Authors:
Magdalena Kaiser,
Rishiraj Saha Roy,
Gerhard Weikum
Abstract:
Models for conversational question answering (ConvQA) over knowledge graphs (KGs) are usually trained and tested on benchmarks of gold QA pairs. This implies that training is limited to surface forms seen in the respective datasets, and evaluation is on a small set of held-out questions. Through our proposed framework REIGN, we take several steps to remedy this restricted learning setup. First, we…
▽ More
Models for conversational question answering (ConvQA) over knowledge graphs (KGs) are usually trained and tested on benchmarks of gold QA pairs. This implies that training is limited to surface forms seen in the respective datasets, and evaluation is on a small set of held-out questions. Through our proposed framework REIGN, we take several steps to remedy this restricted learning setup. First, we systematically generate reformulations of training questions to increase robustness of models to surface form variations. This is a particularly challenging problem, given the incomplete nature of such questions. Second, we guide ConvQA models towards higher performance by feeding it only those reformulations that help improve their answering quality, using deep reinforcement learning. Third, we demonstrate the viability of training major model components on one benchmark and applying them zero-shot to another. Finally, for a rigorous evaluation of robustness for trained models, we use and release large numbers of diverse reformulations generated by prompting GPT for benchmark test sets (resulting in 20x increase in sizes). Our findings show that ConvQA models with robust training via reformulations, significantly outperform those with standard training from gold QA pairs only.
△ Less
Submitted 16 February, 2024; v1 submitted 20 October, 2023;
originally announced October 2023.
-
Multi-fidelity experimental design for ice-sheet simulation
Authors:
Pierre Thodoroff,
Markus Kaiser,
Rosie Williams,
Robert Arthern,
Scott Hosking,
Neil Lawrence,
James Byrne,
Ieva Kazlauskaite
Abstract:
Computer simulations are becoming an essential tool in many scientific fields from molecular dynamics to aeronautics. In glaciology, future predictions of sea level change require input from ice sheet models. Due to uncertainties in the forcings and the parameter choices for such models, many different realisations of the model are needed in order to produce probabilistic forecasts of sea level ch…
▽ More
Computer simulations are becoming an essential tool in many scientific fields from molecular dynamics to aeronautics. In glaciology, future predictions of sea level change require input from ice sheet models. Due to uncertainties in the forcings and the parameter choices for such models, many different realisations of the model are needed in order to produce probabilistic forecasts of sea level change. For these reasons, producing robust probabilistic forecasts from an ensemble of model simulations over regions of interest can be extremely expensive for many ice sheet models. Multi-fidelity experimental design (MFED) is a strategy that models the high-fidelity output of the simulator by combining information from various resolutions in an attempt to minimize the computational costs of the process and maximize the accuracy of the posterior. In this paper, we present an application of MFED to an ice-sheet simulatorand demonstrate potential computational savings by modelling the relationship between spatial resolutions. We also analyze the behavior of MFED strategies using theoretical results from sub-modular maximization.
△ Less
Submitted 17 July, 2023;
originally announced July 2023.
-
Beyond Rankings: Exploring the Impact of SERP Features on Organic Click-through Rates
Authors:
Erik Fubel,
Niclas Michael Groll,
Patrick Gundlach,
Qiwei Han,
Maximilian Kaiser
Abstract:
Search Engine Result Pages (SERPs) serve as the digital gateways to the vast expanse of the internet. Past decades have witnessed a surge in research primarily centered on the influence of website ranking on these pages, to determine the click-through rate (CTR). However, during this period, the landscape of SERPs has undergone a dramatic evolution: SERP features, encompassing elements such as kno…
▽ More
Search Engine Result Pages (SERPs) serve as the digital gateways to the vast expanse of the internet. Past decades have witnessed a surge in research primarily centered on the influence of website ranking on these pages, to determine the click-through rate (CTR). However, during this period, the landscape of SERPs has undergone a dramatic evolution: SERP features, encompassing elements such as knowledge panels, media galleries, FAQs, and more, have emerged as an increasingly prominent facet of these result pages. Our study examines the crucial role of these features, revealing them to be not merely aesthetic components, but strongly influence CTR and the associated behavior of internet users. We demonstrate how these features can significantly modulate web traffic, either amplifying or attenuating it. We dissect these intricate interaction effects leveraging a unique dataset of 67,000 keywords and their respective Google SERPs, spanning over 40 distinct US-based e-commerce domains, generating over 6 million clicks from 24 million views. This cross-website dataset, unprecedented in its scope, enables us to assess the impact of 24 different SERP features on organic CTR. Through an ablation study modeling CTR, we illustrate the incremental predictive power these features hold.
△ Less
Submitted 31 May, 2023;
originally announced June 2023.
-
On the Simultaneous Solution of Structural Membranes on all Level Sets within a Bulk Domain
Authors:
Thomas-Peter Fries,
Michael W. Kaiser
Abstract:
A mechanical model and numerical method for structural membranes implied by all isosurfaces of a level-set function in a three-dimensional bulk domain are proposed. The mechanical model covers large displacements in the context of the finite strain theory and is formulated based on the tangential differential calculus. Alongside curved two-dimensional membranes embedded in three dimensions, also t…
▽ More
A mechanical model and numerical method for structural membranes implied by all isosurfaces of a level-set function in a three-dimensional bulk domain are proposed. The mechanical model covers large displacements in the context of the finite strain theory and is formulated based on the tangential differential calculus. Alongside curved two-dimensional membranes embedded in three dimensions, also the simpler case of curved ropes (cables) in two-dimensional bulk domains is covered. The implicit geometries (shapes) are implied by the level sets and the boundaries of the structures are given by the intersection of the level sets with the boundary of the bulk domain. For the numerical analysis, the bulk domain is discretized using a background mesh composed by (higher-order) elements with the dimensionality of the embedding space. The elements are by no means aligned to the level sets, i.e., the geometries of the structures, which resembles a fictitious domain method, most importantly the Trace FEM. The proposed numerical method is a hybrid of the classical FEM and fictitious domain methods which may be labeled as "Bulk Trace FEM". Numerical studies confirm higher-order convergence rates and the potential for new material models with continuously embedded sub-structures in bulk domains.
△ Less
Submitted 31 May, 2023;
originally announced May 2023.
-
VEDLIoT -- Next generation accelerated AIoT systems and applications
Authors:
Kevin Mika,
René Griessl,
Nils Kucza,
Florian Porrmann,
Martin Kaiser,
Lennart Tigges,
Jens Hagemeyer,
Pedro Trancoso,
Muhammad Waqar Azhar,
Fareed Qararyah,
Stavroula Zouzoula,
Jämes Ménétrey,
Marcelo Pasin,
Pascal Felber,
Carina Marcus,
Oliver Brunnegard,
Olof Eriksson,
Hans Salomonsson,
Daniel Ödman,
Andreas Ask,
Antonio Casimiro,
Alysson Bessani,
Tiago Carvalho,
Karol Gugala,
Piotr Zierhoffer
, et al. (7 additional authors not shown)
Abstract:
The VEDLIoT project aims to develop energy-efficient Deep Learning methodologies for distributed Artificial Intelligence of Things (AIoT) applications. During our project, we propose a holistic approach that focuses on optimizing algorithms while addressing safety and security challenges inherent to AIoT systems. The foundation of this approach lies in a modular and scalable cognitive IoT hardware…
▽ More
The VEDLIoT project aims to develop energy-efficient Deep Learning methodologies for distributed Artificial Intelligence of Things (AIoT) applications. During our project, we propose a holistic approach that focuses on optimizing algorithms while addressing safety and security challenges inherent to AIoT systems. The foundation of this approach lies in a modular and scalable cognitive IoT hardware platform, which leverages microserver technology to enable users to configure the hardware to meet the requirements of a diverse array of applications. Heterogeneous computing is used to boost performance and energy efficiency. In addition, the full spectrum of hardware accelerators is integrated, providing specialized ASICs as well as FPGAs for reconfigurable computing. The project's contributions span across trusted computing, remote attestation, and secure execution environments, with the ultimate goal of facilitating the design and deployment of robust and efficient AIoT systems. The overall architecture is validated on use-cases ranging from Smart Home to Automotive and Industrial IoT appliances. Ten additional use cases are integrated via an open call, broadening the range of application areas.
△ Less
Submitted 9 May, 2023;
originally announced May 2023.
-
Technology-Circuit-Algorithm Tri-Design for Processing-in-Pixel-in-Memory (P2M)
Authors:
Md Abdullah-Al Kaiser,
Gourav Datta,
Sreetama Sarkar,
Souvik Kundu,
Zihan Yin,
Manas Garg,
Ajey P. Jacob,
Peter A. Beerel,
Akhilesh R. Jaiswal
Abstract:
The massive amounts of data generated by camera sensors motivate data processing inside pixel arrays, i.e., at the extreme-edge. Several critical developments have fueled recent interest in the processing-in-pixel-in-memory paradigm for a wide range of visual machine intelligence tasks, including (1) advances in 3D integration technology to enable complex processing inside each pixel in a 3D integ…
▽ More
The massive amounts of data generated by camera sensors motivate data processing inside pixel arrays, i.e., at the extreme-edge. Several critical developments have fueled recent interest in the processing-in-pixel-in-memory paradigm for a wide range of visual machine intelligence tasks, including (1) advances in 3D integration technology to enable complex processing inside each pixel in a 3D integrated manner while maintaining pixel density, (2) analog processing circuit techniques for massively parallel low-energy in-pixel computations, and (3) algorithmic techniques to mitigate non-idealities associated with analog processing through hardware-aware training schemes. This article presents a comprehensive technology-circuit-algorithm landscape that connects technology capabilities, circuit design strategies, and algorithmic optimizations to power, performance, area, bandwidth reduction, and application-level accuracy metrics. We present our results using a comprehensive co-design framework incorporating hardware and algorithmic optimizations for various complex real-life visual intelligence tasks mapped onto our P2M paradigm.
△ Less
Submitted 6 April, 2023;
originally announced April 2023.
-
A Context-Switching/Dual-Context ROM Augmented RAM using Standard 8T SRAM
Authors:
Md Abdullah-Al Kaiser,
Edwin Tieu,
Ajey P. Jacob,
Akhilesh R. Jaiswal
Abstract:
The landscape of emerging applications has been continually widening, encompassing various data-intensive applications like artificial intelligence, machine learning, secure encryption, Internet-of-Things, etc. A sustainable approach toward creating dedicated hardware platforms that can cater to multiple applications often requires the underlying hardware to context-switch or support more than one…
▽ More
The landscape of emerging applications has been continually widening, encompassing various data-intensive applications like artificial intelligence, machine learning, secure encryption, Internet-of-Things, etc. A sustainable approach toward creating dedicated hardware platforms that can cater to multiple applications often requires the underlying hardware to context-switch or support more than one context simultaneously. This paper presents a context-switching and dual-context memory based on the standard 8T SRAM bit-cell. Specifically, we exploit the availability of multi-VT transistors by selectively choosing the read-port transistors of the 8T SRAM cell to be either high-VT or low-VT. The 8T SRAM cell is thus augmented to store ROM data (represented as the VT of the transistors constituting the read-port) while simultaneously storing RAM data. Further, we propose specific sensing methodologies such that the memory array can support RAM-only or ROM-only mode (context-switching (CS) mode) or RAM and ROM mode simultaneously (dual-context (DC) mode). Extensive Monte-Carlo simulations have verified the robustness of our proposed ROM-augmented CS/DC memory on the Globalfoundries 22nm-FDX technology node.
△ Less
Submitted 6 April, 2023;
originally announced April 2023.
-
Object Motion Sensitivity: A Bio-inspired Solution to the Ego-motion Problem for Event-based Cameras
Authors:
Shay Snyder,
Hunter Thompson,
Md Abdullah-Al Kaiser,
Gregory Schwartz,
Akhilesh Jaiswal,
Maryam Parsa
Abstract:
Neuromorphic (event-based) image sensors draw inspiration from the human-retina to create an electronic device that can process visual stimuli in a way that closely resembles its biological counterpart. These sensors process information significantly different than the traditional RGB sensors. Specifically, the sensory information generated by event-based image sensors are orders of magnitude spar…
▽ More
Neuromorphic (event-based) image sensors draw inspiration from the human-retina to create an electronic device that can process visual stimuli in a way that closely resembles its biological counterpart. These sensors process information significantly different than the traditional RGB sensors. Specifically, the sensory information generated by event-based image sensors are orders of magnitude sparser compared to that of RGB sensors. The first generation of neuromorphic image sensors, Dynamic Vision Sensor (DVS), are inspired by the computations confined to the photoreceptors and the first retinal synapse. In this work, we highlight the capability of the second generation of neuromorphic image sensors, Integrated Retinal Functionality in CMOS Image Sensors (IRIS), which aims to mimic full retinal computations from photoreceptors to output of the retina (retinal ganglion cells) for targeted feature-extraction. The feature of choice in this work is Object Motion Sensitivity (OMS) that is processed locally in the IRIS sensor. Our results show that OMS can accomplish standard computer vision tasks with similar efficiency to conventional RGB and DVS solutions but offers drastic bandwidth reduction. This cuts the wireless and computing power budgets and opens up vast opportunities in high-speed, robust, energy-efficient, and low-bandwidth real-time decision making.
△ Less
Submitted 14 April, 2023; v1 submitted 24 March, 2023;
originally announced March 2023.
-
Interpretable Deep Learning for Forecasting Online Advertising Costs: Insights from the Competitive Bidding Landscape
Authors:
Fynn Oldenburg,
Qiwei Han,
Maximilian Kaiser
Abstract:
As advertisers increasingly shift their budgets toward digital advertising, accurately forecasting advertising costs becomes essential for optimizing marketing campaign returns. This paper presents a comprehensive study that employs various time-series forecasting methods to predict daily average CPC in the online advertising market. We evaluate the performance of statistical models, machine learn…
▽ More
As advertisers increasingly shift their budgets toward digital advertising, accurately forecasting advertising costs becomes essential for optimizing marketing campaign returns. This paper presents a comprehensive study that employs various time-series forecasting methods to predict daily average CPC in the online advertising market. We evaluate the performance of statistical models, machine learning techniques, and deep learning approaches, including the Temporal Fusion Transformer (TFT). Our findings reveal that incorporating multivariate models, enriched with covariates derived from competitors' CPC patterns through time-series clustering, significantly improves forecasting accuracy. We interpret the results by analyzing feature importance and temporal attention, demonstrating how the models leverage both the advertiser's data and insights from the competitive landscape. Additionally, our method proves robust during major market shifts, such as the COVID-19 pandemic, consistently outperforming models that rely solely on individual advertisers' data. This study introduces a scalable technique for selecting relevant covariates from a broad pool of advertisers, offering more accurate long-term forecasts and strategic insights into budget allocation and competitive dynamics in digital advertising.
△ Less
Submitted 21 August, 2024; v1 submitted 11 February, 2023;
originally announced February 2023.
-
Neuromorphic-P2M: Processing-in-Pixel-in-Memory Paradigm for Neuromorphic Image Sensors
Authors:
Md Abdullah-Al Kaiser,
Gourav Datta,
Zixu Wang,
Ajey P. Jacob,
Peter A. Beerel,
Akhilesh R. Jaiswal
Abstract:
Edge devices equipped with computer vision must deal with vast amounts of sensory data with limited computing resources. Hence, researchers have been exploring different energy-efficient solutions such as near-sensor processing, in-sensor processing, and in-pixel processing, bringing the computation closer to the sensor. In particular, in-pixel processing embeds the computation capabilities inside…
▽ More
Edge devices equipped with computer vision must deal with vast amounts of sensory data with limited computing resources. Hence, researchers have been exploring different energy-efficient solutions such as near-sensor processing, in-sensor processing, and in-pixel processing, bringing the computation closer to the sensor. In particular, in-pixel processing embeds the computation capabilities inside the pixel array and achieves high energy efficiency by generating low-level features instead of the raw data stream from CMOS image sensors. Many different in-pixel processing techniques and approaches have been demonstrated on conventional frame-based CMOS imagers, however, the processing-in-pixel approach for neuromorphic vision sensors has not been explored so far. In this work, we for the first time, propose an asynchronous non-von-Neumann analog processing-in-pixel paradigm to perform convolution operations by integrating in-situ multi-bit multi-channel convolution inside the pixel array performing analog multiply and accumulate (MAC) operations that consume significantly less energy than their digital MAC alternative. To make this approach viable, we incorporate the circuit's non-ideality, leakage, and process variations into a novel hardware-algorithm co-design framework that leverages extensive HSpice simulations of our proposed circuit using the GF22nm FD-SOI technology node. We verified our framework on state-of-the-art neuromorphic vision sensor datasets and show that our solution consumes ~2x lower backend-processor energy while maintaining almost similar front-end (sensor) energy on the IBM DVS128-Gesture dataset than the state-of-the-art while maintaining a high test accuracy of 88.36%.
△ Less
Submitted 22 January, 2023;
originally announced January 2023.
-
In-Sensor & Neuromorphic Computing are all you need for Energy Efficient Computer Vision
Authors:
Gourav Datta,
Zeyu Liu,
Md Abdullah-Al Kaiser,
Souvik Kundu,
Joe Mathai,
Zihan Yin,
Ajey P. Jacob,
Akhilesh R. Jaiswal,
Peter A. Beerel
Abstract:
Due to the high activation sparsity and use of accumulates (AC) instead of expensive multiply-and-accumulates (MAC), neuromorphic spiking neural networks (SNNs) have emerged as a promising low-power alternative to traditional DNNs for several computer vision (CV) applications. However, most existing SNNs require multiple time steps for acceptable inference accuracy, hindering real-time deployment…
▽ More
Due to the high activation sparsity and use of accumulates (AC) instead of expensive multiply-and-accumulates (MAC), neuromorphic spiking neural networks (SNNs) have emerged as a promising low-power alternative to traditional DNNs for several computer vision (CV) applications. However, most existing SNNs require multiple time steps for acceptable inference accuracy, hindering real-time deployment and increasing spiking activity and, consequently, energy consumption. Recent works proposed direct encoding that directly feeds the analog pixel values in the first layer of the SNN in order to significantly reduce the number of time steps. Although the overhead for the first layer MACs with direct encoding is negligible for deep SNNs and the CV processing is efficient using SNNs, the data transfer between the image sensors and the downstream processing costs significant bandwidth and may dominate the total energy. To mitigate this concern, we propose an in-sensor computing hardware-software co-design framework for SNNs targeting image recognition tasks. Our approach reduces the bandwidth between sensing and processing by 12-96x and the resulting total energy by 2.32x compared to traditional CV processing, with a 3.8% reduction in accuracy on ImageNet.
△ Less
Submitted 21 December, 2022;
originally announced December 2022.
-
A locally time-invariant metric for climate model ensemble predictions of extreme risk
Authors:
Mala Virdee,
Markus Kaiser,
Emily Shuckburgh,
Carl Henrik Ek,
Ieva Kazlauskaite
Abstract:
Adaptation-relevant predictions of climate change are often derived by combining climate model simulations in a multi-model ensemble. Model evaluation methods used in performance-based ensemble weighting schemes have limitations in the context of high-impact extreme events. We introduce a locally time-invariant method for evaluating climate model simulations with a focus on assessing the simulatio…
▽ More
Adaptation-relevant predictions of climate change are often derived by combining climate model simulations in a multi-model ensemble. Model evaluation methods used in performance-based ensemble weighting schemes have limitations in the context of high-impact extreme events. We introduce a locally time-invariant method for evaluating climate model simulations with a focus on assessing the simulation of extremes. We explore the behaviour of the proposed method in predicting extreme heat days in Nairobi and provide comparative results for eight additional cities.
△ Less
Submitted 18 April, 2023; v1 submitted 26 November, 2022;
originally announced November 2022.
-
Ice Core Dating using Probabilistic Programming
Authors:
Aditya Ravuri,
Tom R. Andersson,
Ieva Kazlauskaite,
Will Tebbutt,
Richard E. Turner,
J. Scott Hosking,
Neil D. Lawrence,
Markus Kaiser
Abstract:
Ice cores record crucial information about past climate. However, before ice core data can have scientific value, the chronology must be inferred by estimating the age as a function of depth. Under certain conditions, chemicals locked in the ice display quasi-periodic cycles that delineate annual layers. Manually counting these noisy seasonal patterns to infer the chronology can be an imperfect an…
▽ More
Ice cores record crucial information about past climate. However, before ice core data can have scientific value, the chronology must be inferred by estimating the age as a function of depth. Under certain conditions, chemicals locked in the ice display quasi-periodic cycles that delineate annual layers. Manually counting these noisy seasonal patterns to infer the chronology can be an imperfect and time-consuming process, and does not capture uncertainty in a principled fashion. In addition, several ice cores may be collected from a region, introducing an aspect of spatial correlation between them. We present an exploration of the use of probabilistic models for automatic dating of ice cores, using probabilistic programming to showcase its use for prototyping, automatic inference and maintainability, and demonstrate common failure modes of these tools.
△ Less
Submitted 29 October, 2022;
originally announced October 2022.
-
IRIS: Integrated Retinal Functionality in Image Sensors
Authors:
Zihan Yin,
Md Abdullah-Al Kaiser,
Lamine Ousmane Camara,
Mark Camarena,
Maryam Parsa,
Ajey Jacob,
Gregory Schwartz,
Akhilesh Jaiswal
Abstract:
Neuromorphic image sensors draw inspiration from the biological retina to implement visual computations in electronic hardware. Gain control in phototransduction and temporal differentiation at the first retinal synapse inspired the first generation of neuromorphic sensors, but processing in downstream retinal circuits, much of which has been discovered in the past decade, has not been implemented…
▽ More
Neuromorphic image sensors draw inspiration from the biological retina to implement visual computations in electronic hardware. Gain control in phototransduction and temporal differentiation at the first retinal synapse inspired the first generation of neuromorphic sensors, but processing in downstream retinal circuits, much of which has been discovered in the past decade, has not been implemented in image sensor technology. We present a technology-circuit co-design solution that implements two motion computations occurring at the output of the retina that could have wide applications for vision based decision making in dynamic environments. Our simulations on Globalfoundries 22nm technology node show that, by taking advantage of the recent advances in semiconductor chip stacking technology, the proposed retina-inspired circuits can be fabricated on image sensing platforms in existing semiconductor foundries. Integrated Retinal Functionality in Image Sensors (IRIS) technology could drive advances in machine vision applications that demand robust, high-speed, energy-efficient and low-bandwidth real-time decision making.
△ Less
Submitted 14 August, 2022;
originally announced August 2022.
-
VEDLIoT: Very Efficient Deep Learning in IoT
Authors:
Martin Kaiser,
Rene Griessl,
Nils Kucza,
Carola Haumann,
Lennart Tigges,
Kevin Mika,
Jens Hagemeyer,
Florian Porrmann,
Ulrich Rückert,
Micha vor dem Berge,
Stefan. Krupop,
Mario Porrmann,
Marco Tassemeier,
Pedro Trancoso,
Fareed Quararyah,
Stavroula Zouzoula,
Antonio Casimiro,
Alysson Bessani,
Jose Cecilio,
Stefan Andersson,
Oliver Brunnegard,
Olof Eriksson,
Roland Weiss,
Franz Meierhöfer,
Hans Salomonsson
, et al. (11 additional authors not shown)
Abstract:
The VEDLIoT project targets the development of energy-efficient Deep Learning for distributed AIoT applications. A holistic approach is used to optimize algorithms while also dealing with safety and security challenges. The approach is based on a modular and scalable cognitive IoT hardware platform. Using modular microserver technology enables the user to configure the hardware to satisfy a wide r…
▽ More
The VEDLIoT project targets the development of energy-efficient Deep Learning for distributed AIoT applications. A holistic approach is used to optimize algorithms while also dealing with safety and security challenges. The approach is based on a modular and scalable cognitive IoT hardware platform. Using modular microserver technology enables the user to configure the hardware to satisfy a wide range of applications. VEDLIoT offers a complete design flow for Next-Generation IoT devices required for collaboratively solving complex Deep Learning applications across distributed systems. The methods are tested on various use-cases ranging from Smart Home to Automotive and Industrial IoT appliances. VEDLIoT is an H2020 EU project which started in November 2020. It is currently in an intermediate stage with the first results available.
△ Less
Submitted 1 July, 2022;
originally announced July 2022.
-
Your Face Mirrors Your Deepest Beliefs-Predicting Personality and Morals through Facial Emotion Recognition
Authors:
P. A. Gloor,
A. Fronzetti Colladon,
E. Altuntas,
C. Cetinkaya,
M. F. Kaiser,
L. Ripperger,
T. Schaefer
Abstract:
Can we really "read the mind in the eyes"? Moreover, can AI assist us in this task? This paper answers these two questions by introducing a machine learning system that predicts personality characteristics of individuals on the basis of their face. It does so by tracking the emotional response of the individual's face through facial emotion recognition (FER) while watching a series of 15 short vid…
▽ More
Can we really "read the mind in the eyes"? Moreover, can AI assist us in this task? This paper answers these two questions by introducing a machine learning system that predicts personality characteristics of individuals on the basis of their face. It does so by tracking the emotional response of the individual's face through facial emotion recognition (FER) while watching a series of 15 short videos of different genres. To calibrate the system, we invited 85 people to watch the videos, while their emotional responses were analyzed through their facial expression. At the same time, these individuals also took four well-validated surveys of personality characteristics and moral values: the revised NEO FFI personality inventory, the Haidt moral foundations test, the Schwartz personal value system, and the domain-specific risk-taking scale (DOSPERT). We found that personality characteristics and moral values of an individual can be predicted through their emotional response to the videos as shown in their face, with an accuracy of up to 86% using gradient-boosted trees. We also found that different personality characteristics are better predicted by different videos, in other words, there is no single video that will provide accurate predictions for all personality characteristics, but it is the response to the mix of different videos that allows for accurate prediction.
△ Less
Submitted 23 December, 2021;
originally announced December 2021.
-
Online Advertising Revenue Forecasting: An Interpretable Deep Learning Approach
Authors:
Max Würfel,
Qiwei Han,
Maximilian Kaiser
Abstract:
Online advertising revenues account for an increasing share of publishers' revenue streams, especially for small and medium-sized publishers who depend on the advertisement networks of tech companies such as Google and Facebook. Thus publishers may benefit significantly from accurate online advertising revenue forecasts to better manage their website monetization strategies. However, publishers wh…
▽ More
Online advertising revenues account for an increasing share of publishers' revenue streams, especially for small and medium-sized publishers who depend on the advertisement networks of tech companies such as Google and Facebook. Thus publishers may benefit significantly from accurate online advertising revenue forecasts to better manage their website monetization strategies. However, publishers who only have access to their own revenue data lack a holistic view of the total ad market of publishers, which in turn limits their ability to generate insights into their own future online advertising revenues. To address this business issue, we leverage a proprietary database encompassing Google Adsense revenues from a large collection of publishers in diverse areas. We adopt the Temporal Fusion Transformer (TFT) model, a novel attention-based architecture to predict publishers' advertising revenues. We leverage multiple covariates, including not only the publisher's own characteristics but also other publishers' advertising revenues. Our prediction results outperform several benchmark deep-learning time-series forecast models over multiple time horizons. Moreover, we interpret the results by analyzing variable importance weights to identify significant features and self-attention weights to reveal persistent temporal patterns.
△ Less
Submitted 16 November, 2021;
originally announced November 2021.
-
Simulation of the behavior of the Leopardus guigna using random walkers
Authors:
A. Torres-Hernandez,
Byron C. Guzmán,
Melanie Kaiser,
Julio C. Hernández
Abstract:
Considering that there is very little information on the behavior habits of guigna cats, as well as investigations in which small populations are captured to place radiocollars on them and then release them in the place where they were captured, which is done with the intention of collecting data on their positions in a territory and thus make estimates of the mean distances they usually travel. U…
▽ More
Considering that there is very little information on the behavior habits of guigna cats, as well as investigations in which small populations are captured to place radiocollars on them and then release them in the place where they were captured, which is done with the intention of collecting data on their positions in a territory and thus make estimates of the mean distances they usually travel. Under the hypothesis that guignas maintain a sedentary behavior in a specific area of a given territory, this paper shows one way to simulate a distribution of points in a territory using random walkers to emulate the distribution of the data that would be obtained by placing radiocollars in a population of guignas, with which it is possible to make estimates of the mean distances that move away from a certain fixed position, and the interactions they can have with points in the territory that represent a high probability of lethality, such as farms, packs of dogs, roads, urban areas, etc. It is necessary to mention that by estimating the possible interactions that a guignas population may have with possible predators in a territory with the help of a satellite image, it is possible to evaluate the points of a territory that represent a potentially lethal risk for the guignas, and thus generate relocation strategies that help preserve them.
△ Less
Submitted 5 November, 2021;
originally announced November 2021.
-
Protecting the Edge: Ultrafast Laser Modified C-shaped Glass Edges
Authors:
Daniel Flamm,
Myriam Kaiser,
Marvin Feil,
Max Kahmann,
Michael Lang,
Jonas Kleiner,
Tim Hesse
Abstract:
A procedure and optical concept is introduced for ultrashort pulsed laser cleaving of transparent materials with tailored edges in a single pass. The procedure is based on holographically splitting a number of foci along the desired edge geometry including C-shaped edges with local 45° tangential angles to the surface. Single-pass, full thickness laser modifications are achieved requiring single-s…
▽ More
A procedure and optical concept is introduced for ultrashort pulsed laser cleaving of transparent materials with tailored edges in a single pass. The procedure is based on holographically splitting a number of foci along the desired edge geometry including C-shaped edges with local 45° tangential angles to the surface. Single-pass, full thickness laser modifications are achieved requiring single-side access to the workpiece only without inclining the optical head. After having induced laser modifications with feed rates of 1 m/s actual separation is performed using a selective etching strategy.
△ Less
Submitted 28 December, 2021; v1 submitted 2 November, 2021;
originally announced November 2021.
-
Nonparametric Cointegrating Regression Functions with Endogeneity and Semi-Long Memory
Authors:
Sepideh Mosaferi,
Mark S. Kaiser
Abstract:
This article develops nonparametric cointegrating regression models with endogeneity and semi-long memory. We assume semi-long memory is produced in the regressor process by tempering of random shock coefficients. The fundamental properties of long memory processes are thus retained in the regressor process. Nonparametric nonlinear cointegrating regressions with serially dependent errors and endog…
▽ More
This article develops nonparametric cointegrating regression models with endogeneity and semi-long memory. We assume semi-long memory is produced in the regressor process by tempering of random shock coefficients. The fundamental properties of long memory processes are thus retained in the regressor process. Nonparametric nonlinear cointegrating regressions with serially dependent errors and endogenous regressors that are driven by long memory innovations have been considered in Wang and Phillips (2016). That work also implemented a statistical specification test for testing whether the regression function follows a parametric form. The convergence rate of the proposed test is parameter dependent, and its limit theory involves the local time of fractional Brownian motion. The present paper modifies the test statistic proposed for the long memory case by Wang and Phillips (2016) to be suitable for the semi-long memory case. With this modification, the limit theory for the test involves the local time of standard Brownian motion. Through simulation studies, we investigate properties of nonparametric regression function estimation with semi-long memory regressors as well as long memory regressors.
△ Less
Submitted 26 August, 2022; v1 submitted 1 November, 2021;
originally announced November 2021.
-
Multiple Instance Learning with Auxiliary Task Weighting for Multiple Myeloma Classification
Authors:
Talha Qaiser,
Stefan Winzeck,
Theodore Barfoot,
Tara Barwick,
Simon J. Doran,
Martin F. Kaiser,
Linda Wedlake,
Nina Tunariu,
Dow-Mu Koh,
Christina Messiou,
Andrea Rockall,
Ben Glocker
Abstract:
Whole body magnetic resonance imaging (WB-MRI) is the recommended modality for diagnosis of multiple myeloma (MM). WB-MRI is used to detect sites of disease across the entire skeletal system, but it requires significant expertise and is time-consuming to report due to the great number of images. To aid radiological reading, we propose an auxiliary task-based multiple instance learning approach (AT…
▽ More
Whole body magnetic resonance imaging (WB-MRI) is the recommended modality for diagnosis of multiple myeloma (MM). WB-MRI is used to detect sites of disease across the entire skeletal system, but it requires significant expertise and is time-consuming to report due to the great number of images. To aid radiological reading, we propose an auxiliary task-based multiple instance learning approach (ATMIL) for MM classification with the ability to localize sites of disease. This approach is appealing as it only requires patient-level annotations where an attention mechanism is used to identify local regions with active disease. We borrow ideas from multi-task learning and define an auxiliary task with adaptive reweighting to support and improve learning efficiency in the presence of data scarcity. We validate our approach on both synthetic and real multi-center clinical data. We show that the MIL attention module provides a mechanism to localize bone regions while the adaptive reweighting of the auxiliary task considerably improves the performance.
△ Less
Submitted 16 July, 2021;
originally announced July 2021.
-
Glass tube cutting with aberration-corrected non-diffracting ultrashort laser pulses
Authors:
Henning Rave,
Henning Heiming,
Patrick Szumny,
Myriam Kaiser,
Jonas Kleiner,
Daniel Flamm
Abstract:
The separation of complex inner and outer contours of glass articles with curved surfaces using ultrashort pulsed lasers is reported. Single-pass, full-thickness modifications along the entire substrate are achieved using a processing optics that allows for beam shaping of non-diffracting beams and, additionally, for aberration compensation of phase distortions occurring at the curved interface. T…
▽ More
The separation of complex inner and outer contours of glass articles with curved surfaces using ultrashort pulsed lasers is reported. Single-pass, full-thickness modifications along the entire substrate are achieved using a processing optics that allows for beam shaping of non-diffracting beams and, additionally, for aberration compensation of phase distortions occurring at the curved interface. The glass articles finally separated by thermal stress or via selective etching meet the demands of the medical industry in terms of micro-debris, surface quality and processing speed.
△ Less
Submitted 21 June, 2021; v1 submitted 12 May, 2021;
originally announced May 2021.
-
Cavity driven Rabi oscillations between Rydberg states of atoms trapped on a superconducting atom chip
Authors:
Manuel Kaiser,
Conny Glaser,
Li Yuan Ley,
Jens Grimmel,
Helge Hattermann,
Daniel Bothner,
Dieter Koelle,
Reinhold Kleiner,
David Petrosyan,
Andreas Günther,
József Fortágh
Abstract:
Hybrid quantum systems involving cold atoms and microwave resonators can enable cavity-mediated infinite-range interactions between atomic spin systems and realize atomic quantum memories and transducers for microwave to optical conversion. To achieve strong coupling of atoms to on-chip microwave resonators, it was suggested to use atomic Rydberg states with strong electric dipole transitions. Her…
▽ More
Hybrid quantum systems involving cold atoms and microwave resonators can enable cavity-mediated infinite-range interactions between atomic spin systems and realize atomic quantum memories and transducers for microwave to optical conversion. To achieve strong coupling of atoms to on-chip microwave resonators, it was suggested to use atomic Rydberg states with strong electric dipole transitions. Here we report on the realization of coherent coupling of a Rydberg transition of ultracold atoms trapped on an integrated superconducting atom chip to the microwave field of an on-chip coplanar waveguide resonator. We observe and characterize the cavity driven Rabi oscillations between a pair of Rydberg states of atoms in an inhomogeneous electric field near the chip surface. Our studies demonstrate the feasibility, but also reveal the challenges, of coherent state manipulation of Rydberg atoms interacting with superconducting circuits.
△ Less
Submitted 11 May, 2021;
originally announced May 2021.
-
Reinforcement Learning from Reformulations in Conversational Question Answering over Knowledge Graphs
Authors:
Magdalena Kaiser,
Rishiraj Saha Roy,
Gerhard Weikum
Abstract:
The rise of personal assistants has made conversational question answering (ConvQA) a very popular mechanism for user-system interaction. State-of-the-art methods for ConvQA over knowledge graphs (KGs) can only learn from crisp question-answer pairs found in popular benchmarks. In reality, however, such training data is hard to come by: users would rarely mark answers explicitly as correct or wron…
▽ More
The rise of personal assistants has made conversational question answering (ConvQA) a very popular mechanism for user-system interaction. State-of-the-art methods for ConvQA over knowledge graphs (KGs) can only learn from crisp question-answer pairs found in popular benchmarks. In reality, however, such training data is hard to come by: users would rarely mark answers explicitly as correct or wrong. In this work, we take a step towards a more natural learning paradigm - from noisy and implicit feedback via question reformulations. A reformulation is likely to be triggered by an incorrect system response, whereas a new follow-up question could be a positive signal on the previous turn's answer. We present a reinforcement learning model, termed CONQUER, that can learn from a conversational stream of questions and reformulations. CONQUER models the answering process as multiple agents walking in parallel on the KG, where the walks are determined by actions sampled using a policy network. This policy network takes the question along with the conversational context as inputs and is trained via noisy rewards obtained from the reformulation likelihood. To evaluate CONQUER, we create and release ConvRef, a benchmark with about 11k natural conversations containing around 205k reformulations. Experiments show that CONQUER successfully learns to answer conversational questions from noisy reward signals, significantly improving over a state-of-the-art baseline.
△ Less
Submitted 20 August, 2021; v1 submitted 11 May, 2021;
originally announced May 2021.
-
Data Generating Process to Evaluate Causal Discovery Techniques for Time Series Data
Authors:
Andrew R. Lawrence,
Marcus Kaiser,
Rui Sampaio,
Maksim Sipos
Abstract:
Going beyond correlations, the understanding and identification of causal relationships in observational time series, an important subfield of Causal Discovery, poses a major challenge. The lack of access to a well-defined ground truth for real-world data creates the need to rely on synthetic data for the evaluation of these methods. Existing benchmarks are limited in their scope, as they either a…
▽ More
Going beyond correlations, the understanding and identification of causal relationships in observational time series, an important subfield of Causal Discovery, poses a major challenge. The lack of access to a well-defined ground truth for real-world data creates the need to rely on synthetic data for the evaluation of these methods. Existing benchmarks are limited in their scope, as they either are restricted to a "static" selection of data sets, or do not allow for a granular assessment of the methods' performance when commonly made assumptions are violated. We propose a flexible and simple to use framework for generating time series data, which is aimed at developing, evaluating, and benchmarking time series causal discovery methods. In particular, the framework can be used to fine tune novel methods on vast amounts of data, without "overfitting" them to a benchmark, but rather so they perform well in real-world use cases. Using our framework, we evaluate prominent time series causal discovery methods and demonstrate a notable degradation in performance when their assumptions are invalidated and their sensitivity to choice of hyperparameters. Finally, we propose future research directions and how our framework can support both researchers and practitioners.
△ Less
Submitted 16 April, 2021;
originally announced April 2021.
-
Unsuitability of NOTEARS for Causal Graph Discovery
Authors:
Marcus Kaiser,
Maksim Sipos
Abstract:
Causal Discovery methods aim to identify a DAG structure that represents causal relationships from observational data. In this article, we stress that it is important to test such methods for robustness in practical settings. As our main example, we analyze the NOTEARS method, for which we demonstrate a lack of scale-invariance. We show that NOTEARS is a method that aims to identify a parsimonious…
▽ More
Causal Discovery methods aim to identify a DAG structure that represents causal relationships from observational data. In this article, we stress that it is important to test such methods for robustness in practical settings. As our main example, we analyze the NOTEARS method, for which we demonstrate a lack of scale-invariance. We show that NOTEARS is a method that aims to identify a parsimonious DAG from the data that explains the residual variance. We conclude that NOTEARS is not suitable for identifying truly causal relationships from the data.
△ Less
Submitted 15 June, 2021; v1 submitted 12 April, 2021;
originally announced April 2021.
-
Models and numbers: Representing the world or imposing order?
Authors:
Matthias Kaiser,
Tatjana Buklijas,
Peter Gluckman
Abstract:
We argue for a foundational epistemic claim and a hypothesis about the production and uses of mathematical epidemiological models, exploring the consequences for our political and socio-economic lives. First, in order to make the best use of scientific models, we need to understand why models are not truly representational of our world, but are already pitched towards various uses. Second, we need…
▽ More
We argue for a foundational epistemic claim and a hypothesis about the production and uses of mathematical epidemiological models, exploring the consequences for our political and socio-economic lives. First, in order to make the best use of scientific models, we need to understand why models are not truly representational of our world, but are already pitched towards various uses. Second, we need to understand the implicit power relations in numbers and models in public policy, and, thus, the implications for good governance if numbers and models are used as the exclusive drivers of decision making.
△ Less
Submitted 31 March, 2021;
originally announced April 2021.
-
The Design, Construction, and Commissioning of the KATRIN Experiment
Authors:
M. Aker,
K. Altenmüller,
J. F. Amsbaugh,
M. Arenz,
M. Babutzka,
J. Bast,
S. Bauer,
H. Bechtler,
M. Beck,
A. Beglarian,
J. Behrens,
B. Bender,
R. Berendes,
A. Berlev,
U. Besserer,
C. Bettin,
B. Bieringer,
K. Blaum,
F. Block,
S. Bobien,
J. Bohn,
K. Bokeloh,
H. Bolz,
B. Bornschein,
L. Bornschein
, et al. (204 additional authors not shown)
Abstract:
The KArlsruhe TRItium Neutrino (KATRIN) experiment, which aims to make a direct and model-independent determination of the absolute neutrino mass scale, is a complex experiment with many components. More than 15 years ago, we published a technical design report (TDR) [https://publikationen.bibliothek.kit.edu/270060419] to describe the hardware design and requirements to achieve our sensitivity goa…
▽ More
The KArlsruhe TRItium Neutrino (KATRIN) experiment, which aims to make a direct and model-independent determination of the absolute neutrino mass scale, is a complex experiment with many components. More than 15 years ago, we published a technical design report (TDR) [https://publikationen.bibliothek.kit.edu/270060419] to describe the hardware design and requirements to achieve our sensitivity goal of 0.2 eV at 90% C.L. on the neutrino mass. Since then there has been considerable progress, culminating in the publication of first neutrino mass results with the entire beamline operating [arXiv:1909.06048]. In this paper, we document the current state of all completed beamline components (as of the first neutrino mass measurement campaign), demonstrate our ability to reliably and stably control them over long times, and present details on their respective commissioning campaigns.
△ Less
Submitted 11 June, 2021; v1 submitted 5 March, 2021;
originally announced March 2021.
-
DeFi-ning DeFi: Challenges & Pathway
Authors:
Hendrik Amler,
Lisa Eckey,
Sebastian Faust,
Marcel Kaiser,
Philipp Sandner,
Benjamin Schlosser
Abstract:
The decentralized and trustless nature of cryptocurrencies and blockchain technology leads to a shift in the digital world. The possibility to execute small programs, called smart contracts, on cryptocurrencies like Ethereum opened doors to countless new applications. One particular exciting use case is decentralized finance (DeFi), which aims to revolutionize traditional financial services by fou…
▽ More
The decentralized and trustless nature of cryptocurrencies and blockchain technology leads to a shift in the digital world. The possibility to execute small programs, called smart contracts, on cryptocurrencies like Ethereum opened doors to countless new applications. One particular exciting use case is decentralized finance (DeFi), which aims to revolutionize traditional financial services by founding them on a decentralized infrastructure. We show the potential of DeFi by analyzing its advantages compared to traditional finance. Additionally, we survey the state-of-the-art of DeFi products and categorize existing services. Since DeFi is still in its infancy, there are countless hurdles for mass adoption. We discuss the most prominent challenges and point out possible solutions. Finally, we analyze the economics behind DeFi products. By carefully analyzing the state-of-the-art and discussing current challenges, we give a perspective on how the DeFi space might develop in the near future.
△ Less
Submitted 14 January, 2021;
originally announced January 2021.
-
Structured light for ultrafast laser micro- and nanoprocessing
Authors:
Daniel Flamm,
Daniel Günther Grossmann,
Marc Sailer,
Myriam Kaiser,
Felix Zimmermann,
Keyou Chen,
Michael Jenne,
Jonas Kleiner,
Julian Hellstern,
Christoph Tillkorn,
Dirk H Sutter,
Malte Kumkar
Abstract:
The industrial maturity of ultrashort pulsed lasers has triggered the development of a plethora of material processing strategies. Recently, the combination of these remarkable temporal pulse properties with advanced structured light concepts has led to breakthroughs in the development of novel laser application methods, which will now gradually reach industrial environments. We review the efficie…
▽ More
The industrial maturity of ultrashort pulsed lasers has triggered the development of a plethora of material processing strategies. Recently, the combination of these remarkable temporal pulse properties with advanced structured light concepts has led to breakthroughs in the development of novel laser application methods, which will now gradually reach industrial environments. We review the efficient generation of customized focus distributions from the near infrared down to the deep ultraviolet, e.g., based on non-diffracting beams and 3D-beam splitters, and demonstrate their impact for micro- and nanomachining of a wide range of materials. In the beam shaping concepts presented, special attention was paid to suitability for both high energies and high powers.
△ Less
Submitted 27 February, 2021; v1 submitted 18 December, 2020;
originally announced December 2020.
-
Beam shaping for ultrafast materials processing
Authors:
Daniel Flamm,
Daniel Günther Grossmann,
Michael Jenne,
Felix Zimmermann,
Jonas Kleiner,
Myriam Kaiser,
Julian Hellstern,
Christoph Tillkorn,
Malte Kumkar
Abstract:
The remarkable temporal properties of ultra-short pulsed lasers in combination with novel beam shaping concepts enable the development of completely new material processing strategies. We demonstrate the benefit of employing focus distributions being tailored in all three spatial dimensions. As example advanced Bessel-like beam profiles, 3D-beam splitting concepts and flat-top focus distributions…
▽ More
The remarkable temporal properties of ultra-short pulsed lasers in combination with novel beam shaping concepts enable the development of completely new material processing strategies. We demonstrate the benefit of employing focus distributions being tailored in all three spatial dimensions. As example advanced Bessel-like beam profiles, 3D-beam splitting concepts and flat-top focus distributions are used to achieve high-quality and efficient results for cutting, welding and drilling applications. Spatial and temporal in situ diagnostics is employed to analyze light-matter interaction and, in combination with flexible digital-holographic beam shaping techniques, to find the optimal beam shape for the respective laser application.
△ Less
Submitted 24 October, 2020;
originally announced October 2020.
-
High-quality Tailored-edge Cleaving Using Aberration-corrected Bessel-like Beams
Authors:
Michael Jenne,
Daniel Flamm,
Taofiq Ouaj,
Julian Hellstern,
Jonas Kleiner,
Daniel Grossmann,
Maximilian Koschig,
Myriam Kaiser,
Malte Kumkar,
Stefan Nolte
Abstract:
We report on the usage of ultrashort laser pulses in form of aberration-corrected Bessel-like beams for laser cutting of glass with bevels. Our approach foresees to incline the material's entrance surface with respect to the processing optics. The detailed analysis of phase distortions caused by the beam transition through the tilted glass surface allows to pre-compensate occurring aberrations usi…
▽ More
We report on the usage of ultrashort laser pulses in form of aberration-corrected Bessel-like beams for laser cutting of glass with bevels. Our approach foresees to incline the material's entrance surface with respect to the processing optics. The detailed analysis of phase distortions caused by the beam transition through the tilted glass surface allows to pre-compensate occurring aberrations using digital holography. We verify theoretical considerations by means of pump-probe microscopy and present high-quality edges in non-strengthened silicate glass.
△ Less
Submitted 20 October, 2020;
originally announced October 2020.
-
Should policy makers trust composite indices? A commentary on the pitfalls of inappropriate indices for policy formation
Authors:
Matthias Kaiser,
Andrew Tzer-Yeu Chen,
Peter Gluckman
Abstract:
This paper critically discusses the use and merits of global indices, in particular, the Global Health Security Index or GHSI (Cameron et 2019) in times of an imminent crisis, like the current pandemic. The index ranked 195 countries according to their expected preparedness in case of a pandemic or other biological threat. The Covid-19 pandemic provides the background to compare each country's pre…
▽ More
This paper critically discusses the use and merits of global indices, in particular, the Global Health Security Index or GHSI (Cameron et 2019) in times of an imminent crisis, like the current pandemic. The index ranked 195 countries according to their expected preparedness in case of a pandemic or other biological threat. The Covid-19 pandemic provides the background to compare each country's predicted performance from the GHSI with the actual performance. In general, there is an inverted relation between predicted versus actual performance, i.e. the predicted top performers are among those that are the worst hit. Obviously, this reflects poorly on the potential policy uses of the index in imminent crisis management. The paper also uses two different data sets, one from the Worldmeter on the spread of the Covid-19 pandemics, and the other one from the INGSA policy tracker, to make comparisons between the actual introduction of pandemic response policies and the corresponding death rate in 29 selected countries.
△ Less
Submitted 3 March, 2021; v1 submitted 31 August, 2020;
originally announced August 2020.
-
BioDynaMo: a general platform for scalable agent-based simulation
Authors:
Lukas Breitwieser,
Ahmad Hesam,
Jean de Montigny,
Vasileios Vavourakis,
Alexandros Iosif,
Jack Jennings,
Marcus Kaiser,
Marco Manca,
Alberto Di Meglio,
Zaid Al-Ars,
Fons Rademakers,
Onur Mutlu,
Roman Bauer
Abstract:
Motivation: Agent-based modeling is an indispensable tool for studying complex biological systems. However, existing simulators do not always take full advantage of modern hardware and often have a field-specific software design.
Results: We present a novel simulation platform called BioDynaMo that alleviates both of these problems. BioDynaMo features a general-purpose and high-performance simul…
▽ More
Motivation: Agent-based modeling is an indispensable tool for studying complex biological systems. However, existing simulators do not always take full advantage of modern hardware and often have a field-specific software design.
Results: We present a novel simulation platform called BioDynaMo that alleviates both of these problems. BioDynaMo features a general-purpose and high-performance simulation engine. We demonstrate that BioDynaMo can be used to simulate use cases in: neuroscience, oncology, and epidemiology. For each use case we validate our findings with experimental data or an analytical solution. Our performance results show that BioDynaMo performs up to three orders of magnitude faster than the state-of-the-art baseline. This improvement makes it feasible to simulate each use case with one billion agents on a single server, showcasing the potential BioDynaMo has for computational biology research.
Availability: BioDynaMo is an open-source project under the Apache 2.0 license and is available at www.biodynamo.org. Instructions to reproduce the results are available in supplementary information.
Contact: [email protected], [email protected], [email protected], [email protected]
Supplementary information: Available at https://doi.org/10.5281/zenodo.4501515
△ Less
Submitted 5 February, 2021; v1 submitted 11 June, 2020;
originally announced June 2020.
-
Functional compensation after lesions: Predicting site and extent of recovery
Authors:
Marcus Kaiser
Abstract:
In some cases, the function of a lesioned area can be compensated for by another area. However, it remains unpredictable if and by which other area a lesion can be compensated. We assume that similar incoming and outgoing connections are necessary to encode the same function as the damaged region. The similarity can be measured both locally using the matching index and looking at a more global sca…
▽ More
In some cases, the function of a lesioned area can be compensated for by another area. However, it remains unpredictable if and by which other area a lesion can be compensated. We assume that similar incoming and outgoing connections are necessary to encode the same function as the damaged region. The similarity can be measured both locally using the matching index and looking at a more global scale by non-metric multidimensional scaling (NMDS). We tested how well both measures can predict the compensating area for the loss of the visual cortex in kittens. For this case study, the global comparison of connectivity turns out to be a better method for predicting functional compensation. In future studies, the extent of the similarity between the lesioned and compensating regions might be a measure of the extent to which function can be successfully recovered.
△ Less
Submitted 6 May, 2020;
originally announced May 2020.
-
Conversational Question Answering over Passages by Leveraging Word Proximity Networks
Authors:
Magdalena Kaiser,
Rishiraj Saha Roy,
Gerhard Weikum
Abstract:
Question answering (QA) over text passages is a problem of long-standing interest in information retrieval. Recently, the conversational setting has attracted attention, where a user asks a sequence of questions to satisfy her information needs around a topic. While this setup is a natural one and similar to humans conversing with each other, it introduces two key research challenges: understandin…
▽ More
Question answering (QA) over text passages is a problem of long-standing interest in information retrieval. Recently, the conversational setting has attracted attention, where a user asks a sequence of questions to satisfy her information needs around a topic. While this setup is a natural one and similar to humans conversing with each other, it introduces two key research challenges: understanding the context left implicit by the user in follow-up questions, and dealing with ad hoc question formulations. In this work, we demonstrate CROWN (Conversational passage ranking by Reasoning Over Word Networks): an unsupervised yet effective system for conversational QA with passage responses, that supports several modes of context propagation over multiple turns. To this end, CROWN first builds a word proximity network (WPN) from large corpora to store statistically significant term co-occurrences. At answering time, passages are ranked by a combination of their similarity to the question, and coherence of query terms within: these factors are measured by reading off node and edge weights from the WPN. CROWN provides an interface that is both intuitive for end-users, and insightful for experts for reconfiguration to individual setups. CROWN was evaluated on TREC CAsT data, where it achieved above-median performance in a pool of neural methods.
△ Less
Submitted 25 May, 2020; v1 submitted 27 April, 2020;
originally announced April 2020.
-
Deep Learning in Mining Biological Data
Authors:
Mufti Mahmud,
M Shamim Kaiser,
Amir Hussain
Abstract:
Recent technological advancements in data acquisition tools allowed life scientists to acquire multimodal data from different biological application domains. Broadly categorized in three types (i.e., sequences, images, and signals), these data are huge in amount and complex in nature. Mining such an enormous amount of data for pattern recognition is a big challenge and requires sophisticated data-…
▽ More
Recent technological advancements in data acquisition tools allowed life scientists to acquire multimodal data from different biological application domains. Broadly categorized in three types (i.e., sequences, images, and signals), these data are huge in amount and complex in nature. Mining such an enormous amount of data for pattern recognition is a big challenge and requires sophisticated data-intensive machine learning techniques. Artificial neural network-based learning systems are well known for their pattern recognition capabilities and lately their deep architectures - known as deep learning (DL) - have been successfully applied to solve many complex pattern recognition problems. Highlighting the role of DL in recognizing patterns in biological data, this article provides - applications of DL to biological sequences, images, and signals data; overview of open access sources of these data; description of open source DL tools applicable on these data; and comparison of these tools from qualitative and quantitative perspectives. At the end, it outlines some open research challenges in mining biological data and puts forward a number of possible future perspectives.
△ Less
Submitted 28 February, 2020;
originally announced March 2020.
-
Computation of Dynamic Equilibria in Series-Parallel Networks
Authors:
Marcus Kaiser
Abstract:
We consider dynamic equilibria for flows over time under the fluid queuing model. In this model, queues on the links of a network take care of flow propagation. Flow enters the network at a single source and leaves at a single sink. In a dynamic equilibrium, every infinitesimally small flow particle reaches the sink as early as possible given the pattern of the rest of the flow. While this model h…
▽ More
We consider dynamic equilibria for flows over time under the fluid queuing model. In this model, queues on the links of a network take care of flow propagation. Flow enters the network at a single source and leaves at a single sink. In a dynamic equilibrium, every infinitesimally small flow particle reaches the sink as early as possible given the pattern of the rest of the flow. While this model has been examined for many decades, progress has been relatively recent. In particular, the derivatives of dynamic equilibria have been characterized as thin flows with resetting, which allowed for more structural results. Our two main results are based on the formulation of thin flows with resetting as linear complementarity problem and its analysis. We present a constructive proof of existence for dynamic equilibria if the inflow rate is right-monotone. The complexity of computing thin flows with resetting, which occurs as a subproblem in this method, is still open. We settle it for the class of two-terminal series-parallel networks by giving a recursive algorithm that solves the problem for all flow values simultaneously in polynomial time.
△ Less
Submitted 4 May, 2020; v1 submitted 26 February, 2020;
originally announced February 2020.