Showing 1–12 of 12 results for author: Kolb, C

How Inverse Conditional Flows Can Serve as a Substitute for Distributional Regression

Authors: Lucas Kook, Chris Kolb, Philipp Schiele, Daniel Dold, Marcel Arpogaus, Cornelius Fritz, Philipp F. Baumann, Philipp Kopper, Tobias Pielok, Emilio Dorigatti, David Rügamer

Abstract: Neural network representations of simple models, such as linear regression, are being studied increasingly to better understand the underlying principles of deep learning algorithms. However, neural representations of distributional regression models, such as the Cox model, have received little attention so far. We close this gap by proposing a framework for distributional regression using inverse… ▽ More Neural network representations of simple models, such as linear regression, are being studied increasingly to better understand the underlying principles of deep learning algorithms. However, neural representations of distributional regression models, such as the Cox model, have received little attention so far. We close this gap by proposing a framework for distributional regression using inverse flow transformations (DRIFT), which includes neural representations of the aforementioned models. We empirically demonstrate that the neural representations of models in DRIFT can serve as a substitute for their classical statistical counterparts in several applications involving continuous, ordered, time-series, and survival outcomes. We confirm that models in DRIFT empirically match the performance of several statistical methods in terms of estimation of partial effects, prediction, and aleatoric uncertainty quantification. DRIFT covers both interpretable statistical models and flexible neural networks opening up new avenues in both statistical modeling and deep learning. △ Less

Submitted 10 July, 2024; v1 submitted 8 May, 2024; originally announced May 2024.

Comments: Accepted at UAI 2024 https://www.auai.org/uai2024/accepted_papers

Generalizing Orthogonalization for Models with Non-Linearities

Authors: David Rügamer, Chris Kolb, Tobias Weber, Lucas Kook, Thomas Nagler

Abstract: The complexity of black-box algorithms can lead to various challenges, including the introduction of biases. These biases present immediate risks in the algorithms' application. It was, for instance, shown that neural networks can deduce racial information solely from a patient's X-ray scan, a task beyond the capability of medical experts. If this fact is not known to the medical expert, automatic… ▽ More The complexity of black-box algorithms can lead to various challenges, including the introduction of biases. These biases present immediate risks in the algorithms' application. It was, for instance, shown that neural networks can deduce racial information solely from a patient's X-ray scan, a task beyond the capability of medical experts. If this fact is not known to the medical expert, automatic decision-making based on this algorithm could lead to prescribing a treatment (purely) based on racial information. While current methodologies allow for the "orthogonalization" or "normalization" of neural networks with respect to such information, existing approaches are grounded in linear models. Our paper advances the discourse by introducing corrections for non-linearities such as ReLU activations. Our approach also encompasses scalar and tensor-valued predictions, facilitating its integration into neural network architectures. Through extensive experiments, we validate our method's effectiveness in safeguarding sensitive data in generalized linear models, normalizing convolutional neural networks for metadata, and rectifying pre-existing embeddings for undesired attributes. △ Less

Submitted 2 June, 2024; v1 submitted 3 May, 2024; originally announced May 2024.

Input Data Adaptive Learning (IDAL) for Sub-acute Ischemic Stroke Lesion Segmentation

Authors: Michael Götz, Christian Weber, Christoph Kolb, Klaus Maier-Hein

Abstract: In machine learning larger databases are usually associated with higher classification accuracy due to better generalization. This generalization may lead to non-optimal classifiers in some medical applications with highly variable expressions of pathologies. This paper presents a method for learning from a large training base by adaptively selecting optimal training samples for given input data.… ▽ More In machine learning larger databases are usually associated with higher classification accuracy due to better generalization. This generalization may lead to non-optimal classifiers in some medical applications with highly variable expressions of pathologies. This paper presents a method for learning from a large training base by adaptively selecting optimal training samples for given input data. In this way heterogeneous databases are supported two-fold. First, by being able to deal with sparsely annotated data allows a quick inclusion of new data set and second, by training an input-dependent classifier. The proposed approach is evaluated using the SISS challenge. The proposed algorithm leads to a significant improvement of the classification accuracy. △ Less

Submitted 12 March, 2024; originally announced March 2024.

Journal ref: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. BrainLes 2015

Smoothing the Edges: Smooth Optimization for Sparse Regularization using Hadamard Overparametrization

Authors: Chris Kolb, Christian L. Müller, Bernd Bischl, David Rügamer

Abstract: We present a framework for smooth optimization of explicitly regularized objectives for (structured) sparsity. These non-smooth and possibly non-convex problems typically rely on solvers tailored to specific models and regularizers. In contrast, our method enables fully differentiable and approximation-free optimization and is thus compatible with the ubiquitous gradient descent paradigm in deep l… ▽ More We present a framework for smooth optimization of explicitly regularized objectives for (structured) sparsity. These non-smooth and possibly non-convex problems typically rely on solvers tailored to specific models and regularizers. In contrast, our method enables fully differentiable and approximation-free optimization and is thus compatible with the ubiquitous gradient descent paradigm in deep learning. The proposed optimization transfer comprises an overparameterization of selected parameters and a change of penalties. In the overparametrized problem, smooth surrogate regularization induces non-smooth, sparse regularization in the base parametrization. We prove that the surrogate objective is equivalent in the sense that it not only has identical global minima but also matching local minima, thereby avoiding the introduction of spurious solutions. Additionally, our theory establishes results of independent interest regarding matching local minima for arbitrary, potentially unregularized, objectives. We comprehensively review sparsity-inducing parametrizations across different fields that are covered by our general theory, extend their scope, and propose improvements in several aspects. Numerical experiments further demonstrate the correctness and effectiveness of our approach on several sparse learning problems ranging from high-dimensional regression to sparse neural network training. △ Less

Submitted 26 April, 2024; v1 submitted 7 July, 2023; originally announced July 2023.

Beep-And-Sleep: Message and Energy Efficient Set Cover

Authors: Thorsten Götte, Christina Kolb, Christian Scheideler, Julian Werthmann

Abstract: We observe message-efficient distributed algorithms for the Set Cover problem. Given a ground set $U$ of $n$ elements and $m$ subsets of $U$, we aim to find the minimal number of these subsets that contain all elements. In the default distributed setup of this problem, each set has a bidirected communication link with each element it contains. Our first result is a $\tilde{O}(\log^2(Δ))$-time and… ▽ More We observe message-efficient distributed algorithms for the Set Cover problem. Given a ground set $U$ of $n$ elements and $m$ subsets of $U$, we aim to find the minimal number of these subsets that contain all elements. In the default distributed setup of this problem, each set has a bidirected communication link with each element it contains. Our first result is a $\tilde{O}(\log^2(Δ))$-time and $O(\sqrt{Δ)}(n+m))$-message algorithm with expected approximation ration of $O(\log(Δ))$ in the $KT_0$ model. The value $Δ$ denotes the maximal cardinality of each subset. Our algorithm is \emph{almost} optimal with regard to time and message complexity. Further, we present Set Cover algorithm in the Beeping model that only relies on carrier-sensing and can trade runtime for approximation ratio similar to the celebrated algorithm by Kuhn and Wattenhofer [PODC '03]. △ Less

Submitted 30 July, 2021; originally announced July 2021.

Model-based Joint Analysis of Safety and Security: Survey and Identification of Gaps

Authors: Stefano M. Nicoletti, Marijn Peppelman, Christina Kolb, Mariëlle Stoelinga

Abstract: We survey the state-of-the-art on model-based formalisms for safety and security joint analysis, where safety refers to the absence of unintended failures, and security to absence of malicious attacks. We conduct a thorough literature review and - as a result - we consider fourteen model-based formalisms and compare them with respect to several criteria: (1) Modelling capabilities and Expressivene… ▽ More We survey the state-of-the-art on model-based formalisms for safety and security joint analysis, where safety refers to the absence of unintended failures, and security to absence of malicious attacks. We conduct a thorough literature review and - as a result - we consider fourteen model-based formalisms and compare them with respect to several criteria: (1) Modelling capabilities and Expressiveness: which phenomena can be expressed in these formalisms? To which extent can they capture safety-security interactions? (2) Analytical capabilities: which analysis types are supported? (3) Practical applicability: to what extent have the formalisms been used to analyze small or larger case studies? Furthermore, (1) we present more precise definitions for safety-security dependencies in tree-like formalisms; (2) we showcase the potential of each formalism by modelling the same toy example from the literature and (3) we present our findings and reflect on possible ways to narrow highlighted gaps. In summary, our key findings are the following: (1) the majority of approaches combine tree-like formal models; (2) the exact nature of safety-security interaction is still ill-understood and (3) diverse formalisms can capture different interactions; (4) analyzed formalisms merge modelling constructs from existing safety- and security-specific formalisms, without introducing ad hoc constructs to model safety-security interactions, or (5) metrics to analyze trade offs. Moreover, (6) large case studies representing safety-security interactions are still missing. △ Less

Submitted 23 October, 2023; v1 submitted 11 June, 2021; originally announced June 2021.

deepregression: a Flexible Neural Network Framework for Semi-Structured Deep Distributional Regression

Authors: David Rügamer, Chris Kolb, Cornelius Fritz, Florian Pfisterer, Philipp Kopper, Bernd Bischl, Ruolin Shen, Christina Bukas, Lisa Barros de Andrade e Sousa, Dominik Thalmeier, Philipp Baumann, Lucas Kook, Nadja Klein, Christian L. Müller

Abstract: In this paper we describe the implementation of semi-structured deep distributional regression, a flexible framework to learn conditional distributions based on the combination of additive regression models and deep networks. Our implementation encompasses (1) a modular neural network building system based on the deep learning library \pkg{TensorFlow} for the fusion of various statistical and deep… ▽ More In this paper we describe the implementation of semi-structured deep distributional regression, a flexible framework to learn conditional distributions based on the combination of additive regression models and deep networks. Our implementation encompasses (1) a modular neural network building system based on the deep learning library \pkg{TensorFlow} for the fusion of various statistical and deep learning approaches, (2) an orthogonalization cell to allow for an interpretable combination of different subnetworks, as well as (3) pre-processing steps necessary to set up such models. The software package allows to define models in a user-friendly manner via a formula interface that is inspired by classical statistical model frameworks such as \pkg{mgcv}. The packages' modular design and functionality provides a unique resource for both scalable estimation of complex statistical models and the combination of approaches from deep learning and statistics. This allows for state-of-the-art predictive performance while simultaneously retaining the indispensable interpretability of classical statistical models. △ Less

Submitted 10 March, 2022; v1 submitted 6 April, 2021; originally announced April 2021.

Semi-Structured Distributional Regression -- Extending Structured Additive Models by Arbitrary Deep Neural Networks and Data Modalities

Authors: David Rügamer, Chris Kolb, Nadja Klein

Abstract: Combining additive models and neural networks allows to broaden the scope of statistical regression and extend deep learning-based approaches by interpretable structured additive predictors at the same time. Existing attempts uniting the two modeling approaches are, however, limited to very specific combinations and, more importantly, involve an identifiability issue. As a consequence, interpretab… ▽ More Combining additive models and neural networks allows to broaden the scope of statistical regression and extend deep learning-based approaches by interpretable structured additive predictors at the same time. Existing attempts uniting the two modeling approaches are, however, limited to very specific combinations and, more importantly, involve an identifiability issue. As a consequence, interpretability and stable estimation are typically lost. We propose a general framework to combine structured regression models and deep neural networks into a unifying network architecture. To overcome the inherent identifiability issues between different model parts, we construct an orthogonalization cell that projects the deep neural network into the orthogonal complement of the statistical model predictor. This enables proper estimation of structured model parts and thereby interpretability. We demonstrate the framework's efficacy in numerical experiments and illustrate its special merits in benchmarks and real-world applications. △ Less

Submitted 9 July, 2022; v1 submitted 13 February, 2020; originally announced February 2020.

A Bounding Box Overlay for Competitive Routing in Hybrid Communication Networks

Authors: Jannik Castenow, Christina Kolb, Christian Scheideler

Abstract: In this work, we present a new approach for competitive geometric routing in wireless ad hoc networks. In general, it is well-known that any online routing strategy performs very poor in the worst case. The main difficulty are uncovered regions within the wireless ad hoc network, which we denote as radio holes. Complex shapes of radio holes, for example zig-zag-shapes, make local geometric routing… ▽ More In this work, we present a new approach for competitive geometric routing in wireless ad hoc networks. In general, it is well-known that any online routing strategy performs very poor in the worst case. The main difficulty are uncovered regions within the wireless ad hoc network, which we denote as radio holes. Complex shapes of radio holes, for example zig-zag-shapes, make local geometric routing even more difficult, i.e., forwarded messages in direction to the destination might get stuck in a dead end or are routed along very long detours, when there is no knowledge about the ad hoc network. To obtain knowledge about the position and shape of radio holes, we make use of a hybrid network approach. This approach assumes that we can not just make use of the ad hoc network but also of some cellular infrastructure, which is used to gather knowledge about the underlying ad hoc network. Communication via the cellular infrastructure incurs costs as cell phone providers are involved. Therefore, we use the cellular infrastructure only to compute routing paths in the ad hoc network. The actual data transmission takes place in the ad hoc network. In order to find good routing paths we aim at computing an abstraction of the ad hoc network in which radio holes are abstracted by bounding boxes. The advantage of bounding boxes as hole abstraction is that we only have to consider a constant number of nodes per hole. We prove that bounding boxes are a suitable hole abstraction that allows us to find $c$-competitive paths in the ad hoc network in case of non-intersecting bounding boxes. In case of intersecting bounding boxes, we show via simulations that our routing strategy significantly outperforms the so far best online routing strategies for wireless ad hoc networks. Finally, we also present a routing strategy that is $c$-competitive in case of pairwise intersecting bounding boxes. △ Less

Submitted 10 April, 2019; v1 submitted 12 October, 2018; originally announced October 2018.

Self-stabilizing Overlays for high-dimensional Monotonic Searchability

Authors: Michael Feldmann, Christina Kolb, Christian Scheideler

Abstract: We extend the concept of monotonic searchability for self-stabilizing systems from one to multiple dimensions. A system is self-stabilizing if it can recover to a legitimate state from any initial illegal state. These kind of systems are most often used in distributed applications. Monotonic searchability provides guarantees when searching for nodes while the recovery process is going on. More pre… ▽ More We extend the concept of monotonic searchability for self-stabilizing systems from one to multiple dimensions. A system is self-stabilizing if it can recover to a legitimate state from any initial illegal state. These kind of systems are most often used in distributed applications. Monotonic searchability provides guarantees when searching for nodes while the recovery process is going on. More precisely, if a search request started at some node $u$ succeeds in reaching its destination $v$, then all future search requests from $u$ to $v$ succeed as well. Although there already exists a self-stabilizing protocol for a two-dimensional topology and an universal approach for monotonic searchability, it is not clear how both of these concepts fit together effectively. The latter concept even comes with some restrictive assumptions on messages, which is not the case for our protocol. We propose a simple novel protocol for a self-stabilizing two-dimensional quadtree that satisfies monotonic searchability. Our protocol can easily be extended to higher dimensions and offers routing in $\mathcal O(\log n)$ hops for any search request. △ Less

Submitted 30 August, 2018; originally announced August 2018.

Competitive Routing in Hybrid Communication Networks

Authors: Daniel Jung, Christina Kolb, Christian Scheideler, Jannik Sundermeier

Abstract: Routing is a challenging problem for wireless ad hoc networks, especially when the nodes are mobile and spread so widely that in most cases multiple hops are needed to route a message from one node to another. In fact, it is known that any online routing protocol has a poor performance in the worst case, in a sense that there is a distribution of nodes resulting in bad routing paths for that proto… ▽ More Routing is a challenging problem for wireless ad hoc networks, especially when the nodes are mobile and spread so widely that in most cases multiple hops are needed to route a message from one node to another. In fact, it is known that any online routing protocol has a poor performance in the worst case, in a sense that there is a distribution of nodes resulting in bad routing paths for that protocol, even if the nodes know their geographic positions and the geographic position of the destination of a message is known. The reason for that is that radio holes in the ad hoc network may require messages to take long detours in order to get to a destination, which are hard to find in an online fashion. In this paper, we assume that the wireless ad hoc network can make limited use of long-range links provided by a global communication infrastructure like a cellular infrastructure or a satellite in order to compute an abstraction of the wireless ad hoc network that allows the messages to be sent along near-shortest paths in the ad hoc network. We present distributed algorithms that compute an abstraction of the ad hoc network in $\mathcal{O}\left(\log ^2 n\right)$ time using long-range links, which results in $c$-competitive routing paths between any two nodes of the ad hoc network for some constant $c$ if the convex hulls of the radio holes do not intersect. We also show that the storage needed for the abstraction just depends on the number and size of the radio holes in the wireless ad hoc network and is independent on the total number of nodes, and this information just has to be known to a few nodes for the routing to work. △ Less

Submitted 28 February, 2018; v1 submitted 25 October, 2017; originally announced October 2017.

ACM Class: C.2.4

Self-Stabilizing Supervised Publish-Subscribe Systems

Authors: Michael Feldmann, Christina Kolb, Christian Scheideler, Thim Strothmann

Abstract: In this paper we present two major results: First, we introduce the first self-stabilizing version of a supervised overlay network by presenting a self-stabilizing supervised skip ring. Secondly, we show how to use the self-stabilizing supervised skip ring to construct an efficient self-stabilizing publish-subscribe system. That is, in addition to stabilizing the overlay network, every subscriber… ▽ More In this paper we present two major results: First, we introduce the first self-stabilizing version of a supervised overlay network by presenting a self-stabilizing supervised skip ring. Secondly, we show how to use the self-stabilizing supervised skip ring to construct an efficient self-stabilizing publish-subscribe system. That is, in addition to stabilizing the overlay network, every subscriber of a topic will eventually know all of the publications that have been issued so far for that topic. The communication work needed to processes a subscribe or unsubscribe operation is just a constant in a legitimate state, and the communication work of checking whether the system is still in a legitimate state is just a constant on expectation for the supervisor as well as any process in the system. △ Less

Submitted 22 December, 2017; v1 submitted 23 October, 2017; originally announced October 2017.