Showing 1–50 of 265 results for author: Berger, M

Deriving the Topological Properties of the Magnetic Field of Coronal Mass Ejections from In Situ Measurements: Techniques

Authors: Nada Al-Haddad, Mitchell Berger

Abstract: Coronal mass ejections (CMEs) are magnetized plasma systems with highly complex magnetic topology and evolution. Methods developed to assess their magnetic configuration have primarily focused on reconstructing three-dimensional representations from one-dimensional time series measurements taken in situ using techniques based on the "highly twisted magnetic flux rope" approximations. However, the… ▽ More Coronal mass ejections (CMEs) are magnetized plasma systems with highly complex magnetic topology and evolution. Methods developed to assess their magnetic configuration have primarily focused on reconstructing three-dimensional representations from one-dimensional time series measurements taken in situ using techniques based on the "highly twisted magnetic flux rope" approximations. However, the magnetic fields of CMEs is know to have more complicated geometries. Their structure can be quantified using measures of field line topology, which have been primarily used for solar physics research. In this work, we introduce a novel technique of directly quantifying the various form of magnetic helicity within a CME in the interplanetary space using synthetic in situ measurements. We use a relatively simple three-dimensional simulation of a CME initiated with a highly-twisted flux rope. We find that a significant portion of the magnetic helicity near 1~au is contained in writhe and mutual helicity rather than just in twist. We discuss the implications of this finding for fitting and reconstruction techniques. △ Less

Submitted 8 August, 2024; originally announced August 2024.

Comments: submitted to ApJ

Optimal rates for estimating the covariance kernel from synchronously sampled functional data

Authors: Max Berger, Hajo Holzmann

Abstract: We obtain minimax-optimal convergence rates in the supremum norm, including infor-mation-theoretic lower bounds, for estimating the covariance kernel of a stochastic process which is repeatedly observed at discrete, synchronous design points. In particular, for dense design we obtain the $\sqrt n$-rate of convergence in the supremum norm without additional logarithmic factors which typically occur… ▽ More We obtain minimax-optimal convergence rates in the supremum norm, including infor-mation-theoretic lower bounds, for estimating the covariance kernel of a stochastic process which is repeatedly observed at discrete, synchronous design points. In particular, for dense design we obtain the $\sqrt n$-rate of convergence in the supremum norm without additional logarithmic factors which typically occur in the results in the literature. Surprisingly, in the transition from dense to sparse design the rates do not reflect the two-dimensional nature of the covariance kernel but correspond to those for univariate mean function estimation. Our estimation method can make use of higher-order smoothness of the covariance kernel away from the diagonal, and does not require the same smoothness on the diagonal itself. Hence, as in Mohammadi and Panaretos (2024) we can cover covariance kernels of processes with rough sample paths. Moreover, the estimator does not use mean function estimation to form residuals, and no smoothness assumptions on the mean have to be imposed. In the dense case we also obtain a central limit theorem in the supremum norm, which can be used as the basis for the construction of uniform confidence sets. Simulations and real-data applications illustrate the practical usefulness of the methods. △ Less

Submitted 18 July, 2024; originally announced July 2024.

Concept Lens: Visually Analyzing the Consistency of Semantic Manipulation in GANs

Authors: Sangwon Jeong, Mingwei Li, Matthew Berger, Shusen Liu

Abstract: As applications of generative AI become mainstream, it is important to understand what generative models are capable of producing, and the extent to which one can predictably control their outputs. In this paper, we propose a visualization design, named Concept Lens, for jointly navigating the data distribution of a generative model, and concept manipulations supported by the model. Our work is fo… ▽ More As applications of generative AI become mainstream, it is important to understand what generative models are capable of producing, and the extent to which one can predictably control their outputs. In this paper, we propose a visualization design, named Concept Lens, for jointly navigating the data distribution of a generative model, and concept manipulations supported by the model. Our work is focused on modern vision-based generative adversarial networks (GAN), and their learned latent spaces, wherein concept discovery has gained significant interest as a means of image manipulation. Concept Lens is designed to support users in understanding the diversity of a provided set of concepts, the relationship between concepts, and the suitability of concepts to give semantic controls for image generation. Key to our approach is the hierarchical grouping of concepts, generated images, and the associated joint exploration. We show how Concept Lens can reveal consistent semantic manipulations for editing images, while also serving as a diagnostic tool for studying the limitations and trade-offs of concept discovery methods. △ Less

Submitted 28 June, 2024; originally announced June 2024.

Journal ref: 2023 IEEE Visualization and Visual Analytics (VIS), Melbourne, Australia, 2023, pp. 221-225

Confidence intervals for tree-structured varying coefficients

Authors: Nikolai Spuck, Matthias Schmid, Malte Monin, Moritz Berger

Abstract: The tree-structured varying coefficient model (TSVC) is a flexible regression approach that allows the effects of covariates to vary with the values of the effect modifiers. Relevant effect modifiers are identified inherently using recursive partitioning techniques. To quantify uncertainty in TSVC models, we propose a procedure to construct confidence intervals of the estimated partition-specific… ▽ More The tree-structured varying coefficient model (TSVC) is a flexible regression approach that allows the effects of covariates to vary with the values of the effect modifiers. Relevant effect modifiers are identified inherently using recursive partitioning techniques. To quantify uncertainty in TSVC models, we propose a procedure to construct confidence intervals of the estimated partition-specific coefficients. This task constitutes a selective inference problem as the coefficients of a TSVC model result from data-driven model building. To account for this issue, we introduce a parametric bootstrap approach, which is tailored to the complex structure of TSVC. Finite sample properties, particularly coverage proportions, of the proposed confidence intervals are evaluated in a simulation study. For illustration, we consider applications to data from COVID-19 patients and from patients suffering from acute odontogenic infection. The proposed approach may also be adapted for constructing confidence intervals for other tree-based methods. △ Less

Submitted 28 June, 2024; originally announced June 2024.

Text-based Transfer Function Design for Semantic Volume Rendering

Authors: Sangwon Jeong, Jixian Li, Christopher Johnson, Shusen Liu, Matthew Berger

Abstract: Transfer function design is crucial in volume rendering, as it directly influences the visual representation and interpretation of volumetric data. However, creating effective transfer functions that align with users' visual objectives is often challenging due to the complex parameter space and the semantic gap between transfer function values and features of interest within the volume. In this wo… ▽ More Transfer function design is crucial in volume rendering, as it directly influences the visual representation and interpretation of volumetric data. However, creating effective transfer functions that align with users' visual objectives is often challenging due to the complex parameter space and the semantic gap between transfer function values and features of interest within the volume. In this work, we propose a novel approach that leverages recent advancements in language-vision models to bridge this semantic gap. By employing a fully differentiable rendering pipeline and an image-based loss function guided by language descriptions, our method generates transfer functions that yield volume-rendered images closely matching the user's intent. We demonstrate the effectiveness of our approach in creating meaningful transfer functions from simple descriptions, empowering users to intuitively express their desired visual outcomes with minimal effort. This advancement streamlines the transfer function design process and makes volume rendering more accessible to a wider range of users. △ Less

Submitted 21 June, 2024; originally announced June 2024.

Graphical Perception of Saliency-based Model Explanations

Authors: Yayan Zhao, Mingwei Li, Matthew Berger

Abstract: In recent years, considerable work has been devoted to explaining predictive, deep learning-based models, and in turn how to evaluate explanations. An important class of evaluation methods are ones that are human-centered, which typically require the communication of explanations through visualizations. And while visualization plays a critical role in perceiving and understanding model explanation… ▽ More In recent years, considerable work has been devoted to explaining predictive, deep learning-based models, and in turn how to evaluate explanations. An important class of evaluation methods are ones that are human-centered, which typically require the communication of explanations through visualizations. And while visualization plays a critical role in perceiving and understanding model explanations, how visualization design impacts human perception of explanations remains poorly understood. In this work, we study the graphical perception of model explanations, specifically, saliency-based explanations for visual recognition models. We propose an experimental design to investigate how human perception is influenced by visualization design, wherein we study the task of alignment assessment, or whether a saliency map aligns with an object in an image. Our findings show that factors related to visualization design decisions, the type of alignment, and qualities of the saliency map all play important roles in how humans perceive saliency-based visual explanations. △ Less

Submitted 11 June, 2024; originally announced June 2024.

CUPID: Contextual Understanding of Prompt-conditioned Image Distributions

Authors: Yayan Zhao, Mingwei Li, Matthew Berger

Abstract: We present CUPID: a visualization method for the contextual understanding of prompt-conditioned image distributions. CUPID targets the visual analysis of distributions produced by modern text-to-image generative models, wherein a user can specify a scene via natural language, and the model generates a set of images, each intended to satisfy the user's description. CUPID is designed to help underst… ▽ More We present CUPID: a visualization method for the contextual understanding of prompt-conditioned image distributions. CUPID targets the visual analysis of distributions produced by modern text-to-image generative models, wherein a user can specify a scene via natural language, and the model generates a set of images, each intended to satisfy the user's description. CUPID is designed to help understand the resulting distribution, using contextual cues to facilitate analysis: objects mentioned in the prompt, novel, synthesized objects not explicitly mentioned, and their potential relationships. Central to CUPID is a novel method for visualizing high-dimensional distributions, wherein contextualized embeddings of objects, those found within images, are mapped to a low-dimensional space via density-based embeddings. We show how such embeddings allows one to discover salient styles of objects within a distribution, as well as identify anomalous, or rare, object styles. Moreover, we introduce conditional density embeddings, whereby conditioning on a given object allows one to compare object dependencies within the distribution. We employ CUPID for analyzing image distributions produced by large-scale diffusion models, where our experimental results offer insights on language misunderstanding from such models and biases in object composition, while also providing an interface for discovery of typical, or rare, synthesized scenes. △ Less

Submitted 11 June, 2024; originally announced June 2024.

Improving Memory Dependence Prediction with Static Analysis

Authors: Luke Panayi, Rohan Gandhi, Jim Whittaker, Vassilios Chouliaras, Martin Berger, Paul Kelly

Abstract: This paper explores the potential of communicating information gained by static analysis from compilers to Out-of-Order (OoO) machines, focusing on the memory dependence predictor (MDP). The MDP enables loads to issue without all in-flight store addresses being known, with minimal memory order violations. We use LLVM to find loads with no dependencies and label them via their opcode. These labelle… ▽ More This paper explores the potential of communicating information gained by static analysis from compilers to Out-of-Order (OoO) machines, focusing on the memory dependence predictor (MDP). The MDP enables loads to issue without all in-flight store addresses being known, with minimal memory order violations. We use LLVM to find loads with no dependencies and label them via their opcode. These labelled loads skip making lookups into the MDP, improving prediction accuracy by reducing false dependencies. We communicate this information in a minimally intrusive way, i.e.~without introducing additional hardware costs or instruction bandwidth, providing these improvements without any additional overhead in the CPU. We find that in select cases in Spec2017, a significant number of load instructions can skip interacting with the MDP and lead to a performance gain. These results point to greater possibilities for static analysis as a source of near zero cost performance gains in future CPU designs. △ Less

Submitted 5 June, 2024; v1 submitted 12 March, 2024; originally announced March 2024.

Comments: 15 pages

ACM Class: B.0; B.8; C.1

Quantum computing in civil engineering: Potentials and Limitations

Authors: Joern Ploennigs, Markus Berger, Martin Mevissen, Kay Smarsly

Abstract: Quantum computing is a new computational paradigm with the potential to solve certain computationally challenging problems much faster than traditional approaches. Civil engineering encompasses many computationally challenging problems, which leads to the question of how well quantum computing is suitable for solving civil engineering problems and how much impact and implications to the field of c… ▽ More Quantum computing is a new computational paradigm with the potential to solve certain computationally challenging problems much faster than traditional approaches. Civil engineering encompasses many computationally challenging problems, which leads to the question of how well quantum computing is suitable for solving civil engineering problems and how much impact and implications to the field of civil engineering can be expected when deploying quantum computing for solving these problems. To address these questions, we will, in this paper, first introduce the fundamentals of quantum computing. Thereupon, we will analyze the problem classes to elucidate where quantum computing holds the potential to outperform traditional computers and, focusing on the limitations, where quantum computing is not considered the most suitable solution. Finally, we will review common complex computation use cases in civil engineering and evaluate the potential and the limitations of being improved by quantum computing. △ Less

Submitted 28 March, 2024; v1 submitted 22 February, 2024; originally announced February 2024.

Comments: accepted at ICCCBE 2024

LTL learning on GPUs

Authors: Mojtaba Valizadeh, Nathanaël Fijalkow, Martin Berger

Abstract: Linear temporal logic (LTL) is widely used in industrial verification. LTL formulae can be learned from traces. Scaling LTL formula learning is an open problem. We implement the first GPU-based LTL learner using a novel form of enumerative program synthesis. The learner is sound and complete. Our benchmarks indicate that it handles traces at least 2048 times more numerous, and on average at least… ▽ More Linear temporal logic (LTL) is widely used in industrial verification. LTL formulae can be learned from traces. Scaling LTL formula learning is an open problem. We implement the first GPU-based LTL learner using a novel form of enumerative program synthesis. The learner is sound and complete. Our benchmarks indicate that it handles traces at least 2048 times more numerous, and on average at least 46 times faster than existing state-of-the-art learners. This is achieved with, among others, novel branch-free LTL semantics that has $O(\log n)$ time complexity, where $n$ is trace length, while previous implementations are $O(n^2)$ or worse (assuming bitwise boolean operations and shifts by powers of 2 have unit costs -- a realistic assumption on modern processors). △ Less

Submitted 27 March, 2024; v1 submitted 19 February, 2024; originally announced February 2024.

Comments: 27 pages

MSC Class: 68 ACM Class: D.3

Generative AI and the History of Architecture

Authors: Joern Ploennigs, Markus Berger

Abstract: Recent generative AI platforms are able to create texts or impressive images from simple text prompts. This makes them powerful tools for summarizing knowledge about architectural history or deriving new creative work in early design tasks like ideation, sketching and modelling. But, how good is the understanding of the generative AI models of the history of architecture? Has it learned to properl… ▽ More Recent generative AI platforms are able to create texts or impressive images from simple text prompts. This makes them powerful tools for summarizing knowledge about architectural history or deriving new creative work in early design tasks like ideation, sketching and modelling. But, how good is the understanding of the generative AI models of the history of architecture? Has it learned to properly distinguish styles, or is it hallucinating information? In this chapter, we investigate this question for generative AI platforms for text and image generation for different architectural styles, to understand the capabilities and boundaries of knowledge of those tools. We also analyze how they are already being used by analyzing a data set of 101 million Midjourney queries to see if and how practitioners are already querying for specific architectural concepts. △ Less

Submitted 22 December, 2023; originally announced December 2023.

Comments: chapter to appear in Decoding Cultural Heritage with AI

ArchiGuesser -- AI Art Architecture Educational Game

Authors: Joern Ploennigs, Markus Berger, Eva Carnein

Abstract: The use of generative AI in education is a controversial topic. Current technology offers the potential to create educational content from text, speech, to images based on simple input prompts. This can enhance productivity by summarizing knowledge and improving communication, quickly adjusting to different types of learners. Moreover, generative AI holds the promise of making the learning itself… ▽ More The use of generative AI in education is a controversial topic. Current technology offers the potential to create educational content from text, speech, to images based on simple input prompts. This can enhance productivity by summarizing knowledge and improving communication, quickly adjusting to different types of learners. Moreover, generative AI holds the promise of making the learning itself more fun, by responding to user inputs and dynamically generating high-quality creative material. In this paper we present the multisensory educational game ArchiGuesser that combines various AI technologies from large language models, image generation, to computer vision to serve a single purpose: Teaching students in a playful way the diversity of our architectural history and how generative AI works. △ Less

Submitted 14 December, 2023; originally announced December 2023.

Comments: NeurIPS, Creative AI track

Modeling the Ratio of Correlated Biomarkers Using Copula Regression

Authors: Moritz Berger, Nadja Klein, Michael Wagner, Matthias Schmid

Abstract: Modeling the ratio of two dependent components as a function of covariates is a frequently pursued objective in observational research. Despite the high relevance of this topic in medical studies, where biomarker ratios are often used as surrogate endpoints for specific diseases, existing models are based on oversimplified assumptions, assuming e.g.\@ independence or strictly positive associations… ▽ More Modeling the ratio of two dependent components as a function of covariates is a frequently pursued objective in observational research. Despite the high relevance of this topic in medical studies, where biomarker ratios are often used as surrogate endpoints for specific diseases, existing models are based on oversimplified assumptions, assuming e.g.\@ independence or strictly positive associations between the components. In this paper, we close this gap in the literature and propose a regression model where the marginal distributions of the two components are linked by Frank copula. A key feature of our model is that it allows for both positive and negative correlations between the components, with one of the model parameters being directly interpretable in terms of Kendall's rank correlation coefficient. We study our method theoretically, evaluate finite sample properties in a simulation study and demonstrate its efficacy in an application to diagnosis of Alzheimer's disease via ratios of amyloid-beta and total tau protein biomarkers. △ Less

Submitted 1 December, 2023; originally announced December 2023.

Comments: 32 pages, 6 figures, 5 tables

Introduction to Field Line Helicity

Authors: A. R. Yeates, M. A. Berger

Abstract: Field line helicity measures the net linking of magnetic flux with a single magnetic field line. It offers a finer topological description than the usual global magnetic helicity integral, while still being invariant in an ideal evolution unless there is a flux of helicity through the domain boundary. In this chapter, we explore how to appropriately define field line helicity in different volumes… ▽ More Field line helicity measures the net linking of magnetic flux with a single magnetic field line. It offers a finer topological description than the usual global magnetic helicity integral, while still being invariant in an ideal evolution unless there is a flux of helicity through the domain boundary. In this chapter, we explore how to appropriately define field line helicity in different volumes in a way that preserves a meaningful topological interpretation. We also review the time evolution of field line helicity under both boundary motions and magnetic reconnection. △ Less

Submitted 15 November, 2023; originally announced November 2023.

Comments: 23 pages, 8 figures, to appear as chapter of AGU book "Helicities in Geophysics, Astrophysics and Beyond", published by Wiley, ISBN 1119841682

Detection of nonlinearity, discontinuity and interactions in generalized regression models

Authors: Nikolai Spuck, Matthias Schmid, Moritz Berger

Abstract: In generalized regression models the effect of continuous covariates is commonly assumed to be linear. This assumption, however, may be too restrictive in applications and may lead to biased effect estimates and decreased predictive ability. While a multitude of alternatives for the flexible modeling of continuous covariates have been proposed, methods that provide guidance for choosing a suitable… ▽ More In generalized regression models the effect of continuous covariates is commonly assumed to be linear. This assumption, however, may be too restrictive in applications and may lead to biased effect estimates and decreased predictive ability. While a multitude of alternatives for the flexible modeling of continuous covariates have been proposed, methods that provide guidance for choosing a suitable functional form are still limited. To address this issue, we propose a detection algorithm that evaluates several approaches for modeling continuous covariates and guides practitioners to choose the most appropriate alternative. The algorithm utilizes a unified framework for tree-structured modeling which makes the results easily interpretable. We assessed the performance of the algorithm by conducting a simulation study. To illustrate the proposed algorithm, we analyzed data of patients suffering from chronic kidney disease. △ Less

Submitted 31 October, 2023; originally announced October 2023.

Comments: 30 pages, 8 figures

Proving the Potential of Skeleton Based Action Recognition to Automate the Analysis of Manual Processes

Authors: Marlin Berger, Frederik Cloppenburg, Jens Eufinger, Thomas Gries

Abstract: In manufacturing sectors such as textiles and electronics, manual processes are a fundamental part of production. The analysis and monitoring of the processes is necessary for efficient production design. Traditional methods for analyzing manual processes are complex, expensive, and inflexible. Compared to established approaches such as Methods-Time-Measurement (MTM), machine learning (ML) methods… ▽ More In manufacturing sectors such as textiles and electronics, manual processes are a fundamental part of production. The analysis and monitoring of the processes is necessary for efficient production design. Traditional methods for analyzing manual processes are complex, expensive, and inflexible. Compared to established approaches such as Methods-Time-Measurement (MTM), machine learning (ML) methods promise: Higher flexibility, self-sufficient & permanent use, lower costs. In this work, based on a video stream, the current motion class in a manual assembly process is detected. With information on the current motion, Key-Performance-Indicators (KPIs) can be derived easily. A skeleton-based action recognition approach is taken, as this field recently shows major success in machine vision tasks. For skeleton-based action recognition in manual assembly, no sufficient pre-work could be found. Therefore, a ML pipeline is developed, to enable extensive research on different (pre-) processing methods and neural nets. Suitable well generalizing approaches are found, proving the potential of ML to enhance analyzation of manual processes. Models detect the current motion, performed by an operator in manual assembly, but the results can be transferred to all kinds of manual processes. △ Less

Submitted 12 October, 2023; originally announced October 2023.

Comments: 16 pages, 6 figures. Find peer-reviewed version in Proceedings of IntelliSys 2023

Distribution-free risk assessment of regression-based machine learning algorithms

Authors: Sukrita Singh, Neeraj Sarna, Yuanyuan Li, Yang Li, Agni Orfanoudaki, Michael Berger

Abstract: Machine learning algorithms have grown in sophistication over the years and are increasingly deployed for real-life applications. However, when using machine learning techniques in practical settings, particularly in high-risk applications such as medicine and engineering, obtaining the failure probability of the predictive model is critical. We refer to this problem as the risk-assessment task. W… ▽ More Machine learning algorithms have grown in sophistication over the years and are increasingly deployed for real-life applications. However, when using machine learning techniques in practical settings, particularly in high-risk applications such as medicine and engineering, obtaining the failure probability of the predictive model is critical. We refer to this problem as the risk-assessment task. We focus on regression algorithms and the risk-assessment task of computing the probability of the true label lying inside an interval defined around the model's prediction. We solve the risk-assessment problem using the conformal prediction approach, which provides prediction intervals that are guaranteed to contain the true label with a given probability. Using this coverage property, we prove that our approximated failure probability is conservative in the sense that it is not lower than the true failure probability of the ML algorithm. We conduct extensive experiments to empirically study the accuracy of the proposed method for problems with and without covariate shift. Our analysis focuses on different modeling regimes, dataset sizes, and conformal prediction methodologies. △ Less

Submitted 5 October, 2023; originally announced October 2023.

A new provably stable weighted state redistribution algorithm

Authors: Marsha Berger, Andrew Giuliani

Abstract: We propose a practical finite volume method on cut cells using state redistribution. Our algorithm is provably monotone, total variation diminishing, and GKS stable in many situations, and shuts off continuously as the cut cell size approaches a target value. Our analysis reveals why original state redistribution works so well: it results in a monotone scheme for most configurations, though at tim… ▽ More We propose a practical finite volume method on cut cells using state redistribution. Our algorithm is provably monotone, total variation diminishing, and GKS stable in many situations, and shuts off continuously as the cut cell size approaches a target value. Our analysis reveals why original state redistribution works so well: it results in a monotone scheme for most configurations, though at times subject to a slightly smaller CFL condition. Our analysis also explains why a pre-merging step is beneficial. We show computational experiments in two and three dimensions. △ Less

Submitted 2 April, 2024; v1 submitted 30 August, 2023; originally announced August 2023.

Extending Cross-Modal Retrieval with Interactive Learning to Improve Image Retrieval Performance in Forensics

Authors: Nils Böhne, Mark Berger, Ronald van Velzen

Abstract: Nowadays, one of the critical challenges in forensics is analyzing the enormous amounts of unstructured digital evidence, such as images. Often, unstructured digital evidence contains precious information for forensic investigations. Therefore, a retrieval system that can effectively identify forensically relevant images is paramount. In this work, we explored the effectiveness of interactive lear… ▽ More Nowadays, one of the critical challenges in forensics is analyzing the enormous amounts of unstructured digital evidence, such as images. Often, unstructured digital evidence contains precious information for forensic investigations. Therefore, a retrieval system that can effectively identify forensically relevant images is paramount. In this work, we explored the effectiveness of interactive learning in improving image retrieval performance in the forensic domain by proposing Excalibur - a zero-shot cross-modal image retrieval system extended with interactive learning. Excalibur was evaluated using both simulations and a user study. The simulations reveal that interactive learning is highly effective in improving retrieval performance in the forensic domain. Furthermore, user study participants could effectively leverage the power of interactive learning. Finally, they considered Excalibur effective and straightforward to use and expressed interest in using it in their daily practice. △ Less

Submitted 28 August, 2023; originally announced August 2023.

Comments: Submitted to the AAAI22 conference

Correct and Optimal: the Regular Expression Inference Challenge

Authors: Mojtaba Valizadeh, Philip John Gorinski, Ignacio Iacobacci, Martin Berger

Abstract: We propose regular expression inference (REI) as a challenge for code/language modelling, and the wider machine learning community. REI is a supervised machine learning (ML) and program optimisation task, and poses the problem of finding minimal regular expressions from examples: Given two finite sets of strings $P$ and $N$ and a cost function $cost(\cdot)$, the task is to generate an expression… ▽ More We propose regular expression inference (REI) as a challenge for code/language modelling, and the wider machine learning community. REI is a supervised machine learning (ML) and program optimisation task, and poses the problem of finding minimal regular expressions from examples: Given two finite sets of strings $P$ and $N$ and a cost function $cost(\cdot)$, the task is to generate an expression $r$ that accepts all strings in $P$ and rejects all strings in $N$, while no other such expression $r'$ exists with $cost(r')<cost(r)$. REI has advantages as a challenge problem: (i) regular expressions are well-known, widely used, and a natural idealisation of code; (ii) REI's asymptotic worst-case complexity is well understood; (iii) REI has a small number of easy to understand parameters (e.g. $P$ or $N$ cardinality, string lengths of examples, or the cost function); this lets us easily finetune REI-hardness; (iv) REI, with its emphasis on optimisation, is an unsolved problem for deep learning based ML. Recently, an REI solver was implemented on GPUs, using program synthesis techniques. This enabled, for the first time, fast generation of minimal regular expressions for complex REI instances. Building on this advance, we generate and publish the first large-scale datasets for REI, and devise and evaluate several initial heuristic and machine learning baselines. We invite the community to participate and explore ML methods that learn to solve REI problems. We believe that progress in REI directly translates to progress in code/language modelling. △ Less

Submitted 10 May, 2024; v1 submitted 15 August, 2023; originally announced August 2023.

Comments: accepted to IJCAI 2024; 7 pages with 2 tables, 5 pages appendix with 3 tables, 1 figure

Multi-band metasurface-driven surface-enhanced infrared absorption spectroscopy for improved characterization of in-situ electrochemical reactions

Authors: Malo Duportal, Luca M. Berger, Stefan A. Maier, Andreas Tittl, Katharina Krischer

Abstract: Surface-enhanced spectroscopy techniques are the method-of-choice to characterize adsorbed intermediates occurring during electrochemical reactions, which are crucial in realizing a green sustainable future. Characterizing species with low coverages or short lifetimes have so far been limited by low signal enhancement. Recently, metasurface-driven surface-enhanced infrared absorption spectroscopy… ▽ More Surface-enhanced spectroscopy techniques are the method-of-choice to characterize adsorbed intermediates occurring during electrochemical reactions, which are crucial in realizing a green sustainable future. Characterizing species with low coverages or short lifetimes have so far been limited by low signal enhancement. Recently, metasurface-driven surface-enhanced infrared absorption spectroscopy (SEIRAS) has been pioneered as a promising narrowband technology to study single vibrational modes of electrochemical interfaces during CO oxidation. However, many reactions involve several species or configurations of adsorption that need to be monitored simultaneously requiring reproducible and broadband sensing platforms to provide a clear understanding of the underlying electrochemical processes. Here, we experimentally realize multi-band metasurface-driven SEIRAS for the in-situ study of electrochemical CO2 reduction on a Pt surface. We develop an easily reproducible and spectrally-tunable platinum nano-slot metasurface. Two CO adsorption configurations at 2030 cm-1 and 1840 cm-1 are locally enhanced as a proof of concept that can be extended to more vibrational bands. Our platform provides a 41-fold enhancement in the detection of characteristic absorption signals compared to conventional broadband electrochemically roughened platinum films. A straightforward methodology is outlined starting by baselining our system in CO saturated environment and clearly detecting both configurations of adsorption, in particular the hitherto hardly detectable CO bridge configuration. Then, thanks to the signal enhancement provided by our platform, we find that the CO bridge configuration on platinum does not play a significant role during CO2 reduction in an alkaline environment. We anticipate that our technology will guide researchers in developing similar sensing platforms. △ Less

Submitted 20 July, 2023; originally announced July 2023.

Comments: 21 pages, 4 figures

Implicit Adaptive Mesh Refinement for Dispersive Tsunami Propagation

Authors: Marsha J. Berger, Randall J. LeVeque

Abstract: We present an algorithm to solve the dispersive depth-averaged Serre-Green-Naghdi (SGN) equations using patch-based adaptive mesh refinement. These equations require adding additional higher derivative terms to the nonlinear shallow water equations. This has been implemented as a new component of the open source GeoClaw software that is widely used for modeling tsunamis, storm surge, and related h… ▽ More We present an algorithm to solve the dispersive depth-averaged Serre-Green-Naghdi (SGN) equations using patch-based adaptive mesh refinement. These equations require adding additional higher derivative terms to the nonlinear shallow water equations. This has been implemented as a new component of the open source GeoClaw software that is widely used for modeling tsunamis, storm surge, and related hazards, improving its accuracy on shorter wavelength phenomena. We use a formulation that requires solving an elliptic system of equations at each time step, making the method implicit. The adaptive algorithm allows different time steps on different refinement levels, and solves the implicit equations level by level. Computational examples are presented to illustrate the stability and accuracy on a radially symmetric test case and two realistic tsunami modeling problems, including a hypothetical asteroid impact creating a short wavelength tsunami for which dispersive terms are necessary. △ Less

Submitted 19 March, 2024; v1 submitted 11 July, 2023; originally announced July 2023.

Comments: 25 pages, 16 figures, submitted for publication (revised)

MSC Class: 65M50; 65M08; 86A15

Automating Computational Design with Generative AI

Authors: Joern Ploennigs, Markus Berger

Abstract: AI image generators based on diffusion models have recently garnered attention for their capability to create images from simple text prompts. However, for practical use in civil engineering they need to be able to create specific construction plans for given constraints. This paper investigates the potential of current AI generators in addressing such challenges, specifically for the creation of… ▽ More AI image generators based on diffusion models have recently garnered attention for their capability to create images from simple text prompts. However, for practical use in civil engineering they need to be able to create specific construction plans for given constraints. This paper investigates the potential of current AI generators in addressing such challenges, specifically for the creation of simple floor plans. We explain how the underlying diffusion-models work and propose novel refinement approaches to improve semantic encoding and generation quality. In several experiments we show that we can improve validity of generated floor plans from 6% to 90%. Based on these results we derive future research challenges considering building information modelling. With this we provide: (i) evaluation of current generative AIs; (ii) propose improved refinement approaches; (iii) evaluate them on various examples; (iv) derive future directions for diffusion models in civil engineering. △ Less

Submitted 3 May, 2024; v1 submitted 5 July, 2023; originally announced July 2023.

Survival of the flattest in the quasispecies model

Authors: Maxime Berger, Raphaël Cerf

Abstract: Viruses present an amazing genetic variability. An ensemble of infecting viruses, also called a viral quasispecies, is a cloud of mutants centered around a specific genotype. The simplest model of evolution, whose equilibrium state is described by the quasispecies equation, is the Moran--Kingman model. For the sharp peak landscape, we perform several exact computations and we derive several exact… ▽ More Viruses present an amazing genetic variability. An ensemble of infecting viruses, also called a viral quasispecies, is a cloud of mutants centered around a specific genotype. The simplest model of evolution, whose equilibrium state is described by the quasispecies equation, is the Moran--Kingman model. For the sharp peak landscape, we perform several exact computations and we derive several exact formulas. We obtain also an exact formula for the quasispecies distribution, involving a series and the mean fitness. A very simple formula for the mean Hamming distance is derived, which is exact and which do not require a specific asymptotic expansion (like sending the length of the macromolecules to $\infty$ or the mutation probability to $0$). We try also to extend these formulas to a general fitness landscape. We obtain an equation involving the covariance of the fitness and the Hamming class number in the quasispecies distribution. With the help of these formulas, we discuss the phenomenon of the error threshold and the notion of quasispecies. We recover the limiting quasipecies distribution in the long chain regime. We go beyond the sharp peak landscape and we consider fitness landscapes having finitely many peaks and a plateau--type landscape. We finally prove rigorously within this framework the possible occurrence of the survival of the flattest, a phenomenon which has been previously discovered by Wilke, Wang, Ofria, Lenski and Adami and which has been investigated in several works. △ Less

Submitted 20 August, 2024; v1 submitted 15 June, 2023; originally announced June 2023.

From dense to sparse design: Optimal rates under the supremum norm for estimating the mean function in functional data analysis

Authors: Max Berger, Philipp Hermann, Hajo Holzmann

Abstract: We derive optimal rates of convergence in the supremum norm for estimating the Hölder-smooth mean function of a stochastic process which is repeatedly and discretely observed with additional errors at fixed, multivariate, synchronous design points, the typical scenario for machine recorded functional data. Similarly to the optimal rates in $L_2$ obtained in \citet{cai2011optimal}, for sparse desig… ▽ More We derive optimal rates of convergence in the supremum norm for estimating the Hölder-smooth mean function of a stochastic process which is repeatedly and discretely observed with additional errors at fixed, multivariate, synchronous design points, the typical scenario for machine recorded functional data. Similarly to the optimal rates in $L_2$ obtained in \citet{cai2011optimal}, for sparse design a discretization term dominates, while in the dense case the parametric $\sqrt n$ rate can be achieved as if the $n$ processes were continuously observed without errors. The supremum norm is of practical interest since it corresponds to the visualization of the estimation error, and forms the basis for the construction uniform confidence bands. We show that in contrast to the analysis in $L_2$, there is an intermediate regime between the sparse and dense cases dominated by the contribution of the observation errors. Furthermore, under the supremum norm interpolation estimators which suffice in $L_2$ turn out to be sub-optimal in the dense setting, which helps to explain their poor empirical performance. In contrast to previous contributions involving the supremum norm, we discuss optimality even in the multivariate setting, and for dense design obtain the $\sqrt n$ rate of convergence without additional logarithmic factors. We also obtain a central limit theorem in the supremum norm, and provide simulations and real data applications to illustrate our results. △ Less

Submitted 8 May, 2024; v1 submitted 7 June, 2023; originally announced June 2023.

MSC Class: 62G05 (Primary) 62M09 (Secondary)

Metallic and All-Dielectric Metasurfaces Sustaining Displacement-Mediated Bound States in the Continuum

Authors: Luca M. Berger, Martin Barkey, Stefan A. Maier, Andreas Tittl

Abstract: Bound states in the continuum (BICs) are localized electromagnetic modes within the continuous spectrum of radiating waves. Due to their infinite lifetimes without radiation losses, BICs are driving research directions in lasing, non-linear optical processes, and sensing. However, conventional methods for converting BICs into leaky resonances, or quasi-BICs, with high-quality factors typically rel… ▽ More Bound states in the continuum (BICs) are localized electromagnetic modes within the continuous spectrum of radiating waves. Due to their infinite lifetimes without radiation losses, BICs are driving research directions in lasing, non-linear optical processes, and sensing. However, conventional methods for converting BICs into leaky resonances, or quasi-BICs, with high-quality factors typically rely on breaking the in-plane inversion symmetry of the metasurface and often result in resonances that are strongly dependent on the angle of the incident light, making them unsuitable for many practical applications. Here, we numerically analyze and experimentally demonstrate an emerging class of BIC-driven metasurfaces, where the coupling to the far field is controlled by the displacement of individual resonators. In particular, we investigate both all-dielectric and metallic as well as positive and inverse displacement-mediated metasurfaces sustaining angular-robust quasi-BICs in the mid-infrared spectral region. We explore their behavior with changes in the incidence angle of illumination and experimentally show their superior performance compared to two conventional alternatives: silicon-based tilted ellipses and cylindrical nanoholes in gold. We anticipate our findings to open exciting perspectives for bio-sensing, conformal optical devices, and photonic devices using focused light. △ Less

Submitted 1 June, 2023; originally announced June 2023.

Comments: 27 pages, 7 figures, 1 table

Search-Based Regular Expression Inference on a GPU

Authors: Mojtaba Valizadeh, Martin Berger

Abstract: Regular expression inference (REI) is a supervised machine learning and program synthesis problem that takes a cost metric for regular expressions, and positive and negative examples of strings as input. It outputs a regular expression that is precise (i.e., accepts all positive and rejects all negative examples), and minimal w.r.t. to the cost metric. We present a novel algorithm for REI over arb… ▽ More Regular expression inference (REI) is a supervised machine learning and program synthesis problem that takes a cost metric for regular expressions, and positive and negative examples of strings as input. It outputs a regular expression that is precise (i.e., accepts all positive and rejects all negative examples), and minimal w.r.t. to the cost metric. We present a novel algorithm for REI over arbitrary alphabets that is enumerative and trades off time for space. Our main algorithmic idea is to implement the search space of regular expressions succinctly as a contiguous matrix of bitvectors. Collectively, the bitvectors represent, as characteristic sequences, all sub-languages of the infix-closure of the union of positive and negative examples. Mathematically, this is a semiring of (a variant of) formal power series. Infix-closure enables bottom-up compositional construction of larger from smaller regular expressions using the operations of our semiring. This minimises data movement and data-dependent branching, hence maximises data-parallelism. In addition, the infix-closure remains unchanged during the search, hence search can be staged: first pre-compute various expensive operations, and then run the compute intensive search process. We provide two C++ implementations, one for general purpose CPUs and one for Nvidia GPUs (using CUDA). We benchmark both on Google Colab Pro: the GPU implementation is on average over 1000x faster than the CPU implementation on the hardest benchmarks. △ Less

Submitted 29 May, 2023; originally announced May 2023.

ACM Class: D.3

Segmentation of glioblastomas in early post-operative multi-modal MRI with deep neural networks

Authors: Ragnhild Holden Helland, Alexandros Ferles, André Pedersen, Ivar Kommers, Hilko Ardon, Frederik Barkhof, Lorenzo Bello, Mitchel S. Berger, Tora Dunås, Marco Conti Nibali, Julia Furtner, Shawn Hervey-Jumper, Albert J. S. Idema, Barbara Kiesel, Rishi Nandoe Tewari, Emmanuel Mandonnet, Domenique M. J. Müller, Pierre A. Robe, Marco Rossi, Lisa M. Sagberg, Tommaso Sciortino, Tom Aalders, Michiel Wagemakers, Georg Widhalm, Marnix G. Witte , et al. (8 additional authors not shown)

Abstract: Extent of resection after surgery is one of the main prognostic factors for patients diagnosed with glioblastoma. To achieve this, accurate segmentation and classification of residual tumor from post-operative MR images is essential. The current standard method for estimating it is subject to high inter- and intra-rater variability, and an automated method for segmentation of residual tumor in ear… ▽ More Extent of resection after surgery is one of the main prognostic factors for patients diagnosed with glioblastoma. To achieve this, accurate segmentation and classification of residual tumor from post-operative MR images is essential. The current standard method for estimating it is subject to high inter- and intra-rater variability, and an automated method for segmentation of residual tumor in early post-operative MRI could lead to a more accurate estimation of extent of resection. In this study, two state-of-the-art neural network architectures for pre-operative segmentation were trained for the task. The models were extensively validated on a multicenter dataset with nearly 1000 patients, from 12 hospitals in Europe and the United States. The best performance achieved was a 61\% Dice score, and the best classification performance was about 80\% balanced accuracy, with a demonstrated ability to generalize across hospitals. In addition, the segmentation performance of the best models was on par with human expert raters. The predicted segmentations can be used to accurately classify the patients into those with residual tumor, and those with gross total resection. △ Less

Submitted 18 April, 2023; originally announced April 2023.

Comments: 13 pages, 4 figures, 4 tables

ACM Class: I.4.6; J.3

Synchronous replication initiation of multiple origins

Authors: Mareike Berger, Pieter Rein ten Wolde

Abstract: Initiating replication synchronously at multiple origins of replication allows the bacterium Escherichia coli to divide even faster than the time it takes to replicate the entire chromosome in nutrient-rich environments. What mechanisms give rise to synchronous replication initiation remains however poorly understood. Via mathematical modelling, we identify four distinct synchronization regimes de… ▽ More Initiating replication synchronously at multiple origins of replication allows the bacterium Escherichia coli to divide even faster than the time it takes to replicate the entire chromosome in nutrient-rich environments. What mechanisms give rise to synchronous replication initiation remains however poorly understood. Via mathematical modelling, we identify four distinct synchronization regimes depending on two quantities: the duration of the so-called licensing period during which the initiation potential in the cell remains high after the first origin has fired and the duration of the blocking period during which already initiated origins remain blocked. For synchronous replication initiation, the licensing period must be long enough such that all origins can be initiated, but shorter than the blocking period to prevent reinitiation of origins that have already fired. We find an analytical expression for the degree of synchrony as a function of the duration of the licensing period, which we confirm by simulations. Our model reveals that the delay between the firing of the first and the last origin scales with the coefficient of variation (CV) of the initiation volume. Matching these to the values measured experimentally shows that the firing rate must rise with the cell volume with an effective Hill coefficient that is at least 20; the probability that all origins fire before the blocking period is over is then at least 92%. Our analysis thus reveals that the low CV of the initiation volume is a consequence of synchronous replication initiation. Finally, we show that the previously presented molecular model for the regulation of replication initiation in E. coli can give rise to synchronous replication initiation for biologically realistic parameters. △ Less

Submitted 3 April, 2023; originally announced April 2023.

Comments: 22 pages, 12 figures

Artificial-intelligence-based molecular classification of diffuse gliomas using rapid, label-free optical imaging

Authors: Todd C. Hollon, Cheng Jiang, Asadur Chowdury, Mustafa Nasir-Moin, Akhil Kondepudi, Alexander Aabedi, Arjun Adapa, Wajd Al-Holou, Jason Heth, Oren Sagher, Pedro Lowenstein, Maria Castro, Lisa Irina Wadiura, Georg Widhalm, Volker Neuschmelting, David Reinecke, Niklas von Spreckelsen, Mitchel S. Berger, Shawn L. Hervey-Jumper, John G. Golfinos, Matija Snuderl, Sandra Camelo-Piragua, Christian Freudiger, Honglak Lee, Daniel A. Orringer

Abstract: Molecular classification has transformed the management of brain tumors by enabling more accurate prognostication and personalized treatment. However, timely molecular diagnostic testing for patients with brain tumors is limited, complicating surgical and adjuvant treatment and obstructing clinical trial enrollment. In this study, we developed DeepGlioma, a rapid ($< 90$ seconds), artificial-intel… ▽ More Molecular classification has transformed the management of brain tumors by enabling more accurate prognostication and personalized treatment. However, timely molecular diagnostic testing for patients with brain tumors is limited, complicating surgical and adjuvant treatment and obstructing clinical trial enrollment. In this study, we developed DeepGlioma, a rapid ($< 90$ seconds), artificial-intelligence-based diagnostic screening system to streamline the molecular diagnosis of diffuse gliomas. DeepGlioma is trained using a multimodal dataset that includes stimulated Raman histology (SRH); a rapid, label-free, non-consumptive, optical imaging method; and large-scale, public genomic data. In a prospective, multicenter, international testing cohort of patients with diffuse glioma ($n=153$) who underwent real-time SRH imaging, we demonstrate that DeepGlioma can predict the molecular alterations used by the World Health Organization to define the adult-type diffuse glioma taxonomy (IDH mutation, 1p19q co-deletion and ATRX mutation), achieving a mean molecular classification accuracy of $93.3\pm 1.6\%$. Our results represent how artificial intelligence and optical histology can be used to provide a rapid and scalable adjunct to wet lab methods for the molecular screening of patients with diffuse glioma. △ Less

Submitted 23 March, 2023; originally announced March 2023.

Comments: Paper published in Nature Medicine

A modest proposal: explicit support for foundational pluralism

Authors: Martin Berger, Dominic P. Mulligan

Abstract: Whilst mathematicians assume classical reasoning principles by default they often context switch when working, restricting themselves to various forms of subclassical reasoning. This pattern is especially common amongst logicians and set theorists, but workaday mathematicians also commonly do this too, witnessed by narrative notes accompanying a proof -- "the following proof is constructive", or "… ▽ More Whilst mathematicians assume classical reasoning principles by default they often context switch when working, restricting themselves to various forms of subclassical reasoning. This pattern is especially common amongst logicians and set theorists, but workaday mathematicians also commonly do this too, witnessed by narrative notes accompanying a proof -- "the following proof is constructive", or "the following proof does not use choice", for example. Yet, current proof assistants provide poor support for capturing these narrative notes formally, an observation that is especially true of systems based on Gordon's HOL, a classical higher-order logic. Consequently, HOL and its many implementations seem ironically more committed to classical reasoning than mainstream mathematicians are themselves, limiting the mathematical content that one may easily formalise. To facilitate these context switches, we propose that mathematicians mentally employ a simple tainting system when temporarily working subclassically -- an idea not currently explored in proof assistants. We introduce a series of modest but far-reaching changes to HOL, extending the standard two-place Natural Deduction relation to incorporate a taint-label, taken from a particular lattice, and which describes or limits the "amount" of classical reasoning used within a proof. Taint can be seen either as a simple typing system on HOL proofs, or as a form of static analysis on proof trees, and partitions our logic into various fragments of differing expressivity, sitting side-by-side. Results may pass from a "less classical" fragment into a "more classical" fragment of the logic without modification, but not vice versa, with the flow of results between worlds controlled by an inference rule akin to a subtyping or subsumption rule. △ Less

Submitted 20 February, 2023; originally announced February 2023.

Comments: 18 pages

Classifying Linear Matrix Inequalities via Abstract Operator Systems

Authors: Martin Berger, Tom Drescher, Tim Netzer

Abstract: We systematically study how properties of abstract operator systems help classifying linear matrix inequality definitions of sets. Our main focus is on polyhedral cones, the 3-dimensional Lorentz cone, where we can completely describe all defining linear matrix inequalities, and on the cone of positive semidefinite matrices. Here we use results on isometries between matrix algebras to describe lin… ▽ More We systematically study how properties of abstract operator systems help classifying linear matrix inequality definitions of sets. Our main focus is on polyhedral cones, the 3-dimensional Lorentz cone, where we can completely describe all defining linear matrix inequalities, and on the cone of positive semidefinite matrices. Here we use results on isometries between matrix algebras to describe linear matrix inequality definitions of relatively small size. We conversely use the theory of operator systems to characterize special such isometries. △ Less

Submitted 26 January, 2023; originally announced January 2023.

Improved in-situ characterization of electrochemical interfaces using metasurface-driven surface-enhanced infrared absorption spectroscopy

Authors: Luca M. Berger, Malo Duportal, Leonardo de Souza Menezes, Emiliano Cortes, Stefan A. Maier, Andreas Tittl, Katharina Krischer

Abstract: Electrocatalysis plays a crucial role in realizing the transition towards green energy, driving research directions from hydrogen generation to carbon dioxide reduction. Understanding electrochemical reactions is crucial to improve their efficiency and to bridge the gap toward a sustainable zero-carbon future. Surface-enhanced infrared absorption spectroscopy (SEIRAS) is a suitable method for inve… ▽ More Electrocatalysis plays a crucial role in realizing the transition towards green energy, driving research directions from hydrogen generation to carbon dioxide reduction. Understanding electrochemical reactions is crucial to improve their efficiency and to bridge the gap toward a sustainable zero-carbon future. Surface-enhanced infrared absorption spectroscopy (SEIRAS) is a suitable method for investigating these processes because it can monitor with chemical specificity the mechanisms of the reactions. However, it remains difficult to detect many relevant aspects of electrochemical reactions such as short-lived intermediates. Here, we develop and experimentally realize an integrated nanophotonic-electrochemical SEIRAS platform for the in situ investigation of molecular signal traces emerging during electrochemical experiments. Specifically, we implement a platinum nano-slot metasurface featuring strongly enhanced electromagnetic near fields and spectrally target it at the weak vibrational bending mode of adsorbed CO at ~2033 cm-1. Crucially, our platinum nano-slot metasurface provides high molecular sensitivity. The resonances can be tuned over a broad range in the mid-infrared spectrum. Compared to conventional unstructured platinum layers, our nanophotonic-electrochemical platform delivers a substantial improvement of the experimentally detected characteristic absorption signals by a factor of 27, enabling the detection of new species with weak signals, fast conversions, or low surface concentrations. By providing a deeper understanding of catalytic reactions, we anticipate our nanophotonic-electrochemical platform to open exciting perspectives for electrochemical SEIRAS, surface-enhanced Raman spectroscopy, and the study of reactions in other fields of chemistry such as photoelectrocatalysis. △ Less

Submitted 5 January, 2023; originally announced January 2023.

Comments: Main text and supporting information, 28 pages, 4 figures + 1 supporting figure

AI Art in Architecture

Authors: Joern Ploennigs, Markus Berger

Abstract: Recent diffusion-based AI art platforms are able to create impressive images from simple text descriptions. This makes them powerful tools for concept design in any discipline that requires creativity in visual design tasks. This is also true for early stages of architectural design with multiple stages of ideation, sketching and modelling. In this paper, we investigate how applicable diffusion-ba… ▽ More Recent diffusion-based AI art platforms are able to create impressive images from simple text descriptions. This makes them powerful tools for concept design in any discipline that requires creativity in visual design tasks. This is also true for early stages of architectural design with multiple stages of ideation, sketching and modelling. In this paper, we investigate how applicable diffusion-based models already are to these tasks. We research the applicability of the platforms Midjourney, DALL-E 2 and StableDiffusion to a series of common use cases in architectural design to determine which are already solvable or might soon be. We also analyze how they are already being used by analyzing a data set of 40 million Midjourney queries with NLP methods to extract common usage patterns. With this insights we derived a workflow to interior and exterior design that combines the strengths of the individual platforms. △ Less

Submitted 19 December, 2022; originally announced December 2022.

Journal ref: AI Civ. Eng. 2, 8 (2023)

ALARM: Active LeArning of Rowhammer Mitigations

Authors: Amir Naseredini, Martin Berger, Matteo Sammartino, Shale Xiong

Abstract: Rowhammer is a serious security problem of contemporary dynamic random-access memory (DRAM) where reads or writes of bits can flip other bits. DRAM manufacturers add mitigations, but don't disclose details, making it difficult for customers to evaluate their efficacy. We present a tool, based on active learning, that automatically infers parameter of Rowhammer mitigations against synthetic models… ▽ More Rowhammer is a serious security problem of contemporary dynamic random-access memory (DRAM) where reads or writes of bits can flip other bits. DRAM manufacturers add mitigations, but don't disclose details, making it difficult for customers to evaluate their efficacy. We present a tool, based on active learning, that automatically infers parameter of Rowhammer mitigations against synthetic models of modern DRAM. △ Less

Submitted 30 November, 2022; originally announced November 2022.

Towards an objective characterization of an individual's facial movements using Self-Supervised Person-Specific-Models

Authors: Yanis Tazi, Michael Berger, Winrich A. Freiwald

Abstract: Disentangling facial movements from other facial characteristics, particularly from facial identity, remains a challenging task, as facial movements display great variation between individuals. In this paper, we aim to characterize individual-specific facial movements. We present a novel training approach to learn facial movements independently of other facial characteristics, focusing on each ind… ▽ More Disentangling facial movements from other facial characteristics, particularly from facial identity, remains a challenging task, as facial movements display great variation between individuals. In this paper, we aim to characterize individual-specific facial movements. We present a novel training approach to learn facial movements independently of other facial characteristics, focusing on each individual separately. We propose self-supervised Person-Specific Models (PSMs), in which one model per individual can learn to extract an embedding of the facial movements independently of the person's identity and other structural facial characteristics from unlabeled facial video. These models are trained using encoder-decoder-like architectures. We provide quantitative and qualitative evidence that a PSM learns a meaningful facial embedding that discovers fine-grained movements otherwise not characterized by a General Model (GM), which is trained across individuals and characterizes general patterns of facial movements. We present quantitative and qualitative evidence that this approach is easily scalable and generalizable for new individuals: facial movements knowledge learned on a person can quickly and effectively be transferred to a new person. Lastly, we propose a novel PSM using curriculum temporal learning to leverage the temporal contiguity between video frames. Our code, analysis details, and all pretrained models are available in Github and Supplementary Materials. △ Less

Submitted 15 November, 2022; originally announced November 2022.

Comments: 9 pages for main text with 6 figures + 5 supplementary figures

Integration-free Learning of Flow Maps

Authors: Saroj Sahoo, Matthew Berger

Abstract: We present a method for learning neural representations of flow maps from time-varying vector field data. The flow map is pervasive within the area of flow visualization, as it is foundational to numerous visualization techniques, e.g. integral curve computation for pathlines or streaklines, as well as computing separation/attraction structures within the flow field. Yet bottlenecks in flow map co… ▽ More We present a method for learning neural representations of flow maps from time-varying vector field data. The flow map is pervasive within the area of flow visualization, as it is foundational to numerous visualization techniques, e.g. integral curve computation for pathlines or streaklines, as well as computing separation/attraction structures within the flow field. Yet bottlenecks in flow map computation, namely the numerical integration of vector fields, can easily inhibit their use within interactive visualization settings. In response, in our work we seek neural representations of flow maps that are efficient to evaluate, while remaining scalable to optimize, both in computation cost and data requirements. A key aspect of our approach is that we can frame the process of representation learning not in optimizing for samples of the flow map, but rather, a self-consistency criterion on flow map derivatives that eliminates the need for flow map samples, and thus numerical integration, altogether. Central to realizing this is a novel neural network design for flow maps, coupled with an optimization scheme, wherein our representation only requires the time-varying vector field for learning, encoded as instantaneous velocity. We show the benefits of our method over prior works in terms of accuracy and efficiency across a range of 2D and 3D time-varying vector fields, while showing how our neural representation of flow maps can benefit unsteady flow visualization techniques such as streaklines, and the finite-time Lyapunov exponent. △ Less

Submitted 25 March, 2023; v1 submitted 6 November, 2022; originally announced November 2022.

Cluster-Based Autoencoders for Volumetric Point Clouds

Authors: Stephan Antholzer, Martin Berger, Tobias Hell

Abstract: Autoencoders allow to reconstruct a given input from a small set of parameters. However, the input size is often limited due to computational costs. We therefore propose a clustering and reassembling method for volumetric point clouds, in order to allow high resolution data as input. We furthermore present an autoencoder based on the well-known FoldingNet for volumetric point clouds and discuss ho… ▽ More Autoencoders allow to reconstruct a given input from a small set of parameters. However, the input size is often limited due to computational costs. We therefore propose a clustering and reassembling method for volumetric point clouds, in order to allow high resolution data as input. We furthermore present an autoencoder based on the well-known FoldingNet for volumetric point clouds and discuss how our approach can be utilized for blending between high resolution point clouds as well as for transferring a volumetric design/style onto a pointcloud while maintaining its shape. △ Less

Submitted 2 November, 2022; originally announced November 2022.

OA-SLAM: Leveraging Objects for Camera Relocalization in Visual SLAM

Authors: Matthieu Zins, Gilles Simon, Marie-Odile Berger

Abstract: In this work, we explore the use of objects in Simultaneous Localization and Mapping in unseen worlds and propose an object-aided system (OA-SLAM). More precisely, we show that, compared to low-level points, the major benefit of objects lies in their higher-level semantic and discriminating power. Points, on the contrary, have a better spatial localization accuracy than the generic coarse models u… ▽ More In this work, we explore the use of objects in Simultaneous Localization and Mapping in unseen worlds and propose an object-aided system (OA-SLAM). More precisely, we show that, compared to low-level points, the major benefit of objects lies in their higher-level semantic and discriminating power. Points, on the contrary, have a better spatial localization accuracy than the generic coarse models used to represent objects (cuboid or ellipsoid). We show that combining points and objects is of great interest to address the problem of camera pose recovery. Our main contributions are: (1) we improve the relocalization ability of a SLAM system using high-level object landmarks; (2) we build an automatic system, capable of identifying, tracking and reconstructing objects with 3D ellipsoids; (3) we show that object-based localization can be used to reinitialize or resume camera tracking. Our fully automatic system allows on-the-fly object mapping and enhanced pose tracking recovery, which we think, can significantly benefit to the AR community. Our experiments show that the camera can be relocalized from viewpoints where classical methods fail. We demonstrate that this localization allows a SLAM system to continue working despite a tracking loss, which can happen frequently with an uninitiated user. Our code and test data are released at gitlab.inria.fr/tangram/oa-slam. △ Less

Submitted 17 September, 2022; originally announced September 2022.

Comments: ISMAR 2022

Perspective-1-Ellipsoid: Formulation, Analysis and Solutions of the Camera Pose Estimation Problem from One Ellipse-Ellipsoid Correspondence

Authors: Vincent Gaudillière, Gilles Simon, Marie-Odile Berger

Abstract: In computer vision, camera pose estimation from correspondences between 3D geometric entities and their projections into the image has been a widely investigated problem. Although most state-of-the-art methods exploit low-level primitives such as points or lines, the emergence of very effective CNN-based object detectors in the recent years has paved the way to the use of higher-level features car… ▽ More In computer vision, camera pose estimation from correspondences between 3D geometric entities and their projections into the image has been a widely investigated problem. Although most state-of-the-art methods exploit low-level primitives such as points or lines, the emergence of very effective CNN-based object detectors in the recent years has paved the way to the use of higher-level features carrying semantically meaningful information. Pioneering works in that direction have shown that modelling 3D objects by ellipsoids and 2D detections by ellipses offers a convenient manner to link 2D and 3D data. However, the mathematical formalism most often used in the related litterature does not enable to easily distinguish ellipsoids and ellipses from other quadrics and conics, leading to a loss of specificity potentially detrimental in some developments. Moreover, the linearization process of the projection equation creates an over-representation of the camera parameters, also possibly causing an efficiency loss. In this paper, we therefore introduce an ellipsoid-specific theoretical framework and demonstrate its beneficial properties in the context of pose estimation. More precisely, we first show that the proposed formalism enables to reduce the pose estimation problem to a position or orientation-only estimation problem in which the remaining unknowns can be derived in closed-form. Then, we demonstrate that it can be further reduced to a 1 Degree-of-Freedom (1DoF) problem and provide the analytical derivations of the pose as a function of that unique scalar unknown. We illustrate our theoretical considerations by visual examples and include a discussion on the practical aspects. Finally, we release this paper along with the corresponding source code in order to contribute towards more efficient resolutions of ellipsoid-related pose estimation problems. △ Less

Submitted 14 June, 2023; v1 submitted 26 August, 2022; originally announced August 2022.

Comments: 29 pages, 11 figures

Level Set-Based Camera Pose Estimation From Multiple 2D/3D Ellipse-Ellipsoid Correspondences

Authors: Matthieu Zins, Gilles Simon, Marie-Odile Berger

Abstract: In this paper, we propose an object-based camera pose estimation from a single RGB image and a pre-built map of objects, represented with ellipsoidal models. We show that contrary to point correspondences, the definition of a cost function characterizing the projection of a 3D object onto a 2D object detection is not straightforward. We develop an ellipse-ellipse cost based on level sets sampling,… ▽ More In this paper, we propose an object-based camera pose estimation from a single RGB image and a pre-built map of objects, represented with ellipsoidal models. We show that contrary to point correspondences, the definition of a cost function characterizing the projection of a 3D object onto a 2D object detection is not straightforward. We develop an ellipse-ellipse cost based on level sets sampling, demonstrate its nice properties for handling partially visible objects and compare its performance with other common metrics. Finally, we show that the use of a predictive uncertainty on the detected ellipses allows a fair weighting of the contribution of the correspondences which improves the computed pose. The code is released at https://gitlab.inria.fr/tangram/level-set-based-camera-pose-estimation. △ Less

Submitted 19 August, 2022; v1 submitted 16 July, 2022; originally announced July 2022.

Comments: published at IROS 2022

How Do Drivers Self-Regulate their Secondary Task Engagements? The Effect of Driving Automation on Touchscreen Interactions and Glance Behavior

Authors: Patrick Ebel, Moritz Berger, Christoph Lingenfelder, Andreas Vogelsang

Abstract: With ever-improving driver assistance systems and large touchscreens becoming the main in-vehicle interface, drivers are more tempted than ever to engage in distracting non-driving-related tasks. However, little research exists on how driving automation affects drivers' self-regulation when interacting with center stack touchscreens. To investigate this, we employ multilevel models on a real-world… ▽ More With ever-improving driver assistance systems and large touchscreens becoming the main in-vehicle interface, drivers are more tempted than ever to engage in distracting non-driving-related tasks. However, little research exists on how driving automation affects drivers' self-regulation when interacting with center stack touchscreens. To investigate this, we employ multilevel models on a real-world driving dataset consisting of 10,139 sequences. Our results show significant differences in drivers' interaction and glance behavior in response to varying levels of driving automation, vehicle speed, and road curvature. During partially automated driving, drivers are not only more likely to engage in secondary touchscreen tasks, but their mean glance duration toward the touchscreen also increases by 12% (Level 1) and 20% (Level 2) compared to manual driving. We further show that the effect of driving automation on drivers' self-regulation is larger than that of vehicle speed and road curvature. The derived knowledge can facilitate the safety evaluation of infotainment systems and the development of context-aware driver monitoring systems. △ Less

Submitted 12 July, 2022; v1 submitted 9 July, 2022; originally announced July 2022.

Comments: 14th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications

ACM Class: H.5.2

Preoperative brain tumor imaging: models and software for segmentation and standardized reporting

Authors: D. Bouget, A. Pedersen, A. S. Jakola, V. Kavouridis, K. E. Emblem, R. S. Eijgelaar, I. Kommers, H. Ardon, F. Barkhof, L. Bello, M. S. Berger, M. C. Nibali, J. Furtner, S. Hervey-Jumper, A. J. S. Idema, B. Kiesel, A. Kloet, E. Mandonnet, D. M. J. Müller, P. A. Robe, M. Rossi, T. Sciortino, W. Van den Brink, M. Wagemakers, G. Widhalm , et al. (5 additional authors not shown)

Abstract: For patients suffering from brain tumor, prognosis estimation and treatment decisions are made by a multidisciplinary team based on a set of preoperative MR scans. Currently, the lack of standardized and automatic methods for tumor detection and generation of clinical reports represents a major hurdle. In this study, we investigate glioblastomas, lower grade gliomas, meningiomas, and metastases, t… ▽ More For patients suffering from brain tumor, prognosis estimation and treatment decisions are made by a multidisciplinary team based on a set of preoperative MR scans. Currently, the lack of standardized and automatic methods for tumor detection and generation of clinical reports represents a major hurdle. In this study, we investigate glioblastomas, lower grade gliomas, meningiomas, and metastases, through four cohorts of up to 4000 patients. Tumor segmentation models were trained using the AGU-Net architecture with different preprocessing steps and protocols. Segmentation performances were assessed in-depth using a wide-range of voxel and patient-wise metrics covering volume, distance, and probabilistic aspects. Finally, two software solutions have been developed, enabling an easy use of the trained models and standardized generation of clinical reports: Raidionics and Raidionics-Slicer. Segmentation performances were quite homogeneous across the four different brain tumor types, with an average true positive Dice ranging between 80% and 90%, patient-wise recall between 88% and 98%, and patient-wise precision around 95%. With our Raidionics software, running on a desktop computer with CPU support, tumor segmentation can be performed in 16 to 54 seconds depending on the dimensions of the MRI volume. For the generation of a standardized clinical report, including the tumor segmentation and features computation, 5 to 15 minutes are necessary. All trained models have been made open-access together with the source code for both software solutions and validation metrics computation. In the future, an automatic classification of the brain tumor type would be necessary to replace manual user input. Finally, the inclusion of post-operative segmentation in both software solutions will be key for generating complete post-operative standardized clinical reports. △ Less

Submitted 29 April, 2022; originally announced April 2022.

Comments: 20 pages, 5 figures, 10 tables

ACM Class: I.4.6; J.3

Journal ref: Frontiers in Neurology, Sec. Applied Neuroimaging, Volume 13, 2022

Object-Based Visual Camera Pose Estimation From Ellipsoidal Model and 3D-Aware Ellipse Prediction

Authors: Matthieu Zins, Gilles Simon, Marie-Odile Berger

Abstract: In this paper, we propose a method for initial camera pose estimation from just a single image which is robust to viewing conditions and does not require a detailed model of the scene. This method meets the growing need of easy deployment of robotics or augmented reality applications in any environments, especially those for which no accurate 3D model nor huge amount of ground truth data are avail… ▽ More In this paper, we propose a method for initial camera pose estimation from just a single image which is robust to viewing conditions and does not require a detailed model of the scene. This method meets the growing need of easy deployment of robotics or augmented reality applications in any environments, especially those for which no accurate 3D model nor huge amount of ground truth data are available. It exploits the ability of deep learning techniques to reliably detect objects regardless of viewing conditions. Previous works have also shown that abstracting the geometry of a scene of objects by an ellipsoid cloud allows to compute the camera pose accurately enough for various application needs. Though promising, these approaches use the ellipses fitted to the detection bounding boxes as an approximation of the imaged objects. In this paper, we go one step further and propose a learning-based method which detects improved elliptic approximations of objects which are coherent with the 3D ellipsoids in terms of perspective projection. Experiments prove that the accuracy of the computed pose significantly increases thanks to our method. This is achieved with very little effort in terms of training data acquisition - a few hundred calibrated images of which only three need manual object annotation. Code and models are released at https://gitlab.inria.fr/tangram/3d-aware-ellipses-for-visual-localization △ Less

Submitted 9 March, 2022; originally announced March 2022.

Comments: International Journal of Computer Vision (IJCV)

A Weighted State Redistribution Algorithm for Embedded Boundary Grids

Authors: Andrew Giuliani, Ann S. Almgren, John B. Bell, Marsha J. Berger, Marc T. Henry de Frahan, Deepak Rangarajan

Abstract: State redistribution is an algorithm that stabilizes cut cells for embedded boundary grid methods. This work extends the earlier algorithm in several important ways. First, state redistribution is extended to three spatial dimensions. Second, we discuss several algorithmic changes and improvements motivated by the more complicated cut cell geometries that can occur in higher dimensions. In particu… ▽ More State redistribution is an algorithm that stabilizes cut cells for embedded boundary grid methods. This work extends the earlier algorithm in several important ways. First, state redistribution is extended to three spatial dimensions. Second, we discuss several algorithmic changes and improvements motivated by the more complicated cut cell geometries that can occur in higher dimensions. In particular, we introduce a weighted version with less dissipation in an easily generalizable framework. Third, we demonstrate that state redistribution can also stabilize a solution update that includes both advective and diffusive contributions. The stabilization algorithm is shown to be effective for incompressible as well as compressible reacting flows. Finally, we discuss the implementation of the algorithm for several exascale-ready simulation codes based on AMReX, demonstrating ease of use in combination with domain decomposition, hybrid parallelism and complex physics. △ Less

Submitted 11 May, 2022; v1 submitted 22 December, 2021; originally announced December 2021.

Systematic Analysis of Programming Languages and Their Execution Environments for Spectre Attacks

Authors: Amir Naseredini, Stefan Gast, Martin Schwarzl, Pedro Miguel Sousa Bernardo, Amel Smajic, Claudio Canella, Martin Berger, Daniel Gruss

Abstract: In this paper, we analyze the security of programming languages and their execution environments (compilers and interpreters) with respect to Spectre attacks. The analysis shows that only 16 out of 42 execution environments have mitigations against at least one Spectre variant, i.e., 26 have no mitigations against any Spectre variant. Using our novel tool Speconnector, we develop Spectre proof-of-… ▽ More In this paper, we analyze the security of programming languages and their execution environments (compilers and interpreters) with respect to Spectre attacks. The analysis shows that only 16 out of 42 execution environments have mitigations against at least one Spectre variant, i.e., 26 have no mitigations against any Spectre variant. Using our novel tool Speconnector, we develop Spectre proof-of-concept attacks in 8 programming languages and on code generated by 11 execution environments that were previously not known to be affected. Our results highlight some programming languages that are used to implement security-critical code, but remain entirely unprotected, even three years after the discovery of Spectre. △ Less

Submitted 24 November, 2021; originally announced November 2021.

Towards Adaptive Simulations of Dispersive Tsunami Propagation from an Asteroid Impact

Authors: Marsha J. Berger, Randall J. LeVeque

Abstract: The long-term goal of this work is the development of high-fidelity simulation tools for dispersive tsunami propagation. A dispersive model is especially important for short wavelength phenomena such as an asteroid impact into the ocean, and is also important in modeling other events where the simpler shallow water equations are insufficient. Adaptive simulations are crucial to bridge the scales f… ▽ More The long-term goal of this work is the development of high-fidelity simulation tools for dispersive tsunami propagation. A dispersive model is especially important for short wavelength phenomena such as an asteroid impact into the ocean, and is also important in modeling other events where the simpler shallow water equations are insufficient. Adaptive simulations are crucial to bridge the scales from deep ocean to inundation, but have difficulties with the implicit system of equations that results from dispersive models. We propose a fractional step scheme that advances the solution on separate patches with different spatial resolutions and time steps. We show a simulation with 7 levels of adaptive meshes and onshore inundation resulting from a simulated asteroid impact off the coast of Washington. Finally, we discuss a number of open research questions that need to be resolved for high quality simulations. △ Less

Submitted 29 September, 2021; originally announced October 2021.

Comments: 16 pages, 5 figures, submitted to Proc. International Congress of Mathematicians, 2022

MSC Class: 65M50; 76B15

Abstract Operator Systems over the Cone of Positive Semidefinite Matrices

Authors: Martin Berger, Tim Netzer

Abstract: There are several important abstract operator systems with the convex cone of positive semidefinite matrices at the first level. Well-known are the operator systems of separable matrices, of positive semidefinite matrices, and of block positive matrices. In terms of maps, these are the operator systems of entanglement breaking, completely positive, and positive linear maps, respectively. But there… ▽ More There are several important abstract operator systems with the convex cone of positive semidefinite matrices at the first level. Well-known are the operator systems of separable matrices, of positive semidefinite matrices, and of block positive matrices. In terms of maps, these are the operator systems of entanglement breaking, completely positive, and positive linear maps, respectively. But there exist other interesting and less well-studied such operator systems, for example those of completely copositive maps, doubly completely positive maps, and decomposable maps, which all play an important role in quantum information theory. We investigate which of these systems is finitely generated, and which admits a finite-dimensional realization in the sense of the Choi-Effros Theorem. We answer this question for all of the described systems completely. Our main contribution is that decomposable maps form a system which does not admit a finite-dimensional realization, though being finitely generated, whereas the system of doubly completely positive maps is not finitely generated, though having a finite-dimensional realization. This also implies that there cannot exist a finitary Choi-type characterization of doubly completely positive maps. △ Less

Submitted 29 September, 2021; originally announced September 2021.

STFT-LDA: An Algorithm to Facilitate the Visual Analysis of Building Seismic Responses

Authors: Zhenge Zhao, Danilo Motta, Matthew Berger, Joshua A. Levine, Ismail B. Kuzucu, Robert B. Fleischman, Afonso Paiva, Carlos Scheidegger

Abstract: Civil engineers use numerical simulations of a building's responses to seismic forces to understand the nature of building failures, the limitations of building codes, and how to determine the latter to prevent the former. Such simulations generate large ensembles of multivariate, multiattribute time series. Comprehensive understanding of this data requires techniques that support the multivariate… ▽ More Civil engineers use numerical simulations of a building's responses to seismic forces to understand the nature of building failures, the limitations of building codes, and how to determine the latter to prevent the former. Such simulations generate large ensembles of multivariate, multiattribute time series. Comprehensive understanding of this data requires techniques that support the multivariate nature of the time series and can compare behaviors that are both periodic and non-periodic across multiple time scales and multiple time series themselves. In this paper, we present a novel technique to extract such patterns from time series generated from simulations of seismic responses. The core of our approach is the use of topic modeling, where topics correspond to interpretable and discriminative features of the earthquakes. We transform the raw time series data into a time series of topics, and use this visual summary to compare temporal patterns in earthquakes, query earthquakes via the topics across arbitrary time scales, and enable details on demand by linking the topic visualization with the original earthquake data. We show, through a surrogate task and an expert study, that this technique allows analysts to more easily identify recurring patterns in such time series. By integrating this technique in a prototype system, we show how it enables novel forms of visual interaction. △ Less

Submitted 1 September, 2021; originally announced September 2021.

Comments: 16 pages, 10 figures

Computational Investigations of Selected Enzymes From Two Iron and α-ketoglutarate-Dependent Families

Authors: Madison B. Berger, Alice R. Walker, Erik Antonio Vázquez Montelongo, G. Andrés Cisneros

Abstract: DNA alkylation is used as the key epigenetic mark in eukaryotes, however, most alkylation in DNA can result in deleterious effects. Therefore, this process needs to be tightly regulated. AlkB and TET are families within the Fe and $α$-kg-dependent superfamily of enzymes that are tasked with dealkylating DNA and RNA in cells. Members of these families span all species and are an integral part of tr… ▽ More DNA alkylation is used as the key epigenetic mark in eukaryotes, however, most alkylation in DNA can result in deleterious effects. Therefore, this process needs to be tightly regulated. AlkB and TET are families within the Fe and $α$-kg-dependent superfamily of enzymes that are tasked with dealkylating DNA and RNA in cells. Members of these families span all species and are an integral part of transcriptional regulation. While both families catalyze oxidative dealkylation of various bases, each has specific preference for alkylated base type as well as distinct catalytic mechanisms. This perspective aims to provide an overview of computational work carried out to investigate several members of these enzyme families including AlkB, ALKBH2, ALKBH3 and TET2. Insights into structural details, mutagenesis studies, reaction path analysis, electronic structure features in the active site, and substrate preferences are presented and discussed △ Less

Submitted 24 August, 2021; originally announced August 2021.