-
NFDI4Health workflow and service for synthetic data generation, assessment and risk management
Authors:
Sobhan Moazemi,
Tim Adams,
Hwei Geok NG,
Lisa Kühnel,
Julian Schneider,
Anatol-Fiete Näher,
Juliane Fluck,
Holger Fröhlich
Abstract:
Individual health data is crucial for scientific advancements, particularly in developing Artificial Intelligence (AI); however, sharing real patient information is often restricted due to privacy concerns. A promising solution to this challenge is synthetic data generation. This technique creates entirely new datasets that mimic the statistical properties of real data, while preserving confidenti…
▽ More
Individual health data is crucial for scientific advancements, particularly in developing Artificial Intelligence (AI); however, sharing real patient information is often restricted due to privacy concerns. A promising solution to this challenge is synthetic data generation. This technique creates entirely new datasets that mimic the statistical properties of real data, while preserving confidential patient information. In this paper, we present the workflow and different services developed in the context of Germany's National Data Infrastructure project NFDI4Health. First, two state-of-the-art AI tools (namely, VAMBN and MultiNODEs) for generating synthetic health data are outlined. Further, we introduce SYNDAT (a public web-based tool) which allows users to visualize and assess the quality and risk of synthetic data provided by desired generative models. Additionally, the utility of the proposed methods and the web-based tool is showcased using data from Alzheimer's Disease Neuroimaging Initiative (ADNI) and the Center for Cancer Registry Data of the Robert Koch Institute (RKI).
△ Less
Submitted 8 August, 2024;
originally announced August 2024.
-
The 2010 Census Confidentiality Protections Failed, Here's How and Why
Authors:
John M. Abowd,
Tamara Adams,
Robert Ashmead,
David Darais,
Sourya Dey,
Simson L. Garfinkel,
Nathan Goldschlag,
Daniel Kifer,
Philip Leclerc,
Ethan Lew,
Scott Moore,
Rolando A. Rodríguez,
Ramy N. Tadros,
Lars Vilhuber
Abstract:
Using only 34 published tables, we reconstruct five variables (census block, sex, age, race, and ethnicity) in the confidential 2010 Census person records. Using the 38-bin age variable tabulated at the census block level, at most 20.1% of reconstructed records can differ from their confidential source on even a single value for these five variables. Using only published data, an attacker can veri…
▽ More
Using only 34 published tables, we reconstruct five variables (census block, sex, age, race, and ethnicity) in the confidential 2010 Census person records. Using the 38-bin age variable tabulated at the census block level, at most 20.1% of reconstructed records can differ from their confidential source on even a single value for these five variables. Using only published data, an attacker can verify that all records in 70% of all census blocks (97 million people) are perfectly reconstructed. The tabular publications in Summary File 1 thus have prohibited disclosure risk similar to the unreleased confidential microdata. Reidentification studies confirm that an attacker can, within blocks with perfect reconstruction accuracy, correctly infer the actual census response on race and ethnicity for 3.4 million vulnerable population uniques (persons with nonmodal characteristics) with 95% accuracy, the same precision as the confidential data achieve and far greater than statistical baselines. The flaw in the 2010 Census framework was the assumption that aggregation prevented accurate microdata reconstruction, justifying weaker disclosure limitation methods than were applied to 2010 Census public microdata. The framework used for 2020 Census publications defends against attacks that are based on reconstruction, as we also demonstrate here. Finally, we show that alternatives to the 2020 Census Disclosure Avoidance System with similar accuracy (enhanced swapping) also fail to protect confidentiality, and those that partially defend against reconstruction attacks (incomplete suppression implementations) destroy the primary statutory use case: data for redistricting all legislatures in the country in compliance with the 1965 Voting Rights Act.
△ Less
Submitted 18 December, 2023;
originally announced December 2023.
-
King of the Hill: C2 for Next Generation Swarm Warfare
Authors:
Takuma Adams,
Timothy McLennan-Smith
Abstract:
As the reliability of cheap, off-the-shelf autonomous platforms increases, so does the risk posed by intelligent multi-agent systems to military operations. In the contemporary context of the Russo-Ukrainian war alone, we have seen autonomous aerial vehicles and surface vessels deployed both individually and in multitude to deliver critical effects to both sides. While there is a large body of lit…
▽ More
As the reliability of cheap, off-the-shelf autonomous platforms increases, so does the risk posed by intelligent multi-agent systems to military operations. In the contemporary context of the Russo-Ukrainian war alone, we have seen autonomous aerial vehicles and surface vessels deployed both individually and in multitude to deliver critical effects to both sides. While there is a large body of literature on tactical level communications and interactions between agents, the exploration of high-level command and control (C2) structures that will underpin future autonomous multi-agent military operations is a less explored area of research. We propose a quantitative game-theoretic framework to study effective C2 structures in cooperative and competitive multi-agent swarming scenarios. To test our framework, we construct a virtual environment where two adversarial swarms compete to achieve outcomes comparable to real-world scenarios. The framework we present in this paper enables us to quickly test and interrogate different C2 configurations in multi-agent systems to explore C2 as a force multiplier when at a force disadvantage.
△ Less
Submitted 29 September, 2023;
originally announced October 2023.
-
Synthetic data generation for a longitudinal cohort study -- Evaluation, method extension and reproduction of published data analysis results
Authors:
Lisa Kühnel,
Julian Schneider,
Ines Perrar,
Tim Adams,
Fabian Prasser,
Ute Nöthlings,
Holger Fröhlich,
Juliane Fluck
Abstract:
Access to individual-level health data is essential for gaining new insights and advancing science. In particular, modern methods based on artificial intelligence rely on the availability of and access to large datasets. In the health sector, access to individual-level data is often challenging due to privacy concerns. A promising alternative is the generation of fully synthetic data, i.e. data ge…
▽ More
Access to individual-level health data is essential for gaining new insights and advancing science. In particular, modern methods based on artificial intelligence rely on the availability of and access to large datasets. In the health sector, access to individual-level data is often challenging due to privacy concerns. A promising alternative is the generation of fully synthetic data, i.e. data generated through a randomised process that have similar statistical properties as the original data, but do not have a one-to-one correspondence with the original individual-level records. In this study, we use a state-of-the-art synthetic data generation method and perform in-depth quality analyses of the generated data for a specific use case in the field of nutrition. We demonstrate the need for careful analyses of synthetic data that go beyond descriptive statistics and provide valuable insights into how to realise the full potential of synthetic datasets. By extending the methods, but also by thoroughly analysing the effects of sampling from a trained model, we are able to largely reproduce significant real-world analysis results in the chosen use case.
△ Less
Submitted 12 May, 2023;
originally announced May 2023.
-
Investigating Crowdsourcing Protocols for Evaluating the Factual Consistency of Summaries
Authors:
Xiangru Tang,
Alexander Fabbri,
Haoran Li,
Ziming Mao,
Griffin Thomas Adams,
Borui Wang,
Asli Celikyilmaz,
Yashar Mehdad,
Dragomir Radev
Abstract:
Current pre-trained models applied to summarization are prone to factual inconsistencies which either misrepresent the source text or introduce extraneous information. Thus, comparing the factual consistency of summaries is necessary as we develop improved models. However, the optimal human evaluation setup for factual consistency has not been standardized. To address this issue, we crowdsourced e…
▽ More
Current pre-trained models applied to summarization are prone to factual inconsistencies which either misrepresent the source text or introduce extraneous information. Thus, comparing the factual consistency of summaries is necessary as we develop improved models. However, the optimal human evaluation setup for factual consistency has not been standardized. To address this issue, we crowdsourced evaluations for factual consistency using the rating-based Likert scale and ranking-based Best-Worst Scaling protocols, on 100 articles from each of the CNN-Daily Mail and XSum datasets over four state-of-the-art models, to determine the most reliable evaluation framework. We find that ranking-based protocols offer a more reliable measure of summary quality across datasets, while the reliability of Likert ratings depends on the target dataset and the evaluation design. Our crowdsourcing templates and summary evaluations will be publicly available to facilitate future research on factual consistency in summarization.
△ Less
Submitted 9 July, 2022; v1 submitted 19 September, 2021;
originally announced September 2021.
-
Genericity and Rigidity for Slow Entropy Transformations
Authors:
Terry Adams
Abstract:
The notion of slow entropy, both upper and lower slow entropy, was defined by Katok and Thouvenot as a more refined measure of complexity for dynamical systems, than the classical Kolmogorov-Sinai entropy. For any subexponential rate function $a_n(t)$, we prove there exists a generic class of invertible measure preserving systems such that the lower slow entropy is zero and the upper slow entropy…
▽ More
The notion of slow entropy, both upper and lower slow entropy, was defined by Katok and Thouvenot as a more refined measure of complexity for dynamical systems, than the classical Kolmogorov-Sinai entropy. For any subexponential rate function $a_n(t)$, we prove there exists a generic class of invertible measure preserving systems such that the lower slow entropy is zero and the upper slow entropy is infinite. Also, given any subexponential rate $a_n(t)$, we show there exists a rigid, weak mixing, invertible system such that the lower slow entropy is infinite with respect to $a_n(t)$. This gives a general solution to a question on the existence of rigid transformations with positive polynomial upper slow entropy, Finally, we connect slow entropy with the notion of entropy covergence rate presented by Blume. In particular, we show slow entropy is a strictly stronger notion of complexity and give examples which have zero upper slow entropy, but also have an arbitrary sublinear positive entropy convergence rate.
△ Less
Submitted 15 July, 2020; v1 submitted 27 June, 2020;
originally announced June 2020.
-
A Continuous, Full-scope, Spatio-temporal Tracking Metric based on KL-divergence
Authors:
Terrence Adams
Abstract:
A unified metric is given for the evaluation of object tracking systems. The metric is inspired by KL-divergence or relative entropy, which is commonly used to evaluate clustering techniques. Since tracking problems are fundamentally different from clustering, the components of KL-divergence are recast to handle various types of tracking errors (i.e., false alarms, missed detections, merges, split…
▽ More
A unified metric is given for the evaluation of object tracking systems. The metric is inspired by KL-divergence or relative entropy, which is commonly used to evaluate clustering techniques. Since tracking problems are fundamentally different from clustering, the components of KL-divergence are recast to handle various types of tracking errors (i.e., false alarms, missed detections, merges, splits). Scoring results are given on a standard tracking dataset (Oxford Town Centre Dataset), as well as several simulated scenarios. Also, this new metric is compared with several other metrics including the commonly used Multiple Object Tracking Accuracy metric. In the final section, advantages of this metric are given including the fact that it is continuous, parameter-less and comprehensive.
△ Less
Submitted 1 March, 2019; v1 submitted 9 May, 2018;
originally announced May 2018.
-
AI-Powered Social Bots
Authors:
Terrence Adams
Abstract:
This paper gives an overview of impersonation bots that generate output in one, or possibly, multiple modalities. We also discuss rapidly advancing areas of machine learning and artificial intelligence that could lead to frighteningly powerful new multi-modal social bots. Our main conclusion is that most commonly known bots are one dimensional (i.e., chatterbot), and far from deceiving serious int…
▽ More
This paper gives an overview of impersonation bots that generate output in one, or possibly, multiple modalities. We also discuss rapidly advancing areas of machine learning and artificial intelligence that could lead to frighteningly powerful new multi-modal social bots. Our main conclusion is that most commonly known bots are one dimensional (i.e., chatterbot), and far from deceiving serious interrogators. However, using recent advances in machine learning, it is possible to unleash incredibly powerful, human-like armies of social bots, in potentially well coordinated campaigns of deception and influence.
△ Less
Submitted 16 June, 2017;
originally announced June 2017.
-
Development of a Big Data Framework for Connectomic Research
Authors:
Terrence Adams
Abstract:
This paper outlines research and development of a new Hadoop-based architecture for distributed processing and analysis of electron microscopy of brains. We show development of a new C++ library for implementation of 3D image analysis techniques, and deployment in a distributed map/reduce framework. We demonstrate our new framework on a subset of the Kasthuri11 dataset from the Open Connectome Pro…
▽ More
This paper outlines research and development of a new Hadoop-based architecture for distributed processing and analysis of electron microscopy of brains. We show development of a new C++ library for implementation of 3D image analysis techniques, and deployment in a distributed map/reduce framework. We demonstrate our new framework on a subset of the Kasthuri11 dataset from the Open Connectome Project.
△ Less
Submitted 24 January, 2015;
originally announced January 2015.
