-
Semi-Classical Subspaces, The No Synchronization Law, and More
Authors:
Samuel Epstein
Abstract:
This paper looks at the intersection of algorithmic information theory and physics, namely quantum mechanics, thermodynamics, and black holes. We discuss theorems which characterize the barrier between the quantum world and the classical realm. The notion of a ``semi-classical subspace'' is introduced. Partial signals and partial cloning can be executed on quantum states in semi-classical subspace…
▽ More
This paper looks at the intersection of algorithmic information theory and physics, namely quantum mechanics, thermodynamics, and black holes. We discuss theorems which characterize the barrier between the quantum world and the classical realm. The notion of a ``semi-classical subspace'' is introduced. Partial signals and partial cloning can be executed on quantum states in semi-classical subspaces. The No Synchronization Law is detailed, which says separate and isolated physical systems evolving over time cannot have thermodynamic algorithmic entropies that are in synch. We look at future work involving the Kolmogorov complexity of black holes.
△ Less
Submitted 27 August, 2024; v1 submitted 25 July, 2024;
originally announced July 2024.
-
On Kolmogorov Structure Functions
Authors:
Samuel Epstein
Abstract:
All strings with low mutual information with the halting sequence will have flat Kolmogorov Structure Functions, in the context of Algorithmic Statistics. Assuming the Independence Postulate, strings with non-negligible information with the halting sequence are purely mathematical constructions, and cannot be found in nature. Thus Algorithmic Statistics does not study strings in the physical world…
▽ More
All strings with low mutual information with the halting sequence will have flat Kolmogorov Structure Functions, in the context of Algorithmic Statistics. Assuming the Independence Postulate, strings with non-negligible information with the halting sequence are purely mathematical constructions, and cannot be found in nature. Thus Algorithmic Statistics does not study strings in the physical world. This leads to the general thesis that two part codes require limitations as shown in the Minimum Description Length Principle. We also discuss issues with set-restricted Kolmogorov Structure Functions.
△ Less
Submitted 9 July, 2024; v1 submitted 9 June, 2024;
originally announced June 2024.
-
Game Derandomization
Authors:
Samuel Epstein
Abstract:
Using Kolmogorov Game Derandomization, upper bounds of the Kolmogorov complexity of deterministic winning players against deterministic environments can be proved. This paper gives improved upper bounds of the Kolmogorov complexity of such players. This paper also generalizes this result to probabilistic games. This applies to computable, lower computable, and uncomputable environments. We charact…
▽ More
Using Kolmogorov Game Derandomization, upper bounds of the Kolmogorov complexity of deterministic winning players against deterministic environments can be proved. This paper gives improved upper bounds of the Kolmogorov complexity of such players. This paper also generalizes this result to probabilistic games. This applies to computable, lower computable, and uncomputable environments. We characterize the classic even-odds game and then generalize these results to time bounded players and also to all zero-sum repeated games. We characterize partial game derandomization. But first, we start with an illustrative example of game derandomization, taking place on the island of Crete.
△ Less
Submitted 25 May, 2024;
originally announced May 2024.
-
T-HITL Effectively Addresses Problematic Associations in Image Generation and Maintains Overall Visual Quality
Authors:
Susan Epstein,
Li Chen,
Alessandro Vecchiato,
Ankit Jain
Abstract:
Generative AI image models may inadvertently generate problematic representations of people. Past research has noted that millions of users engage daily across the world with these models and that the models, including through problematic representations of people, have the potential to compound and accelerate real-world discrimination and other harms (Bianchi et al, 2023). In this paper, we focus…
▽ More
Generative AI image models may inadvertently generate problematic representations of people. Past research has noted that millions of users engage daily across the world with these models and that the models, including through problematic representations of people, have the potential to compound and accelerate real-world discrimination and other harms (Bianchi et al, 2023). In this paper, we focus on addressing the generation of problematic associations between demographic groups and semantic concepts that may reflect and reinforce negative narratives embedded in social data. Building on sociological literature (Blumer, 1958) and mapping representations to model behaviors, we have developed a taxonomy to study problematic associations in image generation models. We explore the effectiveness of fine tuning at the model level as a method to address these associations, identifying a potential reduction in visual quality as a limitation of traditional fine tuning. We also propose a new methodology with twice-human-in-the-loop (T-HITL) that promises improvements in both reducing problematic associations and also maintaining visual quality. We demonstrate the effectiveness of T-HITL by providing evidence of three problematic associations addressed by T-HITL at the model level. Our contributions to scholarship are two-fold. By defining problematic associations in the context of machine learning models and generative AI, we introduce a conceptual and technical taxonomy for addressing some of these associations. Finally, we provide a method, T-HITL, that addresses these associations and simultaneously maintains visual quality of image model generations. This mitigation need not be a tradeoff, but rather an enhancement.
△ Less
Submitted 26 February, 2024;
originally announced February 2024.
-
Two Quantum Paradigms, but Still No Signal
Authors:
Samuel Epstein
Abstract:
An overwhelming majority of quantum (pure and mixed) states, when undertaking a POVM measurement, will result in a classical probability with no algorithmic information. Thus most quantum states produce white noise when measured. Furthermore most non-pointer states, when undergoing the decoherence process, will produce white noise. These results can be seen as consequences of the vastness of Hilbe…
▽ More
An overwhelming majority of quantum (pure and mixed) states, when undertaking a POVM measurement, will result in a classical probability with no algorithmic information. Thus most quantum states produce white noise when measured. Furthermore most non-pointer states, when undergoing the decoherence process, will produce white noise. These results can be seen as consequences of the vastness of Hilbert spaces.
△ Less
Submitted 20 February, 2024;
originally announced February 2024.
-
Two Simple Proofs of Müller's Theorem
Authors:
Samuel Epstein
Abstract:
Due to Müller's theorem, the Kolmogorov complexity of a string was shown to be equal to its quantum Kolmogorov complexity. Thus there are no benefits to using quantum mechanics to compress classical information. The quantitative amount of information in classical sources is invariant to the physical model used. These consequences make this theorem arguably the most important result in the intersec…
▽ More
Due to Müller's theorem, the Kolmogorov complexity of a string was shown to be equal to its quantum Kolmogorov complexity. Thus there are no benefits to using quantum mechanics to compress classical information. The quantitative amount of information in classical sources is invariant to the physical model used. These consequences make this theorem arguably the most important result in the intersection of algorithmic information theory and physics. The original proof is quite extensive. This paper contains two simple proofs of this theorem. This paper also contains new bounds for quantum Kolmogorov complexity with error.
△ Less
Submitted 2 July, 2024; v1 submitted 7 February, 2024;
originally announced February 2024.
-
The Randomness Deficiency Function and the Shift Operator
Authors:
Samuel Epstein
Abstract:
Almost surely, the difference between the randomness deficiencies of two infinite sequences will be unbounded with respect to repeated iterations of the shift operator.
Almost surely, the difference between the randomness deficiencies of two infinite sequences will be unbounded with respect to repeated iterations of the shift operator.
△ Less
Submitted 27 August, 2024; v1 submitted 8 September, 2023;
originally announced September 2023.
-
Rician likelihood loss for quantitative MRI using self-supervised deep learning
Authors:
Christopher S. Parker,
Anna Schroder,
Sean C. Epstein,
James Cole,
Daniel C. Alexander,
Hui Zhang
Abstract:
Purpose: Previous quantitative MR imaging studies using self-supervised deep learning have reported biased parameter estimates at low SNR. Such systematic errors arise from the choice of Mean Squared Error (MSE) loss function for network training, which is incompatible with Rician-distributed MR magnitude signals. To address this issue, we introduce the negative log Rician likelihood (NLR) loss. M…
▽ More
Purpose: Previous quantitative MR imaging studies using self-supervised deep learning have reported biased parameter estimates at low SNR. Such systematic errors arise from the choice of Mean Squared Error (MSE) loss function for network training, which is incompatible with Rician-distributed MR magnitude signals. To address this issue, we introduce the negative log Rician likelihood (NLR) loss. Methods: A numerically stable and accurate implementation of the NLR loss was developed to estimate quantitative parameters of the apparent diffusion coefficient (ADC) model and intra-voxel incoherent motion (IVIM) model. Parameter estimation accuracy, precision and overall error were evaluated in terms of bias, variance and root mean squared error and compared against the MSE loss over a range of SNRs (5 - 30). Results: Networks trained with NLR loss show higher estimation accuracy than MSE for the ADC and IVIM diffusion coefficients as SNR decreases, with minimal loss of precision or total error. At high effective SNR (high SNR and small diffusion coefficients), both losses show comparable accuracy and precision for all parameters of both models. Conclusion: The proposed NLR loss is numerically stable and accurate across the full range of tested SNRs and improves parameter estimation accuracy of diffusion coefficients using self-supervised deep learning. We expect the development to benefit quantitative MR imaging techniques broadly, enabling more accurate parameter estimation from noisy data.
△ Less
Submitted 13 July, 2023;
originally announced July 2023.
-
Robots in the Garden: Artificial Intelligence and Adaptive Landscapes
Authors:
Zihao Zhang,
Susan L. Epstein,
Casey Breen,
Sophia Xia,
Zhigang Zhu,
Christian Volkmann
Abstract:
This paper introduces ELUA, the Ecological Laboratory for Urban Agriculture, a collaboration among landscape architects, architects and computer scientists who specialize in artificial intelligence, robotics and computer vision. ELUA has two gantry robots, one indoors and the other outside on the rooftop of a 6-story campus building. Each robot can seed, water, weed, and prune in its garden. To su…
▽ More
This paper introduces ELUA, the Ecological Laboratory for Urban Agriculture, a collaboration among landscape architects, architects and computer scientists who specialize in artificial intelligence, robotics and computer vision. ELUA has two gantry robots, one indoors and the other outside on the rooftop of a 6-story campus building. Each robot can seed, water, weed, and prune in its garden. To support responsive landscape research, ELUA also includes sensor arrays, an AI-powered camera, and an extensive network infrastructure. This project demonstrates a way to integrate artificial intelligence into an evolving urban ecosystem, and encourages landscape architects to develop an adaptive design framework where design becomes a long-term engagement with the environment.
△ Less
Submitted 22 May, 2023;
originally announced May 2023.
-
Regression and Algorithmic Information Theory
Authors:
Samuel Epstein
Abstract:
In this paper we prove a theorem about regression, in that the shortest description of a function consistent with a finite sample of data is less than the combined conditional Kolmogorov complexities over the data in the sample.
In this paper we prove a theorem about regression, in that the shortest description of a function consistent with a finite sample of data is less than the combined conditional Kolmogorov complexities over the data in the sample.
△ Less
Submitted 16 April, 2023;
originally announced April 2023.
-
On the Algorithmic Information Between Probabilities
Authors:
Samuel Epstein
Abstract:
We extend algorithmic conservation inequalities to probability measures. The amount of self information of a probability measure cannot increase when submitted to randomized processing. This includes (potentially non-computable) measures over finite sequences, infinite sequences, and $T_0$, second countable topologies. One example is the convolution of signals over real numbers with probability ke…
▽ More
We extend algorithmic conservation inequalities to probability measures. The amount of self information of a probability measure cannot increase when submitted to randomized processing. This includes (potentially non-computable) measures over finite sequences, infinite sequences, and $T_0$, second countable topologies. One example is the convolution of signals over real numbers with probability kernels. Thus the smoothing of any signal due We show that given a quantum measurement, for an overwhelming majority of pure states, no meaningful information is produced.
△ Less
Submitted 13 March, 2023;
originally announced March 2023.
-
A Quantum Outlier Theorem
Authors:
Samuel Epstein
Abstract:
In recent results, it has been proven that all sampling methods produce outliers. In this paper, we extend these results to quantum information theory. Projectors of large rank must contain pure quantum states in their images that are outlying states. Otherwise, the projectors are exotic, in that they have high mutual information with the halting sequence. Thus quantum coding schemes that use proj…
▽ More
In recent results, it has been proven that all sampling methods produce outliers. In this paper, we extend these results to quantum information theory. Projectors of large rank must contain pure quantum states in their images that are outlying states. Otherwise, the projectors are exotic, in that they have high mutual information with the halting sequence. Thus quantum coding schemes that use projections, such as Schumacher compression, must communicate using outlier quantum states.
△ Less
Submitted 10 March, 2023;
originally announced March 2023.
-
Uniform Tests and Algorithmic Thermodynamic Entropy
Authors:
Samuel Epstein
Abstract:
We prove that given a computable metric space and two computable measures, the set of points that have high universal uniform test scores with respect to the first measure will have a lower bound with respect to the second measure. This result is transferred to thermodynamics, showing that algorithmic thermodynamic entropy must oscillate in the presence of dynamics. Another application is that out…
▽ More
We prove that given a computable metric space and two computable measures, the set of points that have high universal uniform test scores with respect to the first measure will have a lower bound with respect to the second measure. This result is transferred to thermodynamics, showing that algorithmic thermodynamic entropy must oscillate in the presence of dynamics. Another application is that outliers will become emergent in computable dynamics of computable metric spaces.
△ Less
Submitted 30 July, 2023; v1 submitted 9 March, 2023;
originally announced March 2023.
-
How to Compress the Solution
Authors:
Samuel Epstein
Abstract:
Using derandomization, we provide an upper bound on the compression size of solutions to the graph coloring problem. In general, if solutions to a combinatorial problem exist with high probability and the probability is simple, then there exists a simple solution to the problem. Otherwise the problem instance has high mutual information with the halting problem.
Using derandomization, we provide an upper bound on the compression size of solutions to the graph coloring problem. In general, if solutions to a combinatorial problem exist with high probability and the probability is simple, then there exists a simple solution to the problem. Otherwise the problem instance has high mutual information with the halting problem.
△ Less
Submitted 6 September, 2023; v1 submitted 9 March, 2023;
originally announced March 2023.
-
On the Existence of Anomalies, The Reals Case
Authors:
Samuel Epstein
Abstract:
The Independence Postulate (IP) is a finitary Church-Turing Thesis, saying mathematical sequences are independent from physical ones. Modelling observations as infinite sequences of real numbers, IP implies the existence of anomalies.
The Independence Postulate (IP) is a finitary Church-Turing Thesis, saying mathematical sequences are independent from physical ones. Modelling observations as infinite sequences of real numbers, IP implies the existence of anomalies.
△ Less
Submitted 13 May, 2024; v1 submitted 9 March, 2023;
originally announced March 2023.
-
Resolving quantitative MRI model degeneracy with machine learning via training data distribution design
Authors:
Michele Guerreri,
Sean Epstein,
Hojjat Azadbakht,
Hui Zhang
Abstract:
Quantitative MRI (qMRI) aims to map tissue properties non-invasively via models that relate these unknown quantities to measured MRI signals. Estimating these unknowns, which has traditionally required model fitting - an often iterative procedure, can now be done with one-shot machine learning (ML) approaches. Such parameter estimation may be complicated by intrinsic qMRI signal model degeneracy:…
▽ More
Quantitative MRI (qMRI) aims to map tissue properties non-invasively via models that relate these unknown quantities to measured MRI signals. Estimating these unknowns, which has traditionally required model fitting - an often iterative procedure, can now be done with one-shot machine learning (ML) approaches. Such parameter estimation may be complicated by intrinsic qMRI signal model degeneracy: different combinations of tissue properties produce the same signal. Despite their many advantages, it remains unclear whether ML approaches can resolve this issue. Growing empirical evidence appears to suggest ML approaches remain susceptible to model degeneracy. Here we demonstrate under the right circumstances ML can address this issue. Inspired by recent works on the impact of training data distributions on ML-based parameter estimation, we propose to resolve model degeneracy by designing training data distributions. We put forward a classification of model degeneracies and identify one particular kind of degeneracies amenable to the proposed attack. The strategy is demonstrated successfully using the Revised NODDI model with standard multi-shell diffusion MRI data as an exemplar. Our results illustrate the importance of training set design which has the potential to allow accurate estimation of tissue properties with ML.
△ Less
Submitted 9 March, 2023;
originally announced March 2023.
-
Some Implications of the Independence Postulate for Physics
Authors:
Samuel Epstein
Abstract:
The Many Worlds Theory and Constructor Theory are in conflict with the Independence Postulate. The conflict with the Many Worlds Theory is shown through the existence of a finite experiment that measures the spin of a large number of electrons. After the experiment there are branches of positive probability which contain forbidden sequences that break the Independence Postulate. Constructor Theory…
▽ More
The Many Worlds Theory and Constructor Theory are in conflict with the Independence Postulate. The conflict with the Many Worlds Theory is shown through the existence of a finite experiment that measures the spin of a large number of electrons. After the experiment there are branches of positive probability which contain forbidden sequences that break the Independence Postulate. Constructor Theory consists of counterfactuals, decreeing certain processes can or cannot occur. However this binary classification meets challenges when describing whether a forbidden sequence can be found or created.
△ Less
Submitted 15 August, 2024; v1 submitted 14 February, 2023;
originally announced February 2023.
-
On the Existence of Anomalies
Authors:
Samuel Epstein
Abstract:
The Independence Postulate (IP) is a finitary Church-Turing Thesis, saying mathematical sequences are independent from physical ones. IP implies the existence of anomalies.
The Independence Postulate (IP) is a finitary Church-Turing Thesis, saying mathematical sequences are independent from physical ones. IP implies the existence of anomalies.
△ Less
Submitted 12 February, 2023;
originally announced February 2023.
-
A Quantum EL Theorem
Authors:
Samuel Epstein
Abstract:
In this paper, we prove a quantum version of the EL Theorem. It states that non-exotic projections of large rank must have simple quantum states in their images. A consequence to this is there is no way to communicate a quantum source with corresponding large enough von Neumann entropy without using simple quantum states.
In this paper, we prove a quantum version of the EL Theorem. It states that non-exotic projections of large rank must have simple quantum states in their images. A consequence to this is there is no way to communicate a quantum source with corresponding large enough von Neumann entropy without using simple quantum states.
△ Less
Submitted 19 January, 2023;
originally announced January 2023.
-
Derandomization under Different Resource Constraints
Authors:
Samuel Epstein
Abstract:
We provide another proof to the EL Theorem. We show the tradeoff between compressibility of codebooks and their communication capacity. A resource bounded version of the EL Theorem is proven. This is used to prove three instances of resource bounded derandomization. This paper is in support of the general claim that if the existence of an object can be proven with the probabilistic method, then bo…
▽ More
We provide another proof to the EL Theorem. We show the tradeoff between compressibility of codebooks and their communication capacity. A resource bounded version of the EL Theorem is proven. This is used to prove three instances of resource bounded derandomization. This paper is in support of the general claim that if the existence of an object can be proven with the probabilistic method, then bounds on its Kolmogorov complexity can be proven as well.
△ Less
Submitted 12 September, 2023; v1 submitted 26 November, 2022;
originally announced November 2022.
-
Fitting a Directional Microstructure Model to Diffusion-Relaxation MRI Data with Self-Supervised Machine Learning
Authors:
Jason P. Lim,
Stefano B. Blumberg,
Neil Narayan,
Sean C. Epstein,
Daniel C. Alexander,
Marco Palombo,
Paddy J. Slator
Abstract:
Machine learning is a powerful approach for fitting microstructural models to diffusion MRI data. Early machine learning microstructure imaging implementations trained regressors to estimate model parameters in a supervised way, using synthetic training data with known ground truth. However, a drawback of this approach is that the choice of training data impacts fitted parameter values. Self-super…
▽ More
Machine learning is a powerful approach for fitting microstructural models to diffusion MRI data. Early machine learning microstructure imaging implementations trained regressors to estimate model parameters in a supervised way, using synthetic training data with known ground truth. However, a drawback of this approach is that the choice of training data impacts fitted parameter values. Self-supervised learning is emerging as an attractive alternative to supervised learning in this context. Thus far, both supervised and self-supervised learning have typically been applied to isotropic models, such as intravoxel incoherent motion (IVIM), as opposed to models where the directionality of anisotropic structures is also estimated. In this paper, we demonstrate self-supervised machine learning model fitting for a directional microstructural model. In particular, we fit a combined T1-ball-stick model to the multidimensional diffusion (MUDI) challenge diffusion-relaxation dataset. Our self-supervised approach shows clear improvements in parameter estimation and computational time, for both simulated and in-vivo brain data, compared to standard non-linear least squares fitting. Code for the artificial neural net constructed for this study is available for public use from the following GitHub repository: https://github.com/jplte/deep-T1-ball-stick
△ Less
Submitted 5 October, 2022;
originally announced October 2022.
-
22 Examples of Solution Compression via Derandomization
Authors:
Samuel Epstein
Abstract:
We provide bounds on the compression size of the solutions to 22 problems in computer science. For each problem, we show that solutions exist with high probability, for some simple probability measure. Once this is proven, derandomization can be used to prove the existence of a simple solution.
We provide bounds on the compression size of the solutions to 22 problems in computer science. For each problem, we show that solutions exist with high probability, for some simple probability measure. Once this is proven, derandomization can be used to prove the existence of a simple solution.
△ Less
Submitted 24 April, 2023; v1 submitted 24 August, 2022;
originally announced August 2022.
-
The Kolmogorov Birthday Paradox
Authors:
Samuel Epstein
Abstract:
We prove a Kolmogorov complexity variant of the birthday paradox. Sufficiently sized random subsets of strings are guaranteed to have two members x and y with low K(x/y). To prove this, we first show that the minimum conditional Kolmogorov complexity between members of finite sets is very low if they are not exotic. Exotic sets have high mutual information with the halting sequence.
We prove a Kolmogorov complexity variant of the birthday paradox. Sufficiently sized random subsets of strings are guaranteed to have two members x and y with low K(x/y). To prove this, we first show that the minimum conditional Kolmogorov complexity between members of finite sets is very low if they are not exotic. Exotic sets have high mutual information with the halting sequence.
△ Less
Submitted 19 May, 2023; v1 submitted 23 August, 2022;
originally announced August 2022.
-
Outliers, Dynamics, and the Independence Postulate
Authors:
Samuel Epstein
Abstract:
We show that outliers occur almost surely in computable dynamics over infinite sequences. Ever greater outliers can be found as the number of visited states increases. We show the Independence Postulate explains how outliers are found in the physical world. We generalize the outliers theorem to uncomputable sampling methods.
We show that outliers occur almost surely in computable dynamics over infinite sequences. Ever greater outliers can be found as the number of visited states increases. We show the Independence Postulate explains how outliers are found in the physical world. We generalize the outliers theorem to uncomputable sampling methods.
△ Less
Submitted 25 July, 2022; v1 submitted 8 July, 2022;
originally announced July 2022.
-
Choice of training label matters: how to best use deep learning for quantitative MRI parameter estimation
Authors:
Sean C. Epstein,
Timothy J. P. Bray,
Margaret Hall-Craggs,
Hui Zhang
Abstract:
Deep learning (DL) is gaining popularity as a parameter estimation method for quantitative MRI. A range of competing implementations have been proposed, relying on either supervised or self-supervised learning. Self-supervised approaches, sometimes referred to as unsupervised, have been loosely based on auto-encoders, whereas supervised methods have, to date, been trained on groundtruth labels. Th…
▽ More
Deep learning (DL) is gaining popularity as a parameter estimation method for quantitative MRI. A range of competing implementations have been proposed, relying on either supervised or self-supervised learning. Self-supervised approaches, sometimes referred to as unsupervised, have been loosely based on auto-encoders, whereas supervised methods have, to date, been trained on groundtruth labels. These two learning paradigms have been shown to have distinct strengths. Notably, self-supervised approaches have offered lower-bias parameter estimates than their supervised alternatives. This result is counterintuitive - incorporating prior knowledge with supervised labels should, in theory, lead to improved accuracy. In this work, we show that this apparent limitation of supervised approaches stems from the naive choice of groundtruth training labels. By training on labels which are deliberately not groundtruth, we show that the low-bias parameter estimation previously associated with self-supervised methods can be replicated - and improved on - within a supervised learning framework. This approach sets the stage for a single, unifying, deep learning parameter estimation framework, based on supervised learning, where trade-offs between bias and variance are made by careful adjustment of training label.
△ Less
Submitted 23 January, 2024; v1 submitted 11 May, 2022;
originally announced May 2022.
-
The Outlier Theorem Revisited
Authors:
Samuel Epstein
Abstract:
An outlier is a datapoint that is set apart from a sample population. The outlier theorem in algorithmic information theory states that given a computable sampling method, outliers must appear. We present a simple proof to the outlier theorem, with exponentially improved bounds. We extend the outlier theorem to ergodic dynamical systems which are guaranteed to hit ever larger outlier states with d…
▽ More
An outlier is a datapoint that is set apart from a sample population. The outlier theorem in algorithmic information theory states that given a computable sampling method, outliers must appear. We present a simple proof to the outlier theorem, with exponentially improved bounds. We extend the outlier theorem to ergodic dynamical systems which are guaranteed to hit ever larger outlier states with diminishing measures. We show how to construct deterministic functions from random ones, i.e. function derandomization. We also prove that all open sets of the Cantor space with large uniform measure will either have a simple computable member or high mutual information with the halting sequence.
△ Less
Submitted 25 June, 2023; v1 submitted 16 March, 2022;
originally announced March 2022.
-
On the Kolmogorov Complexity of Binary Classifiers
Authors:
Samuel Epstein
Abstract:
We provide tight upper and lower bounds on the expected minimum Kolmogorov complexity of binary classifiers that are consistent with labeled samples. The expected size is not more than complexity of the target concept plus the conditional entropy of the labels given the sample.
We provide tight upper and lower bounds on the expected minimum Kolmogorov complexity of binary classifiers that are consistent with labeled samples. The expected size is not more than complexity of the target concept plus the conditional entropy of the labels given the sample.
△ Less
Submitted 3 February, 2022; v1 submitted 28 January, 2022;
originally announced January 2022.
-
On the Algorithmic Content of Quantum Measurements
Authors:
Samuel Epstein
Abstract:
We show that given a quantum measurement, for an overwhelming majority of pure states, no meaningful information is produced. This is independent of the number of outcomes of the quantum measurement. Due to conservation inequalities, such random noise cannot be processed into coherent data.
We show that given a quantum measurement, for an overwhelming majority of pure states, no meaningful information is produced. This is independent of the number of outcomes of the quantum measurement. Due to conservation inequalities, such random noise cannot be processed into coherent data.
△ Less
Submitted 14 May, 2021; v1 submitted 7 February, 2021;
originally announced February 2021.
-
Epistemic Logic of Know-Who
Authors:
Sophia Epstein,
Pavel Naumov
Abstract:
The paper suggests a definition of "know who" as a modality using Grove-Halpern semantics of names. It also introduces a logical system that describes the interplay between modalities "knows who", "knows", and "for all agents". The main technical result is a completeness theorem for the proposed system.
The paper suggests a definition of "know who" as a modality using Grove-Halpern semantics of names. It also introduces a logical system that describes the interplay between modalities "knows who", "knows", and "for all agents". The main technical result is a completeness theorem for the proposed system.
△ Less
Submitted 2 March, 2021; v1 submitted 11 December, 2020;
originally announced December 2020.
-
Deliberate Exploration Supports Navigation in Unfamiliar Worlds
Authors:
Raj Korpan,
Susan L. Epstein
Abstract:
To perform tasks well in a new domain, one must first know something about it. This paper reports on a robot controller for navigation through unfamiliar indoor worlds. Based on spatial affordances, it integrates planning with reactive heuristics. Before it addresses specific targets, however, the system deliberately explores for high-level connectivity and captures that data in a cognitive spatia…
▽ More
To perform tasks well in a new domain, one must first know something about it. This paper reports on a robot controller for navigation through unfamiliar indoor worlds. Based on spatial affordances, it integrates planning with reactive heuristics. Before it addresses specific targets, however, the system deliberately explores for high-level connectivity and captures that data in a cognitive spatial model. Despite limited exploration time, planning in the resultant model is faster and better supports successful travel in a challenging, realistic space.
△ Less
Submitted 1 July, 2020;
originally announced July 2020.
-
Extending Chaitin's Incompleteness Theorem
Authors:
Samuel Epstein
Abstract:
Chaitin's incompleteness theorem states that sufficiently rich formal systems cannot prove lower bounds on Kolmogorov complexity. In this paper we extend this theorem by showing theories that prove the Kolmogorov complexity of a large (but finite) number of strings are inaccessible. This is done by first showing such theories have large information with the halting sequence. Then, by applying the…
▽ More
Chaitin's incompleteness theorem states that sufficiently rich formal systems cannot prove lower bounds on Kolmogorov complexity. In this paper we extend this theorem by showing theories that prove the Kolmogorov complexity of a large (but finite) number of strings are inaccessible. This is done by first showing such theories have large information with the halting sequence. Then, by applying the independence postulate, such theories are shown to be inaccessible in the physical world.
△ Less
Submitted 4 June, 2023; v1 submitted 29 January, 2020;
originally announced January 2020.
-
Distanced LSTM: Time-Distanced Gates in Long Short-Term Memory Models for Lung Cancer Detection
Authors:
Riqiang Gao,
Yuankai Huo,
Shunxing Bao,
Yucheng Tang,
Sanja L. Antic,
Emily S. Epstein,
Aneri B. Balar,
Steve Deppen,
Alexis B. Paulson,
Kim L. Sandler,
Pierre P. Massion,
Bennett A. Landman
Abstract:
The field of lung nodule detection and cancer prediction has been rapidly developing with the support of large public data archives. Previous studies have largely focused on cross-sectional (single) CT data. Herein, we consider longitudinal data. The Long Short-Term Memory (LSTM) model addresses learning with regularly spaced time points (i.e., equal temporal intervals). However, clinical imaging…
▽ More
The field of lung nodule detection and cancer prediction has been rapidly developing with the support of large public data archives. Previous studies have largely focused on cross-sectional (single) CT data. Herein, we consider longitudinal data. The Long Short-Term Memory (LSTM) model addresses learning with regularly spaced time points (i.e., equal temporal intervals). However, clinical imaging follows patient needs with often heterogeneous, irregular acquisitions. To model both regular and irregular longitudinal samples, we generalize the LSTM model with the Distanced LSTM (DLSTM) for temporally varied acquisitions. The DLSTM includes a Temporal Emphasis Model (TEM) that enables learning across regularly and irregularly sampled intervals. Briefly, (1) the time intervals between longitudinal scans are modeled explicitly, (2) temporally adjustable forget and input gates are introduced for irregular temporal sampling; and (3) the latest longitudinal scan has an additional emphasis term. We evaluate the DLSTM framework in three datasets including simulated data, 1794 National Lung Screening Trial (NLST) scans, and 1420 clinically acquired data with heterogeneous and irregular temporal accession. The experiments on the first two datasets demonstrate that our method achieves competitive performance on both simulated and regularly sampled datasets (e.g. improve LSTM from 0.6785 to 0.7085 on F1 score in NLST). In external validation of clinically and irregularly acquired data, the benchmarks achieved 0.8350 (CNN feature) and 0.8380 (LSTM) on the area under the ROC curve (AUC) score, while the proposed DLSTM achieves 0.8905.
△ Less
Submitted 11 September, 2019;
originally announced September 2019.
-
The EL Theorem
Authors:
Samuel Epstein
Abstract:
The combined universal probability $\mathbf{m}(D)$ of strings $x$ in sets $D$ is close to max $\mathbf{m}(x)$ over $x$ in $D$: their logs differ by at most $D$'s information $\mathbf{I}(D:\mathcal{H})$ about the halting sequence $\mathcal{H}$.
The combined universal probability $\mathbf{m}(D)$ of strings $x$ in sets $D$ is close to max $\mathbf{m}(x)$ over $x$ in $D$: their logs differ by at most $D$'s information $\mathbf{I}(D:\mathcal{H})$ about the halting sequence $\mathcal{H}$.
△ Less
Submitted 10 September, 2023; v1 submitted 10 July, 2019;
originally announced July 2019.
-
On the Conditional Complexity of Sets of Strings
Authors:
Samuel Epstein
Abstract:
Given a set X of finite strings, one interesting question to ask is whether there exists a member of X which is simple conditional to all other members of X. Conditional simplicity is measured by low conditional Kolmogorov complexity. We prove the affirmative to this question for sets that have low mutual information with the halting sequence. There are two results with respect to this question. O…
▽ More
Given a set X of finite strings, one interesting question to ask is whether there exists a member of X which is simple conditional to all other members of X. Conditional simplicity is measured by low conditional Kolmogorov complexity. We prove the affirmative to this question for sets that have low mutual information with the halting sequence. There are two results with respect to this question. One is dependent on the maximum conditional complexity between two elements of X, the other is dependent on the maximum expected value of the conditional complexity of a member of X relative to each member of X.
△ Less
Submitted 7 February, 2021; v1 submitted 1 July, 2019;
originally announced July 2019.
-
Algorithmic No-Cloning Theorem
Authors:
Samuel Epstein
Abstract:
We introduce the notions of algorithmic mutual information and rarity of quantum states. These definitions enjoy conservation inequalities over unitary transformations and partial traces. We show that a large majority of pure states have minute self algorithmic information. We provide an algorithmic variant to the no-cloning theorem, by showing that only a small minority of quantum pure states can…
▽ More
We introduce the notions of algorithmic mutual information and rarity of quantum states. These definitions enjoy conservation inequalities over unitary transformations and partial traces. We show that a large majority of pure states have minute self algorithmic information. We provide an algorithmic variant to the no-cloning theorem, by showing that only a small minority of quantum pure states can clone a non negligible amount of algorithmic information. We also provide a chain rule inequality for quantum algorithmic entropy. We show that rarity does not increase under POVM measurements.
△ Less
Submitted 10 January, 2021; v1 submitted 9 August, 2018;
originally announced August 2018.
-
MengeROS: a Crowd Simulation Tool for Autonomous Robot Navigation
Authors:
Anoop Aroor,
Susan L. Epstein,
Raj Korpan
Abstract:
While effective navigation in large, crowded environments is essential for an autonomous robot, preliminary testing of algorithms to support it requires simulation across a broad range of crowd scenarios. Most available simulation tools provide either realistic crowds without robots or realistic robots without realistic crowds. This paper introduces MengeROS, a 2-D simulator that realistically int…
▽ More
While effective navigation in large, crowded environments is essential for an autonomous robot, preliminary testing of algorithms to support it requires simulation across a broad range of crowd scenarios. Most available simulation tools provide either realistic crowds without robots or realistic robots without realistic crowds. This paper introduces MengeROS, a 2-D simulator that realistically integrates multiple robots and crowds. MengeROS provides a broad range of settings in which to test the capabilities and performance of navigation algorithms designed for large crowded environments.
△ Less
Submitted 26 January, 2018;
originally announced January 2018.
-
Toward Crowd-Sensitive Path Planning
Authors:
Anoop Aroor,
Susan L. Epstein
Abstract:
If a robot can predict crowds in parts of its environment that are inaccessible to its sensors, then it can plan to avoid them. This paper proposes a fast, online algorithm that learns average crowd densities in different areas. It also describes how these densities can be incorporated into existing navigation architectures. In simulation across multiple challenging crowd scenarios, the robot reac…
▽ More
If a robot can predict crowds in parts of its environment that are inaccessible to its sensors, then it can plan to avoid them. This paper proposes a fast, online algorithm that learns average crowd densities in different areas. It also describes how these densities can be incorporated into existing navigation architectures. In simulation across multiple challenging crowd scenarios, the robot reaches its target faster, travels less, and risks fewer collisions than if it were to plan with the traditional A* algorithm.
△ Less
Submitted 16 October, 2017;
originally announced October 2017.
-
WHY: Natural Explanations from a Robot Navigator
Authors:
Raj Korpan,
Susan L. Epstein,
Anoop Aroor,
Gil Dekel
Abstract:
Effective collaboration between a robot and a person requires natural communication. When a robot travels with a human companion, the robot should be able to explain its navigation behavior in natural language. This paper explains how a cognitively-based, autonomous robot navigation system produces informative, intuitive explanations for its decisions. Language generation here is based upon the ro…
▽ More
Effective collaboration between a robot and a person requires natural communication. When a robot travels with a human companion, the robot should be able to explain its navigation behavior in natural language. This paper explains how a cognitively-based, autonomous robot navigation system produces informative, intuitive explanations for its decisions. Language generation here is based upon the robot's commonsense, its qualitative reasoning, and its learned spatial model. This approach produces natural explanations in real time for a robot as it navigates in a large, complex indoor environment.
△ Less
Submitted 27 September, 2017;
originally announced September 2017.
-
An Extended Coding Theorem with Application to Quantum Complexities
Authors:
Samuel Epstein
Abstract:
This paper introduces a new inequality in algorithmic information theory that can be seen as an extended coding theorem. This inequality has applications in new bounds between quantum complexity measures.
This paper introduces a new inequality in algorithmic information theory that can be seen as an extended coding theorem. This inequality has applications in new bounds between quantum complexity measures.
△ Less
Submitted 22 January, 2021; v1 submitted 16 November, 2015;
originally announced November 2015.
-
Randomness Conservation over Algorithms
Authors:
Samuel Epstein
Abstract:
Current discrete randomness and information conservation inequalities are over total recursive functions, i.e. restricted to deterministic processing. This restriction implies that an algorithm can break algorithmic randomness conservation inequalities. We address this issue by proving tight bounds of randomness and information conservation with respect to recursively enumerable transformations, i…
▽ More
Current discrete randomness and information conservation inequalities are over total recursive functions, i.e. restricted to deterministic processing. This restriction implies that an algorithm can break algorithmic randomness conservation inequalities. We address this issue by proving tight bounds of randomness and information conservation with respect to recursively enumerable transformations, i.e. processing by algorithms. We also show conservation of randomness of finite strings with respect to enumerable distributions, i.e. semicomputable semi-measures.
△ Less
Submitted 14 October, 2013; v1 submitted 11 July, 2013;
originally announced July 2013.
-
All Sampling Methods Produce Outliers
Authors:
Samuel Epstein
Abstract:
Given a computable probability measure P over natural numbers or infinite binary sequences, there is no computable, randomized method that can produce an arbitrarily large sample such that none of its members are outliers of P.
Given a computable probability measure P over natural numbers or infinite binary sequences, there is no computable, randomized method that can produce an arbitrarily large sample such that none of its members are outliers of P.
△ Less
Submitted 27 July, 2022; v1 submitted 13 April, 2013;
originally announced April 2013.
-
Sets Have Simple Members
Authors:
Samuel Epstein,
Leonid A. Levin
Abstract:
The combined Universal Probability M(D) of strings x in sets D is close to max M({x}) over x in D: their ~logs differ by at most D's information j=I(D:H) about the halting sequence H. Thus if all x have complexity K(x) >k, D carries >i bits of information on each its x where i+j ~ k. Note that there are no ways to generate D with significant I(D:H).
The combined Universal Probability M(D) of strings x in sets D is close to max M({x}) over x in D: their ~logs differ by at most D's information j=I(D:H) about the halting sequence H. Thus if all x have complexity K(x) >k, D carries >i bits of information on each its x where i+j ~ k. Note that there are no ways to generate D with significant I(D:H).
△ Less
Submitted 20 March, 2014; v1 submitted 7 July, 2011;
originally announced July 2011.
-
An Information Theoretic Representation of Agent Dynamics as Set Intersections
Authors:
Samuel Epstein,
Margrit Betke
Abstract:
We represent agents as sets of strings. Each string encodes a potential interaction with another agent or environment. We represent the total set of dynamics between two agents as the intersection of their respective strings, we prove complexity properties of player interactions using Algorithmic Information Theory. We show how the proposed construction is compatible with Universal Artificial Inte…
▽ More
We represent agents as sets of strings. Each string encodes a potential interaction with another agent or environment. We represent the total set of dynamics between two agents as the intersection of their respective strings, we prove complexity properties of player interactions using Algorithmic Information Theory. We show how the proposed construction is compatible with Universal Artificial Intelligence, in that the AIXI model can be seen as universal with respect to interaction.
△ Less
Submitted 5 July, 2011;
originally announced July 2011.
