Showing 1–32 of 32 results for author: Vyas, N

Quasi-Linear Size PCPs with Small Soundness from HDX

Authors: Mitali Bafna, Dor Minzer, Nikhil Vyas

Abstract: We construct 2-query, quasi-linear sized probabilistically checkable proofs (PCPs) with arbitrarily small constant soundness, improving upon Dinur's 2-query quasi-linear size PCPs with soundness $1-Ω(1)$. As an immediate corollary, we get that under the exponential time hypothesis, for all $ε>0$ no approximation algorithm for $3$-SAT can obtain an approximation ratio of $7/8+ε$ in time… ▽ More We construct 2-query, quasi-linear sized probabilistically checkable proofs (PCPs) with arbitrarily small constant soundness, improving upon Dinur's 2-query quasi-linear size PCPs with soundness $1-Ω(1)$. As an immediate corollary, we get that under the exponential time hypothesis, for all $ε>0$ no approximation algorithm for $3$-SAT can obtain an approximation ratio of $7/8+ε$ in time $2^{n/\log^C n}$, where $C$ is a constant depending on $ε$. Our result builds on a recent line of works showing the existence of linear sized direct product testers with small soundness by independent works of Bafna, Lifshitz, and Minzer, and of Dikstein, Dinur, and Lubotzky. The main new ingredient in our proof is a technique that embeds a given PCP construction into a PCP on a prescribed graph, provided that the latter is a graph underlying a sufficiently good high-dimensional expander. Towards this end, we use ideas from fault-tolerant distributed computing, and more precisely from the literature of the almost everywhere agreement problem starting with the work of Dwork, Peleg, Pippenger, and Upfal (1986). We show that graphs underlying HDXs admit routing protocols that are tolerant to adversarial edge corruptions, and in doing so we also improve the state of the art in this line of work. Our PCP construction requires variants of the aforementioned direct product testers with poly-logarithmic degree. The existence and constructability of these variants is shown in an appendix by Zhiwei Yun. △ Less

Submitted 17 July, 2024; originally announced July 2024.

Comments: Appendix D by Zhiwei Yun showing that variants of the Chapman-Lubotzky complexes can be constructed with $q=\text{polylog}(n)$

Deconstructing What Makes a Good Optimizer for Language Models

Authors: Rosie Zhao, Depen Morwani, David Brandfonbrener, Nikhil Vyas, Sham Kakade

Abstract: Training language models becomes increasingly expensive with scale, prompting numerous attempts to improve optimization efficiency. Despite these efforts, the Adam optimizer remains the most widely used, due to a prevailing view that it is the most effective approach. We aim to compare several optimization algorithms, including SGD, Adafactor, Adam, and Lion, in the context of autoregressive langu… ▽ More Training language models becomes increasingly expensive with scale, prompting numerous attempts to improve optimization efficiency. Despite these efforts, the Adam optimizer remains the most widely used, due to a prevailing view that it is the most effective approach. We aim to compare several optimization algorithms, including SGD, Adafactor, Adam, and Lion, in the context of autoregressive language modeling across a range of model sizes, hyperparameters, and architecture variants. Our findings indicate that, except for SGD, these algorithms all perform comparably both in their optimal performance and also in terms of how they fare across a wide range of hyperparameter choices. Our results suggest to practitioners that the choice of optimizer can be guided by practical considerations like memory constraints and ease of implementation, as no single algorithm emerged as a clear winner in terms of performance or stability to hyperparameter misspecification. Given our findings, we further dissect these approaches, examining two simplified versions of Adam: a) signed momentum (Signum) which we see recovers both the performance and hyperparameter stability of Adam and b) Adalayer, a layerwise variant of Adam which we introduce to study Adam's preconditioning. Examining Adalayer leads us to the conclusion that the largest impact of Adam's preconditioning is restricted to the last layer and LayerNorm parameters, and, perhaps surprisingly, the remaining layers can be trained with SGD. △ Less

Submitted 10 July, 2024; originally announced July 2024.

A New Perspective on Shampoo's Preconditioner

Authors: Depen Morwani, Itai Shapira, Nikhil Vyas, Eran Malach, Sham Kakade, Lucas Janson

Abstract: Shampoo, a second-order optimization algorithm which uses a Kronecker product preconditioner, has recently garnered increasing attention from the machine learning community. The preconditioner used by Shampoo can be viewed either as an approximation of the Gauss--Newton component of the Hessian or the covariance matrix of the gradients maintained by Adagrad. We provide an explicit and novel connec… ▽ More Shampoo, a second-order optimization algorithm which uses a Kronecker product preconditioner, has recently garnered increasing attention from the machine learning community. The preconditioner used by Shampoo can be viewed either as an approximation of the Gauss--Newton component of the Hessian or the covariance matrix of the gradients maintained by Adagrad. We provide an explicit and novel connection between the $\textit{optimal}$ Kronecker product approximation of these matrices and the approximation made by Shampoo. Our connection highlights a subtle but common misconception about Shampoo's approximation. In particular, the $\textit{square}$ of the approximation used by the Shampoo optimizer is equivalent to a single step of the power iteration algorithm for computing the aforementioned optimal Kronecker product approximation. Across a variety of datasets and architectures we empirically demonstrate that this is close to the optimal Kronecker product approximation. Additionally, for the Hessian approximation viewpoint, we empirically study the impact of various practical tricks to make Shampoo more computationally efficient (such as using the batch gradient and the empirical Fisher) on the quality of Hessian approximation. △ Less

Submitted 25 June, 2024; originally announced June 2024.

Relaxing Trust Assumptions on Quantum Key Distribution Networks

Authors: Nilesh Vyas, Paulo Mendes

Abstract: Quantum security over long distances with untrusted relays is largely unfounded and is still an open question for active research. Nevertheless, quantum networks based on trusted relays are being built across the globe. However, standard QKD network architecture implores a complete trust requirement on QKD relays, which is too demanding and limits the use cases for QKD networks. In this work, we e… ▽ More Quantum security over long distances with untrusted relays is largely unfounded and is still an open question for active research. Nevertheless, quantum networks based on trusted relays are being built across the globe. However, standard QKD network architecture implores a complete trust requirement on QKD relays, which is too demanding and limits the use cases for QKD networks. In this work, we explore the possibility to securely relay a secret in a QKD network by relaxing the trust assumptions (if not completely) on the relay. We characterize QKD relays with different trust levels, namely, Full Access Trust (FAT), Partial Access Trust (PAT), and No Access Trust (NAT). As the name suggests, each level defines the degree with which a relay is required to be trusted with the secret provided by the key management system for end-to-end communication. We then review and propose multiple constructions of the QKD key management system based on the different trust levels. Main contribution of the paper is realized by evaluating key management systems with no access trust level. In principle, we review key management with centralized topology and propose a new decentralized key management system. These different topologies provide various advantages based on the QKD network requirements, allowing an operational flexibility in the architecture. We believe this work presents a new perspective to the open problem of providing a confiding and a practical solution for future long range secure communications △ Less

Submitted 20 February, 2024; originally announced February 2024.

Distinguishing the Knowable from the Unknowable with Language Models

Authors: Gustaf Ahdritz, Tian Qin, Nikhil Vyas, Boaz Barak, Benjamin L. Edelman

Abstract: We study the feasibility of identifying epistemic uncertainty (reflecting a lack of knowledge), as opposed to aleatoric uncertainty (reflecting entropy in the underlying distribution), in the outputs of large language models (LLMs) over free-form text. In the absence of ground-truth probabilities, we explore a setting where, in order to (approximately) disentangle a given LLM's uncertainty, a sign… ▽ More We study the feasibility of identifying epistemic uncertainty (reflecting a lack of knowledge), as opposed to aleatoric uncertainty (reflecting entropy in the underlying distribution), in the outputs of large language models (LLMs) over free-form text. In the absence of ground-truth probabilities, we explore a setting where, in order to (approximately) disentangle a given LLM's uncertainty, a significantly larger model stands in as a proxy for the ground truth. We show that small linear probes trained on the embeddings of frozen, pretrained models accurately predict when larger models will be more confident at the token level and that probes trained on one text domain generalize to others. Going further, we propose a fully unsupervised method that achieves non-trivial accuracy on the same task. Taken together, we interpret these results as evidence that LLMs naturally contain internal representations of different types of uncertainty that could potentially be leveraged to devise more informative indicators of model confidence in diverse practical settings. △ Less

Submitted 27 February, 2024; v1 submitted 5 February, 2024; originally announced February 2024.

Gemini: A Family of Highly Capable Multimodal Models

Authors: Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M. Dai, Anja Hauth, Katie Millican, David Silver, Melvin Johnson, Ioannis Antonoglou, Julian Schrittwieser, Amelia Glaese, Jilin Chen, Emily Pitler, Timothy Lillicrap, Angeliki Lazaridou, Orhan Firat, James Molloy, Michael Isard, Paul R. Barham, Tom Hennigan, Benjamin Lee , et al. (1325 additional authors not shown)

Abstract: This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family consists of Ultra, Pro, and Nano sizes, suitable for applications ranging from complex reasoning tasks to on-device memory-constrained use-cases. Evaluation on a broad range of benchmarks shows that our most-capable Gemini Ultr… ▽ More This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family consists of Ultra, Pro, and Nano sizes, suitable for applications ranging from complex reasoning tasks to on-device memory-constrained use-cases. Evaluation on a broad range of benchmarks shows that our most-capable Gemini Ultra model advances the state of the art in 30 of 32 of these benchmarks - notably being the first model to achieve human-expert performance on the well-studied exam benchmark MMLU, and improving the state of the art in every one of the 20 multimodal benchmarks we examined. We believe that the new capabilities of the Gemini family in cross-modal reasoning and language understanding will enable a wide variety of use cases. We discuss our approach toward post-training and deploying Gemini models responsibly to users through services including Gemini, Gemini Advanced, Google AI Studio, and Cloud Vertex AI. △ Less

Submitted 17 June, 2024; v1 submitted 18 December, 2023; originally announced December 2023.

On Privileged and Convergent Bases in Neural Network Representations

Authors: Davis Brown, Nikhil Vyas, Yamini Bansal

Abstract: In this study, we investigate whether the representations learned by neural networks possess a privileged and convergent basis. Specifically, we examine the significance of feature directions represented by individual neurons. First, we establish that arbitrary rotations of neural representations cannot be inverted (unlike linear networks), indicating that they do not exhibit complete rotational i… ▽ More In this study, we investigate whether the representations learned by neural networks possess a privileged and convergent basis. Specifically, we examine the significance of feature directions represented by individual neurons. First, we establish that arbitrary rotations of neural representations cannot be inverted (unlike linear networks), indicating that they do not exhibit complete rotational invariance. Subsequently, we explore the possibility of multiple bases achieving identical performance. To do this, we compare the bases of networks trained with the same parameters but with varying random initializations. Our study reveals two findings: (1) Even in wide networks such as WideResNets, neural networks do not converge to a unique basis; (2) Basis correlation increases significantly when a few early layers of the network are frozen identically. Furthermore, we analyze Linear Mode Connectivity, which has been studied as a measure of basis correlation. Our findings give evidence that while Linear Mode Connectivity improves with increased network width, this improvement is not due to an increase in basis correlation. △ Less

Submitted 24 July, 2023; originally announced July 2023.

Comments: In the Workshop on High-dimensional Learning Dynamics at ICML 2023

Beyond Implicit Bias: The Insignificance of SGD Noise in Online Learning

Authors: Nikhil Vyas, Depen Morwani, Rosie Zhao, Gal Kaplun, Sham Kakade, Boaz Barak

Abstract: The success of SGD in deep learning has been ascribed by prior works to the implicit bias induced by finite batch sizes ("SGD noise"). While prior works focused on offline learning (i.e., multiple-epoch training), we study the impact of SGD noise on online (i.e., single epoch) learning. Through an extensive empirical analysis of image and language data, we demonstrate that small batch sizes do not… ▽ More The success of SGD in deep learning has been ascribed by prior works to the implicit bias induced by finite batch sizes ("SGD noise"). While prior works focused on offline learning (i.e., multiple-epoch training), we study the impact of SGD noise on online (i.e., single epoch) learning. Through an extensive empirical analysis of image and language data, we demonstrate that small batch sizes do not confer any implicit bias advantages in online learning. In contrast to offline learning, the benefits of SGD noise in online learning are strictly computational, facilitating more cost-effective gradient steps. This suggests that SGD in the online regime can be construed as taking noisy steps along the "golden path" of the noiseless gradient descent algorithm. We study this hypothesis and provide supporting evidence in loss and function space. Our findings challenge the prevailing understanding of SGD and offer novel insights into its role in online learning. △ Less

Submitted 7 June, 2024; v1 submitted 14 June, 2023; originally announced June 2023.

Feature-Learning Networks Are Consistent Across Widths At Realistic Scales

Authors: Nikhil Vyas, Alexander Atanasov, Blake Bordelon, Depen Morwani, Sabarish Sainathan, Cengiz Pehlevan

Abstract: We study the effect of width on the dynamics of feature-learning neural networks across a variety of architectures and datasets. Early in training, wide neural networks trained on online data have not only identical loss curves but also agree in their point-wise test predictions throughout training. For simple tasks such as CIFAR-5m this holds throughout training for networks of realistic widths.… ▽ More We study the effect of width on the dynamics of feature-learning neural networks across a variety of architectures and datasets. Early in training, wide neural networks trained on online data have not only identical loss curves but also agree in their point-wise test predictions throughout training. For simple tasks such as CIFAR-5m this holds throughout training for networks of realistic widths. We also show that structural properties of the models, including internal representations, preactivation distributions, edge of stability phenomena, and large learning rate effects are consistent across large widths. This motivates the hypothesis that phenomena seen in realistic models can be captured by infinite-width, feature-learning limits. For harder tasks (such as ImageNet and language modeling), and later training times, finite-width deviations grow systematically. Two distinct effects cause these deviations across widths. First, the network output has initialization-dependent variance scaling inversely with width, which can be removed by ensembling networks. We observe, however, that ensembles of narrower networks perform worse than a single wide network. We call this the bias of narrower width. We conclude with a spectral perspective on the origin of this finite-width bias. △ Less

Submitted 5 December, 2023; v1 submitted 28 May, 2023; originally announced May 2023.

Comments: 24 pages, 19 figures. NeurIPS 2023. Revised based on reviewer feedback

On Provable Copyright Protection for Generative Models

Authors: Nikhil Vyas, Sham Kakade, Boaz Barak

Abstract: There is a growing concern that learned conditional generative models may output samples that are substantially similar to some copyrighted data $C$ that was in their training set. We give a formal definition of $\textit{near access-freeness (NAF)}$ and prove bounds on the probability that a model satisfying this definition outputs a sample similar to $C$, even if $C$ is included in its training s… ▽ More There is a growing concern that learned conditional generative models may output samples that are substantially similar to some copyrighted data $C$ that was in their training set. We give a formal definition of $\textit{near access-freeness (NAF)}$ and prove bounds on the probability that a model satisfying this definition outputs a sample similar to $C$, even if $C$ is included in its training set. Roughly speaking, a generative model $p$ is $\textit{$k$-NAF}$ if for every potentially copyrighted data $C$, the output of $p$ diverges by at most $k$-bits from the output of a model $q$ that $\textit{did not access $C$ at all}$. We also give generative model learning algorithms, which efficiently modify the original generative model learning algorithm in a black box manner, that output generative models with strong bounds on the probability of sampling protected content. Furthermore, we provide promising experiments for both language (transformers) and image (diffusion) generative models, showing minimal degradation in output quality while ensuring strong protections against sampling protected content. △ Less

Submitted 21 July, 2023; v1 submitted 21 February, 2023; originally announced February 2023.

Comments: Accepted at ICML 2023

On the Number of Quantifiers as a Complexity Measure

Authors: Ronald Fagin, Jonathan Lenchner, Nikhil Vyas, Ryan Williams

Abstract: In 1981, Neil Immerman described a two-player game, which he called the "separability game" \cite{Immerman81}, that captures the number of quantifiers needed to describe a property in first-order logic. Immerman's paper laid the groundwork for studying the number of quantifiers needed to express properties in first-order logic, but the game seemed to be too complicated to study, and the arguments… ▽ More In 1981, Neil Immerman described a two-player game, which he called the "separability game" \cite{Immerman81}, that captures the number of quantifiers needed to describe a property in first-order logic. Immerman's paper laid the groundwork for studying the number of quantifiers needed to express properties in first-order logic, but the game seemed to be too complicated to study, and the arguments of the paper almost exclusively used quantifier rank as a lower bound on the total number of quantifiers. However, last year Fagin, Lenchner, Regan and Vyas rediscovered the games, provided some tools for analyzing them, and showed how to utilize them to characterize the number of quantifiers needed to express linear orders of different sizes. In this paper, we push forward in the study of number of quantifiers as a bona fide complexity measure by establishing several new results. First we carefully distinguish minimum number of quantifiers from the more usual descriptive complexity measures, minimum quantifier rank and minimum number of variables. Then, for each positive integer $k$, we give an explicit example of a property of finite structures (in particular, of finite graphs) that can be expressed with a sentence of quantifier rank $k$, but where the same property needs $2^{Ω(k^2)}$ quantifiers to be expressed. △ Less

Submitted 4 July, 2022; v1 submitted 30 June, 2022; originally announced July 2022.

ACM Class: F.4.1

Limitations of the NTK for Understanding Generalization in Deep Learning

Authors: Nikhil Vyas, Yamini Bansal, Preetum Nakkiran

Abstract: The ``Neural Tangent Kernel'' (NTK) (Jacot et al 2018), and its empirical variants have been proposed as a proxy to capture certain behaviors of real neural networks. In this work, we study NTKs through the lens of scaling laws, and demonstrate that they fall short of explaining important aspects of neural network generalization. In particular, we demonstrate realistic settings where finite-width… ▽ More The ``Neural Tangent Kernel'' (NTK) (Jacot et al 2018), and its empirical variants have been proposed as a proxy to capture certain behaviors of real neural networks. In this work, we study NTKs through the lens of scaling laws, and demonstrate that they fall short of explaining important aspects of neural network generalization. In particular, we demonstrate realistic settings where finite-width neural networks have significantly better data scaling exponents as compared to their corresponding empirical and infinite NTKs at initialization. This reveals a more fundamental difference between the real networks and NTKs, beyond just a few percentage points of test accuracy. Further, we show that even if the empirical NTK is allowed to be pre-trained on a constant number of samples, the kernel scaling does not catch up to the neural network scaling. Finally, we show that the empirical NTK continues to evolve throughout most of the training, in contrast with prior work which suggests that it stabilizes after a few epochs of training. Altogether, our work establishes concrete limitations of the NTK approach in understanding generalization of real networks on natural datasets. △ Less

Submitted 20 June, 2022; originally announced June 2022.

Fix your Models by Fixing your Datasets

Authors: Atindriyo Sanyal, Vikram Chatterji, Nidhi Vyas, Ben Epstein, Nikita Demir, Anthony Corletti

Abstract: The quality of underlying training data is very crucial for building performant machine learning models with wider generalizabilty. However, current machine learning (ML) tools lack streamlined processes for improving the data quality. So, getting data quality insights and iteratively pruning the errors to obtain a dataset which is most representative of downstream use cases is still an ad-hoc man… ▽ More The quality of underlying training data is very crucial for building performant machine learning models with wider generalizabilty. However, current machine learning (ML) tools lack streamlined processes for improving the data quality. So, getting data quality insights and iteratively pruning the errors to obtain a dataset which is most representative of downstream use cases is still an ad-hoc manual process. Our work addresses this data tooling gap, required to build improved ML workflows purely through data-centric techniques. More specifically, we introduce a systematic framework for (1) finding noisy or mislabelled samples in the dataset and, (2) identifying the most informative samples, which when included in training would provide maximal model performance lift. We demonstrate the efficacy of our framework on public as well as private enterprise datasets of two Fortune 500 companies, and are confident this work will form the basis for ML teams to perform more intelligent data discovery and pruning. △ Less

Submitted 14 December, 2021; originally announced December 2021.

Namesakes: Ambiguously Named Entities from Wikipedia and News

Authors: Oleg Vasilyev, Aysu Altun, Nidhi Vyas, Vedant Dharnidharka, Erika Lam, John Bohannon

Abstract: We present Namesakes, a dataset of ambiguously named entities obtained from English-language Wikipedia and news articles. It consists of 58862 mentions of 4148 unique entities and their namesakes: 1000 mentions from news, 28843 from Wikipedia articles about the entity, and 29019 Wikipedia backlink mentions. Namesakes should be helpful in establishing challenging benchmarks for the task of named en… ▽ More We present Namesakes, a dataset of ambiguously named entities obtained from English-language Wikipedia and news articles. It consists of 58862 mentions of 4148 unique entities and their namesakes: 1000 mentions from news, 28843 from Wikipedia articles about the entity, and 29019 Wikipedia backlink mentions. Namesakes should be helpful in establishing challenging benchmarks for the task of named entity linking (NEL). △ Less

Submitted 22 November, 2021; originally announced November 2021.

Comments: 11 pages, 6 figures

Optimal Fine-grained Hardness of Approximation of Linear Equations

Authors: Mitali Bafna, Nikhil Vyas

Abstract: The problem of solving linear systems is one of the most fundamental problems in computer science, where given a satisfiable linear system $(A,b)$, for $A \in \mathbb{R}^{n \times n}$ and $b \in \mathbb{R}^n$, we wish to find a vector $x \in \mathbb{R}^n$ such that $Ax = b$. The current best algorithms for solving dense linear systems reduce the problem to matrix multiplication, and run in time… ▽ More The problem of solving linear systems is one of the most fundamental problems in computer science, where given a satisfiable linear system $(A,b)$, for $A \in \mathbb{R}^{n \times n}$ and $b \in \mathbb{R}^n$, we wish to find a vector $x \in \mathbb{R}^n$ such that $Ax = b$. The current best algorithms for solving dense linear systems reduce the problem to matrix multiplication, and run in time $O(n^ω)$. We consider the problem of finding $\varepsilon$-approximate solutions to linear systems with respect to the $L_2$-norm, that is, given a satisfiable linear system $(A \in \mathbb{R}^{n \times n}, b \in \mathbb{R}^n)$, find an $x \in \mathbb{R}^n$ such that $||Ax - b||_2 \leq \varepsilon||b||_2$. Our main result is a fine-grained reduction from computing the rank of a matrix to finding $\varepsilon$-approximate solutions to linear systems. In particular, if the best known $O(n^ω)$ time algorithm for computing the rank of $n \times O(n)$ matrices is optimal (which we conjecture is true), then finding an $\varepsilon$-approximate solution to a dense linear system also requires $\tildeΩ(n^ω)$ time, even for $\varepsilon$ as large as $(1 - 1/\text{poly}(n))$. We also prove (under some modified conjectures for the rank-finding problem) optimal hardness of approximation for sparse linear systems, linear systems over positive semidefinite matrices, well-conditioned linear systems, and approximately solving linear systems with respect to the $L_p$-norm, for $p \geq 1$. At the heart of our results is a novel reduction from the rank problem to a decision version of the approximate linear systems problem. This reduction preserves properties such as matrix sparsity and bit complexity. △ Less

Submitted 24 June, 2021; originally announced June 2021.

Comments: To appear in ICALP 2021

Training With Data Dependent Dynamic Learning Rates

Authors: Shreyas Saxena, Nidhi Vyas, Dennis DeCoste

Abstract: Recently many first and second order variants of SGD have been proposed to facilitate training of Deep Neural Networks (DNNs). A common limitation of these works stem from the fact that they use the same learning rate across all instances present in the dataset. This setting is widely adopted under the assumption that loss functions for each instance are similar in nature, and hence, a common lear… ▽ More Recently many first and second order variants of SGD have been proposed to facilitate training of Deep Neural Networks (DNNs). A common limitation of these works stem from the fact that they use the same learning rate across all instances present in the dataset. This setting is widely adopted under the assumption that loss functions for each instance are similar in nature, and hence, a common learning rate can be used. In this work, we relax this assumption and propose an optimization framework which accounts for difference in loss function characteristics across instances. More specifically, our optimizer learns a dynamic learning rate for each instance present in the dataset. Learning a dynamic learning rate for each instance allows our optimization framework to focus on different modes of training data during optimization. When applied to an image classification task, across different CNN architectures, learning dynamic learning rates leads to consistent gains over standard optimizers. When applied to a dataset containing corrupt instances, our framework reduces the learning rates on noisy instances, and improves over the state-of-the-art. Finally, we show that our optimization framework can be used for personalization of a machine learning model towards a known targeted data distribution. △ Less

Submitted 27 May, 2021; originally announced May 2021.

Multi-Structural Games and Number of Quantifiers

Authors: Ronald Fagin, Jonathan Lenchner, Kenneth W. Regan, Nikhil Vyas

Abstract: We study multi-structural games, played on two sets $\mathcal{A}$ and $\mathcal{B}$ of structures. These games generalize Ehrenfeucht-Fraïssé games. Whereas Ehrenfeucht-Fraïssé games capture the quantifier rank of a first-order sentence, multi-structural games capture the number of quantifiers, in the sense that Spoiler wins the $r$-round game if and only if there is a first-order sentence $φ$ wit… ▽ More We study multi-structural games, played on two sets $\mathcal{A}$ and $\mathcal{B}$ of structures. These games generalize Ehrenfeucht-Fraïssé games. Whereas Ehrenfeucht-Fraïssé games capture the quantifier rank of a first-order sentence, multi-structural games capture the number of quantifiers, in the sense that Spoiler wins the $r$-round game if and only if there is a first-order sentence $φ$ with at most $r$ quantifiers, where every structure in $\mathcal{A}$ satisfies $φ$ and no structure in $\mathcal{B}$ satisfies $φ$. We use these games to give a complete characterization of the number of quantifiers required to distinguish linear orders of different sizes, and develop machinery for analyzing structures beyond linear orders. △ Less

Submitted 3 March, 2022; v1 submitted 29 April, 2021; originally announced April 2021.

Comments: Appeared in LICS 2021

Faster object tracking pipeline for real time tracking

Authors: Parthesh Soni, Falak Shah, Nisarg Vyas

Abstract: Multi-object tracking (MOT) is a challenging practical problem for vision based applications. Most recent approaches for MOT use precomputed detections from models such as Faster RCNN, performing fine-tuning of bounding boxes and association in subsequent phases. However, this is not suitable for actual industrial applications due to unavailability of detections upfront. In their recent work, Wang… ▽ More Multi-object tracking (MOT) is a challenging practical problem for vision based applications. Most recent approaches for MOT use precomputed detections from models such as Faster RCNN, performing fine-tuning of bounding boxes and association in subsequent phases. However, this is not suitable for actual industrial applications due to unavailability of detections upfront. In their recent work, Wang et al. proposed a tracking pipeline that uses a Joint detection and embedding model and performs target localization and association in realtime. Upon investigating the tracking by detection paradigm, we find that the tracking pipeline can be made faster by performing localization and association tasks parallely with model prediction. This, and other computational optimizations such as using mixed precision model and performing batchwise detection result in a speed-up of the tracking pipeline by 57.8\% (19 FPS to 30 FPS) on FullHD resolution. Moreover, the speed is independent of the object density in image sequence. The main contribution of this paper is showcasing a generic pipeline which can be used to speed up detection based object tracking methods. We also reviewed different batch sizes for optimal performance, taking into consideration GPU memory usage and speed. △ Less

Submitted 8 November, 2020; originally announced November 2020.

Comments: 12 pages, 6 figures

Fast Low-Space Algorithms for Subset Sum

Authors: Ce Jin, Nikhil Vyas, Ryan Williams

Abstract: We consider the canonical Subset Sum problem: given a list of positive integers $a_1,\ldots,a_n$ and a target integer $t$ with $t > a_i$ for all $i$, determine if there is an $S \subseteq [n]$ such that $\sum_{i \in S} a_i = t$. The well-known pseudopolynomial-time dynamic programming algorithm [Bellman, 1957] solves Subset Sum in $O(nt)$ time, while requiring $Ω(t)$ space. In this paper we pres… ▽ More We consider the canonical Subset Sum problem: given a list of positive integers $a_1,\ldots,a_n$ and a target integer $t$ with $t > a_i$ for all $i$, determine if there is an $S \subseteq [n]$ such that $\sum_{i \in S} a_i = t$. The well-known pseudopolynomial-time dynamic programming algorithm [Bellman, 1957] solves Subset Sum in $O(nt)$ time, while requiring $Ω(t)$ space. In this paper we present algorithms for Subset Sum with $\tilde O(nt)$ running time and much lower space requirements than Bellman's algorithm, as well as that of prior work. We show that Subset Sum can be solved in $\tilde O(nt)$ time and $O(\log(nt))$ space with access to $O(\log n \log \log n+\log t)$ random bits. This significantly improves upon the $\tilde O(n t^{1+\varepsilon})$-time, $\tilde O(n\log t)$-space algorithm of Bringmann (SODA 2017). We also give an $\tilde O(n^{1+\varepsilon}t)$-time, $O(\log(nt))$-space randomized algorithm, improving upon previous $(nt)^{O(1)}$-time $O(\log(nt))$-space algorithms by Elberfeld, Jakoby, and Tantau (FOCS 2010), and Kane (2010). In addition, we also give a $\mathrm{poly} \log(nt)$-space, $\tilde O(n^2 t)$-time deterministic algorithm. We also study time-space trade-offs for Subset Sum. For parameter $1\le k\le \min\{n,t\}$, we present a randomized algorithm running in $\tilde O((n+t)\cdot k)$ time and $O((t/k) \mathrm{polylog} (nt))$ space. As an application of our results, we give an $\tilde{O}(\min\{n^2/\varepsilon, n/\varepsilon^2\})$-time and $\mathrm{polylog}(nt)$-space algorithm for "weak" $\varepsilon$-approximations of Subset Sum. △ Less

Submitted 7 November, 2020; originally announced November 2020.

Comments: To appear in SODA 2021

Learning Soft Labels via Meta Learning

Authors: Nidhi Vyas, Shreyas Saxena, Thomas Voice

Abstract: One-hot labels do not represent soft decision boundaries among concepts, and hence, models trained on them are prone to overfitting. Using soft labels as targets provide regularization, but different soft labels might be optimal at different stages of optimization. Also, training with fixed labels in the presence of noisy annotations leads to worse generalization. To address these limitations, we… ▽ More One-hot labels do not represent soft decision boundaries among concepts, and hence, models trained on them are prone to overfitting. Using soft labels as targets provide regularization, but different soft labels might be optimal at different stages of optimization. Also, training with fixed labels in the presence of noisy annotations leads to worse generalization. To address these limitations, we propose a framework, where we treat the labels as learnable parameters, and optimize them along with model parameters. The learned labels continuously adapt themselves to the model's state, thereby providing dynamic regularization. When applied to the task of supervised image-classification, our method leads to consistent gains across different datasets and architectures. For instance, dynamically learned labels improve ResNet18 by 2.1% on CIFAR100. When applied to dataset containing noisy labels, the learned labels correct the annotation mistakes, and improves over state-of-the-art by a significant margin. Finally, we show that learned labels capture semantic relationship between classes, and thereby improve teacher models for the downstream task of distillation. △ Less

Submitted 20 September, 2020; originally announced September 2020.

Various Secure Routing Schemes for MANETs: A Survey

Authors: Priya R. Soni, Charmi A. Joshi, Dhwani R. Bhadra, Nikita P. Vyas, Rutvij H. Jhaveri

Abstract: MANET is an infrastructure less as well as self configuring network consisting of mobile nodes communicating with each other using radio medium. Its exclusive properties such as dynamic topology, decentralization, and wireless medium make MANET to become very unique network amongst other traditional networks, thereby determining security to be a major challenge. In this paper, we have carried out… ▽ More MANET is an infrastructure less as well as self configuring network consisting of mobile nodes communicating with each other using radio medium. Its exclusive properties such as dynamic topology, decentralization, and wireless medium make MANET to become very unique network amongst other traditional networks, thereby determining security to be a major challenge. In this paper, we have carried out the survey of various security approaches of Mobile Adhoc Networks and provide a comprehensive study regarding it. We have focused our work on three approaches such as Bayesian watch dog, Trust based systems, and Ant colony optimization. In wireless perspective, security is a crucial term to handle. Therefore it becomes necessary when we are concerning our work with Mobile Adhoc Network. △ Less

Submitted 14 April, 2020; originally announced April 2020.

Lower Bounds Against Sparse Symmetric Functions of ACC Circuits: Expanding the Reach of $\#$SAT Algorithms

Authors: Nikhil Vyas, Ryan Williams

Abstract: We continue the program of proving circuit lower bounds via circuit satisfiability algorithms. So far, this program has yielded several concrete results, proving that functions in $\text{Quasi-NP} = \text{NTIME}[n^{(\log n)^{O(1)}}]$ and $\text{NEXP}$ do not have small circuits from various circuit classes ${\cal C}$, by showing that ${\cal C}$ admits non-trivial satisfiability and/or $\#$SAT algo… ▽ More We continue the program of proving circuit lower bounds via circuit satisfiability algorithms. So far, this program has yielded several concrete results, proving that functions in $\text{Quasi-NP} = \text{NTIME}[n^{(\log n)^{O(1)}}]$ and $\text{NEXP}$ do not have small circuits from various circuit classes ${\cal C}$, by showing that ${\cal C}$ admits non-trivial satisfiability and/or $\#$SAT algorithms which beat exhaustive search by a minor amount. In this paper, we present a new strong lower bound consequence of non-trivial $\#$SAT algorithm for a circuit class ${\mathcal C}$. Say a symmetric Boolean function $f(x_1,\ldots,x_n)$ is sparse if it outputs $1$ on $O(1)$ values of $\sum_i x_i$. We show that for every sparse $f$, and for all "typical" ${\cal C}$, faster $\#$SAT algorithms for ${\cal C}$ circuits actually imply lower bounds against the circuit class $f \circ {\cal C}$, which may be stronger than ${\cal C}$ itself. In particular: $\#$SAT algorithms for $n^k$-size ${\cal C}$-circuits running in $2^n/n^k$ time (for all $k$) imply $\text{NEXP}$ does not have $f \circ {\cal C}$-circuits of polynomial size. $\#$SAT algorithms for $2^{n^ε}$-size ${\cal C}$-circuits running in $2^{n-n^ε}$ time (for some $ε> 0$) imply $\text{Quasi-NP}$ does not have $f \circ {\cal C}$-circuits of polynomial size. Applying $\#$SAT algorithms from the literature, one immediate corollary of our results is that $\text{Quasi-NP}$ does not have $\text{EMAJ} \circ \text{ACC}^0 \circ \text{THR}$ circuits of polynomial size, where $\text{EMAJ}$ is the "exact majority" function, improving previous lower bounds against $\text{ACC}^0$ [Williams JACM'14] and $\text{ACC}^0 \circ \text{THR}$ [Williams STOC'14], [Murray-Williams STOC'18]. This is the first nontrivial lower bound against such a circuit class. △ Less

Submitted 21 January, 2020; originally announced January 2020.

Comments: To appear in STACS 2020

Imperfect Gaps in Gap-ETH and PCPs

Authors: Mitali Bafna, Nikhil Vyas

Abstract: We study the role of perfect completeness in probabilistically checkable proof systems (PCPs) and give a new way to transform a PCP with imperfect completeness to a PCP with perfect completeness when the initial gap is a constant. In particular, we show that $\text{PCP}_{c,s}[r,q] \subseteq \text{PCP}_{1,1-Ω(1)}[r+O(1),q+O(r)]$, for $c-s=Ω(1)$. This implies that one can convert imperfect completen… ▽ More We study the role of perfect completeness in probabilistically checkable proof systems (PCPs) and give a new way to transform a PCP with imperfect completeness to a PCP with perfect completeness when the initial gap is a constant. In particular, we show that $\text{PCP}_{c,s}[r,q] \subseteq \text{PCP}_{1,1-Ω(1)}[r+O(1),q+O(r)]$, for $c-s=Ω(1)$. This implies that one can convert imperfect completeness to perfect in linear-sized PCPs for $NTIME[O(n)]$ with a $O(\log n)$ additive loss in the query complexity $q$. We show our result by constructing a "robust circuit" using threshold gates. These results are a gap amplification procedure for PCPs (when completeness is imperfect), analogous to questions studied in parallel repetition and pseudorandomness. We also investigate the time complexity of approximating perfectly satisfiable instances of 3SAT versus those with imperfect completeness. We show that the Gap-ETH conjecture without perfect completeness is equivalent to Gap-ETH with perfect completeness, i.e. we show that Gap-3SAT, where the gap is not around 1, has a subexponential algorithm, if and only if, Gap-3SAT with perfect completeness has subexponential algorithms. We also relate the time complexities of these two problems in a more fine-grained way, to show that $T_2(n) \leq T_1(n(\log\log n)^{O(1)})$, where $T_1(n),T_2(n)$ denote the randomized time-complexity of approximating MAX 3SAT with perfect and imperfect completeness, respectively. △ Less

Submitted 18 July, 2019; originally announced July 2019.

Comments: To appear in CCC 2019

Measuring Bias in Contextualized Word Representations

Authors: Keita Kurita, Nidhi Vyas, Ayush Pareek, Alan W Black, Yulia Tsvetkov

Abstract: Contextual word embeddings such as BERT have achieved state of the art performance in numerous NLP tasks. Since they are optimized to capture the statistical properties of training data, they tend to pick up on and amplify social stereotypes present in the data as well. In this study, we (1)~propose a template-based method to quantify bias in BERT; (2)~show that this method obtains more consistent… ▽ More Contextual word embeddings such as BERT have achieved state of the art performance in numerous NLP tasks. Since they are optimized to capture the statistical properties of training data, they tend to pick up on and amplify social stereotypes present in the data as well. In this study, we (1)~propose a template-based method to quantify bias in BERT; (2)~show that this method obtains more consistent results in capturing social biases than the traditional cosine based method; and (3)~conduct a case study, evaluating gender bias in a downstream task of Gender Pronoun Resolution. Although our case study focuses on gender bias, the proposed technique is generalizable to unveiling other biases, including in multiclass settings, such as racial and religious biases. △ Less

Submitted 17 June, 2019; originally announced June 2019.

Comments: 1st ACL Workshop on Gender Bias for Natural Language Processing 2019

Approximation Algorithms for Min-Distance Problems

Authors: Mina Dalirrooyfard, Virginia Vassilevska Williams, Nikhil Vyas, Nicole Wein, Yinzhan Xu, Yuancheng Yu

Abstract: We study fundamental graph parameters such as the Diameter and Radius in directed graphs, when distances are measured using a somewhat unorthodox but natural measure: the distance between $u$ and $v$ is the minimum of the shortest path distances from $u$ to $v$ and from $v$ to $u$. The center node in a graph under this measure can for instance represent the optimal location for a hospital to ensur… ▽ More We study fundamental graph parameters such as the Diameter and Radius in directed graphs, when distances are measured using a somewhat unorthodox but natural measure: the distance between $u$ and $v$ is the minimum of the shortest path distances from $u$ to $v$ and from $v$ to $u$. The center node in a graph under this measure can for instance represent the optimal location for a hospital to ensure the fastest medical care for everyone, as one can either go to the hospital, or a doctor can be sent to help. By computing All-Pairs Shortest Paths, all pairwise distances and thus the parameters we study can be computed exactly in $\tilde{O}(mn)$ time for directed graphs on $n$ vertices, $m$ edges and nonnegative edge weights. Furthermore, this time bound is tight under the Strong Exponential Time Hypothesis [Roditty-Vassilevska W. STOC 2013] so it is natural to study how well these parameters can be approximated in $O(mn^{1-ε})$ time for constant $ε>0$. Abboud, Vassilevska Williams, and Wang [SODA 2016] gave a polynomial factor approximation for Diameter and Radius, as well as a constant factor approximation for both problems in the special case where the graph is a DAG. We greatly improve upon these bounds by providing the first constant factor approximations for Diameter, Radius and the related Eccentricities problem in general graphs. Additionally, we provide a hierarchy of algorithms for Diameter that gives a time/accuracy trade-off. △ Less

Submitted 17 June, 2019; v1 submitted 25 April, 2019; originally announced April 2019.

Comments: To appear in ICALP 2019

Tight Approximation Algorithms for Bichromatic Graph Diameter and Related Problems

Authors: Mina Dalirrooyfard, Virginia Vassilevska Williams, Nikhil Vyas, Nicole Wein

Abstract: Some of the most fundamental and well-studied graph parameters are the Diameter (the largest shortest paths distance) and Radius (the smallest distance for which a "center" node can reach all other nodes). The natural and important $ST$-variant considers two subsets $S$ and $T$ of the vertex set and lets the $ST$-diameter be the maximum distance between a node in $S$ and a node in $T$, and the… ▽ More Some of the most fundamental and well-studied graph parameters are the Diameter (the largest shortest paths distance) and Radius (the smallest distance for which a "center" node can reach all other nodes). The natural and important $ST$-variant considers two subsets $S$ and $T$ of the vertex set and lets the $ST$-diameter be the maximum distance between a node in $S$ and a node in $T$, and the $ST$-radius be the minimum distance for a node of $S$ to reach all nodes of $T$. The bichromatic variant is the special case in which $S$ and $T$ partition the vertex set. In this paper we present a comprehensive study of the approximability of $ST$ and Bichromatic Diameter, Radius, and Eccentricities, and variants, in graphs with and without directions and weights. We give the first nontrivial approximation algorithms for most of these problems, including time/accuracy trade-off upper and lower bounds. We show that nearly all of our obtained bounds are tight under the Strong Exponential Time Hypothesis (SETH), or the related Hitting Set Hypothesis. For instance, for Bichromatic Diameter in undirected weighted graphs with $m$ edges, we present an $\tilde{O}(m^{3/2})$ time $5/3$-approximation algorithm, and show that under SETH, neither the running time, nor the approximation factor can be significantly improved while keeping the other unchanged. △ Less

Submitted 25 April, 2019; originally announced April 2019.

Comments: To appear in ICALP 2019

The ARIEL-CMU Systems for LoReHLT18

Authors: Aditi Chaudhary, Siddharth Dalmia, Junjie Hu, Xinjian Li, Austin Matthews, Aldrian Obaja Muis, Naoki Otani, Shruti Rijhwani, Zaid Sheikh, Nidhi Vyas, Xinyi Wang, Jiateng Xie, Ruochen Xu, Chunting Zhou, Peter J. Jansen, Yiming Yang, Lori Levin, Florian Metze, Teruko Mitamura, David R. Mortensen, Graham Neubig, Eduard Hovy, Alan W Black, Jaime Carbonell, Graham V. Horwood , et al. (5 additional authors not shown)

Abstract: This paper describes the ARIEL-CMU submissions to the Low Resource Human Language Technologies (LoReHLT) 2018 evaluations for the tasks Machine Translation (MT), Entity Discovery and Linking (EDL), and detection of Situation Frames in Text and Speech (SF Text and Speech). This paper describes the ARIEL-CMU submissions to the Low Resource Human Language Technologies (LoReHLT) 2018 evaluations for the tasks Machine Translation (MT), Entity Discovery and Linking (EDL), and detection of Situation Frames in Text and Speech (SF Text and Speech). △ Less

Submitted 24 February, 2019; originally announced February 2019.

Thwarting Adversarial Examples: An $L_0$-RobustSparse Fourier Transform

Authors: Mitali Bafna, Jack Murtagh, Nikhil Vyas

Abstract: We give a new algorithm for approximating the Discrete Fourier transform of an approximately sparse signal that has been corrupted by worst-case $L_0$ noise, namely a bounded number of coordinates of the signal have been corrupted arbitrarily. Our techniques generalize to a wide range of linear transformations that are used in data analysis such as the Discrete Cosine and Sine transforms, the Hada… ▽ More We give a new algorithm for approximating the Discrete Fourier transform of an approximately sparse signal that has been corrupted by worst-case $L_0$ noise, namely a bounded number of coordinates of the signal have been corrupted arbitrarily. Our techniques generalize to a wide range of linear transformations that are used in data analysis such as the Discrete Cosine and Sine transforms, the Hadamard transform, and their high-dimensional analogs. We use our algorithm to successfully defend against well known $L_0$ adversaries in the setting of image classification. We give experimental results on the Jacobian-based Saliency Map Attack (JSMA) and the Carlini Wagner (CW) $L_0$ attack on the MNIST and Fashion-MNIST datasets as well as the Adversarial Patch on the ImageNet dataset. △ Less

Submitted 12 December, 2018; originally announced December 2018.

Comments: Accepted at 32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Montréal, Canada

Super Strong ETH is False for Random $k$-SAT

Authors: Nikhil Vyas

Abstract: It has been hypothesized that $k$-SAT is hard to solve for randomly chosen instances near the "critical threshold", where the clause-to-variable ratio is $2^k \ln 2-θ(1)$. Feige's hypothesis for $k$-SAT says that for all sufficiently large clause-to-variable ratios, random $k$-SAT cannot be refuted in polynomial time. It has also been hypothesized that the worst-case $k$-SAT problem cannot be solv… ▽ More It has been hypothesized that $k$-SAT is hard to solve for randomly chosen instances near the "critical threshold", where the clause-to-variable ratio is $2^k \ln 2-θ(1)$. Feige's hypothesis for $k$-SAT says that for all sufficiently large clause-to-variable ratios, random $k$-SAT cannot be refuted in polynomial time. It has also been hypothesized that the worst-case $k$-SAT problem cannot be solved in $2^{n(1-ω_k(1)/k)}$ time, as multiple known algorithmic paradigms (backtracking, local search and the polynomial method) only yield an $2^{n(1-1/O(k))}$ time algorithm. This hypothesis has been called the "Super-Strong ETH", modeled after the ETH and the Strong ETH. Our main result is a randomized algorithm which refutes the Super-Strong ETH for the case of random $k$-SAT, for any clause-to-variable ratio. Given any random $k$-SAT instance $F$ with $n$ variables and $m$ clauses, our algorithm decides satisfiability for $F$ in $2^{n(1-Ω(\log k)/k)}$ time, with high probability. It turns out that a well-known algorithm from the literature on SAT algorithms does the job: the PPZ algorithm of Paturi, Pudlak, and Zane (1998). △ Less

Submitted 14 October, 2018; originally announced October 2018.

Comments: 15 pages

Distribution-based objectives for Markov Decision Processes

Authors: S. Akshay, Blaise Genest, Nikhil Vyas

Abstract: We consider distribution-based objectives for Markov Decision Processes (MDP). This class of objectives gives rise to an interesting trade-off between full and partial information. As in full observation, the strategy in the MDP can depend on the state of the system, but similar to partial information, the strategy needs to account for all the states at the same time. In this paper, we focus on tw… ▽ More We consider distribution-based objectives for Markov Decision Processes (MDP). This class of objectives gives rise to an interesting trade-off between full and partial information. As in full observation, the strategy in the MDP can depend on the state of the system, but similar to partial information, the strategy needs to account for all the states at the same time. In this paper, we focus on two safety problems that arise naturally in this context, namely, existential and universal safety. Given an MDP A and a closed and convex polytope H of probability distributions over the states of A, the existential safety problem asks whether there exists some distribution d in H and a strategy of A, such that starting from d and repeatedly applying this strategy keeps the distribution forever in H. The universal safety problem asks whether for all distributions in H, there exists such a strategy of A which keeps the distribution forever in H. We prove that both problems are decidable, with tight complexity bounds: we show that existential safety is PTIME-complete, while universal safety is co-NP-complete. Further, we compare these results with existential and universal safety problems for Rabin's probabilistic finite-state automata (PFA), the subclass of Partially Observable MDPs which have zero observation. Compared to MDPs, strategies of PFAs are not state-dependent. In sharp contrast to the PTIME result, we show that existential safety for PFAs is undecidable, with H having closed and open boundaries. On the other hand, it turns out that the universal safety for PFAs is decidable in EXPTIME, with a co-NP lower bound. Finally, we show that an alternate representation of the input polytope allows us to improve the complexity of universal safety for MDPs and PFAs. △ Less

Submitted 25 April, 2018; originally announced April 2018.

Comments: An extended abstract of this paper has been accepted in the conference LICS'2018

Is the essence of a quantum game captured completely in the original classical game?

Authors: Muhammed Jabir T, Nilesh Vyas, Colin Benjamin

Abstract: S. J. van Enk and R. Pike in PRA 66, 024306 (2002) argue that the equilibrium solution to a quantum game isn't unique but is already present in the classical game itself. In this work, we contest this assertion by showing that a random strategy in a particular quantum (Hawk-Dove) game is unique to the quantum game. In other words, one cannot obtain the equilibrium solution of the quantum Hawk-Dove… ▽ More S. J. van Enk and R. Pike in PRA 66, 024306 (2002) argue that the equilibrium solution to a quantum game isn't unique but is already present in the classical game itself. In this work, we contest this assertion by showing that a random strategy in a particular quantum (Hawk-Dove) game is unique to the quantum game. In other words, one cannot obtain the equilibrium solution of the quantum Hawk-Dove game in the classical Hawk-Dove game. Moreover, we provide an analytical solution to the quantum $2\times2$ strategic form Hawk-Dove game using randomly mixed strategies. The random strategy which we describe is Pareto optimal with their payoff classically unobtainable. We compare quantum strategies to correlated strategies and find that correlated strategies in the quantum Hawk-Dove game or quantum Prisoner's dilemma yield the Nash equilibrium solution. △ Less

Submitted 13 August, 2021; v1 submitted 30 January, 2017; originally announced January 2017.

Comments: 11 Pages, accepted for publication in Physica A: Statistical Mechanics and its Applications (2021)

Journal ref: Physica A: Statistical Mechanics and its Applications 584, 126360 (2021)

Faster Space-Efficient Algorithms for Subset Sum, k-Sum and Related Problems

Authors: Nikhil Bansal, Shashwat Garg, Jesper Nederlof, Nikhil Vyas

Abstract: We present space efficient Monte Carlo algorithms that solve Subset Sum and Knapsack instances with $n$ items using $O^*(2^{0.86n})$ time and polynomial space, where the $O^*(\cdot)$ notation suppresses factors polynomial in the input size. Both algorithms assume random read-only access to random bits. Modulo this mild assumption, this resolves a long-standing open problem in exact algorithms for… ▽ More We present space efficient Monte Carlo algorithms that solve Subset Sum and Knapsack instances with $n$ items using $O^*(2^{0.86n})$ time and polynomial space, where the $O^*(\cdot)$ notation suppresses factors polynomial in the input size. Both algorithms assume random read-only access to random bits. Modulo this mild assumption, this resolves a long-standing open problem in exact algorithms for NP-hard problems. These results can be extended to solve Binary Linear Programming on $n$ variables with few constraints in a similar running time. We also show that for any constant $k\geq 2$, random instances of $k$-Sum can be solved using $O(n^{k-0.5}polylog(n))$ time and $O(\log n)$ space, without the assumption of random access to random bits. Underlying these results is an algorithm that determines whether two given lists of length $n$ with integers bounded by a polynomial in $n$ share a common value. Assuming random read-only access to random bits, we show that this problem can be solved using $O(\log n)$ space significantly faster than the trivial $O(n^2)$ time algorithm if no value occurs too often in the same list. △ Less

Submitted 24 June, 2017; v1 submitted 8 December, 2016; originally announced December 2016.

Comments: 23 pages, 3 figures