-
Regularized Unconstrained Weakly Submodular Maximization
Authors:
Yanhui Zhu,
Samik Basu,
A. Pavan
Abstract:
Submodular optimization finds applications in machine learning and data mining. In this paper, we study the problem of maximizing functions of the form $h = f-c$, where $f$ is a monotone, non-negative, weakly submodular set function and $c$ is a modular function. We design a deterministic approximation algorithm that runs with ${O}(\frac{n}ε\log \frac{n}{γε})$ oracle calls to function $h$, and out…
▽ More
Submodular optimization finds applications in machine learning and data mining. In this paper, we study the problem of maximizing functions of the form $h = f-c$, where $f$ is a monotone, non-negative, weakly submodular set function and $c$ is a modular function. We design a deterministic approximation algorithm that runs with ${O}(\frac{n}ε\log \frac{n}{γε})$ oracle calls to function $h$, and outputs a set ${S}$ such that $h({S}) \geq γ(1-ε)f(OPT)-c(OPT)-\frac{c(OPT)}{γ(1-ε)}\log\frac{f(OPT)}{c(OPT)}$, where $γ$ is the submodularity ratio of $f$. Existing algorithms for this problem either admit a worse approximation ratio or have quadratic runtime. We also present an approximation ratio of our algorithm for this problem with an approximate oracle of $f$. We validate our theoretical results through extensive empirical evaluations on real-world applications, including vertex cover and influence diffusion problems for submodular utility function $f$, and Bayesian A-Optimal design for weakly submodular $f$. Our experimental results demonstrate that our algorithms efficiently achieve high-quality solutions.
△ Less
Submitted 18 August, 2024; v1 submitted 8 August, 2024;
originally announced August 2024.
-
Total Variation Distance for Product Distributions is $\#\mathsf{P}$-Complete
Authors:
Arnab Bhattacharyya,
Sutanu Gayen,
Kuldeep S. Meel,
Dimitrios Myrisiotis,
A. Pavan,
N. V. Vinodchandran
Abstract:
We show that computing the total variation distance between two product distributions is $\#\mathsf{P}$-complete. This is in stark contrast with other distance measures such as Kullback-Leibler, Chi-square, and Hellinger, which tensorize over the marginals leading to efficient algorithms.
We show that computing the total variation distance between two product distributions is $\#\mathsf{P}$-complete. This is in stark contrast with other distance measures such as Kullback-Leibler, Chi-square, and Hellinger, which tensorize over the marginals leading to efficient algorithms.
△ Less
Submitted 13 May, 2024;
originally announced May 2024.
-
Improved Evolutionary Algorithms for Submodular Maximization with Cost Constraints
Authors:
Yanhui Zhu,
Samik Basu,
A Pavan
Abstract:
We present an evolutionary algorithm evo-SMC for the problem of Submodular Maximization under Cost constraints (SMC). Our algorithm achieves $1/2$-approximation with a high probability $1-1/n$ within $\mathcal{O}(n^2K_β)$ iterations, where $K_β$ denotes the maximum size of a feasible solution set with cost constraint $β$. To the best of our knowledge, this is the best approximation guarantee offer…
▽ More
We present an evolutionary algorithm evo-SMC for the problem of Submodular Maximization under Cost constraints (SMC). Our algorithm achieves $1/2$-approximation with a high probability $1-1/n$ within $\mathcal{O}(n^2K_β)$ iterations, where $K_β$ denotes the maximum size of a feasible solution set with cost constraint $β$. To the best of our knowledge, this is the best approximation guarantee offered by evolutionary algorithms for this problem. We further refine evo-SMC, and develop st-evo-SMC. This stochastic version yields a significantly faster algorithm while maintaining the approximation ratio of $1/2$, with probability $1-ε$. The required number of iterations reduces to $\mathcal{O}(nK_β\log{(1/ε)}/p)$, where the user defined parameters $p \in (0,1]$ represents the stochasticity probability, and $ε\in (0,1]$ denotes the error threshold. Finally, the empirical evaluations carried out through extensive experimentation substantiate the efficiency and effectiveness of our proposed algorithms. Our algorithms consistently outperform existing methods, producing higher-quality solutions.
△ Less
Submitted 18 August, 2024; v1 submitted 9 May, 2024;
originally announced May 2024.
-
Total Variation Distance Meets Probabilistic Inference
Authors:
Arnab Bhattacharyya,
Sutanu Gayen,
Kuldeep S. Meel,
Dimitrios Myrisiotis,
A. Pavan,
N. V. Vinodchandran
Abstract:
In this paper, we establish a novel connection between total variation (TV) distance estimation and probabilistic inference. In particular, we present an efficient, structure-preserving reduction from relative approximation of TV distance to probabilistic inference over directed graphical models. This reduction leads to a fully polynomial randomized approximation scheme (FPRAS) for estimating TV d…
▽ More
In this paper, we establish a novel connection between total variation (TV) distance estimation and probabilistic inference. In particular, we present an efficient, structure-preserving reduction from relative approximation of TV distance to probabilistic inference over directed graphical models. This reduction leads to a fully polynomial randomized approximation scheme (FPRAS) for estimating TV distances between same-structure distributions over any class of Bayes nets for which there is an efficient probabilistic inference algorithm. In particular, it leads to an FPRAS for estimating TV distances between distributions that are defined over a common Bayes net of small treewidth. Prior to this work, such approximation schemes only existed for estimating TV distances between product distributions. Our approach employs a new notion of $partial$ couplings of high-dimensional distributions, which might be of independent interest.
△ Less
Submitted 1 July, 2024; v1 submitted 16 September, 2023;
originally announced September 2023.
-
Geometry of Rounding: Near Optimal Bounds and a New Neighborhood Sperner's Lemma
Authors:
Jason Vander Woude,
Peter Dixon,
A. Pavan,
Jamie Radcliffe,
N. V. Vinodchandran
Abstract:
A partition $\mathcal{P}$ of $\mathbb{R}^d$ is called a $(k,\varepsilon)$-secluded partition if, for every $\vec{p} \in \mathbb{R}^d$, the ball $\overline{B}_{\infty}(\varepsilon, \vec{p})$ intersects at most $k$ members of $\mathcal{P}$. A goal in designing such secluded partitions is to minimize $k$ while making $\varepsilon$ as large as possible. This partition problem has connections to a dive…
▽ More
A partition $\mathcal{P}$ of $\mathbb{R}^d$ is called a $(k,\varepsilon)$-secluded partition if, for every $\vec{p} \in \mathbb{R}^d$, the ball $\overline{B}_{\infty}(\varepsilon, \vec{p})$ intersects at most $k$ members of $\mathcal{P}$. A goal in designing such secluded partitions is to minimize $k$ while making $\varepsilon$ as large as possible. This partition problem has connections to a diverse range of topics, including deterministic rounding schemes, pseudodeterminism, replicability, as well as Sperner/KKM-type results.
In this work, we establish near-optimal relationships between $k$ and $\varepsilon$. We show that, for any bounded measure partitions and for any $d\geq 1$, it must be that $k\geq(1+2\varepsilon)^d$. Thus, when $k=k(d)$ is restricted to ${\rm poly}(d)$, it follows that $\varepsilon=\varepsilon(d)\in O\left(\frac{\ln d}{d}\right)$. This bound is tight up to log factors, as it is known that there exist secluded partitions with $k(d)=d+1$ and $\varepsilon(d)=\frac{1}{2d}$. We also provide new constructions of secluded partitions that work for a broad spectrum of $k(d)$ and $\varepsilon(d)$ parameters. Specifically, we prove that, for any $f:\mathbb{N}\rightarrow\mathbb{N}$, there is a secluded partition with $k(d)=(f(d)+1)^{\lceil\frac{d}{f(d)}\rceil}$ and $\varepsilon(d)=\frac{1}{2f(d)}$. These new partitions are optimal up to $O(\log d)$ factors for various choices of $k(d)$ and $\varepsilon(d)$. Based on the lower bound result, we establish a new neighborhood version of Sperner's lemma over hypercubes, which is of independent interest. In addition, we prove a no-free-lunch theorem about the limitations of rounding schemes in the context of pseudodeterministic/replicable algorithms.
△ Less
Submitted 10 April, 2023;
originally announced April 2023.
-
List and Certificate Complexities in Replicable Learning
Authors:
Peter Dixon,
A. Pavan,
Jason Vander Woude,
N. V. Vinodchandran
Abstract:
We investigate replicable learning algorithms. Ideally, we would like to design algorithms that output the same canonical model over multiple runs, even when different runs observe a different set of samples from the unknown data distribution. In general, such a strong notion of replicability is not achievable. Thus we consider two feasible notions of replicability called list replicability and ce…
▽ More
We investigate replicable learning algorithms. Ideally, we would like to design algorithms that output the same canonical model over multiple runs, even when different runs observe a different set of samples from the unknown data distribution. In general, such a strong notion of replicability is not achievable. Thus we consider two feasible notions of replicability called list replicability and certificate replicability. Intuitively, these notions capture the degree of (non) replicability. We design algorithms for certain learning problems that are optimal in list and certificate complexity. We establish matching impossibility results.
△ Less
Submitted 5 April, 2023;
originally announced April 2023.
-
Constraint Optimization over Semirings
Authors:
A. Pavan,
Kuldeep S. Meel,
N. V. Vinodchandran,
Arnab Bhattacharyya
Abstract:
Interpretations of logical formulas over semirings have applications in various areas of computer science including logic, AI, databases, and security. Such interpretations provide richer information beyond the truth or falsity of a statement. Examples of such semirings include Viterbi semiring, min-max or access control semiring, tropical semiring, and fuzzy semiring.
The present work investiga…
▽ More
Interpretations of logical formulas over semirings have applications in various areas of computer science including logic, AI, databases, and security. Such interpretations provide richer information beyond the truth or falsity of a statement. Examples of such semirings include Viterbi semiring, min-max or access control semiring, tropical semiring, and fuzzy semiring.
The present work investigates the complexity of constraint optimization problems over semirings. The generic optimization problem we study is the following: Given a propositional formula $\varphi$ over $n$ variable and a semiring $(K,+,\cdot,0,1)$, find the maximum value over all possible interpretations of $\varphi$ over $K$. This can be seen as a generalization of the well-known satisfiability problem. A related problem is to find an interpretation that achieves the maximum value. In this work, we first focus on these optimization problems over the Viterbi semiring, which we call optConfVal and optConf.
We show that for general propositional formulas in negation normal form, optConfVal and optConf are in ${\mathrm{FP}}^{\mathrm{NP}}$. We investigate optConf when the input formula $\varphi$ is represented as a CNF. For CNF formulae, we first derive an upper bound on optConfVal as a function of the number of maximum satisfiable clauses. In particular, we show that if $r$ is the maximum number of satisfiable clauses in a CNF formula with $m$ clauses, then its optConfVal is at most $1/4^{m-r}$. Building on this we establish that optConfVal for CNF formulae is hard for the complexity class ${\mathrm{FP}}^{\mathrm{NP}[\log]}$. We also design polynomial-time approximation algorithms and establish an inapproximability for optConfVal. We establish similar complexity results for these optimization problems over other semirings including tropical, fuzzy, and access control semirings.
△ Less
Submitted 24 February, 2023;
originally announced February 2023.
-
Geometry of Rounding
Authors:
Jason Vander Woude,
Peter Dixon,
A. Pavan,
Jamie Radcliffe,
N. V. Vinodchandran
Abstract:
Rounding has proven to be a fundamental tool in theoretical computer science. By observing that rounding and partitioning of $\mathbb{R}^d$ are equivalent, we introduce the following natural partition problem which we call the {\em secluded hypercube partition problem}: Given $k\in \mathbb{N}$ (ideally small) and $ε>0$ (ideally large), is there a partition of $\mathbb{R}^d$ with unit hypercubes su…
▽ More
Rounding has proven to be a fundamental tool in theoretical computer science. By observing that rounding and partitioning of $\mathbb{R}^d$ are equivalent, we introduce the following natural partition problem which we call the {\em secluded hypercube partition problem}: Given $k\in \mathbb{N}$ (ideally small) and $ε>0$ (ideally large), is there a partition of $\mathbb{R}^d$ with unit hypercubes such that for every point $p \in \mathbb{R}^d$, its closed $ε$-neighborhood (in the $\ell_{\infty}$ norm) intersects at most $k$ hypercubes?
We undertake a comprehensive study of this partition problem. We prove that for every $d\in \mathbb{N}$, there is an explicit (and efficiently computable) hypercube partition of $\mathbb{R}^d$ with $k = d+1$ and $ε= \frac{1}{2d}$. We complement this construction by proving that the value of $k=d+1$ is the best possible (for any $ε$) for a broad class of ``reasonable'' partitions including hypercube partitions. We also investigate the optimality of the parameter $ε$ and prove that any partition in this broad class that has $k=d+1$, must have $ε\leq\frac{1}{2\sqrt{d}}$. These bounds imply limitations of certain deterministic rounding schemes existing in the literature. Furthermore, this general bound is based on the currently known lower bounds for the dissection number of the cube, and improvements to this bound will yield improvements to our bounds.
While our work is motivated by the desire to understand rounding algorithms, one of our main conceptual contributions is the introduction of the {\em secluded hypercube partition problem}, which fits well with a long history of investigations by mathematicians on various hypercube partitions/tilings of Euclidean space.
△ Less
Submitted 4 November, 2022;
originally announced November 2022.
-
On Approximating Total Variation Distance
Authors:
Arnab Bhattacharyya,
Sutanu Gayen,
Kuldeep S. Meel,
Dimitrios Myrisiotis,
A. Pavan,
N. V. Vinodchandran
Abstract:
Total variation distance (TV distance) is a fundamental notion of distance between probability distributions. In this work, we introduce and study the problem of computing the TV distance of two product distributions over the domain $\{0,1\}^n$. In particular, we establish the following results.
1. The problem of exactly computing the TV distance of two product distributions is $\#\mathsf{P}$-co…
▽ More
Total variation distance (TV distance) is a fundamental notion of distance between probability distributions. In this work, we introduce and study the problem of computing the TV distance of two product distributions over the domain $\{0,1\}^n$. In particular, we establish the following results.
1. The problem of exactly computing the TV distance of two product distributions is $\#\mathsf{P}$-complete. This is in stark contrast with other distance measures such as KL, Chi-square, and Hellinger which tensorize over the marginals leading to efficient algorithms.
2. There is a fully polynomial-time deterministic approximation scheme (FPTAS) for computing the TV distance of two product distributions $P$ and $Q$ where $Q$ is the uniform distribution. This result is extended to the case where $Q$ has a constant number of distinct marginals. In contrast, we show that when $P$ and $Q$ are Bayes net distributions, the relative approximation of their TV distance is $\mathsf{NP}$-hard.
△ Less
Submitted 16 August, 2023; v1 submitted 14 June, 2022;
originally announced June 2022.
-
Model Counting meets F0 Estimation
Authors:
A. Pavan,
N. V. Vinodchandran,
Arnab Bhattacharyya,
Kuldeep S. Meel
Abstract:
Constraint satisfaction problems (CSP's) and data stream models are two powerful abstractions to capture a wide variety of problems arising in different domains of computer science. Developments in the two communities have mostly occurred independently and with little interaction between them. In this work, we seek to investigate whether bridging the seeming communication gap between the two commu…
▽ More
Constraint satisfaction problems (CSP's) and data stream models are two powerful abstractions to capture a wide variety of problems arising in different domains of computer science. Developments in the two communities have mostly occurred independently and with little interaction between them. In this work, we seek to investigate whether bridging the seeming communication gap between the two communities may pave the way to richer fundamental insights. To this end, we focus on two foundational problems: model counting for CSP's and computation of zeroth frequency moments ($F_0$) for data streams.
Our investigations lead us to observe striking similarity in the core techniques employed in the algorithmic frameworks that have evolved separately for model counting and $F_0$ computation. We design a recipe for translation of algorithms developed for $F_0$ estimation to that of model counting, resulting in new algorithms for model counting. We then observe that algorithms in the context of distributed streaming can be transformed to distributed algorithms for model counting. We next turn our attention to viewing streaming from the lens of counting and show that framing $F_0$ estimation as a special case of #DNF counting allows us to obtain a general recipe for a rich class of streaming problems, which had been subjected to case-specific analysis in prior works. In particular, our view yields a state-of-the art algorithm for multidimensional range efficient $F_0$ estimation with a simpler analysis.
△ Less
Submitted 3 May, 2021;
originally announced May 2021.
-
Promise Problems Meet Pseudodeterminism
Authors:
Peter Dixon,
A. Pavan,
N. V. Vinodchandran
Abstract:
The Acceptance Probability Estimation Problem (APEP) is to additively approximate the acceptance probability of a Boolean circuit. This problem admits a probabilistic approximation scheme. A central question is whether we can design a pseudodeterministic approximation algorithm for this problem: a probabilistic polynomial-time algorithm that outputs a canonical approximation with high probability.…
▽ More
The Acceptance Probability Estimation Problem (APEP) is to additively approximate the acceptance probability of a Boolean circuit. This problem admits a probabilistic approximation scheme. A central question is whether we can design a pseudodeterministic approximation algorithm for this problem: a probabilistic polynomial-time algorithm that outputs a canonical approximation with high probability. Recently, it was shown that such an algorithm would imply that every approximation algorithm can be made pseudodeterministic (Dixon, Pavan, Vinodchandran; ITCS 2021).
The main conceptual contribution of this work is to establish that the existence of a pseudodeterministic algorithm for APEP is fundamentally connected to the relationship between probabilistic promise classes and the corresponding standard complexity classes. In particular, we show the following equivalence: every promise problem in PromiseBPP has a solution in BPP if and only if APEP has a pseudodeterministic algorithm. Based on this intuition, we show that pseudodeterministic algorithms for APEP can shed light on a few central topics in complexity theory such as circuit lowerbounds, probabilistic hierarchy theorems, and multi-pseudodeterminism.
△ Less
Submitted 15 March, 2021;
originally announced March 2021.
-
Parallel Triangle Counting in Massive Streaming Graphs
Authors:
Kanat Tangwongsan,
A. Pavan,
Srikanta Tirthapura
Abstract:
The number of triangles in a graph is a fundamental metric, used in social network analysis, link classification and recommendation, and more. Driven by these applications and the trend that modern graph datasets are both large and dynamic, we present the design and implementation of a fast and cache-efficient parallel algorithm for estimating the number of triangles in a massive undirected graph…
▽ More
The number of triangles in a graph is a fundamental metric, used in social network analysis, link classification and recommendation, and more. Driven by these applications and the trend that modern graph datasets are both large and dynamic, we present the design and implementation of a fast and cache-efficient parallel algorithm for estimating the number of triangles in a massive undirected graph whose edges arrive as a stream. It brings together the benefits of streaming algorithms and parallel algorithms. By building on the streaming algorithms framework, the algorithm has a small memory footprint. By leveraging the paralell cache-oblivious framework, it makes efficient use of the memory hierarchy of modern multicore machines without needing to know its specific parameters. We prove theoretical bounds on accuracy, memory access cost, and parallel runtime complexity, as well as showing empirically that the algorithm yields accurate results and substantial speedups compared to an optimized sequential implementation.
(This is an expanded version of a CIKM'13 paper of the same title.)
△ Less
Submitted 9 August, 2013;
originally announced August 2013.
-
On the Power of Unambiguity in Logspace
Authors:
Aduri Pavan,
Raghunath Tewari,
N. V. Vinodchandran
Abstract:
We report progress on the \NL vs \UL problem. [-] We show unconditionally that the complexity class $\ReachFewL\subseteq\UL$. This improves on the earlier known upper bound $\ReachFewL \subseteq \FewL$. [-] We investigate the complexity of min-uniqueness - a central notion in studying the \NL vs \UL problem. We show that min-uniqueness is necessary and sufficient for showing $\NL =\UL$. We revis…
▽ More
We report progress on the \NL vs \UL problem. [-] We show unconditionally that the complexity class $\ReachFewL\subseteq\UL$. This improves on the earlier known upper bound $\ReachFewL \subseteq \FewL$. [-] We investigate the complexity of min-uniqueness - a central notion in studying the \NL vs \UL problem. We show that min-uniqueness is necessary and sufficient for showing $\NL =\UL$. We revisit the class $\OptL[\log n]$ and show that {\sc ShortestPathLength} - computing the length of the shortest path in a DAG, is complete for $\OptL[\log n]$. We introduce $\UOptL[\log n]$, an unambiguous version of $\OptL[\log n]$, and show that (a) $\NL =\UL$ if and only if $\OptL[\log n] = \UOptL[\log n]$, (b) $\LogFew \leq \UOptL[\log n] \leq \SPL$. [-] We show that the reachability problem over graphs embedded on 3 pages is complete for \NL. This contrasts with the reachability problem over graphs embedded on 2 pages which is logspace equivalent to the reachability problem in planar graphs and hence is in \UL.
△ Less
Submitted 12 January, 2010;
originally announced January 2010.
-
Collapsing and Separating Completeness Notions under Average-Case and Worst-Case Hypotheses
Authors:
Xiaoyang Gu,
John M. Hitchcock,
A. Pavan
Abstract:
This paper presents the following results on sets that are complete for NP.
1. If there is a problem in NP that requires exponential time at almost all lengths, then every many-one NP-complete set is complete under length-increasing reductions that are computed by polynomial-size circuits. 2. If there is a problem in coNP that cannot be solved by polynomial-size nondeterministic circuits, then…
▽ More
This paper presents the following results on sets that are complete for NP.
1. If there is a problem in NP that requires exponential time at almost all lengths, then every many-one NP-complete set is complete under length-increasing reductions that are computed by polynomial-size circuits. 2. If there is a problem in coNP that cannot be solved by polynomial-size nondeterministic circuits, then every many-one complete set is complete under length-increasing reductions that are computed by polynomial-size circuits. 3. If there exist a one-way permutation that is secure against subexponential-size circuits and there is a hard tally language in NP intersect coNP, then there is a Turing complete language for NP that is not many-one complete. Our first two results use worst-case hardness hypotheses whereas earlier work that showed similar results relied on average-case or almost-everywhere hardness assumptions. The use of average-case and worst-case hypotheses in the last result is unique as previous results obtaining the same consequence relied on almost-everywhere hardness results.
△ Less
Submitted 3 February, 2010; v1 submitted 4 January, 2010;
originally announced January 2010.
