-
When to Give Up on a Parallel Implementation
Authors:
Nathan S. Sheffield,
Alek Westover
Abstract:
In the Serial Parallel Decision Problem (SPDP), introduced by Kuszmaul and Westover [SPAA'24], an algorithm receives a series of tasks online, and must choose for each between a serial implementation and a parallelizable (but less efficient) implementation. Kuszmaul and Westover describe three decision models: (1) \defn{Instantly-committing} schedulers must decide on arrival, irrevocably, which im…
▽ More
In the Serial Parallel Decision Problem (SPDP), introduced by Kuszmaul and Westover [SPAA'24], an algorithm receives a series of tasks online, and must choose for each between a serial implementation and a parallelizable (but less efficient) implementation. Kuszmaul and Westover describe three decision models: (1) \defn{Instantly-committing} schedulers must decide on arrival, irrevocably, which implementation of the task to run. (2) \defn{Eventually-committing} schedulers can delay their decision beyond a task's arrival time, but cannot revoke their decision once made. (3) \defn{Never-committing} schedulers are always free to abandon their progress on the task and start over using a different implementation. Kuszmaul and Westover gave a simple instantly-committing scheduler whose total completion time is $3$-competitive with the offline optimal schedule. They conjectured that the three decision models should admit different competitive ratios, but left upper bounds below $3$ in any model as an open problem.
In this paper, we show that the powers of instantly, eventually, and never committing schedulers are distinct, at least in the ``massively parallel regime''. The massively parallel regime of the SPDP is the special case where the number of available processors is asymptotically larger than the number of tasks to process, meaning that the \emph{work} associated with running a task in serial is negligible compared to its \emph{runtime}. In this regime, we show (1) The optimal competitive ratio for instantly-committing schedulers is $2$, (2) The optimal competitive ratio for eventually-committing schedulers lies in $[1.618, 1.678]$, (3) The optimal competitive ratio for never-committing schedulers lies in $[1.366, 1.500]$.
△ Less
Submitted 28 August, 2024;
originally announced August 2024.
-
Complexity of Multiple-Hamiltonicity in Graphs of Bounded Degree
Authors:
Brian Liu,
Nathan S. Sheffield,
Alek Westover
Abstract:
We study the following generalization of the Hamiltonian cycle problem: Given integers $a,b$ and graph $G$, does there exist a closed walk in $G$ that visits every vertex at least $a$ times and at most $b$ times? Equivalently, does there exist a connected $[2a,2b]$ factor of $2b \cdot G$ with all degrees even? This problem is NP-hard for any constants $1 \leq a \leq b$. However, the graphs produce…
▽ More
We study the following generalization of the Hamiltonian cycle problem: Given integers $a,b$ and graph $G$, does there exist a closed walk in $G$ that visits every vertex at least $a$ times and at most $b$ times? Equivalently, does there exist a connected $[2a,2b]$ factor of $2b \cdot G$ with all degrees even? This problem is NP-hard for any constants $1 \leq a \leq b$. However, the graphs produced by known reductions have maximum degree growing linearly in $b$. The case $a = b = 1 $ -- i.e. Hamiltonicity -- remains NP-hard even in $3$-regular graphs; a natural question is whether this is true for other $a$, $b$.
In this work, we study which $a, b$ permit polynomial time algorithms and which lead to NP-hardness in graphs with constrained degrees. We give tight characterizations for regular graphs and graphs of bounded max-degree, both directed and undirected.
△ Less
Submitted 25 May, 2024;
originally announced May 2024.
-
A Nearly Quadratic Improvement for Memory Reallocation
Authors:
Martin Farach-Colton,
William Kuszmaul,
Nathan Sheffield,
Alek Westover
Abstract:
In the Memory Reallocation Problem a set of items of various sizes must be dynamically assigned to non-overlapping contiguous chunks of memory. It is guaranteed that the sum of the sizes of all items present at any time is at most a $(1-\varepsilon)$-fraction of the total size of memory (i.e., the load-factor is at most $1-\varepsilon$). The allocator receives insert and delete requests online, an…
▽ More
In the Memory Reallocation Problem a set of items of various sizes must be dynamically assigned to non-overlapping contiguous chunks of memory. It is guaranteed that the sum of the sizes of all items present at any time is at most a $(1-\varepsilon)$-fraction of the total size of memory (i.e., the load-factor is at most $1-\varepsilon$). The allocator receives insert and delete requests online, and can re-arrange existing items to handle the requests, but at a reallocation cost defined to be the sum of the sizes of items moved divided by the size of the item being inserted/deleted.
The folklore algorithm for Memory Reallocation achieves a cost of $O(\varepsilon^{-1})$ per update. In recent work at FOCS'23, Kuszmaul showed that, in the special case where each item is promised to be smaller than an $\varepsilon^4$-fraction of memory, it is possible to achieve expected update cost $O(\log\varepsilon^{-1})$. Kuszmaul conjectures, however, that for larger items the folklore algorithm is optimal.
In this work we disprove Kuszmaul's conjecture, giving an allocator that achieves expected update cost $O(\varepsilon^{-1/2} \operatorname*{polylog} \varepsilon^{-1})$ on any input sequence. We also give the first non-trivial lower bound for the Memory Reallocation Problem: we demonstrate an input sequence on which any resizable allocator (even offline) must incur amortized update cost at least $Ω(\log\varepsilon^{-1})$.
Finally, we analyze the Memory Reallocation Problem on a stochastic sequence of inserts and deletes, with random sizes in $[δ, 2 δ]$ for some $δ$. We show that, in this simplified setting, it is possible to achieve $O(\log\varepsilon^{-1})$ expected update cost, even in the ``large item'' parameter regime ($δ> \varepsilon^4$).
△ Less
Submitted 20 May, 2024;
originally announced May 2024.
-
Scheduling Jobs with Work-Inefficient Parallel Solutions
Authors:
William Kuszmaul,
Alek Westover
Abstract:
This paper introduces the \emph{serial-parallel decision problem}. Consider an online scheduler that receives a series of tasks, where each task has both a parallel and a serial implementation. The parallel implementation has the advantage that it can make progress concurrently on multiple processors, but the disadvantage that it is (potentially) work-inefficient. As tasks arrive, the scheduler mu…
▽ More
This paper introduces the \emph{serial-parallel decision problem}. Consider an online scheduler that receives a series of tasks, where each task has both a parallel and a serial implementation. The parallel implementation has the advantage that it can make progress concurrently on multiple processors, but the disadvantage that it is (potentially) work-inefficient. As tasks arrive, the scheduler must decide for each task which implementation to use.
We begin by studying \emph{total awake time}. We give a simple \emph{decide-on-arrival} scheduler that achieves a competitive ratio of $3$ for total awake time -- this scheduler makes serial/parallel decisions immediately when jobs arrive. Our second result is an \emph{parallel-work-oblivious} scheduler that achieves a competitive ratio of $6$ for total awake time -- this scheduler makes all of its decisions based only on the size of each serial job and without needing to know anything about the parallel implementations. Finally, we prove a lower bound showing that, if a scheduler wishes to achieve a competitive ratio of $O(1)$, it can have at most one of the two aforementioned properties (decide-on-arrival or parallel-work-oblivious). We also prove lower bounds of the form $1 + Ω(1)$ on the optimal competitive ratio for any scheduler.
Next, we turn our attention to optimizing \emph{mean response time}. Here, we show that it is possible to achieve an $O(1)$ competitive ratio with $O(1)$ speed augmentation. This is the most technically involved of our results. We also prove that, in this setting, it is not possible for a parallel-work-oblivious scheduler to do well.
In addition to these results, we present tight bounds on the optimal competitive ratio if we allow for arrival dependencies between tasks (e.g., tasks are components of a single parallel program), and we give an in-depth discussion of the remaining open questions.
△ Less
Submitted 20 May, 2024;
originally announced May 2024.
-
On the Relationship Between Several Variants of the Linear Hashing Conjecture
Authors:
Alek Westover
Abstract:
In Linear Hashing ($\mathsf{LH}$) with $β$ bins on a size $u$ universe ${\mathcal{U}=\{0,1,\ldots, u-1\}}$, items $\{x_1,x_2,\ldots, x_n\}\subset \mathcal{U}$ are placed in bins by the hash function $$x_i\mapsto (ax_i+b)\mod p \mod β$$ for some prime $p\in [u,2u]$ and randomly chosen integers $a,b \in [1,p]$. The "maxload" of $\mathsf{LH}$ is the number of items assigned to the fullest bin. Expect…
▽ More
In Linear Hashing ($\mathsf{LH}$) with $β$ bins on a size $u$ universe ${\mathcal{U}=\{0,1,\ldots, u-1\}}$, items $\{x_1,x_2,\ldots, x_n\}\subset \mathcal{U}$ are placed in bins by the hash function $$x_i\mapsto (ax_i+b)\mod p \mod β$$ for some prime $p\in [u,2u]$ and randomly chosen integers $a,b \in [1,p]$. The "maxload" of $\mathsf{LH}$ is the number of items assigned to the fullest bin. Expected maxload for a worst-case set of items is a natural measure of how well $\mathsf{LH}$ distributes items amongst the bins.
Fix $β=n$. Despite $\mathsf{LH}$'s simplicity, bounding $\mathsf{LH}$'s worst-case maxload is extremely challenging. It is well-known that on random inputs $\mathsf{LH}$ achieves maxload $Ω\left(\frac{\log n}{\log\log n}\right)$; this is currently the best lower bound for $\mathsf{LH}$'s expected maxload. Recently Knudsen established an upper bound of $\widetilde{O}(n^{1 / 3})$. The question "Is the worst-case expected maxload of $\mathsf{LH}$ $n^{o(1)}$?" is one of the most basic open problems in discrete math.
In this paper we propose a set of intermediate open questions to help researchers make progress on this problem. We establish the relationship between these intermediate open questions and make some partial progress on them.
△ Less
Submitted 17 May, 2024; v1 submitted 24 July, 2023;
originally announced July 2023.
-
The Variable-Processor Cup Game
Authors:
William Kuszmaul,
Alek Westover
Abstract:
The problem of scheduling tasks on $p$ processors so that no task ever gets too far behind is often described as a game with cups and water. In the $p$-processor cup game on $n$ cups, there are two players, a filler and an emptier, that take turns adding and removing water from a set of $n$ cups. In each turn, the filler adds $p$ units of water to the cups, placing at most $1$ unit of water in eac…
▽ More
The problem of scheduling tasks on $p$ processors so that no task ever gets too far behind is often described as a game with cups and water. In the $p$-processor cup game on $n$ cups, there are two players, a filler and an emptier, that take turns adding and removing water from a set of $n$ cups. In each turn, the filler adds $p$ units of water to the cups, placing at most $1$ unit of water in each cup, and then the emptier selects $p$ cups to remove up to $1$ unit of water from. The emptier's goal is to minimize the backlog, which is the height of the fullest cup.
The $p$-processor cup game has been studied in many different settings, dating back to the late 1960's. All of the past work shares one common assumption: that $p$ is fixed. This paper initiates the study of what happens when the number of available processors $p$ varies over time, resulting in what we call the \emph{variable-processor cup game}.
Remarkably, the optimal bounds for the variable-processor cup game differ dramatically from its classical counterpart. Whereas the $p$-processor cup has optimal backlog $Θ(\log n)$, the variable-processor game has optimal backlog $Θ(n)$. Moreover, there is an efficient filling strategy that yields backlog $Ω(n^{1 - ε})$ in quasi-polynomial time against any deterministic emptying strategy.
We additionally show that straightforward uses of randomization cannot be used to help the emptier. In particular, for any positive constant $Δ$, and any $Δ$-greedy-like randomized emptying algorithm $\mathcal{A}$, there is a filling strategy that achieves backlog $Ω(n^{1 - ε})$ against $\mathcal{A}$ in quasi-polynomial time.
△ Less
Submitted 9 February, 2021; v1 submitted 30 November, 2020;
originally announced December 2020.
-
In-Place Parallel-Partition Algorithms using Exclusive-Read-and-Write Memory
Authors:
William Kuszmaul,
Alek Westover
Abstract:
We present an in-place algorithm for the partition problem that has linear work and polylogarithmic span. The algorithm uses only exclusive read/write shared variables, and can be implemented using parallel-for-loops without any additional concurrency considerations (i.e., the algorithm is EREW). A key feature of the algorithm is that it exhibits provably optimal cache behavior, up to small-order…
▽ More
We present an in-place algorithm for the partition problem that has linear work and polylogarithmic span. The algorithm uses only exclusive read/write shared variables, and can be implemented using parallel-for-loops without any additional concurrency considerations (i.e., the algorithm is EREW). A key feature of the algorithm is that it exhibits provably optimal cache behavior, up to small-order factors.
We also present a second in-place EREW algorithm for the partition problem that has linear work and span $O(\log n \cdot \log \log n)$, which is within an $O(\log\log n)$ factor of the optimal span. By using this low-span algorithm as a subroutine within the cache-friendly algorithm, we are able to obtain a single EREW algorithm that combines their theoretical guarantees: the algorithm achieves span $O(\log n \cdot \log \log n)$ and optimal cache behavior. As an immediate consequence, we also get an in-place EREW quicksort algorithm with work $O(n \log n)$, span $O(\log^2 n \cdot \log \log n)$.
Whereas the standard EREW algorithm for parallel partitioning is memory-bandwidth bound on large numbers of cores, our cache-friendly algorithm is able to achieve near-ideal scaling in practice by avoiding the memory-bandwidth bottleneck. The algorithm's performance is comparable to that of the Blocked Strided Algorithm of Francis, Pannan, Frias, and Petit, which is the previous state-of-the art for parallel EREW sorting algorithms, but which lacks theoretical guarantees on its span and cache behavior.
△ Less
Submitted 8 July, 2020; v1 submitted 26 April, 2020;
originally announced April 2020.
