Showing 1–46 of 46 results for author: Leung, K K

Noise-Aware Distributed Quantum Approximate Optimization Algorithm on Near-term Quantum Hardware

Authors: Kuan-Cheng Chen, Xiatian Xu, Felix Burt, Chen-Yu Liu, Shang Yu, Kin K Leung

Abstract: This paper introduces a noise-aware distributed Quantum Approximate Optimization Algorithm (QAOA) tailored for execution on near-term quantum hardware. Leveraging a distributed framework, we address the limitations of current Noisy Intermediate-Scale Quantum (NISQ) devices, which are hindered by limited qubit counts and high error rates. Our approach decomposes large QAOA problems into smaller sub… ▽ More This paper introduces a noise-aware distributed Quantum Approximate Optimization Algorithm (QAOA) tailored for execution on near-term quantum hardware. Leveraging a distributed framework, we address the limitations of current Noisy Intermediate-Scale Quantum (NISQ) devices, which are hindered by limited qubit counts and high error rates. Our approach decomposes large QAOA problems into smaller subproblems, distributing them across multiple Quantum Processing Units (QPUs) to enhance scalability and performance. The noise-aware strategy incorporates error mitigation techniques to optimize qubit fidelity and gate operations, ensuring reliable quantum computations. We evaluate the efficacy of our framework using the HamilToniQ Benchmarking Toolkit, which quantifies the performance across various quantum hardware configurations. The results demonstrate that our distributed QAOA framework achieves significant improvements in computational speed and accuracy, showcasing its potential to solve complex optimization problems efficiently in the NISQ era. This work sets the stage for advanced algorithmic strategies and practical quantum system enhancements, contributing to the broader goal of achieving quantum advantage. △ Less

Submitted 9 August, 2024; v1 submitted 24 July, 2024; originally announced July 2024.

AdaptSFL: Adaptive Split Federated Learning in Resource-constrained Edge Networks

Authors: Zheng Lin, Guanqiao Qu, Wei Wei, Xianhao Chen, Kin K. Leung

Abstract: The increasing complexity of deep neural networks poses significant barriers to democratizing them to resource-limited edge devices. To address this challenge, split federated learning (SFL) has emerged as a promising solution by of floading the primary training workload to a server via model partitioning while enabling parallel training among edge devices. However, although system optimization su… ▽ More The increasing complexity of deep neural networks poses significant barriers to democratizing them to resource-limited edge devices. To address this challenge, split federated learning (SFL) has emerged as a promising solution by of floading the primary training workload to a server via model partitioning while enabling parallel training among edge devices. However, although system optimization substantially influences the performance of SFL under resource-constrained systems, the problem remains largely uncharted. In this paper, we provide a convergence analysis of SFL which quantifies the impact of model splitting (MS) and client-side model aggregation (MA) on the learning performance, serving as a theoretical foundation. Then, we propose AdaptSFL, a novel resource-adaptive SFL framework, to expedite SFL under resource-constrained edge computing systems. Specifically, AdaptSFL adaptively controls client-side MA and MS to balance communication-computing latency and training convergence. Extensive simulations across various datasets validate that our proposed AdaptSFL framework takes considerably less time to achieve a target accuracy than benchmarks, demonstrating the effectiveness of the proposed strategies. △ Less

Submitted 22 May, 2024; v1 submitted 19 March, 2024; originally announced March 2024.

Comments: 15 pages, 10 figures

Temporal Dependencies in Feature Importance for Time Series Predictions

Authors: Kin Kwan Leung, Clayton Rooke, Jonathan Smith, Saba Zuberi, Maksims Volkovs

Abstract: Time series data introduces two key challenges for explainability methods: firstly, observations of the same feature over subsequent time steps are not independent, and secondly, the same feature can have varying importance to model predictions over time. In this paper, we propose Windowed Feature Importance in Time (WinIT), a feature removal based explainability approach to address these issues.… ▽ More Time series data introduces two key challenges for explainability methods: firstly, observations of the same feature over subsequent time steps are not independent, and secondly, the same feature can have varying importance to model predictions over time. In this paper, we propose Windowed Feature Importance in Time (WinIT), a feature removal based explainability approach to address these issues. Unlike existing feature removal explanation methods, WinIT explicitly accounts for the temporal dependence between different observations of the same feature in the construction of its importance score. Furthermore, WinIT captures the varying importance of a feature over time, by summarizing its importance over a window of past time steps. We conduct an extensive empirical study on synthetic and real-world data, compare against a wide range of leading explainability methods, and explore the impact of various evaluation strategies. Our results show that WinIT achieves significant gains over existing methods, with more consistent performance across different evaluation metrics. The code for our work is publicly available at \url{https://github.com/layer6ai-labs/WinIT}. △ Less

Submitted 6 March, 2023; v1 submitted 29 July, 2021; originally announced July 2021.

Comments: International Conference on Learning Representations 2023 (ICLR'23)

Additive Link Metrics Identification: Proof of Selected Lemmas and Propositions

Authors: Liang Ma, Ting He, Kin K. Leung, Don Towsley, Ananthram Swami

Abstract: This is a technical report, containing all the lemma and proposition proofs in paper "Topological Constraints on Identifying Additive Link Metrics via End-to-end Paths Measurements" by Liang Ma, Ting He, Kin K. Leung, Don Towsley, and Ananthram Swami, published in Annual Conference of The International Technology Alliance (ACITA), 2012. This is a technical report, containing all the lemma and proposition proofs in paper "Topological Constraints on Identifying Additive Link Metrics via End-to-end Paths Measurements" by Liang Ma, Ting He, Kin K. Leung, Don Towsley, and Ananthram Swami, published in Annual Conference of The International Technology Alliance (ACITA), 2012. △ Less

Submitted 17 December, 2020; originally announced January 2021.

Comments: arXiv admin note: substantial text overlap with arXiv:2012.12190

Efficient Identification of Additive Link Metrics: Theorem Proof and Evaluations

Authors: Liang Ma, Ting He, Kin K. Leung, Don Towsley, Ananthram Swami

Abstract: This is a technical report, containing all the theorem proofs and additional evaluations in paper "Efficient Identification of Additive Link Metrics via Network Tomography" by Liang Ma, Ting He, Kin K. Leung, Don Towsley, and Ananthram Swami, published in IEEE ICDCS, 2013. This is a technical report, containing all the theorem proofs and additional evaluations in paper "Efficient Identification of Additive Link Metrics via Network Tomography" by Liang Ma, Ting He, Kin K. Leung, Don Towsley, and Ananthram Swami, published in IEEE ICDCS, 2013. △ Less

Submitted 17 December, 2020; originally announced December 2020.

Identification of Additive Link Metrics: Proof of Selected Theorems

Authors: Liang Ma, Ting He, Kin K. Leung, Ananthram Swami, Don Towsley

Abstract: This is a technical report, containing all the theorem proofs in the following two papers: (1) Liang Ma, Ting He, Kin K. Leung, Ananthram Swami, and Don Towsley, "Identifiability of Link Metrics Based on End-to-end Path Measurements," in ACM IMC, 2013. (2) Liang Ma, Ting He, Kin K. Leung, Ananthram Swami, and Don Towsley, "Inferring Link Metrics from End-to-end Path Measurements: Identifiability a… ▽ More This is a technical report, containing all the theorem proofs in the following two papers: (1) Liang Ma, Ting He, Kin K. Leung, Ananthram Swami, and Don Towsley, "Identifiability of Link Metrics Based on End-to-end Path Measurements," in ACM IMC, 2013. (2) Liang Ma, Ting He, Kin K. Leung, Ananthram Swami, and Don Towsley, "Inferring Link Metrics from End-to-end Path Measurements: Identifiability and Monitor Placement," IEEE/ACM Transactions on Networking, vol. 22, no. 4, pp. 1351-1368, 2014. △ Less

Submitted 23 December, 2020; v1 submitted 17 December, 2020; originally announced December 2020.

Comments: References are updated

Partial Network Identifiability: Theorem Proof and Evaluation

Authors: Liang Ma, Ting He, Kin K. Leung, Ananthram Swami, Don Towsley

Abstract: This is a technical report, containing all the theorem proofs and additional evaluations in paper "Monitor Placement for Maximal Identifiability in Network Tomography" by Liang Ma, Ting He, Kin K. Leung, Ananthram Swami, Don Towsley, published in IEEE INFOCOM, 2014. This is a technical report, containing all the theorem proofs and additional evaluations in paper "Monitor Placement for Maximal Identifiability in Network Tomography" by Liang Ma, Ting He, Kin K. Leung, Ananthram Swami, Don Towsley, published in IEEE INFOCOM, 2014. △ Less

Submitted 17 December, 2020; originally announced December 2020.

Link Identifiability with Two Monitors: Proof of Selected Theorems

Authors: Liang Ma, Ting He, Kin K. Leung, Ananthram Swami, Don Towsley

Abstract: This is a technical report, containing all the theorem proofs in paper "Link Identifiability in Communication Networks with Two Monitors" by Liang Ma, Ting He, Kin K. Leung, Ananthram Swami, and Don Towsley, published in IEEE Globecom, 2013. This is a technical report, containing all the theorem proofs in paper "Link Identifiability in Communication Networks with Two Monitors" by Liang Ma, Ting He, Kin K. Leung, Ananthram Swami, and Don Towsley, published in IEEE Globecom, 2013. △ Less

Submitted 24 December, 2020; v1 submitted 17 December, 2020; originally announced December 2020.

Comments: Auxiliary algorithms are removed from this report as they exist in the main (IEEE Globecom'13) paper. arXiv admin note: substantial text overlap with arXiv:2012.11378

Fundamental Theories in Node Failure Localization

Authors: Liang Ma, Ting He, Ananthram Swami, Don Towsley, Kin K. Leung, Jessica Lowe

Abstract: This is a technical report, containing all the theorem proofs in paper "Node Failure Localization in Communication Networks via Network Tomography" by Liang Ma, Ting He, Ananthram Swami, Don Towsley, Kin K. Leung, and Jessica Lowe, published in ITA Annual Fall Meeting, 2014. This is a technical report, containing all the theorem proofs in paper "Node Failure Localization in Communication Networks via Network Tomography" by Liang Ma, Ting He, Ananthram Swami, Don Towsley, Kin K. Leung, and Jessica Lowe, published in ITA Annual Fall Meeting, 2014. △ Less

Submitted 17 December, 2020; originally announced December 2020.

Comments: arXiv admin note: text overlap with arXiv:2012.09959

Failure Localization Capability: Theorem Proof and Evaluation

Authors: Liang Ma, Ting He, Ananthram Swami, Don Towsley, Kin K. Leung

Abstract: This is a technical report, containing all the theorem proofs and additional evaluations in paper "Network Capability in Localizing Node Failures via End-to-end Path Measurements" by Liang Ma, Ting He, Ananthram Swami, Don Towsley, and Kin K. Leung, published in IEEE/ACM Transactions on Networking, vol. 25, no. 1, pp. 434-450, 2017. This is a technical report, containing all the theorem proofs and additional evaluations in paper "Network Capability in Localizing Node Failures via End-to-end Path Measurements" by Liang Ma, Ting He, Ananthram Swami, Don Towsley, and Kin K. Leung, published in IEEE/ACM Transactions on Networking, vol. 25, no. 1, pp. 434-450, 2017. △ Less

Submitted 26 December, 2020; v1 submitted 17 December, 2020; originally announced December 2020.

Comments: Updated references

Node Failure Localization: Theorem Proof

Authors: Liang Ma, Ting He, Ananthram Swami, Don Towsley, Kin K. Leung

Abstract: This is a technical report, containing all the theorem proofs in paper "On Optimal Monitor Placement for Localizing Node Failures via Network Tomography" by Liang Ma, Ting He, Ananthram Swami, Don Towsley, and Kin K. Leung, published in IFIP WG 7.3 Performance, 2015. This is a technical report, containing all the theorem proofs in paper "On Optimal Monitor Placement for Localizing Node Failures via Network Tomography" by Liang Ma, Ting He, Ananthram Swami, Don Towsley, and Kin K. Leung, published in IFIP WG 7.3 Performance, 2015. △ Less

Submitted 16 December, 2020; originally announced December 2020.

Resource Allocation in One-dimensional Distributed Service Networks with Applications

Authors: Nitish K. Panigrahy, Prithwish Basu, Philippe Nain, Don Towsley, Ananthram Swami, Kevin S. Chan, Kin K. Leung

Abstract: We consider assignment policies that allocate resources to users, where both resources and users are located on a one-dimensional line. First, we consider unidirectional assignment policies that allocate resources only to users located to their left. We propose the Move to Right (MTR) policy, which scans from left to right assigning nearest rightmost available resource to a user, and contrast it t… ▽ More We consider assignment policies that allocate resources to users, where both resources and users are located on a one-dimensional line. First, we consider unidirectional assignment policies that allocate resources only to users located to their left. We propose the Move to Right (MTR) policy, which scans from left to right assigning nearest rightmost available resource to a user, and contrast it to the Unidirectional Gale-Shapley (UGS) matching policy. While both policies among all unidirectional policies, minimize the expected distance traveled by a request (request distance), MTR is fairer. Moreover, we show that when user and resource locations are modeled by statistical point processes, and resources are allowed to satisfy more than one user, the spatial system under unidirectional policies can be mapped into bulk service queueing systems, thus allowing the application of many queueing theory results that yield closed form expressions. As we consider a case where different resources can satisfy different numbers of users, we also generate new results for bulk service queues. We also consider bidirectional policies where there are no directional restrictions on resource allocation and develop an algorithm for computing the optimal assignment which is more efficient than known algorithms in the literature when there are more resources than users. Numerical evaluation of performance of unidirectional and bidirectional allocation schemes yields design guidelines beneficial for resource placement. \np{Finally, we present a heuristic algorithm, which leverages the optimal dynamic programming scheme for one-dimensional inputs to obtain approximate solutions to the optimal assignment problem for the two-dimensional scenario and empirically yields request distances within a constant factor of the optimal solution. △ Less

Submitted 8 November, 2020; originally announced November 2020.

Comments: arXiv admin note: text overlap with arXiv:1901.02414

On the Analysis of Spatially Constrained Power of Two Choice Policies

Authors: Nitish K. Panigrahy, Prithwish Basu, Don Towsley, Ananthram Swami, Kin K. Leung

Abstract: We consider a class of power of two choice based assignment policies for allocating users to servers, where both users and servers are located on a two-dimensional Euclidean plane. In this framework, we investigate the inherent tradeoff between the communication cost, and load balancing performance of different allocation policies. To this end, we first design and evaluate a Spatial Power of two (… ▽ More We consider a class of power of two choice based assignment policies for allocating users to servers, where both users and servers are located on a two-dimensional Euclidean plane. In this framework, we investigate the inherent tradeoff between the communication cost, and load balancing performance of different allocation policies. To this end, we first design and evaluate a Spatial Power of two (sPOT) policy in which each user is allocated to the least loaded server among its two geographically nearest servers sequentially. When servers are placed on a two-dimensional square grid, sPOT maps to the classical Power of two (POT) policy on the Delaunay graph associated with the Voronoi tessellation of the set of servers. We show that the associated Delaunay graph is 4-regular and provide expressions for asymptotic maximum load using results from the literature. For uniform placement of servers, we map sPOT to a classical balls and bins allocation policy with bins corresponding to the Voronoi regions associated with the second order Voronoi diagram of the set of servers. We provide expressions for the lower bound on the asymptotic expected maximum load on the servers and prove that sPOT does not achieve POT load balancing benefits. However, experimental results suggest the efficacy of sPOT with respect to expected communication cost. Finally, we propose two non-uniform server sampling based POT policies that achieve the best of both the performance metrics. Experimental results validate the effctiveness of our proposed policies. △ Less

Submitted 4 November, 2020; originally announced November 2020.

Jointly-Learned State-Action Embedding for Efficient Reinforcement Learning

Authors: Paul J. Pritz, Liang Ma, Kin K. Leung

Abstract: While reinforcement learning has achieved considerable successes in recent years, state-of-the-art models are often still limited by the size of state and action spaces. Model-free reinforcement learning approaches use some form of state representations and the latest work has explored embedding techniques for actions, both with the aim of achieving better generalization and applicability. However… ▽ More While reinforcement learning has achieved considerable successes in recent years, state-of-the-art models are often still limited by the size of state and action spaces. Model-free reinforcement learning approaches use some form of state representations and the latest work has explored embedding techniques for actions, both with the aim of achieving better generalization and applicability. However, these approaches consider only states or actions, ignoring the interaction between them when generating embedded representations. In this work, we establish the theoretical foundations for the validity of training a reinforcement learning agent using embedded states and actions. We then propose a new approach for jointly learning embeddings for states and actions that combines aspects of model-free and model-based reinforcement learning, which can be applied in both discrete and continuous domains. Specifically, we use a model of the environment to obtain embeddings for states and actions and present a generic architecture that leverages these to learn a policy. In this way, the embedded representations obtained via our approach enable better generalization over both states and actions by capturing similarities in the embedding spaces. Evaluations of our approach on several gaming, robotic control, and recommender systems show it significantly outperforms state-of-the-art models in both discrete/continuous domains with large state/action spaces, thus confirming its efficacy. △ Less

Submitted 20 August, 2021; v1 submitted 9 October, 2020; originally announced October 2020.

Energy-Efficient Resource Management for Federated Edge Learning with CPU-GPU Heterogeneous Computing

Authors: Qunsong Zeng, Yuqing Du, Kaibin Huang, Kin K. Leung

Abstract: Edge machine learning involves the deployment of learning algorithms at the network edge to leverage massive distributed data and computation resources to train artificial intelligence (AI) models. Among others, the framework of federated edge learning (FEEL) is popular for its data-privacy preservation. FEEL coordinates global model training at an edge server and local model training at edge devi… ▽ More Edge machine learning involves the deployment of learning algorithms at the network edge to leverage massive distributed data and computation resources to train artificial intelligence (AI) models. Among others, the framework of federated edge learning (FEEL) is popular for its data-privacy preservation. FEEL coordinates global model training at an edge server and local model training at edge devices that are connected by wireless links. This work contributes to the energy-efficient implementation of FEEL in wireless networks by designing joint computation-and-communication resource management ($\text{C}^2$RM). The design targets the state-of-the-art heterogeneous mobile architecture where parallel computing using both a CPU and a GPU, called heterogeneous computing, can significantly improve both the performance and energy efficiency. To minimize the sum energy consumption of devices, we propose a novel $\text{C}^2$RM framework featuring multi-dimensional control including bandwidth allocation, CPU-GPU workload partitioning and speed scaling at each device, and $\text{C}^2$ time division for each link. The key component of the framework is a set of equilibriums in energy rates with respect to different control variables that are proved to exist among devices or between processing units at each device. The results are applied to designing efficient algorithms for computing the optimal $\text{C}^2$RM policies faster than the standard optimization tools. Based on the equilibriums, we further design energy-efficient schemes for device scheduling and greedy spectrum sharing that scavenges "spectrum holes" resulting from heterogeneous $\text{C}^2$ time divisions among devices. Using a real dataset, experiments are conducted to demonstrate the effectiveness of $\text{C}^2$RM on improving the energy efficiency of a FEEL system. △ Less

Submitted 15 July, 2020; v1 submitted 14 July, 2020; originally announced July 2020.

State Action Separable Reinforcement Learning

Authors: Ziyao Zhang, Liang Ma, Kin K. Leung, Konstantinos Poularakis, Mudhakar Srivatsa

Abstract: Reinforcement Learning (RL) based methods have seen their paramount successes in solving serial decision-making and control problems in recent years. For conventional RL formulations, Markov Decision Process (MDP) and state-action-value function are the basis for the problem modeling and policy evaluation. However, several challenging issues still remain. Among most cited issues, the enormity of s… ▽ More Reinforcement Learning (RL) based methods have seen their paramount successes in solving serial decision-making and control problems in recent years. For conventional RL formulations, Markov Decision Process (MDP) and state-action-value function are the basis for the problem modeling and policy evaluation. However, several challenging issues still remain. Among most cited issues, the enormity of state/action space is an important factor that causes inefficiency in accurately approximating the state-action-value function. We observe that although actions directly define the agents' behaviors, for many problems the next state after a state transition matters more than the action taken, in determining the return of such a state transition. In this regard, we propose a new learning paradigm, State Action Separable Reinforcement Learning (sasRL), wherein the action space is decoupled from the value function learning process for higher efficiency. Then, a light-weight transition model is learned to assist the agent to determine the action that triggers the associated state transition. In addition, our convergence analysis reveals that under certain conditions, the convergence time of sasRL is $O(T^{1/k})$, where $T$ is the convergence time for updating the value function in the MDP-based formulation and $k$ is a weighting factor. Experiments on several gaming scenarios show that sasRL outperforms state-of-the-art MDP-based RL algorithms by up to $75\%$. △ Less

Submitted 5 June, 2020; originally announced June 2020.

Comments: 16 pages

Overcoming Noisy and Irrelevant Data in Federated Learning

Authors: Tiffany Tuor, Shiqiang Wang, Bong Jun Ko, Changchang Liu, Kin K. Leung

Abstract: Many image and vision applications require a large amount of data for model training. Collecting all such data at a central location can be challenging due to data privacy and communication bandwidth restrictions. Federated learning is an effective way of training a machine learning model in a distributed manner from local data collected by client devices, which does not require exchanging the raw… ▽ More Many image and vision applications require a large amount of data for model training. Collecting all such data at a central location can be challenging due to data privacy and communication bandwidth restrictions. Federated learning is an effective way of training a machine learning model in a distributed manner from local data collected by client devices, which does not require exchanging the raw data among clients. A challenge is that among the large variety of data collected at each client, it is likely that only a subset is relevant for a learning task while the rest of data has a negative impact on model training. Therefore, before starting the learning process, it is important to select the subset of data that is relevant to the given federated learning task. In this paper, we propose a method for distributedly selecting relevant data, where we use a benchmark model trained on a small benchmark dataset that is task-specific, to evaluate the relevance of individual data samples at each client and select the data with sufficiently high relevance. Then, each client only uses the selected subset of its data in the federated learning process. The effectiveness of our proposed approach is evaluated on multiple real-world image datasets in a simulated system with a large number of clients, showing up to $25\%$ improvement in model accuracy compared to training with all data. △ Less

Submitted 22 June, 2020; v1 submitted 22 January, 2020; originally announced January 2020.

Comments: Accepted version in the 25th International Conference on Pattern Recognition (ICPR)

Adaptive Gradient Sparsification for Efficient Federated Learning: An Online Learning Approach

Authors: Pengchao Han, Shiqiang Wang, Kin K. Leung

Abstract: Federated learning (FL) is an emerging technique for training machine learning models using geographically dispersed data collected by local entities. It includes local computation and synchronization steps. To reduce the communication overhead and improve the overall efficiency of FL, gradient sparsification (GS) can be applied, where instead of the full gradient, only a small subset of important… ▽ More Federated learning (FL) is an emerging technique for training machine learning models using geographically dispersed data collected by local entities. It includes local computation and synchronization steps. To reduce the communication overhead and improve the overall efficiency of FL, gradient sparsification (GS) can be applied, where instead of the full gradient, only a small subset of important elements of the gradient is communicated. Existing work on GS uses a fixed degree of gradient sparsity for i.i.d.-distributed data within a datacenter. In this paper, we consider adaptive degree of sparsity and non-i.i.d. local datasets. We first present a fairness-aware GS method which ensures that different clients provide a similar amount of updates. Then, with the goal of minimizing the overall training time, we propose a novel online learning formulation and algorithm for automatically determining the near-optimal communication and computation trade-off that is controlled by the degree of gradient sparsity. The online learning algorithm uses an estimated sign of the derivative of the objective function, which gives a regret bound that is asymptotically equal to the case where exact derivative is available. Experiments with real datasets confirm the benefits of our proposed approaches, showing up to $40\%$ improvement in model accuracy for a finite training time. △ Less

Submitted 20 March, 2020; v1 submitted 14 January, 2020; originally announced January 2020.

Comments: Accepted at IEEE ICDCS 2020

Fast-Fourier-Forecasting Resource Utilisation in Distributed Systems

Authors: Paul J. Pritz, Daniel Perez, Kin K. Leung

Abstract: Distributed computing systems often consist of hundreds of nodes, executing tasks with different resource requirements. Efficient resource provisioning and task scheduling in such systems are non-trivial and require close monitoring and accurate forecasting of the state of the system, specifically resource utilisation at its constituent machines. Two challenges present themselves towards these obj… ▽ More Distributed computing systems often consist of hundreds of nodes, executing tasks with different resource requirements. Efficient resource provisioning and task scheduling in such systems are non-trivial and require close monitoring and accurate forecasting of the state of the system, specifically resource utilisation at its constituent machines. Two challenges present themselves towards these objectives. First, collecting monitoring data entails substantial communication overhead. This overhead can be prohibitively high, especially in networks where bandwidth is limited. Second, forecasting models to predict resource utilisation should be accurate and need to exhibit high inference speed. Mission critical scheduling and resource allocation algorithms use these predictions and rely on their immediate availability. To address the first challenge, we present a communication-efficient data collection mechanism. Resource utilisation data is collected at the individual machines in the system and transmitted to a central controller in batches. Each batch is processed by an adaptive data-reduction algorithm based on Fourier transforms and truncation in the frequency domain. We show that the proposed mechanism leads to a significant reduction in communication overhead while incurring only minimal error and adhering to accuracy guarantees. To address the second challenge, we propose a deep learning architecture using complex Gated Recurrent Units to forecast resource utilisation. This architecture is directly integrated with the above data collection mechanism to improve inference speed of our forecasting model. Using two real-world datasets, we demonstrate the effectiveness of our approach, both in terms of forecasting accuracy and inference speed. Our approach resolves challenges encountered in resource provisioning frameworks and can be applied to other forecasting problems. △ Less

Submitted 7 August, 2020; v1 submitted 13 January, 2020; originally announced January 2020.

Resource Sharing in the Edge: A Distributed Bargaining-Theoretic Approach

Authors: Faheem Zafari, Prithwish Basu, Kin K. Leung, Jian Li, Ananthram Swami, Don Towsley

Abstract: The growing demand for edge computing resources, particularly due to increasing popularity of Internet of Things (IoT), and distributed machine/deep learning applications poses a significant challenge. On the one hand, certain edge service providers (ESPs) may not have sufficient resources to satisfy their applications according to the associated service-level agreements. On the other hand, some E… ▽ More The growing demand for edge computing resources, particularly due to increasing popularity of Internet of Things (IoT), and distributed machine/deep learning applications poses a significant challenge. On the one hand, certain edge service providers (ESPs) may not have sufficient resources to satisfy their applications according to the associated service-level agreements. On the other hand, some ESPs may have additional unused resources. In this paper, we propose a resource-sharing framework that allows different ESPs to optimally utilize their resources and improve the satisfaction level of applications subject to constraints such as communication cost for sharing resources across ESPs. Our framework considers that different ESPs have their own objectives for utilizing their resources, thus resulting in a multi-objective optimization problem. We present an $N$-person \emph{Nash Bargaining Solution} (NBS) for resource allocation and sharing among ESPs with \emph{Pareto} optimality guarantee. Furthermore, we propose a \emph{distributed}, primal-dual algorithm to obtain the NBS by proving that the strong-duality property holds for the resultant resource sharing optimization problem. Using synthetic and real-world data traces, we show numerically that the proposed NBS based framework not only enhances the ability to satisfy applications' resource demands, but also improves utilities of different ESPs. △ Less

Submitted 4 July, 2020; v1 submitted 13 January, 2020; originally announced January 2020.

Comments: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

Let's Share: A Game-Theoretic Framework for Resource Sharing in Mobile Edge Clouds

Authors: Faheem Zafari, Kin K. Leung, Don Towsley, Prithwish Basu, Ananthram Swami, Jian Li

Abstract: Mobile edge computing seeks to provide resources to different delay-sensitive applications. This is a challenging problem as an edge cloud-service provider may not have sufficient resources to satisfy all resource requests. Furthermore, allocating available resources optimally to different applications is also challenging. Resource sharing among different edge cloud-service providers can address t… ▽ More Mobile edge computing seeks to provide resources to different delay-sensitive applications. This is a challenging problem as an edge cloud-service provider may not have sufficient resources to satisfy all resource requests. Furthermore, allocating available resources optimally to different applications is also challenging. Resource sharing among different edge cloud-service providers can address the aforementioned limitation as certain service providers may have resources available that can be ``rented'' by other service providers. However, edge cloud service providers can have different objectives or \emph{utilities}. Therefore, there is a need for an efficient and effective mechanism to share resources among service providers, while considering the different objectives of various providers. We model resource sharing as a multi-objective optimization problem and present a solution framework based on \emph{Cooperative Game Theory} (CGT). We consider the strategy where each service provider allocates resources to its native applications first and shares the remaining resources with applications from other service providers. We prove that for a monotonic, non-decreasing utility function, the game is canonical and convex. Hence, the \emph{core} is not empty and the grand coalition is stable. We propose two algorithms \emph{Game-theoretic Pareto optimal allocation} (GPOA) and \emph{Polyandrous-Polygamous Matching based Pareto Optimal Allocation} (PPMPOA) that provide allocations from the core. Hence the obtained allocations are \emph{Pareto} optimal and the grand coalition of all the service providers is stable. Experimental results confirm that our proposed resource sharing framework improves utilities of edge cloud-service providers and application request satisfaction. △ Less

Submitted 2 January, 2020; originally announced January 2020.

Comments: The paper is currently under review in IEEE Transactions on Network and Service Management

Model Pruning Enables Efficient Federated Learning on Edge Devices

Authors: Yuang Jiang, Shiqiang Wang, Victor Valls, Bong Jun Ko, Wei-Han Lee, Kin K. Leung, Leandros Tassiulas

Abstract: Federated learning (FL) allows model training from local data collected by edge/mobile devices while preserving data privacy, which has wide applicability to image and vision applications. A challenge is that client devices in FL usually have much more limited computation and communication resources compared to servers in a datacenter. To overcome this challenge, we propose PruneFL -- a novel FL a… ▽ More Federated learning (FL) allows model training from local data collected by edge/mobile devices while preserving data privacy, which has wide applicability to image and vision applications. A challenge is that client devices in FL usually have much more limited computation and communication resources compared to servers in a datacenter. To overcome this challenge, we propose PruneFL -- a novel FL approach with adaptive and distributed parameter pruning, which adapts the model size during FL to reduce both communication and computation overhead and minimize the overall training time, while maintaining a similar accuracy as the original model. PruneFL includes initial pruning at a selected client and further pruning as part of the FL process. The model size is adapted during this process, which includes maximizing the approximate empirical risk reduction divided by the time of one FL round. Our experiments with various datasets on edge devices (e.g., Raspberry Pi) show that: (i) we significantly reduce the training time compared to conventional FL and various other pruning-based methods; (ii) the pruned model with automatically determined size converges to an accuracy that is very similar to the original model, and it is also a lottery ticket of the original model. △ Less

Submitted 6 April, 2022; v1 submitted 26 September, 2019; originally announced September 2019.

Comments: Accepted for publication in IEEE Transactions on Neural Networks and Learning Systems (TNNLS)

MACS: Deep Reinforcement Learning based SDN Controller Synchronization Policy Design

Authors: Ziyao Zhang, Liang Ma, Konstantinos Poularakis, Kin K. Leung, Jeremy Tucker, Ananthram Swami

Abstract: In distributed software-defined networks (SDN), multiple physical SDN controllers, each managing a network domain, are implemented to balance centralised control, scalability, and reliability requirements. In such networking paradigms, controllers synchronize with each other, in attempts to maintain a logically centralised network view. Despite the presence of various design proposals for distribu… ▽ More In distributed software-defined networks (SDN), multiple physical SDN controllers, each managing a network domain, are implemented to balance centralised control, scalability, and reliability requirements. In such networking paradigms, controllers synchronize with each other, in attempts to maintain a logically centralised network view. Despite the presence of various design proposals for distributed SDN controller architectures, most existing works only aim at eliminating anomalies arising from the inconsistencies in different controllers' network views. However, the performance aspect of controller synchronization designs with respect to given SDN applications are generally missing. To fill this gap, we formulate the controller synchronization problem as a Markov decision process (MDP) and apply reinforcement learning techniques combined with deep neural networks (DNNs) to train a smart, scalable, and fine-grained controller synchronization policy, called the Multi-Armed Cooperative Synchronization (MACS), whose goal is to maximise the performance enhancements brought by controller synchronizations. Evaluation results confirm the DNN's exceptional ability in abstracting latent patterns in the distributed SDN environment, rendering significant superiority to MACS-based synchronization policy, which are 56% and 30% performance improvements over ONOS and greedy SDN controller synchronization heuristics. △ Less

Submitted 19 September, 2019; originally announced September 2019.

Comments: Published as a conference paper at ICNP 2019

Energy-Efficient Radio Resource Allocation for Federated Edge Learning

Authors: Qunsong Zeng, Yuqing Du, Kin K. Leung, Kaibin Huang

Abstract: Edge machine learning involves the development of learning algorithms at the network edge to leverage massive distributed data and computation resources. Among others, the framework of federated edge learning (FEEL) is particularly promising for its data-privacy preservation. FEEL coordinates global model training at a server and local model training at edge devices over wireless links. In this wo… ▽ More Edge machine learning involves the development of learning algorithms at the network edge to leverage massive distributed data and computation resources. Among others, the framework of federated edge learning (FEEL) is particularly promising for its data-privacy preservation. FEEL coordinates global model training at a server and local model training at edge devices over wireless links. In this work, we explore the new direction of energy-efficient radio resource management (RRM) for FEEL. To reduce devices' energy consumption, we propose energy-efficient strategies for bandwidth allocation and scheduling. They adapt to devices' channel states and computation capacities so as to reduce their sum energy consumption while warranting learning performance. In contrast with the traditional rate-maximization designs, the derived optimal policies allocate more bandwidth to those scheduled devices with weaker channels or poorer computation capacities, which are the bottlenecks of synchronized model updates in FEEL. On the other hand, the scheduling priority function derived in closed form gives preferences to devices with better channels and computation capacities. Substantial energy reduction contributed by the proposed strategies is demonstrated in learning experiments. △ Less

Submitted 13 July, 2019; originally announced July 2019.

Automated Computer Evaluation of Acute Ischemic Stroke and Large Vessel Occlusion

Authors: Jia You, Philip L. H. Yu, Anderson C. O. Tsang, Eva L. H. Tsui, Pauline P. S. Woo, Gilberto K. K. Leung

Abstract: Large vessel occlusion (LVO) plays an important role in the diagnosis of acute ischemic stroke. Identifying LVO of patients in the early stage on admission would significantly lower the probabilities of suffering from severe effects due to stroke or even save their lives. In this paper, we utilized both structural and imaging data from all recorded acute ischemic stroke patients in Hong Kong. Tota… ▽ More Large vessel occlusion (LVO) plays an important role in the diagnosis of acute ischemic stroke. Identifying LVO of patients in the early stage on admission would significantly lower the probabilities of suffering from severe effects due to stroke or even save their lives. In this paper, we utilized both structural and imaging data from all recorded acute ischemic stroke patients in Hong Kong. Total 300 patients (200 training and 100 testing) are used in this study. We established three hierarchical models based on demographic data, clinical data and features obtained from computerized tomography (CT) scans. The first two stages of modeling are merely based on demographic and clinical data. Besides, the third model utilized extra CT imaging features obtained from deep learning model. The optimal cutoff is determined at the maximal Youden index based on 10-fold cross-validation. With both clinical and imaging features, the Level-3 model achieved the best performance on testing data. The sensitivity, specificity, Youden index, accuracy and area under the curve (AUC) are 0.930, 0.684, 0.614, 0.790 and 0.850 respectively. △ Less

Submitted 18 June, 2019; originally announced June 2019.

Online Collection and Forecasting of Resource Utilization in Large-Scale Distributed Systems

Authors: Tiffany Tuor, Shiqiang Wang, Kin K. Leung, Bong Jun Ko

Abstract: Large-scale distributed computing systems often contain thousands of distributed nodes (machines). Monitoring the conditions of these nodes is important for system management purposes, which, however, can be extremely resource demanding as this requires collecting local measurements of each individual node and constantly sending those measurements to a central controller. Meanwhile, it is often us… ▽ More Large-scale distributed computing systems often contain thousands of distributed nodes (machines). Monitoring the conditions of these nodes is important for system management purposes, which, however, can be extremely resource demanding as this requires collecting local measurements of each individual node and constantly sending those measurements to a central controller. Meanwhile, it is often useful to forecast the future system conditions for various purposes such as resource planning/allocation and anomaly detection, but it is usually too resource-consuming to have one forecasting model running for each node, which may also neglect correlations in observed metrics across different nodes. In this paper, we propose a mechanism for collecting and forecasting the resource utilization of machines in a distributed computing system in a scalable manner. We present an algorithm that allows each local node to decide when to transmit its most recent measurement to the central node, so that the transmission frequency is kept below a given constraint value. Based on the measurements received from local nodes, the central node summarizes the received data into a small number of clusters. Since the cluster partitioning can change over time, we also present a method to capture the evolution of clusters and their centroids. As an effective way to reduce the amount of computation, time-series forecasting models are trained on the time-varying centroids of each cluster, to forecast the future resource utilizations of a group of local nodes. The effectiveness of our proposed approach is confirmed by extensive experiments using multiple real-world datasets. △ Less

Submitted 22 May, 2019; originally announced May 2019.

Comments: Accepted at IEEE International Conference on Distributed Computing Systems (ICDCS) 2019

Automated Segmentation for Hyperdense Middle Cerebral Artery Sign of Acute Ischemic Stroke on Non-Contrast CT Images

Authors: Jia You, Philip L. H. Yu, Anderson C. O. Tsang, Eva L. H. Tsui, Pauline P. S. Woo, Gilberto K. K. Leung

Abstract: The hyperdense middle cerebral artery (MCA) dot sign has been reported as an important factor in the diagnosis of acute ischemic stroke due to large vessel occlusion. Interpreting the initial CT brain scan in these patients requires high level of expertise, and has high inter-observer variability. An automated computerized interpretation of the urgent CT brain image, with an emphasis to pick up ea… ▽ More The hyperdense middle cerebral artery (MCA) dot sign has been reported as an important factor in the diagnosis of acute ischemic stroke due to large vessel occlusion. Interpreting the initial CT brain scan in these patients requires high level of expertise, and has high inter-observer variability. An automated computerized interpretation of the urgent CT brain image, with an emphasis to pick up early signs of ischemic stroke will facilitate early patient diagnosis, triage, and shorten the door-to-revascularization time for these group of patients. In this paper, we present an automated detection method of segmenting the MCA dot sign on non-contrast CT brain image scans based on powerful deep learning technique. △ Less

Submitted 22 May, 2019; originally announced May 2019.

Diabetes Mellitus Forecasting Using Population Health Data in Ontario, Canada

Authors: Mathieu Ravaut, Hamed Sadeghi, Kin Kwan Leung, Maksims Volkovs, Laura C. Rosella

Abstract: Leveraging health administrative data (HAD) datasets for predicting the risk of chronic diseases including diabetes has gained a lot of attention in the machine learning community recently. In this paper, we use the largest health records datasets of patients in Ontario,Canada. Provided by the Institute of Clinical Evaluative Sciences (ICES), this database is age, gender and ethnicity-diverse. The… ▽ More Leveraging health administrative data (HAD) datasets for predicting the risk of chronic diseases including diabetes has gained a lot of attention in the machine learning community recently. In this paper, we use the largest health records datasets of patients in Ontario,Canada. Provided by the Institute of Clinical Evaluative Sciences (ICES), this database is age, gender and ethnicity-diverse. The datasets include demographics, lab measurements,drug benefits, healthcare system interactions, ambulatory and hospitalizations records. We perform one of the first large-scale machine learning studies with this data to study the task of predicting diabetes in a range of 1-10 years ahead, which requires no additional screening of individuals.In the best setup, we reach a test AUC of 80.3 with a single-model trained on an observation window of 5 years with a one-year buffer using all datasets. A subset of top 15 features alone (out of a total of 963) could provide a test AUC of 79.1. In this paper, we provide extensive machine learning model performance and feature contribution analysis, which enables us to narrow down to the most important features useful for diabetes forecasting. Examples include chronic conditions such as asthma and hypertension, lab results, diagnostic codes in insurance claims, age and geographical information. △ Less

Submitted 8 April, 2019; originally announced April 2019.

Comments: 18 pages, 3 figures, 8 Tables, Submitted to 2019 ML for Healthcare conference

A Game-Theoretic Framework for Resource Sharing in Clouds

Authors: Faheem Zafari, Kin K. Leung, Don Towsley, Prithwish Basu, Ananthram Swami

Abstract: Providing resources to different users or applications is fundamental to cloud computing. This is a challenging problem as a cloud service provider may have insufficient resources to satisfy all user requests. Furthermore, allocating available resources optimally to different applications is also challenging. Resource sharing among different cloud service providers can improve resource availabilit… ▽ More Providing resources to different users or applications is fundamental to cloud computing. This is a challenging problem as a cloud service provider may have insufficient resources to satisfy all user requests. Furthermore, allocating available resources optimally to different applications is also challenging. Resource sharing among different cloud service providers can improve resource availability and resource utilization as certain cloud service providers may have free resources available that can be ``rented'' by other service providers. However, different cloud service providers can have different objectives or \emph{utilities}. Therefore, there is a need for a framework that can share and allocate resources in an efficient and effective way, while taking into account the objectives of various service providers that results in a \emph{multi-objective optimization} problem. In this paper, we present a \emph{Cooperative Game Theory} (CGT) based framework for resource sharing and allocation among different service providers with varying objectives that form a coalition. We show that the resource sharing problem can be modeled as an $N-$player \emph{canonical} cooperative game with \emph{non-transferable utility} (NTU) and prove that the game is convex for monotonic non-decreasing utilities. We propose an $\mathcal{O}({N})$ algorithm that provides an allocation from the \emph{core}, hence guaranteeing \emph{Pareto optimality}. We evaluate the performance of our proposed resource sharing framework in a number of simulation settings and show that our proposed framework improves user satisfaction and utility of service providers. △ Less

Submitted 28 May, 2019; v1 submitted 1 April, 2019; originally announced April 2019.

Comments: The paper has been accepted for publication in IFIP WMNC 2019, Paris France

Learning the Optimal Synchronization Rates in Distributed SDN Control Architectures

Authors: Konstantinos Poularakis, Qiaofeng Qin, Liang Ma, Sastry Kompella, Kin K. Leung, Leandros Tassiulas

Abstract: Since the early development of Software-Defined Network (SDN) technology, researchers have been concerned with the idea of physical distribution of the control plane to address scalability and reliability challenges of centralized designs. However, having multiple controllers managing the network while maintaining a "logically-centralized" network view brings additional challenges. One such challe… ▽ More Since the early development of Software-Defined Network (SDN) technology, researchers have been concerned with the idea of physical distribution of the control plane to address scalability and reliability challenges of centralized designs. However, having multiple controllers managing the network while maintaining a "logically-centralized" network view brings additional challenges. One such challenge is how to coordinate the management decisions made by the controllers which is usually achieved by disseminating synchronization messages in a peer-to-peer manner. While there exist many architectures and protocols to ensure synchronized network views and drive coordination among controllers, there is no systematic methodology for deciding the optimal frequency (or rate) of message dissemination. In this paper, we fill this gap by introducing the SDN synchronization problem: how often to synchronize the network views for each controller pair. We consider two different objectives; first, the maximization of the number of controller pairs that are synchronized, and second, the maximization of the performance of applications of interest which may be affected by the synchronization rate. Using techniques from knapsack optimization and learning theory, we derive algorithms with provable performance guarantees for each objective. Evaluation results demonstrate significant benefits over baseline schemes that synchronize all controller pairs at equal rate. △ Less

Submitted 25 January, 2019; originally announced January 2019.

Comments: IEEE Infocom 2019

Resource Allocation in One-dimensional Distributed Service Networks

Authors: Nitish K. Panigrahy, Prithwish Basu, Philippe Nain, Don Towsley, Ananthram Swami, Kevin S. Chan, Kin K. Leung

Abstract: We consider assignment policies that allocate resources to users, where both resources and users are located on a one-dimensional line. First, we consider unidirectional assignment policies that allocate resources only to users located to their left. We propose the Move to Right (MTR) policy, which scans from left to right assigning nearest rightmost available resource to a user, and contrast it t… ▽ More We consider assignment policies that allocate resources to users, where both resources and users are located on a one-dimensional line. First, we consider unidirectional assignment policies that allocate resources only to users located to their left. We propose the Move to Right (MTR) policy, which scans from left to right assigning nearest rightmost available resource to a user, and contrast it to the Unidirectional Gale-Shapley (UGS) matching policy. While both these policies are optimal among all unidirectional policies, we show that they are equivalent with respect to the expected distance traveled by a request (request distance), although MTR is fairer. Moreover, we show that when user and resource locations are modeled by statistical point processes, and resources are allowed to satisfy more than one user, the spatial system under unidirectional policies can be mapped into bulk service queuing systems, thus allowing the application of a plethora of queuing theory results that yield closed form expressions. As we consider a case where different resources can satisfy different numbers of users, we also generate new results for bulk service queues. We also consider bidirectional policies where there are no directional restrictions on resource allocation and develop an algorithm for computing the optimal assignment which is more efficient than known algorithms in the literature when there are more resources than users. Finally, numerical evaluation of performance of unidirectional and bidirectional allocation schemes yields design guidelines beneficial for resource placement. △ Less

Submitted 11 February, 2020; v1 submitted 8 January, 2019; originally announced January 2019.

DQ Scheduler: Deep Reinforcement Learning Based Controller Synchronization in Distributed SDN

Authors: Ziyao Zhang, Liang Ma, Konstantinos Poularakis, Kin K. Leung, Lingfei Wu

Abstract: In distributed software-defined networks (SDN), multiple physical SDN controllers, each managing a network domain, are implemented to balance centralized control, scalability and reliability requirements. In such networking paradigm, controllers synchronize with each other to maintain a logically centralized network view. Despite various proposals of distributed SDN controller architectures, most… ▽ More In distributed software-defined networks (SDN), multiple physical SDN controllers, each managing a network domain, are implemented to balance centralized control, scalability and reliability requirements. In such networking paradigm, controllers synchronize with each other to maintain a logically centralized network view. Despite various proposals of distributed SDN controller architectures, most existing works only assume that such logically centralized network view can be achieved with some synchronization designs, but the question of how exactly controllers should synchronize with each other to maximize the benefits of synchronization under the eventual consistency assumptions is largely overlooked. To this end, we formulate the controller synchronization problem as a Markov Decision Process (MDP) and apply reinforcement learning techniques combined with deep neural network to train a smart controller synchronization policy, which we call the Deep-Q (DQ) Scheduler. Evaluation results show that DQ Scheduler outperforms the antientropy algorithm implemented in the ONOS controller by up to 95.2% for inter-domain routing tasks. △ Less

Submitted 3 December, 2018; originally announced December 2018.

Comments: 7 pages

A Game-Theoretic Approach to Multi-Objective Resource Sharing and Allocation in Mobile Edge Clouds

Authors: Faheem Zafari, Jian Li, Kin K Leung, Don Towsley, Ananthram Swami

Abstract: Mobile edge computing seeks to provide resources to different delay-sensitive applications. However, allocating the limited edge resources to a number of applications is a challenging problem. To alleviate the resource scarcity problem, we propose sharing of resources among multiple edge computing service providers where each service provider has a particular utility to optimize. We model the reso… ▽ More Mobile edge computing seeks to provide resources to different delay-sensitive applications. However, allocating the limited edge resources to a number of applications is a challenging problem. To alleviate the resource scarcity problem, we propose sharing of resources among multiple edge computing service providers where each service provider has a particular utility to optimize. We model the resource allocation and sharing problem as a multi-objective optimization problem and present a \emph{Cooperative Game Theory} (CGT) based framework, where each edge service provider first satisfies its native applications and then shares its remaining resources (if available) with users of other providers. Furthermore, we propose an $\mathcal{O}(N)$ algorithm that provides allocation decisions from the \emph{core}, hence the obtained allocations are \emph{Pareto} optimal and the grand coalition of all the service providers is stable. Experimental results show that our proposed resource allocation and sharing framework improves the utility of all the service providers compared with the case where the service providers are working alone (no resource sharing). Our $\mathcal{O}(N)$ algorithm reduces the time complexity of obtaining a solution from the core by as much as 71.67\% when compared with the \emph{Shapley value}. △ Less

Submitted 25 August, 2018; v1 submitted 21 August, 2018; originally announced August 2018.

Comments: The paper has been accepted for publication in ACM Mobicom workshop "Technologies for the Wireless Edge" 2018

Adaptive Federated Learning in Resource Constrained Edge Computing Systems

Authors: Shiqiang Wang, Tiffany Tuor, Theodoros Salonidis, Kin K. Leung, Christian Makaya, Ting He, Kevin Chan

Abstract: Emerging technologies and applications including Internet of Things (IoT), social networking, and crowd-sourcing generate large amounts of data at the network edge. Machine learning models are often built from the collected data, to enable the detection, classification, and prediction of future events. Due to bandwidth, storage, and privacy concerns, it is often impractical to send all the data to… ▽ More Emerging technologies and applications including Internet of Things (IoT), social networking, and crowd-sourcing generate large amounts of data at the network edge. Machine learning models are often built from the collected data, to enable the detection, classification, and prediction of future events. Due to bandwidth, storage, and privacy concerns, it is often impractical to send all the data to a centralized location. In this paper, we consider the problem of learning model parameters from data distributed across multiple edge nodes, without sending raw data to a centralized place. Our focus is on a generic class of machine learning models that are trained using gradient-descent based approaches. We analyze the convergence bound of distributed gradient descent from a theoretical point of view, based on which we propose a control algorithm that determines the best trade-off between local update and global parameter aggregation to minimize the loss function under a given resource budget. The performance of the proposed algorithm is evaluated via extensive experiments with real datasets, both on a networked prototype system and in a larger-scale simulated environment. The experimentation results show that our proposed approach performs near to the optimum with various machine learning models and different data distributions. △ Less

Submitted 16 February, 2019; v1 submitted 14 April, 2018; originally announced April 2018.

Comments: This version (excluding appendices) has been accepted for publication in the IEEE Journal on Selected Areas in Communications

Joint Data Compression and Caching: Approaching Optimality with Guarantees

Authors: Jian Li, Faheem Zafari, Don Towsley, Kin K. Leung, Ananthram Swami

Abstract: We consider the problem of optimally compressing and caching data across a communication network. Given the data generated at edge nodes and a routing path, our goal is to determine the optimal data compression ratios and caching decisions across the network in order to minimize average latency, which can be shown to be equivalent to maximizing the compression and caching gain under an energy cons… ▽ More We consider the problem of optimally compressing and caching data across a communication network. Given the data generated at edge nodes and a routing path, our goal is to determine the optimal data compression ratios and caching decisions across the network in order to minimize average latency, which can be shown to be equivalent to maximizing the compression and caching gain under an energy consumption constraint. We show that this problem is NP-hard in general and the hardness is caused by the caching decision subproblem, while the compression sub-problem is polynomial-time solvable. We then propose an approximation algorithm that achieves a $(1-1/e)$-approximation solution to the optimum in strongly polynomial time. We show that our proposed algorithm achieve the near-optimal performance in synthetic-based evaluations. In this paper, we consider a tree-structured network as an illustrative example, but our results easily extend to general network topology at the expense of more complicated notations. △ Less

Submitted 24 January, 2018; v1 submitted 6 January, 2018; originally announced January 2018.

How Better is Distributed SDN? An Analytical Approach

Authors: Ziyao Zhang, Liang Ma, Kin K. Leung, Franck Le, Sastry Kompella, Leandros Tassiulas

Abstract: Distributed software-defined networks (SDN), consisting of multiple inter-connected network domains, each managed by one SDN controller, is an emerging networking architecture that offers balanced centralized control and distributed operations. Under such networking paradigm, most existing works focus on designing sophisticated controller-synchronization strategies to improve joint controller-deci… ▽ More Distributed software-defined networks (SDN), consisting of multiple inter-connected network domains, each managed by one SDN controller, is an emerging networking architecture that offers balanced centralized control and distributed operations. Under such networking paradigm, most existing works focus on designing sophisticated controller-synchronization strategies to improve joint controller-decision-making for inter-domain routing. However, there is still a lack of fundamental understanding of how the performance of distributed SDN is related to network attributes, thus impossible to justify the necessity of complicated strategies. In this regard, we analyze and quantify the performance enhancement of distributed SDN architectures, influenced by intra-/inter-domain synchronization levels and network structural properties. Based on a generic weighted network model, we establish analytical methods for performance estimation under four synchronization scenarios with increasing synchronization cost. Moreover, two of these synchronization scenarios correspond to extreme cases, i.e., minimum/maximum synchronization, which are, therefore, capable of bounding the performance of distributed SDN with any given synchronization levels. Our theoretical results reveal how network performance is related to synchronization levels and inter-domain connections, the accuracy of which are confirmed by simulations based on both real and synthetic networks. To the best of our knowledge, this is the first work quantifying the performance of distributed SDN analytically, which provides fundamental guidance for future SDN protocol designs and performance estimation. △ Less

Submitted 12 December, 2017; originally announced December 2017.

Optimal Energy Consumption with Communication, Computation, Caching and QoI-Guarantee

Authors: Faheem Zafari, Jian Li, Kin K. Leung, Don Towsley, Ananthram Swami

Abstract: Energy efficiency is a fundamental requirement of modern data communication systems, and its importance is reflected in much recent work on performance analysis of system energy consumption. However, most works have only focused on communication and computation costs, but do not account for caching costs. Given the increasing interest in cache networks, this is a serious deficiency. {In this paper… ▽ More Energy efficiency is a fundamental requirement of modern data communication systems, and its importance is reflected in much recent work on performance analysis of system energy consumption. However, most works have only focused on communication and computation costs, but do not account for caching costs. Given the increasing interest in cache networks, this is a serious deficiency. {In this paper, we consider the problem of energy consumption in data communication, compression and caching (C$3$) with a Quality of Information (QoI) guarantee in a communication network. {Our goal is to identify the optimal data compression rate and data placement over the network to minimize the overall energy consumption in the network.} he formulated problem is a \emph{Mixed Integer Non-Linear Programming} (MINLP) problem with non-convex functions, which is NP-hard in general. } {We} propose a variant of spatial branch and bound algorithm (V-SBB), that can {provide} the $ε$-global optimal solution to {our problem}. {We numerically show that our C3 optimization framework can improve the energy efficiency up to 88\% compared to any C2 optimization between communication and computation or caching. Furthermore, for our energy consumption problem, V-SBB {provides comparatively better solution than some other MINLP solvers.}} △ Less

Submitted 26 January, 2019; v1 submitted 10 December, 2017; originally announced December 2017.

Live Service Migration in Mobile Edge Clouds

Authors: Andrew Machen, Shiqiang Wang, Kin K. Leung, Bong Jun Ko, Theodoros Salonidis

Abstract: Mobile edge clouds (MECs) bring the benefits of the cloud closer to the user, by installing small cloud infrastructures at the network edge. This enables a new breed of real-time applications, such as instantaneous object recognition and safety assistance in intelligent transportation systems, that require very low latency. One key issue that comes with proximity is how to ensure that users always… ▽ More Mobile edge clouds (MECs) bring the benefits of the cloud closer to the user, by installing small cloud infrastructures at the network edge. This enables a new breed of real-time applications, such as instantaneous object recognition and safety assistance in intelligent transportation systems, that require very low latency. One key issue that comes with proximity is how to ensure that users always receive good performance as they move across different locations. Migrating services between MECs is seen as the means to achieve this. This article presents a layered framework for migrating active service applications that are encapsulated either in virtual machines (VMs) or containers. This layering approach allows a substantial reduction in service downtime. The framework is easy to implement using readily available technologies, and one of its key advantages is that it supports containers, which is a promising emerging technology that offers tangible benefits over VMs. The migration performance of various real applications is evaluated by experiments under the presented framework. Insights drawn from the experimentation results are discussed. △ Less

Submitted 2 August, 2017; v1 submitted 13 June, 2017; originally announced June 2017.

Comments: This is the author's version of the paper accepted for publication in IEEE Wireless Communications

Online Placement of Multi-Component Applications in Edge Computing Environments

Authors: Shiqiang Wang, Murtaza Zafer, Kin K. Leung

Abstract: Mobile edge computing is a new cloud computing paradigm which makes use of small-sized edge-clouds to provide real-time services to users. These mobile edge-clouds (MECs) are located in close proximity to users, thus enabling users to seamlessly access applications running on MECs. Due to the co-existence of the core (centralized) cloud, users, and one or multiple layers of MECs, an important prob… ▽ More Mobile edge computing is a new cloud computing paradigm which makes use of small-sized edge-clouds to provide real-time services to users. These mobile edge-clouds (MECs) are located in close proximity to users, thus enabling users to seamlessly access applications running on MECs. Due to the co-existence of the core (centralized) cloud, users, and one or multiple layers of MECs, an important problem is to decide where (on which computational entity) to place different components of an application. This problem, known as the application or workload placement problem, is notoriously hard, and therefore, heuristic algorithms without performance guarantees are generally employed in common practice, which may unknowingly suffer from poor performance as compared to the optimal solution. In this paper, we address the application placement problem and focus on developing algorithms with provable performance bounds. We model the user application as an application graph and the physical computing system as a physical graph, with resource demands/availabilities annotated on these graphs. We first consider the placement of a linear application graph and propose an algorithm for finding its optimal solution. Using this result, we then generalize the formulation and obtain online approximation algorithms with polynomial-logarithmic (poly-log) competitive ratio for tree application graph placement. We jointly consider node and link assignment, and incorporate multiple types of computational resources at nodes. △ Less

Submitted 9 February, 2017; v1 submitted 25 May, 2016; originally announced May 2016.

Comments: This is the author's version of the paper accepted for publication in IEEE Access

Network Capability in Localizing Node Failures via End-to-end Path Measurements

Authors: Liang Ma, Ting He, Ananthram Swami, Don Towsley, Kin K. Leung

Abstract: We investigate the capability of localizing node failures in communication networks from binary states (normal/failed) of end-to-end paths. Given a set of nodes of interest, uniquely localizing failures within this set requires that different observable path states associate with different node failure events. However, this condition is difficult to test on large networks due to the need to enumer… ▽ More We investigate the capability of localizing node failures in communication networks from binary states (normal/failed) of end-to-end paths. Given a set of nodes of interest, uniquely localizing failures within this set requires that different observable path states associate with different node failure events. However, this condition is difficult to test on large networks due to the need to enumerate all possible node failures. Our first contribution is a set of sufficient/necessary conditions for identifying a bounded number of failures within an arbitrary node set that can be tested in polynomial time. In addition to network topology and locations of monitors, our conditions also incorporate constraints imposed by the probing mechanism used. We consider three probing mechanisms that differ according to whether measurement paths are (i) arbitrarily controllable, (ii) controllable but cycle-free, or (iii) uncontrollable (determined by the default routing protocol). Our second contribution is to quantify the capability of failure localization through (1) the maximum number of failures (anywhere in the network) such that failures within a given node set can be uniquely localized, and (2) the largest node set within which failures can be uniquely localized under a given bound on the total number of failures. Both measures in (1-2) can be converted into functions of a per-node property, which can be computed efficiently based on the above sufficient/necessary conditions. We demonstrate how measures (1-2) proposed for quantifying failure localization capability can be used to evaluate the impact of various parameters, including topology, number of monitors, and probing mechanisms. △ Less

Submitted 21 September, 2015; originally announced September 2015.

Dynamic Service Migration in Mobile Edge Computing Based on Markov Decision Process

Authors: Shiqiang Wang, Rahul Urgaonkar, Murtaza Zafer, Ting He, Kevin Chan, Kin K. Leung

Abstract: In mobile edge computing, local edge servers can host cloud-based services, which reduces network overhead and latency but requires service migrations as users move to new locations. It is challenging to make migration decisions optimally because of the uncertainty in such a dynamic cloud environment. In this paper, we formulate the service migration problem as a Markov Decision Process (MDP). Our… ▽ More In mobile edge computing, local edge servers can host cloud-based services, which reduces network overhead and latency but requires service migrations as users move to new locations. It is challenging to make migration decisions optimally because of the uncertainty in such a dynamic cloud environment. In this paper, we formulate the service migration problem as a Markov Decision Process (MDP). Our formulation captures general cost models and provides a mathematical framework to design optimal service migration policies. In order to overcome the complexity associated with computing the optimal policy, we approximate the underlying state space by the distance between the user and service locations. We show that the resulting MDP is exact for uniform one-dimensional user mobility while it provides a close approximation for uniform two-dimensional mobility with a constant additive error. We also propose a new algorithm and a numerical technique for computing the optimal solution which is significantly faster than traditional methods based on standard value or policy iteration. We illustrate the application of our solution in practical scenarios where many theoretical assumptions are relaxed. Our evaluations based on real-world mobility traces of San Francisco taxis show superior performance of the proposed solution compared to baseline solutions. △ Less

Submitted 8 May, 2019; v1 submitted 17 June, 2015; originally announced June 2015.

Comments: Journal version

Mobility-Induced Service Migration in Mobile Micro-Clouds

Authors: Shiqiang Wang, Rahul Urgaonkar, Ting He, Murtaza Zafer, Kevin Chan, Kin K. Leung

Abstract: Mobile micro-cloud is an emerging technology in distributed computing, which is aimed at providing seamless computing/data access to the edge of the network when a centralized service may suffer from poor connectivity and long latency. Different from the traditional cloud, a mobile micro-cloud is smaller and deployed closer to users, typically attached to a cellular basestation or wireless network… ▽ More Mobile micro-cloud is an emerging technology in distributed computing, which is aimed at providing seamless computing/data access to the edge of the network when a centralized service may suffer from poor connectivity and long latency. Different from the traditional cloud, a mobile micro-cloud is smaller and deployed closer to users, typically attached to a cellular basestation or wireless network access point. Due to the relatively small coverage area of each basestation or access point, when a user moves across areas covered by different basestations or access points which are attached to different micro-clouds, issues of service performance and service migration become important. In this paper, we consider such migration issues. We model the general problem as a Markov decision process (MDP), and show that, in the special case where the mobile user follows a one-dimensional asymmetric random walk mobility model, the optimal policy for service migration is a threshold policy. We obtain the analytical solution for the cost resulting from arbitrary thresholds, and then propose an algorithm for finding the optimal thresholds. The proposed algorithm is more efficient than standard mechanisms for solving MDPs. △ Less

Submitted 17 March, 2015; originally announced March 2015.

Comments: in Proc. of IEEE MILCOM 2014, Oct. 2014

Dynamic Service Placement for Mobile Micro-Clouds with Predicted Future Costs

Authors: Shiqiang Wang, Rahul Urgaonkar, Ting He, Kevin Chan, Murtaza Zafer, Kin K. Leung

Abstract: Mobile micro-clouds are promising for enabling performance-critical cloud applications. However, one challenge therein is the dynamics at the network edge. In this paper, we study how to place service instances to cope with these dynamics, where multiple users and service instances coexist in the system. Our goal is to find the optimal placement (configuration) of instances to minimize the average… ▽ More Mobile micro-clouds are promising for enabling performance-critical cloud applications. However, one challenge therein is the dynamics at the network edge. In this paper, we study how to place service instances to cope with these dynamics, where multiple users and service instances coexist in the system. Our goal is to find the optimal placement (configuration) of instances to minimize the average cost over time, leveraging the ability of predicting future cost parameters with known accuracy. We first propose an offline algorithm that solves for the optimal configuration in a specific look-ahead time-window. Then, we propose an online approximation algorithm with polynomial time-complexity to find the placement in real-time whenever an instance arrives. We analytically show that the online algorithm is $O(1)$-competitive for a broad family of cost functions. Afterwards, the impact of prediction errors is considered and a method for finding the optimal look-ahead window size is proposed, which minimizes an upper bound of the average actual cost. The effectiveness of the proposed approach is evaluated by simulations with both synthetic and real-world (San Francisco taxi) user-mobility traces. The theoretical methodology used in this paper can potentially be applied to a larger class of dynamic resource allocation problems. △ Less

Submitted 16 September, 2016; v1 submitted 9 March, 2015; originally announced March 2015.

Comments: This is the author's version of the paper accepted for publication in the IEEE Transactions on Parallel and Distributed Systems

A Methodology for Studying VANET Performance with Practical Vehicle Distribution in Urban Environment

Authors: Ivan Wang-Hei Ho, Kin K. Leung, John W. Polak

Abstract: In a Vehicular Ad-hoc Network (VANET), the amount of interference from neighboring nodes to a communication link is governed by the vehicle density dynamics in vicinity and transmission probabilities of terminals. It is obvious that vehicles are distributed non-homogeneously along a road segment due to traffic controls and speed limits at different portions of the road. The common assumption of ho… ▽ More In a Vehicular Ad-hoc Network (VANET), the amount of interference from neighboring nodes to a communication link is governed by the vehicle density dynamics in vicinity and transmission probabilities of terminals. It is obvious that vehicles are distributed non-homogeneously along a road segment due to traffic controls and speed limits at different portions of the road. The common assumption of homogeneous node distribution in the network in most of the previous work in mobile ad-hoc networks thus appears to be inappropriate in VANETs. In light of the inadequacy, we present in this paper an original methodology to study the performance of VANETs with practical vehicle distribution in urban environment. Specifically, we introduce the stochastic traffic model to characterize the general vehicular traffic flow as well as the randomness of individual vehicles, from which we can acquire the mean dynamics and the probability distribution of vehicular density. As illustrative examples, we demonstrate how the density knowledge from the stochastic traffic model can be utilized to derive the throughput and progress performance of three routing strategies in different channel access protocols. We confirm the accuracy of the analytical results through extensive simulations. Our results demonstrate the applicability of the proposed methodology on modeling protocol performance, and shed insight into the performance analysis of other transmission protocols and network configurations in vehicular networks. Furthermore, we illustrate that the optimal transmission probability for optimized network performance can be obtained as a function of the location space from our results. Such information can be computed by road-side nodes and then broadcasted to road users for optimized multi-hop packet transmission in the communication network. △ Less

Submitted 27 November, 2012; originally announced November 2012.

On the Universality of Sequential Slotted Amplify and Forward Strategy in Cooperative Communications

Authors: Haishi Ning, Cong Ling, Kin K. Leung

Abstract: While cooperative communication has many benefits and is expected to play an important role in future wireless networks, many challenges are still unsolved. Previous research has developed different relaying strategies for cooperative multiple access channels (CMA), cooperative multiple relay channels (CMR) and cooperative broadcast channels (CBC). However, there lacks a unifying strategy that is… ▽ More While cooperative communication has many benefits and is expected to play an important role in future wireless networks, many challenges are still unsolved. Previous research has developed different relaying strategies for cooperative multiple access channels (CMA), cooperative multiple relay channels (CMR) and cooperative broadcast channels (CBC). However, there lacks a unifying strategy that is universally optimal for these three classical channel models. Sequential slotted amplify and forward (SSAF) strategy was previously proposed to achieve the optimal diversity and multiplexing tradeoff (DMT) for CMR. In this paper, the use of SSAF strategy is extended to CBC and CMA, and its optimality for both of them is shown. For CBC, a CBC-SSAF strategy is proposed which can asymptotically achieve the DMT upper bound when the number of cooperative users is large. For CMA, a CMA-SSAF strategy is proposed which even can exactly achieve the DMT upper bound with any number of cooperative users. In this way, SSAF strategy is shown to be universally optimal for all these three classical channel models and has great potential to provide universal optimality for wireless cooperative networks. △ Less

Submitted 8 June, 2010; originally announced June 2010.

Comments: 22 pages, 6 figures

Wireless Network Coding with Imperfect Overhearing

Authors: Haishi Ning, Cong Ling, Kin K. Leung

Abstract: Not only is network coding essential to achieve the capacity of a single-session multicast network, it can also help to improve the throughput of wireless networks with multiple unicast sessions when overheard information is available. Most previous research aimed at realizing such improvement by using perfectly overheard information, while in practice, especially for wireless networks, overheard… ▽ More Not only is network coding essential to achieve the capacity of a single-session multicast network, it can also help to improve the throughput of wireless networks with multiple unicast sessions when overheard information is available. Most previous research aimed at realizing such improvement by using perfectly overheard information, while in practice, especially for wireless networks, overheard information is often imperfect. To date, it is unclear whether network coding should still be used in such situations with imperfect overhearing. In this paper, a simple but ubiquitous wireless network model with two unicast sessions is used to investigate this problem. From the diversity and multiplexing tradeoff perspective, it is proved that even when overheard information is imperfect, network coding can still help to improve the overall system performance. This result implies that network coding should be used actively regardless of the reception quality of overheard information. △ Less

Submitted 22 March, 2010; originally announced March 2010.

Comments: 21 pages, 6 figures, IEEE Trans. Commun.