Showing 1–50 of 56 results for author: Zomaya, A

Analysis of DNS Dependencies and their Security Implications in Australia: A Comparative Study of General and Indigenous Populations

Authors: Niousha Nazemi, Omid Tavallaie, Anna Maria Mandalari, Hamed Haddadi, Ralph Holz, Albert Y. Zomaya

Abstract: This paper investigates the impact of internet centralization on DNS provisioning, particularly its effects on vulnerable populations such as the indigenous people of Australia. We analyze the DNS dependencies of Australian government domains that serve indigenous communities compared to those serving the general population. Our study categorizes DNS providers into leading (hyperscaler, US-headqua… ▽ More This paper investigates the impact of internet centralization on DNS provisioning, particularly its effects on vulnerable populations such as the indigenous people of Australia. We analyze the DNS dependencies of Australian government domains that serve indigenous communities compared to those serving the general population. Our study categorizes DNS providers into leading (hyperscaler, US-headquartered companies), non-leading (smaller Australian-headquartered or non-Australian companies), and Australian government-hosted providers. Then, we build dependency graphs to demonstrate the direct dependency between Australian government domains and their DNS providers and the indirect dependency involving further layers of providers. Additionally, we conduct an IP location analysis of DNS providers to map out the geographical distribution of DNS servers, revealing the extent of centralization on DNS services within or outside of Australia. Finally, we introduce an attacker model to categorize potential cyber attackers based on their intentions and resources. By considering attacker models and DNS dependency results, we discuss the security vulnerability of each population group against any group of attackers and analyze whether the current setup of the DNS services of Australian government services contributes to a digital divide. △ Less

Submitted 23 August, 2024; originally announced August 2024.

Comments: 12 pages, 13 figures, 5 tables Submitted to IEEE Transactions on Network and Service Management

RBLA: Rank-Based-LoRA-Aggregation for Fine-tuning Heterogeneous Models in FLaaS

Authors: Shuaijun Chen, Omid Tavallaie, Niousha Nazemi, Albert Y. Zomaya

Abstract: Federated Learning (FL) is a promising privacy-aware distributed learning framework that can be deployed on various devices, such as mobile phones, desktops, and devices equipped with CPUs or GPUs. In the context of server-based Federated Learning as a Service (FLaas), FL enables the central server to coordinate the training process across multiple devices without direct access to the local data,… ▽ More Federated Learning (FL) is a promising privacy-aware distributed learning framework that can be deployed on various devices, such as mobile phones, desktops, and devices equipped with CPUs or GPUs. In the context of server-based Federated Learning as a Service (FLaas), FL enables the central server to coordinate the training process across multiple devices without direct access to the local data, thereby enhancing privacy and data security. Low-Rank Adaptation (LoRA) is a method that fine-tunes models efficiently by focusing on a low-dimensional subspace of the model's parameters. This approach significantly reduces computational and memory costs compared to fine-tuning all parameters from scratch. When integrated with FL, especially in a FLaas environment, LoRA allows for flexible and efficient deployment across diverse hardware with varying computational capabilities by adjusting the local model's rank. However, in LoRA-enabled FL, different clients may train models with varying ranks, which poses a challenge for model aggregation on the server. Current methods of aggregating models of different ranks require padding weights to a uniform shape, which can degrade the global model's performance. To address this issue, we propose Rank-Based LoRA Aggregation (RBLA), a novel model aggregation method designed for heterogeneous LoRA structures. RBLA preserves key features across models with different ranks. This paper analyzes the issues with current padding methods that reshape models for aggregation in a FLaas environment. Then, we introduce RBLA, a rank-based aggregation method that maintains both low-rank and high-rank features. Finally, we demonstrate the effectiveness of RBLA through comparative experiments with state-of-the-art methods. △ Less

Submitted 16 August, 2024; originally announced August 2024.

Federated Learning as a Service for Hierarchical Edge Networks with Heterogeneous Models

Authors: Wentao Gao, Omid Tavallaie, Shuaijun Chen, Albert Zomaya

Abstract: Federated learning (FL) is a distributed Machine Learning (ML) framework that is capable of training a new global model by aggregating clients' locally trained models without sharing users' original data. Federated learning as a service (FLaaS) offers a privacy-preserving approach for training machine learning models on devices with various computational resources. Most proposed FL-based methods t… ▽ More Federated learning (FL) is a distributed Machine Learning (ML) framework that is capable of training a new global model by aggregating clients' locally trained models without sharing users' original data. Federated learning as a service (FLaaS) offers a privacy-preserving approach for training machine learning models on devices with various computational resources. Most proposed FL-based methods train the same model in all client devices regardless of their computational resources. However, in practical Internet of Things (IoT) scenarios, IoT devices with limited computational resources may not be capable of training models that client devices with greater hardware performance hosted. Most of the existing FL frameworks that aim to solve the problem of aggregating heterogeneous models are designed for Independent and Identical Distributed (IID) data, which may make it hard to reach the target algorithm performance when encountering non-IID scenarios. To address these problems in hierarchical networks, in this paper, we propose a heterogeneous aggregation framework for hierarchical edge systems called HAF-Edge. In our proposed framework, we introduce a communication-efficient model aggregation method designed for FL systems with two-level model aggregations running at the edge and cloud levels. This approach enhances the convergence rate of the global model by leveraging selective knowledge transfer during the aggregation of heterogeneous models. To the best of our knowledge, this work is pioneering in addressing the problem of aggregating heterogeneous models within hierarchical FL systems spanning IoT, edge, and cloud environments. We conducted extensive experiments to validate the performance of our proposed method. The evaluation results demonstrate that HAF-Edge significantly outperforms state-of-the-art methods. △ Less

Submitted 30 July, 2024; originally announced July 2024.

GriDB: Scaling Blockchain Database via Sharding and Off-Chain Cross-Shard Mechanism

Authors: Zicong Hong, Song Guo, Enyuan Zhou, Wuhui Chen, Huawei Huang, Albert Zomaya

Abstract: Blockchain databases have attracted widespread attention but suffer from poor scalability due to underlying non-scalable blockchains. While blockchain sharding is necessary for a scalable blockchain database, it poses a new challenge named on-chain cross-shard database services. Each cross-shard database service (e.g., cross-shard queries or inter-shard load balancing) involves massive cross-shard… ▽ More Blockchain databases have attracted widespread attention but suffer from poor scalability due to underlying non-scalable blockchains. While blockchain sharding is necessary for a scalable blockchain database, it poses a new challenge named on-chain cross-shard database services. Each cross-shard database service (e.g., cross-shard queries or inter-shard load balancing) involves massive cross-shard data exchanges, while the existing cross-shard mechanisms need to process each cross-shard data exchange via the consensus of all nodes in the related shards (i.e., on-chain) to resist a Byzantine environment of blockchain, which eliminates sharding benefits. To tackle the challenge, this paper presents GriDB, the first scalable blockchain database, by designing a novel off-chain cross-shard mechanism for efficient cross-shard database services. Borrowing the idea of off-chain payments, GriDB delegates massive cross-shard data exchange to a few nodes, each of which is randomly picked from a different shard. Considering the Byzantine environment, the untrusted delegates cooperate to generate succinct proof for cross-shard data exchanges, while the consensus is only responsible for the low-cost proof verification. However, different from payments, the database services' verification has more requirements (e.g., completeness, correctness, freshness, and availability); thus, we introduce several new authenticated data structures (ADS). Particularly, we utilize consensus to extend the threat model and reduce the complexity of traditional accumulator-based ADS for verifiable cross-shard queries with a rich set of relational operators. Moreover, we study the necessity of inter-shard load balancing for a scalable blockchain database and design an off-chain and live approach for both efficiency and availability during balancing. △ Less

Submitted 4 July, 2024; originally announced July 2024.

Boosting Communication Efficiency of Federated Learning's Secure Aggregation

Authors: Niousha Nazemi, Omid Tavallaie, Shuaijun Chen, Albert Y. Zomaya, Ralph Holz

Abstract: Federated Learning (FL) is a decentralized machine learning approach where client devices train models locally and send them to a server that performs aggregation to generate a global model. FL is vulnerable to model inversion attacks, where the server can infer sensitive client data from trained models. Google's Secure Aggregation (SecAgg) protocol addresses this data privacy issue by masking eac… ▽ More Federated Learning (FL) is a decentralized machine learning approach where client devices train models locally and send them to a server that performs aggregation to generate a global model. FL is vulnerable to model inversion attacks, where the server can infer sensitive client data from trained models. Google's Secure Aggregation (SecAgg) protocol addresses this data privacy issue by masking each client's trained model using shared secrets and individual elements generated locally on the client's device. Although SecAgg effectively preserves privacy, it imposes considerable communication and computation overhead, especially as network size increases. Building upon SecAgg, this poster introduces a Communication-Efficient Secure Aggregation (CESA) protocol that substantially reduces this overhead by using only two shared secrets per client to mask the model. We propose our method for stable networks with low delay variation and limited client dropouts. CESA is independent of the data distribution and network size (for higher than 6 nodes), preventing the honest-but-curious server from accessing unmasked models. Our initial evaluation reveals that CESA significantly reduces the communication cost compared to SecAgg. △ Less

Submitted 2 May, 2024; originally announced May 2024.

Comments: 2 pages, 4 figures, The 54th Annual IEEE/IFIP International Conference on Dependable Systems and Networks

CGS-Mask: Making Time Series Predictions Intuitive for All

Authors: Feng Lu, Wei Li, Yifei Sun, Cheng Song, Yufei Ren, Albert Y. Zomaya

Abstract: Artificial intelligence (AI) has immense potential in time series prediction, but most explainable tools have limited capabilities in providing a systematic understanding of important features over time. These tools typically rely on evaluating a single time point, overlook the time ordering of inputs, and neglect the time-sensitive nature of time series applications. These factors make it difficu… ▽ More Artificial intelligence (AI) has immense potential in time series prediction, but most explainable tools have limited capabilities in providing a systematic understanding of important features over time. These tools typically rely on evaluating a single time point, overlook the time ordering of inputs, and neglect the time-sensitive nature of time series applications. These factors make it difficult for users, particularly those without domain knowledge, to comprehend AI model decisions and obtain meaningful explanations. We propose CGS-Mask, a post-hoc and model-agnostic cellular genetic strip mask-based saliency approach to address these challenges. CGS-Mask uses consecutive time steps as a cohesive entity to evaluate the impact of features on the final prediction, providing binary and sustained feature importance scores over time. Our algorithm optimizes the mask population iteratively to obtain the optimal mask in a reasonable time. We evaluated CGS-Mask on synthetic and real-world datasets, and it outperformed state-of-the-art methods in elucidating the importance of features over time. According to our pilot user study via a questionnaire survey, CGS-Mask is the most effective approach in presenting easily understandable time series prediction results, enabling users to comprehend the decision-making process of AI models with ease. △ Less

Submitted 12 April, 2024; v1 submitted 14 December, 2023; originally announced December 2023.

Comments: Accepted by AAAI24

Optimization of Federated Learning's Client Selection for Non-IID Data Based on Grey Relational Analysis

Authors: Shuaijun Chen, Omid Tavallaie, Michael Henri Hambali, Seid Miad Zandavi, Hamed Haddadi, Nicholas Lane, Song Guo, Albert Y. Zomaya

Abstract: Federated learning (FL) is a novel distributed learning framework designed for applications with privacy-sensitive data. Without sharing data, FL trains local models on individual devices and constructs the global model on the server by performing model aggregation. However, to reduce the communication cost, the participants in each training round are randomly selected, which significantly decreas… ▽ More Federated learning (FL) is a novel distributed learning framework designed for applications with privacy-sensitive data. Without sharing data, FL trains local models on individual devices and constructs the global model on the server by performing model aggregation. However, to reduce the communication cost, the participants in each training round are randomly selected, which significantly decreases the training efficiency under data and device heterogeneity. To address this issue, in this paper, we introduce a novel approach that considers the data distribution and computational resources of devices to select the clients for each training round. Our proposed method performs client selection based on the Grey Relational Analysis (GRA) theory by considering available computational resources for each client, the training loss, and weight divergence. To examine the usability of our proposed method, we implement our contribution on Amazon Web Services (AWS) by using the TensorFlow library of Python. We evaluate our algorithm's performance in different setups by varying the learning rate, network size, the number of selected clients, and the client selection round. The evaluation results show that our proposed algorithm enhances the performance significantly in terms of test accuracy and the average client's waiting time compared to state-of-the-art methods, federated averaging and Pow-d. △ Less

Submitted 23 January, 2024; v1 submitted 12 October, 2023; originally announced October 2023.

Cloud-Native Computing: A Survey from the Perspective of Services

Authors: Shuiguang Deng, Hailiang Zhao, Binbin Huang, Cheng Zhang, Feiyi Chen, Yinuo Deng, Jianwei Yin, Schahram Dustdar, Albert Y. Zomaya

Abstract: The development of cloud computing delivery models inspires the emergence of cloud-native computing. Cloud-native computing, as the most influential development principle for web applications, has already attracted increasingly more attention in both industry and academia. Despite the momentum in the cloud-native industrial community, a clear research roadmap on this topic is still missing. As a c… ▽ More The development of cloud computing delivery models inspires the emergence of cloud-native computing. Cloud-native computing, as the most influential development principle for web applications, has already attracted increasingly more attention in both industry and academia. Despite the momentum in the cloud-native industrial community, a clear research roadmap on this topic is still missing. As a contribution to this knowledge, this paper surveys key issues during the life-cycle of cloud-native applications, from the perspective of services. Specifically, we elaborate the research domains by decoupling the life-cycle of cloud-native applications into four states: building, orchestration, operate, and maintenance. We also discuss the fundamental necessities and summarize the key performance metrics that play critical roles during the development and management of cloud-native applications. We highlight the key implications and limitations of existing works in each state. The challenges, future directions, and research opportunities are also discussed. △ Less

Submitted 25 June, 2023; originally announced June 2023.

GT-TSCH: Game-Theoretic Distributed TSCH Scheduler for Low-Power IoT Networks

Authors: Omid Tavallaie, Seid Miad Zandavi, Hamed Haddadi, Albert Y. Zomaya

Abstract: Time-Slotted Channel Hopping (TSCH) is a synchronous medium access mode of the IEEE 802.15.4e standard designed for providing low-latency and highly-reliable end-to-end communication. TSCH constructs a communication schedule by combining frequency channel hopping with Time Division Multiple Access (TDMA). In recent years, IETF designed several standards to define general mechanisms for the impleme… ▽ More Time-Slotted Channel Hopping (TSCH) is a synchronous medium access mode of the IEEE 802.15.4e standard designed for providing low-latency and highly-reliable end-to-end communication. TSCH constructs a communication schedule by combining frequency channel hopping with Time Division Multiple Access (TDMA). In recent years, IETF designed several standards to define general mechanisms for the implementation of TSCH. However, the problem of updating the TSCH schedule according to the changes of the wireless link quality and node's traffic load left unresolved. In this paper, we use non-cooperative game theory to propose GT-TSCH, a distributed TSCH scheduler designed for low-power IoT applications. By considering selfish behavior of nodes in packet forwarding, GT-TSCH updates the TSCH schedule in a distributed approach with low control overhead by monitoring the queue length, the place of the node in the Directed Acyclic Graph (DAG) topology, the quality of the wireless link, and the data packet generation rate. We prove the existence and uniqueness of Nash equilibrium in our game model and we find the optimal number of TSCH Tx timeslots to update the TSCH slotframe. To examine the performance of our contribution, we implement GT-TSCH on Zolertia Firefly IoT motes and the Contiki-NG Operating System (OS). The evaluation results reveal that GT-TSCH improves performance in terms of throughput and end-to-end delay compared to the state-of-the-art method. △ Less

Submitted 22 June, 2023; originally announced June 2023.

Comments: 43rd IEEE International Conference on Distributed Computing Systems

Scheduling Multi-Server Jobs with Sublinear Regrets via Online Learning

Authors: Hailiang Zhao, Shuiguang Deng, Zhengzhe Xiang, Xueqiang Yan, Jianwei Yin, Schahram Dustdar, Albert Y. Zomaya

Abstract: Multi-server jobs that request multiple computing resources and hold onto them during their execution dominate modern computing clusters. When allocating the multi-type resources to several co-located multi-server jobs simultaneously in online settings, it is difficult to make the tradeoff between the parallel computation gain and the internal communication overhead, apart from the resource conten… ▽ More Multi-server jobs that request multiple computing resources and hold onto them during their execution dominate modern computing clusters. When allocating the multi-type resources to several co-located multi-server jobs simultaneously in online settings, it is difficult to make the tradeoff between the parallel computation gain and the internal communication overhead, apart from the resource contention between jobs. To study the computation-communication tradeoff, we model the computation gain as the speedup on the job completion time when it is executed in parallelism on multiple computing instances, and fit it with utilities of different concavities. Meanwhile, we take the dominant communication overhead as the penalty to be subtracted. To achieve a better gain-overhead tradeoff, we formulate an cumulative reward maximization program and design an online algorithm, named OGASched, to schedule multi-server jobs. OGASched allocates the multi-type resources to each arrived job in the ascending direction of the reward gradients. It has several parallel sub-procedures to accelerate its computation, which greatly reduces the complexity. We proved that it has a sublinear regret with general concave rewards. We also conduct extensive trace-driven simulations to validate the performance of OGASched. The results demonstrate that OGASched outperforms widely used heuristics by $11.33\%$, $7.75\%$, $13.89\%$, and $13.44\%$, respectively. △ Less

Submitted 5 August, 2023; v1 submitted 11 May, 2023; originally announced May 2023.

Prophet: Conflict-Free Sharding Blockchain via Byzantine-Tolerant Deterministic Ordering

Authors: Zicong Hong, Song Guo, Enyuan Zhou, Jianting Zhang, Wuhui Chen, Jinwen Liang, Jie Zhang, Albert Zomaya

Abstract: Sharding scales throughput by splitting blockchain nodes into parallel groups. However, different shards' independent and random scheduling for cross-shard transactions results in numerous conflicts and aborts, since cross-shard transactions from different shards may access the same account. A deterministic ordering can eliminate conflicts by determining a global order for transactions before proc… ▽ More Sharding scales throughput by splitting blockchain nodes into parallel groups. However, different shards' independent and random scheduling for cross-shard transactions results in numerous conflicts and aborts, since cross-shard transactions from different shards may access the same account. A deterministic ordering can eliminate conflicts by determining a global order for transactions before processing, as proved in the database field. Unfortunately, due to the intertwining of the Byzantine environment and information isolation among shards, there is no trusted party able to predetermine such an order for cross-shard transactions. To tackle this challenge, this paper proposes Prophet, a conflict-free sharding blockchain based on Byzantine-tolerant deterministic ordering. It first depends on untrusted self-organizing coalitions of nodes from different shards to pre-execute cross-shard transactions for prerequisite information about ordering. It then determines a trusted global order based on stateless ordering and post-verification for pre-executed results, through shard cooperation. Following the order, the shards thus orderly execute and commit transactions without conflicts. Prophet orchestrates the pre-execution, ordering, and execution processes in the sharding consensus for minimal overhead. We rigorously prove the determinism and serializability of transactions under the Byzantine and sharded environment. An evaluation of our prototype shows that Prophet improves the throughput by $3.11\times$ and achieves nearly no aborts on 1 million Ethereum transactions compared with state-of-the-art sharding. △ Less

Submitted 17 April, 2023; originally announced April 2023.

Hierarchical Federated Learning with Momentum Acceleration in Multi-Tier Networks

Authors: Zhengjie Yang, Sen Fu, Wei Bao, Dong Yuan, Albert Y. Zomaya

Abstract: In this paper, we propose Hierarchical Federated Learning with Momentum Acceleration (HierMo), a three-tier worker-edge-cloud federated learning algorithm that applies momentum for training acceleration. Momentum is calculated and aggregated in the three tiers. We provide convergence analysis for HierMo, showing a convergence rate of O(1/T). In the analysis, we develop a new approach to characteri… ▽ More In this paper, we propose Hierarchical Federated Learning with Momentum Acceleration (HierMo), a three-tier worker-edge-cloud federated learning algorithm that applies momentum for training acceleration. Momentum is calculated and aggregated in the three tiers. We provide convergence analysis for HierMo, showing a convergence rate of O(1/T). In the analysis, we develop a new approach to characterize model aggregation, momentum aggregation, and their interactions. Based on this result, {we prove that HierMo achieves a tighter convergence upper bound compared with HierFAVG without momentum}. We also propose HierOPT, which optimizes the aggregation periods (worker-edge and edge-cloud aggregation periods) to minimize the loss given a limited training time. △ Less

Submitted 26 October, 2022; originally announced October 2022.

Comments: 18 pages, 5 figures

Multi-level Explanation of Deep Reinforcement Learning-based Scheduling

Authors: Shaojun Zhang, Chen Wang, Albert Zomaya

Abstract: Dependency-aware job scheduling in the cluster is NP-hard. Recent work shows that Deep Reinforcement Learning (DRL) is capable of solving it. It is difficult for the administrator to understand the DRL-based policy even though it achieves remarkable performance gain. Therefore the complex model-based scheduler is not easy to gain trust in the system where simplicity is favored. In this paper, we g… ▽ More Dependency-aware job scheduling in the cluster is NP-hard. Recent work shows that Deep Reinforcement Learning (DRL) is capable of solving it. It is difficult for the administrator to understand the DRL-based policy even though it achieves remarkable performance gain. Therefore the complex model-based scheduler is not easy to gain trust in the system where simplicity is favored. In this paper, we give the multi-level explanation framework to interpret the policy of DRL-based scheduling. We dissect its decision-making process to job level and task level and approximate each level with interpretable models and rules, which align with operational practices. We show that the framework gives the system administrator insights into the state-of-the-art scheduler and reveals the robustness issue in regards to its behavior pattern. △ Less

Submitted 18 September, 2022; originally announced September 2022.

Comments: Accepted in the MLSys'22 Workshop on Cloud Intelligence / AIOps

Intelligent Blockchain-based Edge Computing via Deep Reinforcement Learning: Solutions and Challenges

Authors: Dinh C. Nguyen, Van-Dinh Nguyen, Ming Ding, Symeon Chatzinotas, Pubudu N. Pathirana, Aruna Seneviratne, Octavia Dobre, Albert Y. Zomaya

Abstract: The convergence of mobile edge computing (MEC) and blockchain is transforming the current computing services in wireless Internet-of-Things networks, by enabling task offloading with security enhancement based on blockchain mining. Yet the existing approaches for these enabling technologies are isolated, providing only tailored solutions for specific services and scenarios. To fill this gap, we pr… ▽ More The convergence of mobile edge computing (MEC) and blockchain is transforming the current computing services in wireless Internet-of-Things networks, by enabling task offloading with security enhancement based on blockchain mining. Yet the existing approaches for these enabling technologies are isolated, providing only tailored solutions for specific services and scenarios. To fill this gap, we propose a novel cooperative task offloading and blockchain mining (TOBM) scheme for a blockchain-based MEC system, where each edge device not only handles computation tasks but also deals with block mining for improving system utility. To address the latency issues caused by the blockchain operation in MEC, we develop a new Proof-of-Reputation consensus mechanism based on a lightweight block verification strategy. To accommodate the highly dynamic environment and high-dimensional system state space, we apply a novel distributed deep reinforcement learning-based approach by using a multi-agent deep deterministic policy gradient algorithm. Experimental results demonstrate the superior performance of the proposed TOBM scheme in terms of enhanced system reward, improved offloading utility with lower blockchain mining latency, and better system utility, compared to the existing cooperative and non-cooperative schemes. The paper concludes with key technical challenges and possible directions for future blockchain-based MEC research. △ Less

Submitted 17 June, 2022; originally announced June 2022.

Comments: Accepted at IEEE Network Magazine, 8 pages. arXiv admin note: substantial text overlap with arXiv:2109.14263

Multi-Component Optimization and Efficient Deployment of Neural-Networks on Resource-Constrained IoT Hardware

Authors: Bharath Sudharsan, Dineshkumar Sundaram, Pankesh Patel, John G. Breslin, Muhammad Intizar Ali, Schahram Dustdar, Albert Zomaya, Rajiv Ranjan

Abstract: The majority of IoT devices like smartwatches, smart plugs, HVAC controllers, etc., are powered by hardware with a constrained specification (low memory, clock speed and processor) which is insufficient to accommodate and execute large, high-quality models. On such resource-constrained devices, manufacturers still manage to provide attractive functionalities (to boost sales) by following the tradi… ▽ More The majority of IoT devices like smartwatches, smart plugs, HVAC controllers, etc., are powered by hardware with a constrained specification (low memory, clock speed and processor) which is insufficient to accommodate and execute large, high-quality models. On such resource-constrained devices, manufacturers still manage to provide attractive functionalities (to boost sales) by following the traditional approach of programming IoT devices/products to collect and transmit data (image, audio, sensor readings, etc.) to their cloud-based ML analytics platforms. For decades, this online approach has been facing issues such as compromised data streams, non-real-time analytics due to latency, bandwidth constraints, costly subscriptions, recent privacy issues raised by users and the GDPR guidelines, etc. In this paper, to enable ultra-fast and accurate AI-based offline analytics on resource-constrained IoT devices, we present an end-to-end multi-component model optimization sequence and open-source its implementation. Researchers and developers can use our optimization sequence to optimize high memory, computation demanding models in multiple aspects in order to produce small size, low latency, low-power consuming models that can comfortably fit and execute on resource-constrained hardware. The experimental results show that our optimization components can produce models that are; (i) 12.06 x times compressed; (ii) 0.13% to 0.27% more accurate; (iii) Orders of magnitude faster unit inference at 0.06 ms. Our optimization sequence is generic and can be applied to any state-of-the-art models trained for anomaly detection, predictive maintenance, robotics, voice recognition, and machine vision. △ Less

Submitted 20 April, 2022; originally announced April 2022.

Learning to Schedule Multi-Server Jobs with Fluctuated Processing Speeds

Authors: Hailiang Zhao, Shuiguang Deng, Feiyi Chen, Jianwei Yin, Schahram Dustdar, Albert Y. Zomaya

Abstract: Multi-server jobs are imperative in modern cloud computing systems. A noteworthy feature of multi-server jobs is that, they usually request multiple computing devices simultaneously for their execution. How to schedule multi-server jobs online with a high system efficiency is a topic of great concern. Firstly, the scheduling decisions have to satisfy the service locality constraints. Secondly, the… ▽ More Multi-server jobs are imperative in modern cloud computing systems. A noteworthy feature of multi-server jobs is that, they usually request multiple computing devices simultaneously for their execution. How to schedule multi-server jobs online with a high system efficiency is a topic of great concern. Firstly, the scheduling decisions have to satisfy the service locality constraints. Secondly, the scheduling decisions needs to be made online without the knowledge of future job arrivals. Thirdly, and most importantly, the actual service rate experienced by a job is usually in fluctuation because of the dynamic voltage and frequency scaling (DVFS) and power oversubscription techniques when multiple types of jobs co-locate. A majority of online algorithms with theoretical performance guarantees are proposed. However, most of them require the processing speeds to be knowable, thereby the job completion times can be exactly calculated. To present a theoretically guaranteed online scheduling algorithm for multi-server jobs without knowing actual processing speeds apriori, in this paper, we propose ESDP (Efficient Sampling-based Dynamic Programming), which learns the distribution of the fluctuated processing speeds over time and simultaneously seeks to maximize the cumulative overall utility. The cumulative overall utility is formulated as the sum of the utilities of successfully serving each multi-server job minus the penalty on the operating, maintaining, and energy cost. ESDP is proved to have a polynomial complexity and a logarithmic regret, which is a State-of-the-Art result. We also validate it with extensive simulations and the results show that the proposed algorithm outperforms several benchmark policies with improvements by up to 73%, 36%, and 28%, respectively. △ Less

Submitted 17 October, 2022; v1 submitted 8 April, 2022; originally announced April 2022.

Theoretically Guaranteed Online Workload Dispatching for Deadline-Aware Multi-Server Jobs

Authors: Hailiang Zhao, Shuiguang Deng, Jianwei Yin, Schahram Dustdar, Albert Y. Zomaya

Abstract: Multi-server jobs are imperative in modern computing clusters. A multi-server job has multiple task components and each of the task components is responsible for processing a specific size of workloads. Efficient online workload dispatching is crucial but challenging to co-located heterogeneous multi-server jobs. The dispatching policy should decide $(i)$ where to launch each task component instan… ▽ More Multi-server jobs are imperative in modern computing clusters. A multi-server job has multiple task components and each of the task components is responsible for processing a specific size of workloads. Efficient online workload dispatching is crucial but challenging to co-located heterogeneous multi-server jobs. The dispatching policy should decide $(i)$ where to launch each task component instance of the arrived jobs and $(ii)$ the size of workloads that each task component processes. Existing policies are explicit and effective when facing service locality and resource contention in both offline and online settings. However, when adding the deadline-aware constraint, the theoretical superiority of these policies could not be guaranteed. To fill the theoretical gap, in this paper, we design an $α$-competitive online workload dispatching policy for deadline-aware multi-server jobs based on the spatio-temporal resource mesh model. We formulate the problem as a social welfare maximization program and solve it online with several well designed pseudo functions. The social welfare is formulated as the sum of the utilities of jobs and the utility of the computing cluster. The proposed policy is rigorously proved to be $α$-competitive for some $α\geq 2$. We also validate the theoretical superiority of it with simulations and the results show that it distinctly outperforms two handcrafted baseline policies on the social welfare. △ Less

Submitted 12 June, 2022; v1 submitted 4 December, 2021; originally announced December 2021.

Federated Learning for COVID-19 Detection with Generative Adversarial Networks in Edge Cloud Computing

Authors: Dinh C. Nguyen, Ming Ding, Pubudu N. Pathirana, Aruna Seneviratne, Albert Y. Zomaya

Abstract: COVID-19 has spread rapidly across the globe and become a deadly pandemic. Recently, many artificial intelligence-based approaches have been used for COVID-19 detection, but they often require public data sharing with cloud datacentres and thus remain privacy concerns. This paper proposes a new federated learning scheme, called FedGAN, to generate realistic COVID-19 images for facilitating privacy… ▽ More COVID-19 has spread rapidly across the globe and become a deadly pandemic. Recently, many artificial intelligence-based approaches have been used for COVID-19 detection, but they often require public data sharing with cloud datacentres and thus remain privacy concerns. This paper proposes a new federated learning scheme, called FedGAN, to generate realistic COVID-19 images for facilitating privacy-enhanced COVID-19 detection with generative adversarial networks (GANs) in edge cloud computing. Particularly, we first propose a GAN where a discriminator and a generator based on convolutional neural networks (CNNs) at each edge-based medical institution alternatively are trained to mimic the real COVID-19 data distribution. Then, we propose a new federated learning solution which allows local GANs to collaborate and exchange learned parameters with a cloud server, aiming to enrich the global GAN model for generating realistic COVID-19 images without the need for sharing actual data. To enhance the privacy in federated COVID-19 data analytics, we integrate a differential privacy solution at each hospital institution. Moreover, we propose a new blockchain-based FedGAN framework for secure COVID-19 data analytics, by decentralizing the FL process with a new mining solution for low running latency. Simulations results demonstrate the superiority of our approach for COVID-19 detection over the state-of-the-art schemes. △ Less

Submitted 13 October, 2021; originally announced October 2021.

Comments: Accepted at IEEE Internet of Things Journal, 14 pages

User configurable 3D object regeneration for spatial privacy

Authors: Arpit Nama, Amaya Dharmasiri, Kanchana Thilakarathna, Albert Zomaya, Jaybie Agullo de Guzman

Abstract: Environmental understanding capability of $\textit{augmented}$ (AR) and $\textit{mixed reality}$ (MR) devices are continuously improving through advances in sensing, computer vision, and machine learning. Various AR/MR applications demonstrate such capabilities i.e. scanning a space using a handheld or head mounted device and capturing a digital representation of the space that are accurate copies… ▽ More Environmental understanding capability of $\textit{augmented}$ (AR) and $\textit{mixed reality}$ (MR) devices are continuously improving through advances in sensing, computer vision, and machine learning. Various AR/MR applications demonstrate such capabilities i.e. scanning a space using a handheld or head mounted device and capturing a digital representation of the space that are accurate copies of the real space. However, these capabilities impose privacy risks to users: personally identifiable information can leak from captured 3D maps of the sensitive spaces and/or captured sensitive objects within the mapped space. Thus, in this work, we demonstrate how we can leverage 3D object regeneration for preserving privacy of 3D point clouds. That is, we employ an intermediary layer of protection to transform the 3D point cloud before providing it to the third-party applications. Specifically, we use an existing adversarial autoencoder to generate copies of 3D objects where the likeness of the copies from the original can be varied. To test the viability and performance of this method as a privacy preserving mechanism, we use a 3D classifier to classify and identify these transformed point clouds i.e. perform $\textit{super}$-class and $\textit{intra}$-class classification. To measure the performance of the proposed privacy framework, we define privacy, $Π\in[0,1]$, and utility metrics, $Q\in[0,1]$, which are desired to be maximized. Experimental evaluation shows that the privacy framework can indeed variably effect the privacy of a 3D object by varying the privilege level $l\in[0,1]$ i.e. if a low $l<0.17$ is maintained, $Π_1,Π_2>0.4$ is ensured where $Π_1,Π_2$ are super- and intra-class privacy. Lastly, the privacy framework can ensure relatively high intra-class privacy and utility i.e. $Π_2>0.63$ and $Q>0.70$, if the privilege level is kept within the range of $0.17<l<0.25$. △ Less

Submitted 10 August, 2021; originally announced August 2021.

ACM Class: K.6.5

InaudibleKey: Generic Inaudible Acoustic Signal based Key Agreement Protocol for Mobile Devices

Authors: Weitao Xu, Zhenjiang Li, Wanli Xue, Xiaotong Yu, Bo Wei, Jia Wang, Chengwen Luo, Wei Li, Albert Y. Zomaya

Abstract: Secure Device-to-Device (D2D) communication is becoming increasingly important with the ever-growing number of Internet-of-Things (IoT) devices in our daily life. To achieve secure D2D communication, the key agreement between different IoT devices without any prior knowledge is becoming desirable. Although various approaches have been proposed in the literature, they suffer from a number of limita… ▽ More Secure Device-to-Device (D2D) communication is becoming increasingly important with the ever-growing number of Internet-of-Things (IoT) devices in our daily life. To achieve secure D2D communication, the key agreement between different IoT devices without any prior knowledge is becoming desirable. Although various approaches have been proposed in the literature, they suffer from a number of limitations, such as low key generation rate and short pairing distance. In this paper, we present InaudibleKey, an inaudible acoustic signal-based key generation protocol for mobile devices. Based on acoustic channel reciprocity, InaudibleKey exploits the acoustic channel frequency response of two legitimate devices as a common secret to generating keys. InaudibleKey employs several novel technologies to significantly improve its performance. We conduct extensive experiments to evaluate the proposed system in different real environments. Compared to state-of-the-art works, InaudibleKey improves key generation rate by 3-145 times, extends pairing distance by 3.2-44 times, and reduces information reconciliation counts by 2.5-16 times. Security analysis demonstrates that InaudibleKey is resilient to a number of malicious attacks. We also implement InaudibleKey on modern smartphones and resource-limited IoT devices. Results show that it is energy-efficient and can run on both powerful and resource-limited IoT devices without incurring excessive resource consumption. △ Less

Submitted 26 March, 2021; v1 submitted 22 February, 2021; originally announced February 2021.

Comments: 13 pages, 11 figures, IPSN 2021

ACM Class: C.2.0

Adaptive Processor Frequency Adjustment for Mobile Edge Computing with Intermittent Energy Supply

Authors: Tiansheng Huang, Weiwei Lin, Xiaobin Hong, Xiumin Wang, Qingbo Wu, Rui Li, Ching-Hsien Hsu, Albert Y. Zomaya

Abstract: With astonishing speed, bandwidth, and scale, Mobile Edge Computing (MEC) has played an increasingly important role in the next generation of connectivity and service delivery. Yet, along with the massive deployment of MEC servers, the ensuing energy issue is now on an increasingly urgent agenda. In the current context, the large scale deployment of renewable-energy-supplied MEC servers is perhaps… ▽ More With astonishing speed, bandwidth, and scale, Mobile Edge Computing (MEC) has played an increasingly important role in the next generation of connectivity and service delivery. Yet, along with the massive deployment of MEC servers, the ensuing energy issue is now on an increasingly urgent agenda. In the current context, the large scale deployment of renewable-energy-supplied MEC servers is perhaps the most promising solution for the incoming energy issue. Nonetheless, as a result of the intermittent nature of their power sources, these special design MEC server must be more cautious about their energy usage, in a bid to maintain their service sustainability as well as service standard. Targeting optimization on a single-server MEC scenario, we in this paper propose NAFA, an adaptive processor frequency adjustment solution, to enable an effective plan of the server's energy usage. By learning from the historical data revealing request arrival and energy harvest pattern, the deep reinforcement learning-based solution is capable of making intelligent schedules on the server's processor frequency, so as to strike a good balance between service sustainability and service quality. The superior performance of NAFA is substantiated by real-data-based experiments, wherein NAFA demonstrates up to 20% increase in average request acceptance ratio and up to 50% reduction in average request processing time. △ Less

Submitted 9 October, 2021; v1 submitted 10 February, 2021; originally announced February 2021.

DPoS: Decentralized, Privacy-Preserving, and Low-Complexity Online Slicing for Multi-Tenant Networks

Authors: Hailiang Zhao, Shuiguang Deng, Zijie Liu, Zhengzhe Xiang, Jianwei Yin, Schahram Dustdar, Albert Y. Zomaya

Abstract: Network slicing is the key to enable virtualized resource sharing among vertical industries in the era of 5G communication. Efficient resource allocation is of vital importance to realize network slicing in real-world business scenarios. To deal with the high algorithm complexity, privacy leakage, and unrealistic offline setting of current network slicing algorithms, in this paper we propose a ful… ▽ More Network slicing is the key to enable virtualized resource sharing among vertical industries in the era of 5G communication. Efficient resource allocation is of vital importance to realize network slicing in real-world business scenarios. To deal with the high algorithm complexity, privacy leakage, and unrealistic offline setting of current network slicing algorithms, in this paper we propose a fully decentralized and low-complexity online algorithm, DPoS, for multi-resource slicing. We first formulate the problem as a global social welfare maximization problem. Next, we design the online algorithm DPoS based on the primal-dual approach and posted price mechanism. In DPoS, each tenant is incentivized to make its own decision based on its true preferences without disclosing any private information to the mobile virtual network operator and other tenants. We provide a rigorous theoretical analysis to show that DPoS has the optimal competitive ratio when the cost function of each resource is linear. Extensive simulation experiments are conducted to evaluate the performance of DPoS. The results show that DPoS can not only achieve close-to-offline-optimal performance, but also have low algorithmic overheads. △ Less

Submitted 7 April, 2021; v1 submitted 28 December, 2020; originally announced December 2020.

DONE: Distributed Approximate Newton-type Method for Federated Edge Learning

Authors: Canh T. Dinh, Nguyen H. Tran, Tuan Dung Nguyen, Wei Bao, Amir Rezaei Balef, Bing B. Zhou, Albert Y. Zomaya

Abstract: There is growing interest in applying distributed machine learning to edge computing, forming federated edge learning. Federated edge learning faces non-i.i.d. and heterogeneous data, and the communication between edge workers, possibly through distant locations and with unstable wireless networks, is more costly than their local computational overhead. In this work, we propose DONE, a distributed… ▽ More There is growing interest in applying distributed machine learning to edge computing, forming federated edge learning. Federated edge learning faces non-i.i.d. and heterogeneous data, and the communication between edge workers, possibly through distant locations and with unstable wireless networks, is more costly than their local computational overhead. In this work, we propose DONE, a distributed approximate Newton-type algorithm with fast convergence rate for communication-efficient federated edge learning. First, with strongly convex and smooth loss functions, DONE approximates the Newton direction in a distributed manner using the classical Richardson iteration on each edge worker. Second, we prove that DONE has linear-quadratic convergence and analyze its communication complexities. Finally, the experimental results with non-i.i.d. and heterogeneous data show that DONE attains a comparable performance to the Newton's method. Notably, DONE requires fewer communication iterations compared to distributed gradient descent and outperforms DANE and FEDL, state-of-the-art approaches, in the case of non-quadratic loss functions. △ Less

Submitted 25 January, 2022; v1 submitted 10 December, 2020; originally announced December 2020.

Stochastic Client Selection for Federated Learning with Volatile Clients

Authors: Tiansheng Huang, Weiwei Lin, Li Shen, Keqin Li, Albert Y. Zomaya

Abstract: Federated Learning (FL), arising as a privacy-preserving machine learning paradigm, has received notable attention from the public. In each round of synchronous FL training, only a fraction of available clients are chosen to participate, and the selection decision might have a significant effect on the training efficiency, as well as the final model performance. In this paper, we investigate the c… ▽ More Federated Learning (FL), arising as a privacy-preserving machine learning paradigm, has received notable attention from the public. In each round of synchronous FL training, only a fraction of available clients are chosen to participate, and the selection decision might have a significant effect on the training efficiency, as well as the final model performance. In this paper, we investigate the client selection problem under a volatile context, in which the local training of heterogeneous clients is likely to fail due to various kinds of reasons and in different levels of frequency. {\color{black}Intuitively, too much training failure might potentially reduce the training efficiency, while too much selection on clients with greater stability might introduce bias, thereby resulting in degradation of the training effectiveness. To tackle this tradeoff, we in this paper formulate the client selection problem under joint consideration of effective participation and fairness.} Further, we propose E3CS, a stochastic client selection scheme to solve the problem, and we corroborate its effectiveness by conducting real data-based experiments. According to our experimental results, the proposed selection scheme is able to achieve up to 2x faster convergence to a fixed model accuracy while maintaining the same level of final model accuracy, compared with the state-of-the-art selection schemes. △ Less

Submitted 12 February, 2022; v1 submitted 17 November, 2020; originally announced November 2020.

Comments: Under review by IEEE Internet of Things Journal

Optimal governance and implementation of vaccination programmes to contain the COVID-19 pandemic

Authors: Mahendra Piraveenan, Shailendra Sawleshwarkar, Michael Walsh, Iryna Zablotska, Samit Bhattacharyya, Habib Hassan Farooqui, Tarun Bhatnagar, Anup Karan, Manoj Murhekar, Sanjay Zodpey, K. S. Mallikarjuna Rao, Philippa Pattison, Albert Zomaya, Matjaz Perc

Abstract: Since the recent introduction of several viable vaccines for SARS-CoV-2, vaccination uptake has become the key factor that will determine our success in containing the COVID-19 pandemic. We argue that game theory and social network models should be used to guide decisions pertaining to vaccination programmes for the best possible results. In the months following the introduction of vaccines, their… ▽ More Since the recent introduction of several viable vaccines for SARS-CoV-2, vaccination uptake has become the key factor that will determine our success in containing the COVID-19 pandemic. We argue that game theory and social network models should be used to guide decisions pertaining to vaccination programmes for the best possible results. In the months following the introduction of vaccines, their availability and the human resources needed to run the vaccination programmes have been scarce in many countries. Vaccine hesitancy is also being encountered from some sections of the general public. We emphasize that decision-making under uncertainty and imperfect information, and with only conditionally optimal outcomes, is a unique forte of established game-theoretic modelling. Therefore, we can use this approach to obtain the best framework for modelling and simulating vaccination prioritization and uptake that will be readily available to inform important policy decisions for the optimal control of the COVID-19 pandemic. △ Less

Submitted 9 June, 2021; v1 submitted 12 November, 2020; originally announced November 2020.

Comments: 15 pages, 1 figure; published in Royal Society Open Science

Journal ref: R. Soc. Open Sci. 8, 210429 (2021)

An Efficiency-boosting Client Selection Scheme for Federated Learning with Fairness Guarantee

Authors: Tiansheng Huang, Weiwei Lin, Wentai Wu, Ligang He, Keqin Li, Albert Y. Zomaya

Abstract: The issue of potential privacy leakage during centralized AI's model training has drawn intensive concern from the public. A Parallel and Distributed Computing (or PDC) scheme, termed Federated Learning (FL), has emerged as a new paradigm to cope with the privacy issue by allowing clients to perform model training locally, without the necessity to upload their personal sensitive data. In FL, the n… ▽ More The issue of potential privacy leakage during centralized AI's model training has drawn intensive concern from the public. A Parallel and Distributed Computing (or PDC) scheme, termed Federated Learning (FL), has emerged as a new paradigm to cope with the privacy issue by allowing clients to perform model training locally, without the necessity to upload their personal sensitive data. In FL, the number of clients could be sufficiently large, but the bandwidth available for model distribution and re-upload is quite limited, making it sensible to only involve part of the volunteers to participate in the training process. The client selection policy is critical to an FL process in terms of training efficiency, the final model's quality as well as fairness. In this paper, we will model the fairness guaranteed client selection as a Lyapunov optimization problem and then a C2MAB-based method is proposed for estimation of the model exchange time between each client and the server, based on which we design a fairness guaranteed algorithm termed RBCS-F for problem-solving. The regret of RBCS-F is strictly bounded by a finite constant, justifying its theoretical feasibility. Barring the theoretical results, more empirical data can be derived from our real training experiments on public datasets. △ Less

Submitted 4 September, 2023; v1 submitted 3 November, 2020; originally announced November 2020.

Comments: Accepted by IEEE TPDS. DOI: 10.1109/TPDS.2020.3040887

Semi-Disentangled Representation Learning in Recommendation System

Authors: Weiguang Chen, Wenjun Jiang, Xueqi Li, Kenli Li, Albert Zomaya, Guojun Wang

Abstract: Disentangled representation has been widely explored in many fields due to its maximal compactness, interpretability and versatility. Recommendation system also needs disentanglement to make representation more explainable and general for downstream tasks. However, some challenges slow its broader application -- the lack of fine-grained labels and the complexity of user-item interactions. To allev… ▽ More Disentangled representation has been widely explored in many fields due to its maximal compactness, interpretability and versatility. Recommendation system also needs disentanglement to make representation more explainable and general for downstream tasks. However, some challenges slow its broader application -- the lack of fine-grained labels and the complexity of user-item interactions. To alleviate these problems, we propose a Semi-Disentangled Representation Learning method (SDRL) based on autoencoders. SDRL divides each user/item embedding into two parts: the explainable and the unexplainable, so as to improve proper disentanglement while preserving complex information in representation. The explainable part consists of $internal\ block$ for individual-based features and $external\ block$ for interaction-based features. The unexplainable part is composed by $other\ block$ for other remaining information. Experimental results on three real-world datasets demonstrate that the proposed SDRL could not only effectively express user and item features but also improve the explainability and generality compared with existing representation methods. △ Less

Submitted 25 October, 2020; originally announced October 2020.

Federated Learning with Nesterov Accelerated Gradient

Authors: Zhengjie Yang, Wei Bao, Dong Yuan, Nguyen H. Tran, Albert Y. Zomaya

Abstract: Federated learning (FL) is a fast-developing technique that allows multiple workers to train a global model based on a distributed dataset. Conventional FL (FedAvg) employs gradient descent algorithm, which may not be efficient enough. Momentum is able to improve the situation by adding an additional momentum step to accelerate the convergence and has demonstrated its benefits in both centralized… ▽ More Federated learning (FL) is a fast-developing technique that allows multiple workers to train a global model based on a distributed dataset. Conventional FL (FedAvg) employs gradient descent algorithm, which may not be efficient enough. Momentum is able to improve the situation by adding an additional momentum step to accelerate the convergence and has demonstrated its benefits in both centralized and FL environments. It is well-known that Nesterov Accelerated Gradient (NAG) is a more advantageous form of momentum, but it is not clear how to quantify the benefits of NAG in FL so far. This motives us to propose FedNAG, which employs NAG in each worker as well as NAG momentum and model aggregation in the aggregator. We provide a detailed convergence analysis of FedNAG and compare it with FedAvg. Extensive experiments based on real-world datasets and trace-driven simulation are conducted, demonstrating that FedNAG increases the learning accuracy by 3-24% and decreases the total training time by 11-70% compared with the benchmarks under a wide range of settings. △ Less

Submitted 25 October, 2022; v1 submitted 18 September, 2020; originally announced September 2020.

Comments: publised in TPDS. 18 pages, 6 figures

Fast Adaptive Task Offloading in Edge Computing based on Meta Reinforcement Learning

Authors: Jin Wang, Jia Hu, Geyong Min, Albert Y. Zomaya, Nektarios Georgalas

Abstract: Multi-access edge computing (MEC) aims to extend cloud service to the network edge to reduce network traffic and service latency. A fundamental problem in MEC is how to efficiently offload heterogeneous tasks of mobile applications from user equipment (UE) to MEC hosts. Recently, many deep reinforcement learning (DRL) based methods have been proposed to learn offloading policies through interactin… ▽ More Multi-access edge computing (MEC) aims to extend cloud service to the network edge to reduce network traffic and service latency. A fundamental problem in MEC is how to efficiently offload heterogeneous tasks of mobile applications from user equipment (UE) to MEC hosts. Recently, many deep reinforcement learning (DRL) based methods have been proposed to learn offloading policies through interacting with the MEC environment that consists of UE, wireless channels, and MEC hosts. However, these methods have weak adaptability to new environments because they have low sample efficiency and need full retraining to learn updated policies for new environments. To overcome this weakness, we propose a task offloading method based on meta reinforcement learning, which can adapt fast to new environments with a small number of gradient updates and samples. We model mobile applications as Directed Acyclic Graphs (DAGs) and the offloading policy by a custom sequence-to-sequence (seq2seq) neural network. To efficiently train the seq2seq network, we propose a method that synergizes the first order approximation and clipped surrogate objective. The experimental results demonstrate that this new offloading method can reduce the latency by up to 25% compared to three baselines while being able to adapt fast to new environments. △ Less

Submitted 24 October, 2020; v1 submitted 5 August, 2020; originally announced August 2020.

Comments: Accepted by IEEE Transaction on Parallel and Distributed Systems

MusicID: A Brainwave-based User Authentication System for Internet of Things

Authors: Jinani Sooriyaarachchi, Suranga Seneviratne, Kanchana Thilakarathna, Albert Y. Zomaya

Abstract: We propose MusicID, an authentication solution for smart devices that uses music-induced brainwave patterns as a behavioral biometric modality. We experimentally evaluate MusicID using data collected from real users whilst they are listening to two forms of music; a popular English song and individual's favorite song. We show that an accuracy over 98% for user identification and an accuracy over 9… ▽ More We propose MusicID, an authentication solution for smart devices that uses music-induced brainwave patterns as a behavioral biometric modality. We experimentally evaluate MusicID using data collected from real users whilst they are listening to two forms of music; a popular English song and individual's favorite song. We show that an accuracy over 98% for user identification and an accuracy over 97% for user verification can be achieved by using data collected from a 4-electrode commodity brainwave headset. We further show that a single electrode is able to provide an accuracy of approximately 85% and the use of two electrodes provides an accuracy of approximately 95%. As already shown by commodity brain-sensing headsets for meditation applications, we believe including dry EEG electrodes in smart-headsets is feasible and MusicID has the potential of providing an entry point and continuous authentication framework for upcoming surge of smart-devices mainly driven by Augmented Reality (AR)/Virtual Reality (VR) applications. △ Less

Submitted 2 June, 2020; originally announced June 2020.

Federated Learning over Wireless Networks: Convergence Analysis and Resource Allocation

Authors: Canh T. Dinh, Nguyen H. Tran, Minh N. H. Nguyen, Choong Seon Hong, Wei Bao, Albert Y. Zomaya, Vincent Gramoli

Abstract: There is an increasing interest in a fast-growing machine learning technique called Federated Learning, in which the model training is distributed over mobile user equipments (UEs), exploiting UEs' local computation and training data. Despite its advantages in data privacy-preserving, Federated Learning (FL) still has challenges in heterogeneity across UEs' data and physical resources. We first pr… ▽ More There is an increasing interest in a fast-growing machine learning technique called Federated Learning, in which the model training is distributed over mobile user equipments (UEs), exploiting UEs' local computation and training data. Despite its advantages in data privacy-preserving, Federated Learning (FL) still has challenges in heterogeneity across UEs' data and physical resources. We first propose a FL algorithm which can handle the heterogeneous UEs' data challenge without further assumptions except strongly convex and smooth loss functions. We provide the convergence rate characterizing the trade-off between local computation rounds of UE to update its local model and global communication rounds to update the FL global model. We then employ the proposed FL algorithm in wireless networks as a resource allocation optimization problem that captures the trade-off between the FL convergence wall clock time and energy consumption of UEs with heterogeneous computing and power resources. Even though the wireless resource allocation problem of FL is non-convex, we exploit this problem's structure to decompose it into three sub-problems and analyze their closed-form solutions as well as insights to problem design. Finally, we illustrate the theoretical analysis for the new algorithm with Tensorflow experiments and extensive numerical results for the wireless resource allocation sub-problems. The experiment results not only verify the theoretical convergence but also show that our proposed algorithm outperforms the vanilla FedAvg algorithm in terms of convergence rate and testing accuracy. △ Less

Submitted 28 October, 2020; v1 submitted 28 October, 2019; originally announced October 2019.

Orchestrating the Development Lifecycle of Machine Learning-Based IoT Applications: A Taxonomy and Survey

Authors: Bin Qian, Jie Su, Zhenyu Wen, Devki Nandan Jha, Yinhao Li, Yu Guan, Deepak Puthal, Philip James, Renyu Yang, Albert Y. Zomaya, Omer Rana, Lizhe Wang, Maciej Koutny, Rajiv Ranjan

Abstract: Machine Learning (ML) and Internet of Things (IoT) are complementary advances: ML techniques unlock complete potentials of IoT with intelligence, and IoT applications increasingly feed data collected by sensors into ML models, thereby employing results to improve their business processes and services. Hence, orchestrating ML pipelines that encompasses model training and implication involved in hol… ▽ More Machine Learning (ML) and Internet of Things (IoT) are complementary advances: ML techniques unlock complete potentials of IoT with intelligence, and IoT applications increasingly feed data collected by sensors into ML models, thereby employing results to improve their business processes and services. Hence, orchestrating ML pipelines that encompasses model training and implication involved in holistic development lifecycle of an IoT application often leads to complex system integration. This paper provides a comprehensive and systematic survey on the development lifecycle of ML-based IoT application. We outline core roadmap and taxonomy, and subsequently assess and compare existing standard techniques used in individual stage. △ Less

Submitted 29 May, 2020; v1 submitted 11 October, 2019; originally announced October 2019.

Comments: 50 pages, Accepted by ACM Computing Surveys (CSUR)

IoTSim-Edge: A Simulation Framework for Modeling the Behaviour of IoT and Edge Computing Environments

Authors: Devki Nandan Jha, Khaled Alwasel, Areeb Alshoshan, Xianghua Huang, Ranesh Kumar Naha, Sudheer Kumar Battula, Saurabh Garg, Deepak Puthal, Philip James, Albert Y. Zomaya, Schahram Dustdar, Rajiv Ranjan

Abstract: This paper proposes a novel simulator IoTSim-Edge, which captures the behavior of heterogeneous IoT and edge computing infrastructure and allows users to test their infrastructure and framework in an easy and configurable manner. IoTSim-Edge extends the capability of CloudSim to incorporate the different features of edge and IoT devices. The effectiveness of IoTSim-Edge is described using three te… ▽ More This paper proposes a novel simulator IoTSim-Edge, which captures the behavior of heterogeneous IoT and edge computing infrastructure and allows users to test their infrastructure and framework in an easy and configurable manner. IoTSim-Edge extends the capability of CloudSim to incorporate the different features of edge and IoT devices. The effectiveness of IoTSim-Edge is described using three test cases. The results show the varying capability of IoTSim-Edge in terms of application composition, battery-oriented modeling, heterogeneous protocols modeling and mobility modeling along with the resources provisioning for IoT applications. △ Less

Submitted 7 October, 2019; originally announced October 2019.

Edge Intelligence: The Confluence of Edge Computing and Artificial Intelligence

Authors: Shuiguang Deng, Hailiang Zhao, Weijia Fang, Jianwei Yin, Schahram Dustdar, Albert Y. Zomaya

Abstract: Along with the rapid developments in communication technologies and the surge in the use of mobile devices, a brand-new computation paradigm, Edge Computing, is surging in popularity. Meanwhile, Artificial Intelligence (AI) applications are thriving with the breakthroughs in deep learning and the many improvements in hardware architectures. Billions of data bytes, generated at the network edge, pu… ▽ More Along with the rapid developments in communication technologies and the surge in the use of mobile devices, a brand-new computation paradigm, Edge Computing, is surging in popularity. Meanwhile, Artificial Intelligence (AI) applications are thriving with the breakthroughs in deep learning and the many improvements in hardware architectures. Billions of data bytes, generated at the network edge, put massive demands on data processing and structural optimization. Thus, there exists a strong demand to integrate Edge Computing and AI, which gives birth to Edge Intelligence. In this paper, we divide Edge Intelligence into AI for edge (Intelligence-enabled Edge Computing) and AI on edge (Artificial Intelligence on Edge). The former focuses on providing more optimal solutions to key problems in Edge Computing with the help of popular and effective AI technologies while the latter studies how to carry out the entire process of building AI models, i.e., model training and inference, on the edge. This paper provides insights into this new inter-disciplinary field from a broader perspective. It discusses the core concepts and the research road-map, which should provide the necessary background for potential future research initiatives in Edge Intelligence. △ Less

Submitted 10 February, 2020; v1 submitted 2 September, 2019; originally announced September 2019.

Comments: 13 pages, 3 figures

CreativeBioMan: Brain and Body Wearable Computing based Creative Gaming System

Authors: Min Chen, Yingying Jiang, Yong Cao, Albert Y. Zomaya

Abstract: Current artificial intelligence (AI) technology is mainly used in rational work such as computation and logical analysis. How to make the machine as aesthetic and creative as humans has gradually gained attention. This paper presents a creative game system (i.e., CreativeBioMan) for the first time. It combines brain wave data and multimodal emotion data, and then uses an AI algorithm for intellige… ▽ More Current artificial intelligence (AI) technology is mainly used in rational work such as computation and logical analysis. How to make the machine as aesthetic and creative as humans has gradually gained attention. This paper presents a creative game system (i.e., CreativeBioMan) for the first time. It combines brain wave data and multimodal emotion data, and then uses an AI algorithm for intelligent decision fusion, which can be used in artistic creation, aiming at separating the artist from repeated labor creation. To imitate the process of humans' artistic creation, the creation process of the algorithm is related to artists' previous artworks and their emotion. EEG data is used to analyze the style of artists and then match them with a style from a data set of historical works. Then, universal AI algorithms are combined with the unique creativity of each artist that evolve into a personalized creation algorithm. According to the results of cloud emotion recognition, the color of the artworks is corrected so that the artist's emotions are fully reflected in the works, and thus novel works of art are created. This allows the machine to integrate the understanding of past art and emotions with the ability to create new art forms, in the same manner as humans. This paper introduces the system architecture of CreativeBioMan from two aspects: data collection of the brain and body wearable devices, as well as the intelligent decision-making fusion of models. A Testbed platform is built for an experiment and the creativity of the works produced by the system is analyzed. △ Less

Submitted 4 June, 2019; originally announced June 2019.

Proofware: Proof of Useful Work Blockchain Consensus Protocol for Decentralized Applications

Authors: Zhongli Dong, Young Choon Lee, Albert Y. Zomaya

Abstract: In a blockchain system, consensus protocol as an incentive and security mechanism, is to ensure the participants to build the block honestly and effectively. There are different consensus protocols for blockchain, like Proof of work (PoW), Proof of Stake (PoS), Proof of Space (PoSpace), Proof of Activities etc. But most of these consensus protocols are not designed for doing some useful jobs for s… ▽ More In a blockchain system, consensus protocol as an incentive and security mechanism, is to ensure the participants to build the block honestly and effectively. There are different consensus protocols for blockchain, like Proof of work (PoW), Proof of Stake (PoS), Proof of Space (PoSpace), Proof of Activities etc. But most of these consensus protocols are not designed for doing some useful jobs for society because of too much competition and scalability limitation. Massive electric power and computing resources, including CPU, RAM, storage and sensors have been wasted to run blockchain network based on these consensus protocols. Current frameworks and middleware for building decentralised applications (dApps) are largely limited to simple and less useful jobs. In this paper, we present Proofware which is designed for developers to build their dApps easily with existing public/crowd-based computing resources. Under Proofware, developers can develop and test their own Proof of Useful Work (PoUW) consensus protocols. Also, rather than depending on a centralised accounting system, each dApp has an embedded currency system to keep the whole incentive system decentralised, fair, transparent, stable and sustainable. Based on Proofware, we have built a crowd based video sharing application, called OurTube, as a case study. By the OurTube example, it has shown Proofware significantly improves the productivity to build crowd-based computing system with the features of cost-effectiveness, anti-censorship, elasticity and financial sustainability. △ Less

Submitted 21 March, 2019; originally announced March 2019.

Comments: 10 pages

Diversity, Productivity, and Growth of Open Source Developer Communities

Authors: Qingye Jiang, Young Choon Lee, Joseph G. Davis, Ablert Y. Zomaya

Abstract: The open source development model has become a paradigm shift from traditional in-house/closed-source software development model, with many successes. Traditionally, open source projects were characterized essentially by their individual volunteer developers. Such tradition has changed significantly with the participation of many organizations in particular. However, there exists a knowledge gap c… ▽ More The open source development model has become a paradigm shift from traditional in-house/closed-source software development model, with many successes. Traditionally, open source projects were characterized essentially by their individual volunteer developers. Such tradition has changed significantly with the participation of many organizations in particular. However, there exists a knowledge gap concerning how open source developer communities evolve. In this paper, we present some observations on open source developer communities. In particular, we analyze git repositories of 20 well-known open source projects, with over 3 million commit activities in total. The analysis has been carried out in three respects, productivity, diversity and growth using the Spearman's rank correlation coefficient, diversity index and the Gompertz/logistic curves, respectively. We find out that (a) the Spearman's rank correlation coefficient between active contributors and commit activities reveals how changes in the size of the developer community impacts the productivity of the community; (b) the diversity index of an open source developer community reveals the structure of the community; and (c) the growth of open source developer communities can be described using different phases of growth curves as in many organic matters. △ Less

Submitted 11 September, 2018; originally announced September 2018.

Comments: 10 pages, 6 figures

NHAD: Neuro-Fuzzy Based Horizontal Anomaly Detection In Online Social Networks

Authors: Vishal Sharma, Ravinder Kumar, Wen-Huang Cheng, Mohammed Atiquzzaman, Kathiravan Srinivasan, Albert Y. Zomaya

Abstract: Use of social network is the basic functionality of today's life. With the advent of more and more online social media, the information available and its utilization have come under the threat of several anomalies. Anomalies are the major cause of online frauds which allow information access by unauthorized users as well as information forging. One of the anomalies that act as a silent attacker is… ▽ More Use of social network is the basic functionality of today's life. With the advent of more and more online social media, the information available and its utilization have come under the threat of several anomalies. Anomalies are the major cause of online frauds which allow information access by unauthorized users as well as information forging. One of the anomalies that act as a silent attacker is the horizontal anomaly. These are the anomalies caused by a user because of his/her variable behaviour towards different sources. Horizontal anomalies are difficult to detect and hazardous for any network. In this paper, a self-healing neuro-fuzzy approach (NHAD) is used for the detection, recovery, and removal of horizontal anomalies efficiently and accurately. The proposed approach operates over the five paradigms, namely, missing links, reputation gain, significant difference, trust properties, and trust score. The proposed approach is evaluated with three datasets: DARPA'98 benchmark dataset, synthetic dataset, and real-time traffic. Results show that the accuracy of the proposed NHAD model for 10% to 30% anomalies in synthetic dataset ranges between 98.08% and 99.88%. The evaluation over DARPA'98 dataset demonstrates that the proposed approach is better than the existing solutions as it provides 99.97% detection rate for anomalous class. For real-time traffic, the proposed NHAD model operates with an average accuracy of 99.42% at 99.90% detection rate. △ Less

Submitted 18 April, 2018; v1 submitted 18 April, 2018; originally announced April 2018.

Comments: 14 Pages, 16 Figures, 5 Tables, Accepted in IEEE Transactions on Knowledge and Data Engineering (2018)

Journal ref: IEEE Transactions on Knowledge and Data Engineering. doi: 10.1109/TKDE.2018.2818163

A Manifesto for Future Generation Cloud Computing: Research Directions for the Next Decade

Authors: Rajkumar Buyya, Satish Narayana Srirama, Giuliano Casale, Rodrigo Calheiros, Yogesh Simmhan, Blesson Varghese, Erol Gelenbe, Bahman Javadi, Luis Miguel Vaquero, Marco A. S. Netto, Adel Nadjaran Toosi, Maria Alejandra Rodriguez, Ignacio M. Llorente, Sabrina De Capitani di Vimercati, Pierangela Samarati, Dejan Milojicic, Carlos Varela, Rami Bahsoon, Marcos Dias de Assuncao, Omer Rana, Wanlei Zhou, Hai Jin, Wolfgang Gentzsch, Albert Y. Zomaya, Haiying Shen

Abstract: The Cloud computing paradigm has revolutionised the computer science horizon during the past decade and has enabled the emergence of computing as the fifth utility. It has captured significant attention of academia, industries, and government bodies. Now, it has emerged as the backbone of modern economy by offering subscription-based services anytime, anywhere following a pay-as-you-go model. This… ▽ More The Cloud computing paradigm has revolutionised the computer science horizon during the past decade and has enabled the emergence of computing as the fifth utility. It has captured significant attention of academia, industries, and government bodies. Now, it has emerged as the backbone of modern economy by offering subscription-based services anytime, anywhere following a pay-as-you-go model. This has instigated (1) shorter establishment times for start-ups, (2) creation of scalable global enterprise applications, (3) better cost-to-value associativity for scientific and high performance computing applications, and (4) different invocation/execution models for pervasive and ubiquitous applications. The recent technological developments and paradigms such as serverless computing, software-defined networking, Internet of Things, and processing at network edge are creating new opportunities for Cloud computing. However, they are also posing several new challenges and creating the need for new approaches and research strategies, as well as the re-evaluation of the models that were developed to address issues such as scalability, elasticity, reliability, security, sustainability, and application models. The proposed manifesto addresses them by identifying the major open challenges in Cloud computing, emerging trends, and impact areas. It then offers research directions for the next decade, thus helping in the realisation of Future Generation Cloud Computing. △ Less

Submitted 24 August, 2018; v1 submitted 24 November, 2017; originally announced November 2017.

Comments: 51 pages, 3 figures

Throughput of Infrastructure-based Cooperative Vehicular Networks

Authors: Jieqiong Chen, Guoqiang Mao, Changle Li, Ammar Zafar, Albert Y. Zomaya

Abstract: In this paper, we provide detailed analysis of the achievable throughput of infrastructure-based vehicular network with a finite traffic density under a cooperative communication strategy, which explores combined use of vehicle-to-infrastructure (V2I) communications, vehicle-to-vehicle (V2V) communications, mobility of vehicles and cooperations among vehicles and infrastructure to facilitate the d… ▽ More In this paper, we provide detailed analysis of the achievable throughput of infrastructure-based vehicular network with a finite traffic density under a cooperative communication strategy, which explores combined use of vehicle-to-infrastructure (V2I) communications, vehicle-to-vehicle (V2V) communications, mobility of vehicles and cooperations among vehicles and infrastructure to facilitate the data transmission. A closed form expression of the achievable throughput is obtained, which reveals the relationship between the achievable throughput and its major performance-impacting parameters such as distance between adjacent infrastructure points, radio ranges of infrastructure and vehicles, transmission rates of V2I and V2V communications and vehicular density. Numerical and simulation results show that the proposed cooperative communication strategy significantly increases the throughput of vehicular networks, compared with its non-cooperative counterpart, even when the traffic density is low. Our results shed insight on the optimum deployment of vehicular network infrastructure and optimum design of cooperative communication strategies in vehicular networks to maximize the throughput. △ Less

Submitted 15 January, 2017; v1 submitted 4 December, 2016; originally announced December 2016.

Journal ref: IEEE Transactions on Intelligent Transportation Systems, 2017

Uplink Performance Analysis of Dense Cellular Networks with LoS and NLoS Transmissions

Authors: Tian Ding, Ming Ding, Guoqiang Mao, Zihuai Lin, David Lopez-Perez, Albert Zomaya

Abstract: In this paper, we analyse the coverage probability and the area spectral efficiency (ASE) for the uplink (UL) of dense small cell networks (SCNs) considering a practical path loss model incorporating both line-of-sight (LoS) and non-line-of-sight (NLoS) transmissions. Compared with the existing work, we adopt the following novel approaches in our study: (i) we assume a practical user association s… ▽ More In this paper, we analyse the coverage probability and the area spectral efficiency (ASE) for the uplink (UL) of dense small cell networks (SCNs) considering a practical path loss model incorporating both line-of-sight (LoS) and non-line-of-sight (NLoS) transmissions. Compared with the existing work, we adopt the following novel approaches in our study: (i) we assume a practical user association strategy (UAS) based on the smallest path loss, or equivalently the strongest received signal strength; (ii) we model the positions of both base stations (BSs) and the user equipments (UEs) as two independent Homogeneous Poisson point processes (HPPPs); and (iii) the correlation of BSs' and UEs' positions is considered, thus making our analytical results more accurate. The performance impact of LoS and NLoS transmissions on the ASE for the UL of dense SCNs is shown to be significant, both quantitatively and qualitatively, compared with existing work that does not differentiate LoS and NLoS transmissions. In particular, existing work predicted that a larger UL power compensation factor would always result in a better ASE in the practical range of BS density, i.e., 10^1-10^3 BSs/km^2. However, our results show that a smaller UL power compensation factor can greatly boost the ASE in the UL of dense SCNs, i.e., 10^2-10^3 BSs/km^2, while a larger UL power compensation factor is more suitable for sparse SCNs, i.e., 10^1-10^2 BSs/km^2. △ Less

Submitted 3 January, 2017; v1 submitted 25 September, 2016; originally announced September 2016.

Privacy Knowledge Modelling for Internet of Things: A Look Back

Authors: Charith Perera, Chang Liu, Rajiv Ranjan, Lizhe Wang, Albert Y. Zomaya

Abstract: Internet of Things (IoT) and cloud computing together give us the ability to sense, collect, process, and analyse data so we can use them to better understand behaviours, habits, preferences and life patterns of users and lead them to consume resources more efficiently. In such knowledge discovery activities, privacy becomes a significant challenge due to the extremely personal nature of the knowl… ▽ More Internet of Things (IoT) and cloud computing together give us the ability to sense, collect, process, and analyse data so we can use them to better understand behaviours, habits, preferences and life patterns of users and lead them to consume resources more efficiently. In such knowledge discovery activities, privacy becomes a significant challenge due to the extremely personal nature of the knowledge that can be derived from the data and the potential risks involved. Therefore, understanding the privacy expectations and preferences of stakeholders is an important task in the IoT domain. In this paper, we review how privacy knowledge has been modelled and used in the past in different domains. Our goal is not only to analyse, compare and consolidate past research work but also to appreciate their findings and discuss their applicability towards the IoT. Finally, we discuss major research challenges and opportunities. △ Less

Submitted 27 June, 2016; originally announced June 2016.

Comments: IEEE Computer Magazine 2016

Improving Raw Image Storage Efficiency by Exploiting Similarity

Authors: Binqi Zhang, Chen Wang, Bing Bing Zhou, Albert Y. Zomaya

Abstract: To improve the temporal and spatial storage efficiency, researchers have intensively studied various techniques, including compression and deduplication. Through our evaluation, we find that methods such as photo tags or local features help to identify the content-based similar- ity between raw images. The images can then be com- pressed more efficiently to get better storage space sav- ings. Furt… ▽ More To improve the temporal and spatial storage efficiency, researchers have intensively studied various techniques, including compression and deduplication. Through our evaluation, we find that methods such as photo tags or local features help to identify the content-based similar- ity between raw images. The images can then be com- pressed more efficiently to get better storage space sav- ings. Furthermore, storing similar raw images together enables rapid data sorting, searching and retrieval if the images are stored in a distributed and large-scale envi- ronment by reducing fragmentation. In this paper, we evaluated the compressibility by designing experiments and observing the results. We found that on a statistical basis the higher similarity photos have, the better com- pression results are. This research helps provide a clue for future large-scale storage system design. △ Less

Submitted 19 April, 2016; originally announced April 2016.

Big Data Privacy in the Internet of Things Era

Authors: Charith Perera, Rajiv Ranjan, Lizhe Wang, Samee U. Khan, Albert Y. Zomaya

Abstract: Over the last few years, we have seen a plethora of Internet of Things (IoT) solutions, products and services, making their way into the industry's market-place. All such solution will capture a large amount of data pertaining to the environment, as well as their users. The objective of the IoT is to learn more and to serve better the system users. Some of these solutions may store the data locall… ▽ More Over the last few years, we have seen a plethora of Internet of Things (IoT) solutions, products and services, making their way into the industry's market-place. All such solution will capture a large amount of data pertaining to the environment, as well as their users. The objective of the IoT is to learn more and to serve better the system users. Some of these solutions may store the data locally on the devices ('things'), and others may store in the Cloud. The real value of collecting data comes through data processing and aggregation in large-scale where new knowledge can be extracted. However, such procedures can also lead to user privacy issues. This article discusses some of the main challenges of privacy in IoT, and opportunities for research and innovation. We also introduce some of the ongoing research efforts that address IoT privacy issues. △ Less

Submitted 8 June, 2015; v1 submitted 29 December, 2014; originally announced December 2014.

Comments: Accepted to be published in IEEE IT Professional Magazine: Special Issue Internet of Anything 2015

Data-Intensive Workload Consolidation on Hadoop Distributed File System

Authors: Reza Moraveji, Javid Taheri, MohammadReza HosseinyFarahabady, Nikzad Babaii Rizvandi, Albert Y. Zomaya

Abstract: Workload consolidation, sharing physical resources among multiple workloads, is a promising technique to save cost and energy in cluster computing systems. This paper highlights a few challenges of workload consolidation for Hadoop as one of the current state-of-the-art data-intensive cluster computing system. Through a systematic step-by-step procedure, we investigate challenges for efficient ser… ▽ More Workload consolidation, sharing physical resources among multiple workloads, is a promising technique to save cost and energy in cluster computing systems. This paper highlights a few challenges of workload consolidation for Hadoop as one of the current state-of-the-art data-intensive cluster computing system. Through a systematic step-by-step procedure, we investigate challenges for efficient server consolidation in Hadoop environments. To this end, we first investigate the inter-relationship between last level cache (LLC) contention and throughput degradation for consolidated workloads on a single physical server employing Hadoop distributed file system (HDFS). We then investigate the general case of consolidation on multiple physical servers so that their throughput never falls below a desired/predefined utilization level. We use our empirical results to model consolidation as a classic two-dimensional bin packing problem and then design a computationally efficient greedy algorithm to achieve minimum throughput degradation on multiple servers. Results are very promising and show that our greedy approach is able to achieve near optimal solution in all experimented cases. △ Less

Submitted 28 March, 2013; originally announced March 2013.

Comments: Published at IEEE Grid 2012

Statistical Regression to Predict Total Cumulative CPU Usage of MapReduce Jobs

Authors: Nikzad Babaii Rizvandi, Javid Taheri, Reza Moraveji, Albert Y. Zomaya

Abstract: Recently, businesses have started using MapReduce as a popular computation framework for processing large amount of data, such as spam detection, and different data mining tasks, in both public and private clouds. Two of the challenging questions in such environments are (1) choosing suitable values for MapReduce configuration parameters e.g., number of mappers, number of reducers, and DFS block s… ▽ More Recently, businesses have started using MapReduce as a popular computation framework for processing large amount of data, such as spam detection, and different data mining tasks, in both public and private clouds. Two of the challenging questions in such environments are (1) choosing suitable values for MapReduce configuration parameters e.g., number of mappers, number of reducers, and DFS block size, and (2) predicting the amount of resources that a user should lease from the service provider. Currently, the tasks of both choosing configuration parameters and estimating required resources are solely the users responsibilities. In this paper, we present an approach to provision the total CPU usage in clock cycles of jobs in MapReduce environment. For a MapReduce job, a profile of total CPU usage in clock cycles is built from the job past executions with different values of two configuration parameters e.g., number of mappers, and number of reducers. Then, a polynomial regression is used to model the relation between these configuration parameters and total CPU usage in clock cycles of the job. We also briefly study the influence of input data scaling on measured total CPU usage in clock cycles. This derived model along with the scaling result can then be used to provision the total CPU usage in clock cycles of the same jobs with different input data size. We validate the accuracy of our models using three realistic applications (WordCount, Exim MainLog parsing, and TeraSort). Results show that the predicted total CPU usage in clock cycles of generated resource provisioning options are less than 8% of the measured total CPU usage in clock cycles in our 20-node virtual Hadoop cluster. △ Less

Submitted 14 March, 2013; originally announced March 2013.

Comments: 16 pages- previously published as "On Modelling and Prediction of Total CPU Usage for Applications in MapReduce Enviornments" in IEEE 12th International Conference on Algorithms and Architectures for Parallel Processing (ICA3PP-12), Fukuoka, Japan, 4-7 September, 2012

Pattern Matching for Self- Tuning of MapReduce Jobs

Authors: Nikzad Babaii Rizvandi, Javid Taheri, Albert Y. Zomaya

Abstract: In this paper, we study CPU utilization time patterns of several MapReduce applications. After extracting running patterns of several applications, they are saved in a reference database to be later used to tweak system parameters to efficiently execute unknown applications in future. To achieve this goal, CPU utilization patterns of new applications are compared with the already known ones in the… ▽ More In this paper, we study CPU utilization time patterns of several MapReduce applications. After extracting running patterns of several applications, they are saved in a reference database to be later used to tweak system parameters to efficiently execute unknown applications in future. To achieve this goal, CPU utilization patterns of new applications are compared with the already known ones in the reference database to find/predict their most probable execution patterns. Because of different patterns lengths, the Dynamic Time Warping (DTW) is utilized for such comparison; a correlation analysis is then applied to DTWs outcomes to produce feasible similarity patterns. Three real applications (WordCount, Exim Mainlog parsing and Terasort) are used to evaluate our hypothesis in tweaking system parameters in executing similar applications. Results were very promising and showed effectiveness of our approach on pseudo-distributed MapReduce platforms. △ Less

Submitted 20 January, 2013; originally announced January 2013.

Comments: 7 pages, previously published as "On Using Pattern Matching Algorithms in MapReduce Applications" at ISPA 2011. arXiv admin note: substantial text overlap with arXiv:1112.5505

A Primarily Survey on Energy Efficiency in Cloud and Distributed Computing Systems

Authors: Nikzad Babaii Rizvandi, Albert Y. Zomaya

Abstract: A survey of available techniques in hardware to reduce energy consumption A survey of available techniques in hardware to reduce energy consumption △ Less

Submitted 15 November, 2012; v1 submitted 17 October, 2012; originally announced October 2012.

Comments: 18 pages

Network Load Analysis and Provisioning of MapReduce Applications

Authors: Nikzad Babaii Rizvandi, Javid Taheri, Reza Moraveji, Albert Y. Zomaya

Abstract: In this paper, we study the dependency between configuration parameters and network load of fixed-size MapReduce applications in shuffle phase and then propose an analytical method to model this dependency. Our approach consists of three key phases: profiling, modeling, and prediction. In the first stage, an application is run several times with different sets of MapReduce configuration parameters… ▽ More In this paper, we study the dependency between configuration parameters and network load of fixed-size MapReduce applications in shuffle phase and then propose an analytical method to model this dependency. Our approach consists of three key phases: profiling, modeling, and prediction. In the first stage, an application is run several times with different sets of MapReduce configuration parameters (here number of mappers and number of reducers) to profile the network load of the application in the shuffle phase on a given cluster. Then, the relation between these parameters and the network load is modeled by multivariate linear regression. For evaluation, three applications (WordCount, Exim Mainlog parsing, and TeraSort) are utilized to evaluate our technique on a 4-node MapReduce private cluster. △ Less

Submitted 26 July, 2012; v1 submitted 10 June, 2012; originally announced June 2012.

Comments: 6 pages-submitted to The Thirteenth International Conference on Parallel and Distributed Computing, Applications and Technologies(PDCAT-12), Beijing, China

Multiple Frequency Selection in DVFS-Enabled Processors to Minimize Energy Consumption

Authors: Nikzad Babaii Rizvandi, Albert Y. Zomaya, Young Choon Lee, Ali Javadzadeh Boloori, Javid Taheri

Abstract: In this chapter we focus on slack reclamation and propose a new slack reclamation technique, Multiple Frequency Selection DVFS (MFS-DVFS). The key idea is to execute each task with a linear combination of more than one frequency such that this combination results in using the lowest energy by covering the whole slack time of the task. We have tested our algorithm with both random and real-world ap… ▽ More In this chapter we focus on slack reclamation and propose a new slack reclamation technique, Multiple Frequency Selection DVFS (MFS-DVFS). The key idea is to execute each task with a linear combination of more than one frequency such that this combination results in using the lowest energy by covering the whole slack time of the task. We have tested our algorithm with both random and real-world application task graphs and compared with the results in previous researches in [9] and [12-13]. The experimental results show that our approach can achieve energy almost identical to the optimum energy saving. △ Less

Submitted 3 September, 2012; v1 submitted 22 March, 2012; originally announced March 2012.

Comments: Chapter 17- Book title: "Energy Efficient Distributed Computing", Edited by Albert Y.Zomaya, Young Choon Lee Wiley