-
Reinforcement Learning-driven Data-intensive Workflow Scheduling for Volunteer Edge-Cloud
Authors:
Motahare Mounesan,
Mauro Lemus,
Hemanth Yeddulapalli,
Prasad Calyam,
Saptarshi Debroy
Abstract:
In recent times, Volunteer Edge-Cloud (VEC) has gained traction as a cost-effective, community computing paradigm to support data-intensive scientific workflows. However, due to the highly distributed and heterogeneous nature of VEC resources, centralized workflow task scheduling remains a challenge. In this paper, we propose a Reinforcement Learning (RL)-driven data-intensive scientific workflow…
▽ More
In recent times, Volunteer Edge-Cloud (VEC) has gained traction as a cost-effective, community computing paradigm to support data-intensive scientific workflows. However, due to the highly distributed and heterogeneous nature of VEC resources, centralized workflow task scheduling remains a challenge. In this paper, we propose a Reinforcement Learning (RL)-driven data-intensive scientific workflow scheduling approach that takes into consideration: i) workflow requirements, ii) VEC resources' preference on workflows, and iii) diverse VEC resource policies, to ensure robust resource allocation. We formulate the long-term average performance optimization problem as a Markov Decision Process, which is solved using an event-based Asynchronous Advantage Actor-Critic RL approach. Our extensive simulations and testbed implementations demonstrate our approach's benefits over popular baseline strategies in terms of workflow requirement satisfaction, VEC preference satisfaction, and available VEC resource utilization.
△ Less
Submitted 1 July, 2024;
originally announced July 2024.
-
End-to-End Latency Optimization of Multi-view 3D Reconstruction for Disaster Response
Authors:
Xiaojie Zhang,
Mingjun Li,
Andrew Hilton,
Amitangshu Pal,
Soumyabrata Dey,
Saptarshi Debroy
Abstract:
In order to plan rapid response during disasters, first responder agencies often adopt `bring your own device' (BYOD) model with inexpensive mobile edge devices (e.g., drones, robots, tablets) for complex video analytics applications, e.g., 3D reconstruction of a disaster scene. Unlike simpler video applications, widely used Multi-view Stereo (MVS) based 3D reconstruction applications (e.g., openM…
▽ More
In order to plan rapid response during disasters, first responder agencies often adopt `bring your own device' (BYOD) model with inexpensive mobile edge devices (e.g., drones, robots, tablets) for complex video analytics applications, e.g., 3D reconstruction of a disaster scene. Unlike simpler video applications, widely used Multi-view Stereo (MVS) based 3D reconstruction applications (e.g., openMVG/openMVS) are exceedingly time consuming, especially when run on such computationally constrained mobile edge devices. Additionally, reducing the reconstruction latency of such inherently sequential algorithms is challenging as unintelligent, application-agnostic strategies can drastically degrade the reconstruction (i.e., application outcome) quality making them useless. In this paper, we aim to design a latency optimized MVS algorithm pipeline, with the objective to best balance the end-to-end latency and reconstruction quality by running the pipeline on a collaborative mobile edge environment. The overall optimization approach is two-pronged where: (a) application optimizations introduce data-level parallelism by splitting the pipeline into high frequency and low frequency reconstruction components and (b) system optimizations incorporate task-level parallelism to the pipelines by running them opportunistically on available resources with online quality control in order to balance both latency and quality. Our evaluation on a hardware testbed using publicly available datasets shows upto ~54% reduction in latency with negligible loss (~4-7%) in reconstruction quality.
△ Less
Submitted 3 April, 2023;
originally announced April 2023.
-
DNN-based Denial of Quality of Service Attack on Software-defined Hybrid Edge-Cloud Systems
Authors:
Minh Nguyen,
Jacob Gately,
Swati Kar,
Soumyabrata Dey,
Saptarshi Debroy
Abstract:
In order to satisfy diverse quality-of-service (QoS) requirements of complex real-time video applications, civilian and tactical use cases are employing software-defined hybrid edge-cloud systems. One of the primary QoS requirements of such applications is ultra-low end-to-end latency for video applications that necessitates rapid frame transfer between end-devices and edge servers using software-…
▽ More
In order to satisfy diverse quality-of-service (QoS) requirements of complex real-time video applications, civilian and tactical use cases are employing software-defined hybrid edge-cloud systems. One of the primary QoS requirements of such applications is ultra-low end-to-end latency for video applications that necessitates rapid frame transfer between end-devices and edge servers using software-defined networking (SDN). Failing to guarantee such strict requirements leads to quality degradation of video applications and subsequently mission failure. In this paper, we show how a collaborative group of attackers can exploit SDN's control communications to launch Denial of Quality of Service (DQoS) attack that artificially increases end-to-end latency of video frames and yet evades detection. In particular, we show how Deep Neural Network (DNN) model training on all or partial network state information can help predict network packet drop rates with reasonable accuracy. We also show how such predictions can help design an attack model that can inflict just the right amount of added latency to the end-to-end video processing that is enough to cause considerable QoS degradation but not too much to raise suspicion. We use a realistic edge-cloud testbed on GENI platform for training data collection and demonstration of high model accuracy and attack success rate.
△ Less
Submitted 2 April, 2023;
originally announced April 2023.
