Showing 1–29 of 29 results for author: Pajic, M

Bayesian Methods for Trust in Collaborative Multi-Agent Autonomy

Authors: R. Spencer Hallyburton, Miroslav Pajic

Abstract: Multi-agent, collaborative sensor fusion is a vital component of a multi-national intelligence toolkit. In safety-critical and/or contested environments, adversaries may infiltrate and compromise a number of agents. We analyze state of the art multi-target tracking algorithms under this compromised agent threat model. We prove that the track existence probability test ("track score") is significan… ▽ More Multi-agent, collaborative sensor fusion is a vital component of a multi-national intelligence toolkit. In safety-critical and/or contested environments, adversaries may infiltrate and compromise a number of agents. We analyze state of the art multi-target tracking algorithms under this compromised agent threat model. We prove that the track existence probability test ("track score") is significantly vulnerable to even small numbers of adversaries. To add security awareness, we design a trust estimation framework using hierarchical Bayesian updating. Our framework builds beliefs of trust on tracks and agents by mapping sensor measurements to trust pseudomeasurements (PSMs) and incorporating prior trust beliefs in a Bayesian context. In case studies, our trust estimation algorithm accurately estimates the trustworthiness of tracks/agents, subject to observability limitations. △ Less

Submitted 25 March, 2024; originally announced March 2024.

RadCloud: Real-Time High-Resolution Point Cloud Generation Using Low-Cost Radars for Aerial and Ground Vehicles

Authors: David Hunt, Shaocheng Luo, Amir Khazraei, Xiao Zhang, Spencer Hallyburton, Tingjun Chen, Miroslav Pajic

Abstract: In this work, we present RadCloud, a novel real time framework for directly obtaining higher-resolution lidar-like 2D point clouds from low-resolution radar frames on resource-constrained platforms commonly used in unmanned aerial and ground vehicles (UAVs and UGVs, respectively); such point clouds can then be used for accurate environmental mapping, navigating unknown environments, and other robo… ▽ More In this work, we present RadCloud, a novel real time framework for directly obtaining higher-resolution lidar-like 2D point clouds from low-resolution radar frames on resource-constrained platforms commonly used in unmanned aerial and ground vehicles (UAVs and UGVs, respectively); such point clouds can then be used for accurate environmental mapping, navigating unknown environments, and other robotics tasks. While high-resolution sensing using radar data has been previously reported, existing methods cannot be used on most UAVs, which have limited computational power and energy; thus, existing demonstrations focus on offline radar processing. RadCloud overcomes these challenges by using a radar configuration with 1/4th of the range resolution and employing a deep learning model with 2.25x fewer parameters. Additionally, RadCloud utilizes a novel chirp-based approach that makes obtained point clouds resilient to rapid movements (e.g., aggressive turns or spins), which commonly occur during UAV flights. In real-world experiments, we demonstrate the accuracy and applicability of RadCloud on commercially available UAVs and UGVs, with off-the-shelf radar platforms on-board. △ Less

Submitted 9 March, 2024; originally announced March 2024.

Comments: $©$ 2024 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works

A Multi-Agent Security Testbed for the Analysis of Attacks and Defenses in Collaborative Sensor Fusion

Authors: R. Spencer Hallyburton, David Hunt, Shaocheng Luo, Miroslav Pajic

Abstract: The performance and safety of autonomous vehicles (AVs) deteriorates under adverse environments and adversarial actors. The investment in multi-sensor, multi-agent (MSMA) AVs is meant to promote improved efficiency of travel and mitigate safety risks. Unfortunately, minimal investment has been made to develop security-aware MSMA sensor fusion pipelines leaving them vulnerable to adversaries. To ad… ▽ More The performance and safety of autonomous vehicles (AVs) deteriorates under adverse environments and adversarial actors. The investment in multi-sensor, multi-agent (MSMA) AVs is meant to promote improved efficiency of travel and mitigate safety risks. Unfortunately, minimal investment has been made to develop security-aware MSMA sensor fusion pipelines leaving them vulnerable to adversaries. To advance security analysis of AVs, we develop the Multi-Agent Security Testbed, MAST, in the Robot Operating System (ROS2). Our framework is scalable for general AV scenarios and is integrated with recent multi-agent datasets. We construct the first bridge between AVstack and ROS and develop automated AV pipeline builds to enable rapid AV prototyping. We tackle the challenge of deploying variable numbers of agent/adversary nodes at launch-time with dynamic topic remapping. Using this testbed, we motivate the need for security-aware AV architectures by exposing the vulnerability of centralized multi-agent fusion pipelines to (un)coordinated adversary models in case studies and Monte Carlo analysis. △ Less

Submitted 17 January, 2024; originally announced January 2024.

Spectral Statistics of the Sample Covariance Matrix for High Dimensional Linear Gaussians

Authors: Muhammad Abdullah Naeem, Miroslav Pajic

Abstract: Performance of ordinary least squares(OLS) method for the \emph{estimation of high dimensional stable state transition matrix} $A$(i.e., spectral radius $ρ(A)<1$) from a single noisy observed trajectory of the linear time invariant(LTI)\footnote{Linear Gaussian (LG) in Markov chain literature} system $X_{-}:(x_0,x_1, \ldots,x_{N-1})$ satisfying \begin{equation} x_{t+1}=Ax_{t}+w_{t}, \hspace{10pt… ▽ More Performance of ordinary least squares(OLS) method for the \emph{estimation of high dimensional stable state transition matrix} $A$(i.e., spectral radius $ρ(A)<1$) from a single noisy observed trajectory of the linear time invariant(LTI)\footnote{Linear Gaussian (LG) in Markov chain literature} system $X_{-}:(x_0,x_1, \ldots,x_{N-1})$ satisfying \begin{equation} x_{t+1}=Ax_{t}+w_{t}, \hspace{10pt} \text{ where } w_{t} \thicksim N(0,I_{n}), \end{equation} heavily rely on negative moments of the sample covariance matrix: $(X_{-}X_{-}^{*})=\sum_{i=0}^{N-1}x_{i}x_{i}^{*}$ and singular values of $EX_{-}^{*}$, where $E$ is a rectangular Gaussian ensemble $E=[w_0, \ldots, w_{N-1}]$. Negative moments requires sharp estimates on all the eigenvalues $λ_{1}\big(X_{-}X_{-}^{*}\big) \geq \ldots \geq λ_{n}\big(X_{-}X_{-}^{*}\big) \geq 0$. Leveraging upon recent results on spectral theorem for non-Hermitian operators in \cite{naeem2023spectral}, along with concentration of measure phenomenon and perturbation theory(Gershgorins' and Cauchys' interlacing theorem) we show that only when $A=A^{*}$, typical order of $λ_{j}\big(X_{-}X_{-}^{*}\big) \in \big[N-n\sqrt{N}, N+n\sqrt{N}\big]$ for all $j \in [n]$. However, in \emph{high dimensions} when $A$ has only one distinct eigenvalue $λ$ with geometric multiplicity of one, then as soon as eigenvalue leaves \emph{complex half unit disc}, largest eigenvalue suffers from curse of dimensionality: $λ_{1}\big(X_{-}X_{-}^{*}\big)=Ω\big( \lfloor\frac{N}{n}\rfloor e^{α_λn} \big)$, while smallest eigenvalue $λ_{n}\big(X_{-}X_{-}^{*}\big) \in (0, N+\sqrt{N}]$. Consequently, OLS estimator incurs a \emph{phase transition} and becomes \emph{transient: increasing iteration only worsens estimation error}, all of this happening when the dynamics are generated from stable systems. △ Less

Submitted 10 December, 2023; originally announced December 2023.

Comments: arXiv admin note: text overlap with arXiv:2310.10523

MadRadar: A Black-Box Physical Layer Attack Framework on mmWave Automotive FMCW Radars

Authors: David Hunt, Kristen Angell, Zhenzhou Qi, Tingjun Chen, Miroslav Pajic

Abstract: Frequency modulated continuous wave (FMCW) millimeter-wave (mmWave) radars play a critical role in many of the advanced driver assistance systems (ADAS) featured on today's vehicles. While previous works have demonstrated (only) successful false-positive spoofing attacks against these sensors, all but one assumed that an attacker had the runtime knowledge of the victim radar's configuration. In th… ▽ More Frequency modulated continuous wave (FMCW) millimeter-wave (mmWave) radars play a critical role in many of the advanced driver assistance systems (ADAS) featured on today's vehicles. While previous works have demonstrated (only) successful false-positive spoofing attacks against these sensors, all but one assumed that an attacker had the runtime knowledge of the victim radar's configuration. In this work, we introduce MadRadar, a general black-box radar attack framework for automotive mmWave FMCW radars capable of estimating the victim radar's configuration in real-time, and then executing an attack based on the estimates. We evaluate the impact of such attacks maliciously manipulating a victim radar's point cloud, and show the novel ability to effectively `add' (i.e., false positive attacks), `remove' (i.e., false negative attacks), or `move' (i.e., translation attacks) object detections from a victim vehicle's scene. Finally, we experimentally demonstrate the feasibility of our attacks on real-world case studies performed using a real-time physical prototype on a software-defined radio platform. △ Less

Submitted 27 November, 2023; originally announced November 2023.

From Spectral Theorem to Statistical Independence with Application to System Identification

Authors: Muhammad Abdullah Naeem, Amir Khazraei, Miroslav Pajic

Abstract: High dimensional random dynamical systems are ubiquitous, including -- but not limited to -- cyber-physical systems, daily return on different stocks of S&P 1500 and velocity profile of interacting particle systems around McKeanVlasov limit. Mathematically, underlying phenomenon can be captured via a stable $n$-dimensional linear transformation `$A$' and additive randomness. System identification… ▽ More High dimensional random dynamical systems are ubiquitous, including -- but not limited to -- cyber-physical systems, daily return on different stocks of S&P 1500 and velocity profile of interacting particle systems around McKeanVlasov limit. Mathematically, underlying phenomenon can be captured via a stable $n$-dimensional linear transformation `$A$' and additive randomness. System identification aims at extracting useful information about underlying dynamical system, given a length $N$ trajectory from it (corresponds to an $n \times N$ dimensional data matrix). We use spectral theorem for non-Hermitian operators to show that spatio-temperal correlations are dictated by the discrepancy between algebraic and geometric multiplicity of distinct eigenvalues corresponding to state transition matrix. Small discrepancies imply that original trajectory essentially comprises of multiple lower dimensional random dynamical systems living on $A$ invariant subspaces and are statistically independent of each other. In the process, we provide first quantitative handle on decay rate of finite powers of state transition matrix $\|A^{k}\|$ . It is shown that when a stable dynamical system has only one distinct eigenvalue and discrepancy of $n-1$: $\|A\|$ has a dependence on $n$, resulting dynamics are spatially inseparable and consequently there exist at least one row with covariates of typical size $Θ\big(\sqrt{N-n+1}$ $e^{n}\big)$ i.e., even under stability assumption, covariates can suffer from curse of dimensionality. In the light of these findings we set the stage for non-asymptotic error analysis in estimation of state transition matrix $A$ via least squares regression on observed trajectory by showing that element-wise error is essentially a variant of well-know Littlewood-Offord problem. △ Less

Submitted 16 October, 2023; originally announced October 2023.

Vulnerability Analysis of Nonlinear Control Systems to Stealthy False Data Injection Attacks

Authors: Amir Khazraei, Miroslav Pajic

Abstract: In this work, we focus on analyzing vulnerability of nonlinear dynamical control systems to stealthy false data injection attacks on sensors. We start by defining the stealthiness notion in the most general form where an attack is considered stealthy if it would be undetected by any intrusion detector, i.e., any intrusion detector could not do better than a random guess. Depending on the level of… ▽ More In this work, we focus on analyzing vulnerability of nonlinear dynamical control systems to stealthy false data injection attacks on sensors. We start by defining the stealthiness notion in the most general form where an attack is considered stealthy if it would be undetected by any intrusion detector, i.e., any intrusion detector could not do better than a random guess. Depending on the level of attacker's knowledge about the plant model, controller, and the system states, two different attack models are considered. For each attack model, we derive the conditions for which the system will be vulnerable to stealthy impactful attacks, in addition to finding a methodology for designing such sequence of false data injection attacks. When the attacker has complete knowledge about the system, we show that if the closed loop system is incrementally exponentially stable while the open loop plant is incrementally unstable, then the system is vulnerable to stealthy yet impactful attacks on sensors. However, in the second attack model, with less knowledge about the system, additional conditions need to be satisfied and the level of stealthiness depends on the accuracy of attacker's knowledge about the system. We also consider the impact of stealthy attacks on state estimation, and show that if the closed loop control system including the estimator is incrementally stable, then the state estimation in the presence of attack converges to the attack free estimates. Finally, we illustrate our results on numerical case studies. △ Less

Submitted 6 October, 2023; originally announced October 2023.

Comments: arXiv admin note: substantial text overlap with arXiv:2204.03217

High Dimensional Geometry and Limitations in System Identification

Authors: Muhammad Abdullah Naeem, Miroslav Pajic

Abstract: We study the problem of identification of linear dynamical system from a single trajectory, via excitations of isotropic Gaussian. In stark contrast with previously reported results, Ordinary Least Squares (OLS) estimator for even \emph{stable} dynamical system contains non-vanishing error in \emph{high dimensions}; which stems from the fact that realizations of non-diagonalizable dynamics can hav… ▽ More We study the problem of identification of linear dynamical system from a single trajectory, via excitations of isotropic Gaussian. In stark contrast with previously reported results, Ordinary Least Squares (OLS) estimator for even \emph{stable} dynamical system contains non-vanishing error in \emph{high dimensions}; which stems from the fact that realizations of non-diagonalizable dynamics can have strong \emph{spatial correlations} and a variance, of order $O(e^{n})$, where $n$ is the dimension of the underlying state space. Employing \emph{concentration of measure phenomenon}, in particular tensorization of \emph{Talagrands inequality} for random dynamical systems we show that observed trajectory of dynamical system of length-$N$ can have a variance of order $O(e^{nN})$. Consequently, showing some or most of the $n$ distances between an $N-$ dimensional random vector and an $(n-1)$ dimensional hyperplane in $\mathbb{R}^{N}$ can be close to zero with positive probability and these estimates become stronger in high dimensions and more iterations via \emph{Isoperimetry}. \emph{Negative second moment identity}, along with distance estimates give a control on all the singular values of \emph{Random matrix} of data, revealing limitations of OLS for stable non-diagonalizable and explosive diagonalizable systems. △ Less

Submitted 19 May, 2023; originally announced May 2023.

A Modular Platform For Collaborative, Distributed Sensor Fusion

Authors: R. Spencer Hallyburton, Nate Zelter, David Hunt, Kristen Angell, Miroslav Pajic

Abstract: Leading autonomous vehicle (AV) platforms and testing infrastructures are, unfortunately, proprietary and closed-source. Thus, it is difficult to evaluate how well safety-critical AVs perform and how safe they truly are. Similarly, few platforms exist for much-needed multi-agent analysis. To provide a starting point for analysis of sensor fusion and collaborative & distributed sensing, we design a… ▽ More Leading autonomous vehicle (AV) platforms and testing infrastructures are, unfortunately, proprietary and closed-source. Thus, it is difficult to evaluate how well safety-critical AVs perform and how safe they truly are. Similarly, few platforms exist for much-needed multi-agent analysis. To provide a starting point for analysis of sensor fusion and collaborative & distributed sensing, we design an accessible, modular sensing platform with AVstack. We build collaborative and distributed camera-radar fusion algorithms and demonstrate an evaluation ecosystem of AV datasets, physics-based simulators, and hardware in the physical world. This three-part ecosystem enables testing next-generation configurations that are prohibitively challenging in existing development platforms. △ Less

Submitted 29 March, 2023; v1 submitted 13 March, 2023; originally announced March 2023.

Partial-Information, Longitudinal Cyber Attacks on LiDAR in Autonomous Vehicles

Authors: R. Spencer Hallyburton, Qingzhao Zhang, Z. Morley Mao, Miroslav Pajic

Abstract: What happens to an autonomous vehicle (AV) if its data are adversarially compromised? Prior security studies have addressed this question through mostly unrealistic threat models, with limited practical relevance, such as white-box adversarial learning or nanometer-scale laser aiming and spoofing. With growing evidence that cyber threats pose real, imminent danger to AVs and cyber-physical systems… ▽ More What happens to an autonomous vehicle (AV) if its data are adversarially compromised? Prior security studies have addressed this question through mostly unrealistic threat models, with limited practical relevance, such as white-box adversarial learning or nanometer-scale laser aiming and spoofing. With growing evidence that cyber threats pose real, imminent danger to AVs and cyber-physical systems (CPS) in general, we present and evaluate a novel AV threat model: a cyber-level attacker capable of disrupting sensor data but lacking any situational awareness. We demonstrate that even though the attacker has minimal knowledge and only access to raw data from a single sensor (i.e., LiDAR), she can design several attacks that critically compromise perception and tracking in multi-sensor AVs. To mitigate vulnerabilities and advance secure architectures in AVs, we introduce two improvements for security-aware fusion: a probabilistic data-asymmetry monitor and a scalable track-to-track fusion of 3D LiDAR and monocular detections (T2T-3DLM); we demonstrate that the approaches significantly reduce attack effectiveness. To support objective safety and security evaluations in AVs, we release our security evaluation platform, AVsec, which is built on security-relevant metrics to benchmark AVs on gold-standard longitudinal AV datasets and AV simulators. △ Less

Submitted 8 December, 2023; v1 submitted 6 March, 2023; originally announced March 2023.

Stealthy Perception-based Attacks on Unmanned Aerial Vehicles

Authors: Amir Khazraei, Haocheng Meng, Miroslav Pajic

Abstract: In this work, we study vulnerability of unmanned aerial vehicles (UAVs) to stealthy attacks on perception-based control. To guide our analysis, we consider two specific missions: ($i$) ground vehicle tracking (GVT), and ($ii$) vertical take-off and landing (VTOL) of a quadcopter on a moving ground vehicle. Specifically, we introduce a method to consistently attack both the sensors measurements and… ▽ More In this work, we study vulnerability of unmanned aerial vehicles (UAVs) to stealthy attacks on perception-based control. To guide our analysis, we consider two specific missions: ($i$) ground vehicle tracking (GVT), and ($ii$) vertical take-off and landing (VTOL) of a quadcopter on a moving ground vehicle. Specifically, we introduce a method to consistently attack both the sensors measurements and camera images over time, in order to cause control performance degradation (e.g., by failing the mission) while remaining stealthy (i.e., undetected by the deployed anomaly detector). Unlike existing attacks that mainly rely on vulnerability of deep neural networks to small input perturbations (e.g., by adding small patches and/or noise to the images), we show that stealthy yet effective attacks can be designed by changing images of the ground vehicle's landing markers as well as suitably falsifying sensing data. We illustrate the effectiveness of our attacks in Gazebo 3D robotics simulator. △ Less

Submitted 3 March, 2023; originally announced March 2023.

Offline Learning of Closed-Loop Deep Brain Stimulation Controllers for Parkinson Disease Treatment

Authors: Qitong Gao, Stephen L. Schimdt, Afsana Chowdhury, Guangyu Feng, Jennifer J. Peters, Katherine Genty, Warren M. Grill, Dennis A. Turner, Miroslav Pajic

Abstract: Deep brain stimulation (DBS) has shown great promise toward treating motor symptoms caused by Parkinson's disease (PD), by delivering electrical pulses to the Basal Ganglia (BG) region of the brain. However, DBS devices approved by the U.S. Food and Drug Administration (FDA) can only deliver continuous DBS (cDBS) stimuli at a fixed amplitude; this energy inefficient operation reduces battery lifet… ▽ More Deep brain stimulation (DBS) has shown great promise toward treating motor symptoms caused by Parkinson's disease (PD), by delivering electrical pulses to the Basal Ganglia (BG) region of the brain. However, DBS devices approved by the U.S. Food and Drug Administration (FDA) can only deliver continuous DBS (cDBS) stimuli at a fixed amplitude; this energy inefficient operation reduces battery lifetime of the device, cannot adapt treatment dynamically for activity, and may cause significant side-effects (e.g., gait impairment). In this work, we introduce an offline reinforcement learning (RL) framework, allowing the use of past clinical data to train an RL policy to adjust the stimulation amplitude in real time, with the goal of reducing energy use while maintaining the same level of treatment (i.e., control) efficacy as cDBS. Moreover, clinical protocols require the safety and performance of such RL controllers to be demonstrated ahead of deployments in patients. Thus, we also introduce an offline policy evaluation (OPE) method to estimate the performance of RL policies using historical data, before deploying them on patients. We evaluated our framework on four PD patients equipped with the RC+S DBS system, employing the RL controllers during monthly clinical visits, with the overall control efficacy evaluated by severity of symptoms (i.e., bradykinesia and tremor), changes in PD biomakers (i.e., local field potentials), and patient ratings. The results from clinical experiments show that our RL-based controller maintains the same level of control efficacy as cDBS, but with significantly reduced stimulation energy. Further, the OPE method is shown effective in accurately estimating and ranking the expected returns of RL controllers. △ Less

Submitted 15 March, 2023; v1 submitted 5 February, 2023; originally announced February 2023.

Comments: Accepted to International Conference on Cyber Physical Systems (ICCPS) 2023

AVstack: An Open-Source, Reconfigurable Platform for Autonomous Vehicle Development

Authors: R. Spencer Hallyburton, Shucheng Zhang, Miroslav Pajic

Abstract: Pioneers of autonomous vehicles (AVs) promised to revolutionize the driving experience and driving safety. However, milestones in AVs have materialized slower than forecast. Two culprits are (1) the lack of verifiability of proposed state-of-the-art AV components, and (2) stagnation of pursuing next-level evaluations, e.g., vehicle-to-infrastructure (V2I) and multi-agent collaboration. In part, pr… ▽ More Pioneers of autonomous vehicles (AVs) promised to revolutionize the driving experience and driving safety. However, milestones in AVs have materialized slower than forecast. Two culprits are (1) the lack of verifiability of proposed state-of-the-art AV components, and (2) stagnation of pursuing next-level evaluations, e.g., vehicle-to-infrastructure (V2I) and multi-agent collaboration. In part, progress has been hampered by: the large volume of software in AVs, the multiple disparate conventions, the difficulty of testing across datasets and simulators, and the inflexibility of state-of-the-art AV components. To address these challenges, we present AVstack, an open-source, reconfigurable software platform for AV design, implementation, test, and analysis. AVstack solves the validation problem by enabling first-of-a-kind trade studies on datasets and physics-based simulators. AVstack solves the stagnation problem as a reconfigurable AV platform built on dozens of open-source AV components in a high-level programming language. We demonstrate the power of AVstack through longitudinal testing across multiple benchmark datasets and V2I-collaboration case studies that explore trade-offs of designing multi-sensor, multi-agent algorithms. △ Less

Submitted 10 March, 2023; v1 submitted 28 December, 2022; originally announced December 2022.

Attacks on Perception-Based Control Systems: Modeling and Fundamental Limits

Authors: Amir Khazraei, Henry Pfister, Miroslav Pajic

Abstract: We study the performance of perception-based control systems in the presence of attacks, and provide methods for modeling and analysis of their resiliency to stealthy attacks on both physical and perception-based sensing. Specifically, we consider a general setup with a nonlinear affine physical plant controlled with a perception-based controller that maps both the physical (e.g., IMUs) and percep… ▽ More We study the performance of perception-based control systems in the presence of attacks, and provide methods for modeling and analysis of their resiliency to stealthy attacks on both physical and perception-based sensing. Specifically, we consider a general setup with a nonlinear affine physical plant controlled with a perception-based controller that maps both the physical (e.g., IMUs) and perceptual (e.g., camera) sensing to the control input; the system is also equipped with a statistical or learning-based anomaly detector (AD). We model the attacks in the most general form, and introduce the notions of attack effectiveness and stealthiness independent of the used AD. In such setting, we consider attacks with different levels of runtime knowledge about the plant. We find sufficient conditions for existence of stealthy effective attacks that force the plant into an unsafe region without being detected by any AD. We show that as the open-loop unstable plant dynamics diverges faster and the closed-loop system converges faster to an equilibrium point, the system is more vulnerable to effective stealthy attacks. Also, depending on runtime information available to the attacker, the probability of attack remaining stealthy can be arbitrarily close to one, if the attacker's estimate of the plant's state is arbitrarily close to the true state; when an accurate estimate of the plant state is not available, the stealthiness level depends on the control performance in attack-free operation. △ Less

Submitted 27 August, 2023; v1 submitted 14 June, 2022; originally announced June 2022.

Transportation-Inequalities, Lyapunov Stability and Sampling for Dynamical Systems on Continuous State Space

Authors: Muhammad Abdullah Naeem, Miroslav Pajic

Abstract: We study the concentration phenomenon for discrete-time random dynamical systems with an unbounded state space. We develop a heuristic approach towards obtaining exponential concentration inequalities for dynamical systems using an entirely functional analytic framework. We also show that existence of exponential-type Lyapunov function, compared to the purely deterministic setting, not only implie… ▽ More We study the concentration phenomenon for discrete-time random dynamical systems with an unbounded state space. We develop a heuristic approach towards obtaining exponential concentration inequalities for dynamical systems using an entirely functional analytic framework. We also show that existence of exponential-type Lyapunov function, compared to the purely deterministic setting, not only implies stability but also exponential concentration inequalities for sampling from the stationary distribution, via \emph{transport-entropy inequality} (T-E). These results have significant impact in \emph{reinforcement learning} (RL) and \emph{controls}, leading to exponential concentration inequalities even for unbounded observables, while neither assuming reversibility nor exact knowledge of random dynamical system (assumptions at heart of concentration inequalities in statistical mechanics and Markov diffusion processes). △ Less

Submitted 7 December, 2022; v1 submitted 24 May, 2022; originally announced May 2022.

Optimal Myopic Attacks on Nonlinear Estimation

Authors: R. Spencer Hallyburton, Amir Khazraei, Miroslav Pajic

Abstract: Recent high-profile incidents have exposed security risks in control systems. Particularly important and safety-critical modules for security analysis are estimation and control (E&C). Prior works have analyzed the security of E&C for linear, time-invariant systems; however, there are few analyses of nonlinear systems despite their broad use. In an effort to facilitate identifying vulnerabilities… ▽ More Recent high-profile incidents have exposed security risks in control systems. Particularly important and safety-critical modules for security analysis are estimation and control (E&C). Prior works have analyzed the security of E&C for linear, time-invariant systems; however, there are few analyses of nonlinear systems despite their broad use. In an effort to facilitate identifying vulnerabilities in control systems, in this work we establish a class of optimal attacks on nonlinear E&C. Specifically, we define two attack objectives and illustrate that realizing the optimal attacks against the widely-adopted extended Kalman filter with industry-standard $χ^2$ anomaly detection is equivalent to solving convex quadratically-constrained quadratic programs. Given an appropriate information model for the attacker (i.e.,~a specified amount of attacker knowledge), we provide practical relaxations on the optimal attacks to allow for their computation at runtime. We also show that the difference between the optimal and relaxed attacks is bounded. Finally, we illustrate the use of the introduced attack designs on a case-study. △ Less

Submitted 12 September, 2022; v1 submitted 14 April, 2022; originally announced April 2022.

Resiliency of Nonlinear Control Systems to Stealthy Sensor Attacks

Authors: Amir Khazraei, Miroslav Pajic

Abstract: In this work, we focus on analyzing vulnerability of nonlinear dynamical control systems to stealthy sensor attacks. We start by defining the notion of stealthy attacks in the most general form by leveraging Neyman-Pearson lemma; specifically, an attack is considered to be stealthy if it is stealthy from (i.e., undetected by) any intrusion detector -- i.e., the probability of the detection is not… ▽ More In this work, we focus on analyzing vulnerability of nonlinear dynamical control systems to stealthy sensor attacks. We start by defining the notion of stealthy attacks in the most general form by leveraging Neyman-Pearson lemma; specifically, an attack is considered to be stealthy if it is stealthy from (i.e., undetected by) any intrusion detector -- i.e., the probability of the detection is not better than a random guess. We then provide a sufficient condition under which a nonlinear control system is vulnerable to stealthy attacks, in terms of moving the system to an unsafe region due to the attacks. In particular, we show that if the closed-loop system is incrementally exponentially stable while the open-loop plant is incrementally unstable, then the system is vulnerable to stealthy yet impactful attacks on sensors. Finally, we illustrate our results on a case study. △ Less

Submitted 7 April, 2022; originally announced April 2022.

Security Analysis of Camera-LiDAR Fusion Against Black-Box Attacks on Autonomous Vehicles

Authors: R. Spencer Hallyburton, Yupei Liu, Yulong Cao, Z. Morley Mao, Miroslav Pajic

Abstract: To enable safe and reliable decision-making, autonomous vehicles (AVs) feed sensor data to perception algorithms to understand the environment. Sensor fusion with multi-frame tracking is becoming increasingly popular for detecting 3D objects. Thus, in this work, we perform an analysis of camera-LiDAR fusion, in the AV context, under LiDAR spoofing attacks. Recently, LiDAR-only perception was shown… ▽ More To enable safe and reliable decision-making, autonomous vehicles (AVs) feed sensor data to perception algorithms to understand the environment. Sensor fusion with multi-frame tracking is becoming increasingly popular for detecting 3D objects. Thus, in this work, we perform an analysis of camera-LiDAR fusion, in the AV context, under LiDAR spoofing attacks. Recently, LiDAR-only perception was shown vulnerable to LiDAR spoofing attacks; however, we demonstrate these attacks are not capable of disrupting camera-LiDAR fusion. We then define a novel, context-aware attack: frustum attack, and show that out of 8 widely used perception algorithms - across 3 architectures of LiDAR-only and 3 architectures of camera-LiDAR fusion - all are significantly vulnerable to the frustum attack. In addition, we demonstrate that the frustum attack is stealthy to existing defenses against LiDAR spoofing as it preserves consistencies between camera and LiDAR semantics. Finally, we show that the frustum attack can be exercised consistently over time to form stealthy longitudinal attack sequences, compromising the tracking module and creating adverse outcomes on end-to-end AV control. △ Less

Submitted 21 February, 2022; v1 submitted 13 June, 2021; originally announced June 2021.

Learning-Based Vulnerability Analysis of Cyber-Physical Systems

Authors: Amir Khazraei, Spencer Hallyburton, Qitong Gao, Yu Wang, Miroslav Pajic

Abstract: This work focuses on the use of deep learning for vulnerability analysis of cyber-physical systems (CPS). Specifically, we consider a control architecture widely used in CPS (e.g., robotics), where the low-level control is based on e.g., the extended Kalman filter (EKF) and an anomaly detector. To facilitate analyzing the impact potential sensing attacks could have, our objective is to develop lea… ▽ More This work focuses on the use of deep learning for vulnerability analysis of cyber-physical systems (CPS). Specifically, we consider a control architecture widely used in CPS (e.g., robotics), where the low-level control is based on e.g., the extended Kalman filter (EKF) and an anomaly detector. To facilitate analyzing the impact potential sensing attacks could have, our objective is to develop learning-enabled attack generators capable of designing stealthy attacks that maximally degrade system operation. We show how such problem can be cast within a learning-based grey-box framework where parts of the runtime information are known to the attacker, and introduce two models based on feed-forward neural networks (FNN); both models are trained offline, using a cost function that combines the attack effects on the estimation error and the residual signal used for anomaly detection, so that the trained models are capable of recursively generating such effective sensor attacks in real-time. The effectiveness of the proposed methods is illustrated on several case studies. △ Less

Submitted 7 April, 2022; v1 submitted 10 March, 2021; originally announced March 2021.

Formal Verification of Stochastic Systems with ReLU Neural Network Controllers

Authors: Shiqi Sun, Yan Zhang, Xusheng Luo, Panagiotis Vlantis, Miroslav Pajic, Michael M. Zavlanos

Abstract: In this work, we address the problem of formal safety verification for stochastic cyber-physical systems (CPS) equipped with ReLU neural network (NN) controllers. Our goal is to find the set of initial states from where, with a predetermined confidence, the system will not reach an unsafe configuration within a specified time horizon. Specifically, we consider discrete-time LTI systems with Gaussi… ▽ More In this work, we address the problem of formal safety verification for stochastic cyber-physical systems (CPS) equipped with ReLU neural network (NN) controllers. Our goal is to find the set of initial states from where, with a predetermined confidence, the system will not reach an unsafe configuration within a specified time horizon. Specifically, we consider discrete-time LTI systems with Gaussian noise, which we abstract by a suitable graph. Then, we formulate a Satisfiability Modulo Convex (SMC) problem to estimate upper bounds on the transition probabilities between nodes in the graph. Using this abstraction, we propose a method to compute tight bounds on the safety probabilities of nodes in this graph, despite possible over-approximations of the transition probabilities between these nodes. Additionally, using the proposed SMC formula, we devise a heuristic method to refine the abstraction of the system in order to further improve the estimated safety bounds. Finally, we corroborate the efficacy of the proposed method with simulation results considering a robot navigation example and comparison against a state-of-the-art verification scheme. △ Less

Submitted 8 March, 2021; originally announced March 2021.

Probabilistic Conformance for Cyber-Physical Systems

Authors: Yu Wang, Mojtaba Zarei, Borzoo Bonakdarpoor, Miroslav Pajic

Abstract: In system analysis, conformance indicates that two systems simultaneously satisfy the same set of specifications of interest; thus, the results from analyzing one system automatically transfer to the other, or one system can safely replace the other in practice. In this work, we study the probabilistic conformance of cyber-physical systems (CPS). We propose a notion of (approximate) probabilistic… ▽ More In system analysis, conformance indicates that two systems simultaneously satisfy the same set of specifications of interest; thus, the results from analyzing one system automatically transfer to the other, or one system can safely replace the other in practice. In this work, we study the probabilistic conformance of cyber-physical systems (CPS). We propose a notion of (approximate) probabilistic conformance for sets of complex specifications expressed by the Signal Temporal Logic (STL). Based on a novel statistical test, we develop the first statistical verification methods for the probabilistic conformance of a wide class of CPS. Using this method, we verify the conformance of the startup time of the widely-used full and simplified model of Toyota powertrain systems, the settling time of model-predictive-control-based and neural-network-based automotive lane-keeping controllers, as well as the maximal voltage deviation of full and simplified power grid systems. △ Less

Submitted 2 March, 2021; v1 submitted 3 August, 2020; originally announced August 2020.

Learning Expected Reward for Switched Linear Control Systems: A Non-Asymptotic View

Authors: Muhammad Abdullah Naeem, Miroslav Pajic

Abstract: In this work, we show existence of invariant ergodic measure for switched linear dynamical systems (SLDSs) under a norm-stability assumption of system dynamics in some unbounded subset of $\mathbb{R}^{n}$. Consequently, given a stationary Markov control policy, we derive non-asymptotic bounds for learning expected reward (w.r.t the invariant ergodic measure our closed-loop system mixes to) from ti… ▽ More In this work, we show existence of invariant ergodic measure for switched linear dynamical systems (SLDSs) under a norm-stability assumption of system dynamics in some unbounded subset of $\mathbb{R}^{n}$. Consequently, given a stationary Markov control policy, we derive non-asymptotic bounds for learning expected reward (w.r.t the invariant ergodic measure our closed-loop system mixes to) from time-averages using Birkhoff's Ergodic Theorem. The presented results provide a foundation for deriving non-asymptotic analysis for average reward-based optimal control of SLDSs. Finally, we illustrate the presented theoretical results in two case-studies. △ Less

Submitted 14 June, 2020; originally announced June 2020.

Security Analysis for Distributed IoT-Based Industrial Automation

Authors: Vuk Lesi, Zivana Jakovljevic, Miroslav Pajic

Abstract: With ever-expanding computation and communication capabilities of modern embedded platforms, Internet of Things (IoT) technologies enable development of Reconfigurable Manufacturing Systems---a new generation of highly modularized industrial equipment suitable for highly-customized manufacturing. Sequential control in these systems is largely based on discrete events, while their formal execution… ▽ More With ever-expanding computation and communication capabilities of modern embedded platforms, Internet of Things (IoT) technologies enable development of Reconfigurable Manufacturing Systems---a new generation of highly modularized industrial equipment suitable for highly-customized manufacturing. Sequential control in these systems is largely based on discrete events, while their formal execution semantics is specified as Control Interpreted Petri Nets (CIPN). Despite industry-wide use of programming languages based on the CIPN formalism, formal verification of such control applications in the presence of adversarial activity is not supported. Consequently, in this paper we focus on security-aware modeling and verification challenges for CIPN-based sequential control applications. Specifically, we show how CIPN models of networked industrial IoT controllers can be transformed into Time Petri Net (TPN)-based models, and composed with plant and security-aware channel models in order to enable system-level verification of safety properties in the presence of network-based attacks. Additionally, we introduce realistic channel-specific attack models that capture adversarial behavior using nondeterminism. Moreover, we show how verification results can be utilized to introduce security patches and motivate design of attack detectors that improve overall system resiliency, and allow satisfaction of critical safety properties. Finally, we evaluate our framework on an industrial case study. △ Less

Submitted 29 May, 2020; originally announced June 2020.

Attack-Resilient State Estimation with Intermittent Data Authentication

Authors: Amir Khazraei, Miroslav Pajic

Abstract: Network-based attacks on control systems may alter sensor data delivered to the controller, effectively causing degradation in control performance. As a result, having access to accurate state estimates, even in the presence of attacks on sensor measurements, is of critical importance. In this paper, we analyze performance of resilient state estimators (RSEs) when any subset of sensors may be comp… ▽ More Network-based attacks on control systems may alter sensor data delivered to the controller, effectively causing degradation in control performance. As a result, having access to accurate state estimates, even in the presence of attacks on sensor measurements, is of critical importance. In this paper, we analyze performance of resilient state estimators (RSEs) when any subset of sensors may be compromised by a stealthy attacker. Specifically, we consider systems with the well-known l0-based RSE and two commonly used sound intrusion detectors (IDs). For linear time-invariant plants with bounded noise, we define the notion of perfect attackability (PA) when attacks may result in unbounded estimation errors while remaining undetected by the employed ID (i.e., stealthy). We derive necessary and sufficient PA conditions, showing that a system can be perfectly attackable even if the plant is stable. While PA can be prevented with the use the standard cryptographic mechanisms (e.g.,message authentication) that ensure data integrity under network-based attacks, their continuous use imposes significant communication and computational overhead. Consequently, we also study the impact that even intermittent use of data authentication has on RSE performance guarantees in the presence of stealthy attacks. We show that if messages from some of the sensors are even intermittently authenticated, stealthy attacks could not result in unbounded state estimation errors. △ Less

Submitted 16 May, 2020; originally announced May 2020.

Comments: 12 pages, 7 figures, submitted to Journal

Attack-Resilient Supervisory Control of Discrete-Event Systems: A Finite-State Transducer Approach

Authors: Yu Wang, Alper Kamil Bozkurt, Nathan Smith, Miroslav Pajic

Abstract: Resilience to sensor and actuator attacks is a major concern in the supervisory control of discrete events in cyber-physical systems (CPS). In this work, we propose a new framework to design supervisors for CPS under attacks using finite-state transducers (FSTs) to model the effects of the discrete events. FSTs can capture a general class of regular-rewriting attacks in which an attacker can nonde… ▽ More Resilience to sensor and actuator attacks is a major concern in the supervisory control of discrete events in cyber-physical systems (CPS). In this work, we propose a new framework to design supervisors for CPS under attacks using finite-state transducers (FSTs) to model the effects of the discrete events. FSTs can capture a general class of regular-rewriting attacks in which an attacker can nondeterministically rewrite sensing/actuation events according to a given regular relation. These include common insertion, deletion, event-wise replacement, and finite-memory replay attacks. We propose new theorems and algorithms with polynomial complexity to design resilient supervisors against these attacks. We also develop an open-source tool in Python based on the results and illustrate its applicability through a case study △ Less

Submitted 29 June, 2023; v1 submitted 5 April, 2019; originally announced April 2019.

An Optimal Graph-Search Method for Secure State Estimation

Authors: Xusheng Luo, Miroslav Pajic, Michael M. Zavlanos

Abstract: The growing complexity of modern Cyber-Physical Systems (CPS) and the frequent communication between their components make them vulnerable to malicious attacks. As a result, secure state estimation is a critical requirement for the control of these systems. Many existing secure state estimation methods suffer from combinatorial complexity which grows with the number of states and sensors in the sy… ▽ More The growing complexity of modern Cyber-Physical Systems (CPS) and the frequent communication between their components make them vulnerable to malicious attacks. As a result, secure state estimation is a critical requirement for the control of these systems. Many existing secure state estimation methods suffer from combinatorial complexity which grows with the number of states and sensors in the system. This complexity can be mitigated using optimization-based methods that relax the original state estimation problem, although at the cost of optimality as these methods often identify attack-free sensors as attacked. In this paper, we propose a new optimal graph-search algorithm to correctly identify malicious attacks and to securely estimate the states even in large-scale CPS modeled as linear time-invariant systems. The graph consists of layers, each one containing two nodes capturing a truth assignment of any given sensor, and directed edges connecting adjacent layers only. Then, our algorithm searches the layers of this graph incrementally, favoring directions at higher layers with more attack-free assignments, while actively managing a repository of nodes to be expanded at later iterations. The proposed search bias and the ability to revisit nodes in the repository and self-correct, allow our graph-search algorithm to reach the optimal assignment faster and tackle larger problems. We show that our algorithm is complete and optimal provided that process and measurement noises do not dominate the attack signal. Moreover, we provide numerical simulations that demonstrate the ability of our algorithm to correctly identify attacked sensors and securely reconstruct the state. Our simulations show that our method outperforms existing algorithms both in terms of optimality and execution time. △ Less

Submitted 7 October, 2020; v1 submitted 25 March, 2019; originally announced March 2019.

Comments: 16 pages, 10 figures

Security-Aware Synthesis Using Delayed-Action Games

Authors: Mahmoud Elfar, Yu Wang, Miroslav Pajic

Abstract: Stochastic multiplayer games (SMGs) have gained attention in the field of strategy synthesis for multi-agent reactive systems. However, standard SMGs are limited to modeling systems where all agents have full knowledge of the state of the game. In this paper, we introduce delayed-action games (DAGs) formalism that simulates hidden-information games (HIGs) as SMGs, where hidden information is captu… ▽ More Stochastic multiplayer games (SMGs) have gained attention in the field of strategy synthesis for multi-agent reactive systems. However, standard SMGs are limited to modeling systems where all agents have full knowledge of the state of the game. In this paper, we introduce delayed-action games (DAGs) formalism that simulates hidden-information games (HIGs) as SMGs, where hidden information is captured by delaying a player's actions. The elimination of private variables enables the usage of SMG off-the-shelf model checkers to implement HIGs. Furthermore, we demonstrate how a DAG can be decomposed into subgames that can be independently explored, utilizing parallel computation to reduce the model checking time, while alleviating the state space explosion problem that SMGs are notorious for. In addition, we propose a DAG-based framework for strategy synthesis and analysis. Finally, we demonstrate applicability of the DAG-based synthesis framework on a case study of a human-on-the-loop unmanned-aerial vehicle system under stealthy attacks, where the proposed framework is used to formally model, analyze and synthesize security-aware strategies for the system. △ Less

Submitted 29 May, 2019; v1 submitted 12 February, 2019; originally announced February 2019.

Relaxing Integrity Requirements for Attack-Resilient Cyber-Physical Systems

Authors: Ilija Jovanov, Miroslav Pajic

Abstract: The increase in network connectivity has also resulted in several high-profile attacks on cyber-physical systems. An attacker that manages to access a local network could remotely affect control performance by tampering with sensor measurements delivered to the controller. Recent results have shown that with network-based attacks, such as Man-in-the-Middle attacks, the attacker can introduce an un… ▽ More The increase in network connectivity has also resulted in several high-profile attacks on cyber-physical systems. An attacker that manages to access a local network could remotely affect control performance by tampering with sensor measurements delivered to the controller. Recent results have shown that with network-based attacks, such as Man-in-the-Middle attacks, the attacker can introduce an unbounded state estimation error if measurements from a suitable subset of sensors contain false data when delivered to the controller. While these attacks can be addressed with the standard cryptographic tools that ensure data integrity, their continuous use would introduce significant communication and computation overhead. Consequently, we study effects of intermittent data integrity guarantees on system performance under stealthy attacks. We consider linear estimators equipped with a general type of residual-based intrusion detectors (including $χ^2$ and SPRT detectors), and show that even when integrity of sensor measurements is enforced only intermittently, the attack impact is significantly limited; specifically, the state estimation error is bounded or the attacker cannot remain stealthy. Furthermore, we present methods to: (1) evaluate the effects of any given integrity enforcement policy in terms of reachable state-estimation errors for any type of stealthy attacks, and (2) design an enforcement policy that provides the desired estimation error guarantees under attack. Finally, on three automotive case studies we show that even with less than 10% of authenticated messages we can ensure satisfiable control performance in the presence of attacks. △ Less

Submitted 11 January, 2018; v1 submitted 10 July, 2017; originally announced July 2017.

Coding Schemes for Securing Cyber-Physical Systems Against Stealthy Data Injection Attacks

Authors: Fei Miao, Quanyan Zhu, Miroslav Pajic, George J. Pappas

Abstract: This paper considers a method of coding the sensor outputs in order to detect stealthy false data injection attacks. An intelligent attacker can design a sequence of data injection to sensors and actuators that pass the state estimator and statistical fault detector, based on knowledge of the system parameters. To stay undetected, the injected data should increase the state estimation errors while… ▽ More This paper considers a method of coding the sensor outputs in order to detect stealthy false data injection attacks. An intelligent attacker can design a sequence of data injection to sensors and actuators that pass the state estimator and statistical fault detector, based on knowledge of the system parameters. To stay undetected, the injected data should increase the state estimation errors while keep the estimation residues small. We employ a coding matrix to change the original sensor outputs to increase the estimation residues under intelligent data injection attacks. This is a low cost method compared with encryption schemes over all sensor measurements in communication networks. We show the conditions of a feasible coding matrix under the assumption that the attacker does not have knowledge of the exact coding matrix. An algorithm is developed to compute a feasible coding matrix, and, we show that in general, multiple feasible coding matrices exist. To defend against attackers who estimates the coding matrix via sensor and actuator measurements, time-varying coding matrices are designed according to the detection requirements. A heuristic algorithm to decide the time length of updating a coding matrix is then proposed. △ Less

Submitted 29 May, 2016; originally announced May 2016.

Comments: 12 pages, accepted, IEEE Transactions on Control of Network Systems