-
Surveillance Evasion Through Bayesian Reinforcement Learning
Authors:
Dongping Qi,
David Bindel,
Alexander Vladimirsky
Abstract:
We consider a task of surveillance-evading path-planning in a continuous setting. An Evader strives to escape from a 2D domain while minimizing the risk of detection (and immediate capture). The probability of detection is path-dependent and determined by the spatially inhomogeneous surveillance intensity, which is fixed but a priori unknown and gradually learned in the multi-episodic setting. We…
▽ More
We consider a task of surveillance-evading path-planning in a continuous setting. An Evader strives to escape from a 2D domain while minimizing the risk of detection (and immediate capture). The probability of detection is path-dependent and determined by the spatially inhomogeneous surveillance intensity, which is fixed but a priori unknown and gradually learned in the multi-episodic setting. We introduce a Bayesian reinforcement learning algorithm that relies on a Gaussian Process regression (to model the surveillance intensity function based on the information from prior episodes), numerical methods for Hamilton-Jacobi PDEs (to plan the best continuous trajectories based on the current model), and Confidence Bounds (to balance the exploration vs exploitation). We use numerical experiments and regret metrics to highlight the significant advantages of our approach compared to traditional graph-based algorithms of reinforcement learning.
△ Less
Submitted 23 February, 2023; v1 submitted 29 September, 2021;
originally announced September 2021.
-
Evasive path planning under surveillance uncertainty
Authors:
Marc Aurèle Gilles,
Alexander Vladimirsky
Abstract:
The classical setting of optimal control theory assumes full knowledge of the process dynamics and the costs associated with every control strategy. The problem becomes much harder if the controller only knows a finite set of possible running cost functions, but has no way of checking which of these running costs is actually in place. In this paper we address this challenge for a class of evasive…
▽ More
The classical setting of optimal control theory assumes full knowledge of the process dynamics and the costs associated with every control strategy. The problem becomes much harder if the controller only knows a finite set of possible running cost functions, but has no way of checking which of these running costs is actually in place. In this paper we address this challenge for a class of evasive path planning problems on a continuous domain, in which an Evader needs to reach a target while minimizing his exposure to an enemy Observer, who is in turn selecting from a finite set of known surveillance plans. Our key assumption is that both the evader and the observer need to commit to their (possibly probabilistic) strategies in advance and cannot immediately change their actions based on any newly discovered information about the opponent's current position. We consider two types of evader behavior: in the first one, a completely risk-averse evader seeks a trajectory minimizing his {\em worst-case} cumulative observability, and in the second, the evader is concerned with minimizing the {\em average-case} cumulative observability. The latter version is naturally interpreted as a semi-infinite strategic game, and we provide an efficient method for approximating its Nash equilibrium. The proposed approach draws on methods from game theory, convex optimization, optimal control, and multiobjective dynamic programming. We illustrate our algorithm using numerical examples and discuss the computational complexity, including for the generalized version with multiple evaders.
△ Less
Submitted 25 August, 2019; v1 submitted 26 December, 2018;
originally announced December 2018.
-
A bi-criteria path planning algorithm for robotics applications
Authors:
Zachary Clawson,
Xuchu Ding,
Brendan Englot,
Thomas A. Frewen,
William M. Sisson,
Alexander Vladimirsky
Abstract:
Realistic path planning applications often require optimizing with respect to several criteria simultaneously. Here we introduce an efficient algorithm for bi-criteria path planning on graphs. Our approach is based on augmenting the state space to keep track of the "budget" remaining to satisfy the constraints on secondary cost. The resulting augmented graph is acyclic and the primary cost can be…
▽ More
Realistic path planning applications often require optimizing with respect to several criteria simultaneously. Here we introduce an efficient algorithm for bi-criteria path planning on graphs. Our approach is based on augmenting the state space to keep track of the "budget" remaining to satisfy the constraints on secondary cost. The resulting augmented graph is acyclic and the primary cost can be then minimized by a simple upward sweep through budget levels. The efficiency and accuracy of our algorithm is tested on Probabilistic Roadmap graphs to minimize the distance of travel subject to a constraint on the overall threat exposure of the robot. We also present the results from field experiments illustrating the use of this approach on realistic robotic systems.
△ Less
Submitted 8 January, 2017; v1 submitted 3 November, 2015;
originally announced November 2015.
-
Optimal control with reset-renewable resources
Authors:
Ryo Takei,
Weiyan Chen,
Zachary Clawson,
Slav Kirov,
Alexander Vladimirsky
Abstract:
We consider both discrete and continuous control problems constrained by a fixed budget of some resource, which may be renewed upon entering a preferred subset of the state space. In the discrete case, we consider both deterministic and stochastic shortest path problems with full budget resets in all preferred nodes. In the continuous case, we derive augmented PDEs of optimal control, which are th…
▽ More
We consider both discrete and continuous control problems constrained by a fixed budget of some resource, which may be renewed upon entering a preferred subset of the state space. In the discrete case, we consider both deterministic and stochastic shortest path problems with full budget resets in all preferred nodes. In the continuous case, we derive augmented PDEs of optimal control, which are then solved numerically on the extended state space with a full/instantaneous budget reset on the preferred subset. We introduce an iterative algorithm for solving these problems efficiently. The method's performance is demonstrated on a range of computational examples, including the optimal path planning with constraints on prolonged visibility by a static enemy observer.
In addition, we also develop an algorithm that works on the original state space to solve a related but simpler problem: finding the subsets of the domain "reachable-within-the-budget".
This manuscript is an extended version of the paper accepted for publication by SIAM J. on Control and Optimization. In the journal version, Section 3 and the Appendix were omitted due to space limitations.
△ Less
Submitted 26 September, 2014; v1 submitted 27 October, 2011;
originally announced October 2011.
