Showing 1–32 of 32 results for author: Dawson, C

RADIUM: Predicting and Repairing End-to-End Robot Failures using Gradient-Accelerated Sampling

Authors: Charles Dawson, Anjali Parashar, Chuchu Fan

Abstract: Before autonomous systems can be deployed in safety-critical applications, we must be able to understand and verify the safety of these systems. For cases where the risk or cost of real-world testing is prohibitive, we propose a simulation-based framework for a) predicting ways in which an autonomous system is likely to fail and b) automatically adjusting the system's design and control policy to… ▽ More Before autonomous systems can be deployed in safety-critical applications, we must be able to understand and verify the safety of these systems. For cases where the risk or cost of real-world testing is prohibitive, we propose a simulation-based framework for a) predicting ways in which an autonomous system is likely to fail and b) automatically adjusting the system's design and control policy to preemptively mitigate those failures. Existing tools for failure prediction struggle to search over high-dimensional environmental parameters, cannot efficiently handle end-to-end testing for systems with vision in the loop, and provide little guidance on how to mitigate failures once they are discovered. We approach this problem through the lens of approximate Bayesian inference and use differentiable simulation and rendering for efficient failure case prediction and repair. For cases where a differentiable simulator is not available, we provide a gradient-free version of our algorithm, and we include a theoretical and empirical evaluation of the trade-offs between gradient-based and gradient-free methods. We apply our approach on a range of robotics and control problems, including optimizing search patterns for robot swarms, UAV formation control, and robust network control. Compared to optimization-based falsification methods, our method predicts a more diverse, representative set of failure modes, and we find that our use of differentiable simulation yields solutions that have up to 10x lower cost and requires up to 2x fewer iterations to converge relative to gradient-free techniques. In hardware experiments, we find that repairing control policies using our method leads to a 5x robustness improvement. Accompanying code and video can be found at https://mit-realm.github.io/radium/ △ Less

Submitted 4 April, 2024; originally announced April 2024.

Storm Surge Modeling in the AI ERA: Using LSTM-based Machine Learning for Enhancing Forecasting Accuracy

Authors: Stefanos Giaremis, Noujoud Nader, Clint Dawson, Hartmut Kaiser, Carola Kaiser, Efstratios Nikidis

Abstract: Physics simulation results of natural processes usually do not fully capture the real world. This is caused for instance by limits in what physical processes are simulated and to what accuracy. In this work we propose and analyze the use of an LSTM-based deep learning network machine learning (ML) architecture for capturing and predicting the behavior of the systemic error for storm surge forecast… ▽ More Physics simulation results of natural processes usually do not fully capture the real world. This is caused for instance by limits in what physical processes are simulated and to what accuracy. In this work we propose and analyze the use of an LSTM-based deep learning network machine learning (ML) architecture for capturing and predicting the behavior of the systemic error for storm surge forecast models with respect to real-world water height observations from gauge stations during hurricane events. The overall goal of this work is to predict the systemic error of the physics model and use it to improve the accuracy of the simulation results post factum. We trained our proposed ML model on a dataset of 61 historical storms in the coastal regions of the U.S. and we tested its performance in bias correcting modeled water level data predictions from hurricane Ian (2022). We show that our model can consistently improve the forecasting accuracy for hurricane Ian -- unknown to the ML model -- at all gauge station coordinates used for the initial data. Moreover, by examining the impact of using different subsets of the initial training dataset, containing a number of relatively similar or different hurricanes in terms of hurricane track, we found that we can obtain similar quality of bias correction by only using a subset of six hurricanes. This is an important result that implies the possibility to apply a pre-trained ML model to real-time hurricane forecasting results with the goal of bias correcting and improving the produced simulation accuracy. The presented work is an important first step in creating a bias correction system for real-time storm surge forecasting applicable to the full simulation area. It also presents a highly transferable and operationally applicable methodology for improving the accuracy in a wide range of physics simulation scenarios beyond storm surge forecasting. △ Less

Submitted 7 March, 2024; originally announced March 2024.

Active Disruption Avoidance and Trajectory Design for Tokamak Ramp-downs with Neural Differential Equations and Reinforcement Learning

Authors: Allen M. Wang, Oswin So, Charles Dawson, Darren T. Garnier, Cristina Rea, Chuchu Fan

Abstract: The tokamak offers a promising path to fusion energy, but plasma disruptions pose a major economic risk, motivating considerable advances in disruption avoidance. This work develops a reinforcement learning approach to this problem by training a policy to safely ramp-down the plasma current while avoiding limits on a number of quantities correlated with disruptions. The policy training environment… ▽ More The tokamak offers a promising path to fusion energy, but plasma disruptions pose a major economic risk, motivating considerable advances in disruption avoidance. This work develops a reinforcement learning approach to this problem by training a policy to safely ramp-down the plasma current while avoiding limits on a number of quantities correlated with disruptions. The policy training environment is a hybrid physics and machine learning model trained on simulations of the SPARC primary reference discharge (PRD) ramp-down, an upcoming burning plasma scenario which we use as a testbed. To address physics uncertainty and model inaccuracies, the simulation environment is massively parallelized on GPU with randomized physics parameters during policy training. The trained policy is then successfully transferred to a higher fidelity simulator where it successfully ramps down the plasma while avoiding user-specified disruptive limits. We also address the crucial issue of safety criticality by demonstrating that a constraint-conditioned policy can be used as a trajectory design assistant to design a library of feed-forward trajectories to handle different physics conditions and user settings. As a library of trajectories is more interpretable and verifiable offline, we argue such an approach is a promising path for leveraging the capabilities of reinforcement learning in the safety-critical context of burning plasma tokamaks. Finally, we demonstrate how the training environment can be a useful platform for other feed-forward optimization approaches by using an evolutionary algorithm to perform optimization of feed-forward trajectories that are robust to physics uncertainty △ Less

Submitted 14 February, 2024; originally announced February 2024.

Learning Safe Control for Multi-Robot Systems: Methods, Verification, and Open Challenges

Authors: Kunal Garg, Songyuan Zhang, Oswin So, Charles Dawson, Chuchu Fan

Abstract: In this survey, we review the recent advances in control design methods for robotic multi-agent systems (MAS), focussing on learning-based methods with safety considerations. We start by reviewing various notions of safety and liveness properties, and modeling frameworks used for problem formulation of MAS. Then we provide a comprehensive review of learning-based methods for safe control design fo… ▽ More In this survey, we review the recent advances in control design methods for robotic multi-agent systems (MAS), focussing on learning-based methods with safety considerations. We start by reviewing various notions of safety and liveness properties, and modeling frameworks used for problem formulation of MAS. Then we provide a comprehensive review of learning-based methods for safe control design for multi-robot systems. We start with various types of shielding-based methods, such as safety certificates, predictive filters, and reachability tools. Then, we review the current state of control barrier certificate learning in both a centralized and distributed manner, followed by a comprehensive review of multi-agent reinforcement learning with a particular focus on safety. Next, we discuss the state-of-the-art verification tools for the correctness of learning-based methods. Based on the capabilities and the limitations of the state of the art methods in learning and verification for MAS, we identify various broad themes for open challenges: how to design methods that can achieve good performance along with safety guarantees; how to decompose single-agent based centralized methods for MAS; how to account for communication-related practical issues; and how to assess transfer of theoretical guarantees to practice. △ Less

Submitted 22 November, 2023; originally announced November 2023.

Comments: Submitted to Annual Reviews in Control

Evaluating Adversarial Robustness with Expected Viable Performance

Authors: Ryan McCoppin, Colin Dawson, Sean M. Kennedy, Leslie M. Blaha

Abstract: We introduce a metric for evaluating the robustness of a classifier, with particular attention to adversarial perturbations, in terms of expected functionality with respect to possible adversarial perturbations. A classifier is assumed to be non-functional (that is, has a functionality of zero) with respect to a perturbation bound if a conventional measure of performance, such as classification ac… ▽ More We introduce a metric for evaluating the robustness of a classifier, with particular attention to adversarial perturbations, in terms of expected functionality with respect to possible adversarial perturbations. A classifier is assumed to be non-functional (that is, has a functionality of zero) with respect to a perturbation bound if a conventional measure of performance, such as classification accuracy, is less than a minimally viable threshold when the classifier is tested on examples from that perturbation bound. Defining robustness in terms of an expected value is motivated by a domain general approach to robustness quantification. △ Less

Submitted 18 September, 2023; originally announced September 2023.

Comments: Accepted at the 22nd International Conference on Machine Learning and Applications (IEEE 2023)

A Bayesian approach to breaking things: efficiently predicting and repairing failure modes via sampling

Authors: Charles Dawson, Chuchu Fan

Abstract: Before autonomous systems can be deployed in safety-critical applications, we must be able to understand and verify the safety of these systems. For cases where the risk or cost of real-world testing is prohibitive, we propose a simulation-based framework for a) predicting ways in which an autonomous system is likely to fail and b) automatically adjusting the system's design to preemptively mitiga… ▽ More Before autonomous systems can be deployed in safety-critical applications, we must be able to understand and verify the safety of these systems. For cases where the risk or cost of real-world testing is prohibitive, we propose a simulation-based framework for a) predicting ways in which an autonomous system is likely to fail and b) automatically adjusting the system's design to preemptively mitigate those failures. We frame this problem through the lens of approximate Bayesian inference and use differentiable simulation for efficient failure case prediction and repair. We apply our approach on a range of robotics and control problems, including optimizing search patterns for robot swarms and reducing the severity of outages in power transmission networks. Compared to optimization-based falsification techniques, our method predicts a more diverse, representative set of failure modes, and we also find that our use of differentiable simulation yields solutions that have up to 10x lower cost and requires up to 2x fewer iterations to converge relative to gradient-free techniques. Code and videos can be found at https://mit-realm.github.io/breaking-things/ △ Less

Submitted 14 September, 2023; originally announced September 2023.

Comments: To appear at the 2023 Conference on Robot Learning (CoRL)

Rapid Flood Inundation Forecast Using Fourier Neural Operator

Authors: Alexander Y. Sun, Zhi Li, Wonhyun Lee, Qixing Huang, Bridget R. Scanlon, Clint Dawson

Abstract: Flood inundation forecast provides critical information for emergency planning before and during flood events. Real time flood inundation forecast tools are still lacking. High-resolution hydrodynamic modeling has become more accessible in recent years, however, predicting flood extents at the street and building levels in real-time is still computationally demanding. Here we present a hybrid proc… ▽ More Flood inundation forecast provides critical information for emergency planning before and during flood events. Real time flood inundation forecast tools are still lacking. High-resolution hydrodynamic modeling has become more accessible in recent years, however, predicting flood extents at the street and building levels in real-time is still computationally demanding. Here we present a hybrid process-based and data-driven machine learning (ML) approach for flood extent and inundation depth prediction. We used the Fourier neural operator (FNO), a highly efficient ML method, for surrogate modeling. The FNO model is demonstrated over an urban area in Houston (Texas, U.S.) by training using simulated water depths (in 15-min intervals) from six historical storm events and then tested over two holdout events. Results show FNO outperforms the baseline U-Net model. It maintains high predictability at all lead times tested (up to 3 hrs) and performs well when applying to new sites, suggesting strong generalization skill. △ Less

Submitted 29 July, 2023; originally announced July 2023.

Comments: Artificial Intelligence (AI) and Humanitarian Assistance and Disaster Recovery (HADR) workshop, ICCV 2023 in Paris, France

A Discontinuous Galerkin Finite Element Model for Compound Flood Simulations

Authors: Chayanon Wichitrnithed, Eirik Valseth, Ethan J. Kubatko, Younghun Kang, Mackenzie Hudson, Clint Dawson

Abstract: Recent tropical cyclones, e.g., Hurricane Harvey (2017), have lead to significant rainfall and resulting runoff with accompanying flooding. When the runoff interacts with storm surge, the resulting floods can be greatly amplified and lead to effects that cannot be modeled by simple superposition of its distinctive sources. In an effort to develop accurate numerical simulations of runoff, surge, an… ▽ More Recent tropical cyclones, e.g., Hurricane Harvey (2017), have lead to significant rainfall and resulting runoff with accompanying flooding. When the runoff interacts with storm surge, the resulting floods can be greatly amplified and lead to effects that cannot be modeled by simple superposition of its distinctive sources. In an effort to develop accurate numerical simulations of runoff, surge, and compounding floods, we develop a local discontinuous Galerkin method for modified shallow water equations. In this modification, nonzero sources to the continuity equation are included to incorporate rainfall into the model using parametric rainfall models from literature as well as hindcast data. The discontinuous Galerkin spatial discretization is accompanied with a strong stability preserving explicit Runge Kutta time integrator. Hence, temporal stability is ensured through the CFL condition and we exploit the embarrassingly parallel nature of the developed method using MPI parallelization. We demonstrate the capabilities of the developed method though a sequence of physically relevant numerical tests, including small scale test cases based on laboratory measurements and large scale experiments with Hurricane Harvey in the Gulf of Mexico. The results highlight the conservation properties and robustness of the developed method and show the potential of compound flood modeling using our approach. △ Less

Submitted 17 December, 2023; v1 submitted 26 July, 2023; originally announced July 2023.

AutoTAMP: Autoregressive Task and Motion Planning with LLMs as Translators and Checkers

Authors: Yongchao Chen, Jacob Arkin, Charles Dawson, Yang Zhang, Nicholas Roy, Chuchu Fan

Abstract: For effective human-robot interaction, robots need to understand, plan, and execute complex, long-horizon tasks described by natural language. Recent advances in large language models (LLMs) have shown promise for translating natural language into robot action sequences for complex tasks. However, existing approaches either translate the natural language directly into robot trajectories or factor… ▽ More For effective human-robot interaction, robots need to understand, plan, and execute complex, long-horizon tasks described by natural language. Recent advances in large language models (LLMs) have shown promise for translating natural language into robot action sequences for complex tasks. However, existing approaches either translate the natural language directly into robot trajectories or factor the inference process by decomposing language into task sub-goals and relying on a motion planner to execute each sub-goal. When complex environmental and temporal constraints are involved, inference over planning tasks must be performed jointly with motion plans using traditional task-and-motion planning (TAMP) algorithms, making factorization into subgoals untenable. Rather than using LLMs to directly plan task sub-goals, we instead perform few-shot translation from natural language task descriptions to an intermediate task representation that can then be consumed by a TAMP algorithm to jointly solve the task and motion plan. To improve translation, we automatically detect and correct both syntactic and semantic errors via autoregressive re-prompting, resulting in significant improvements in task completion. We show that our approach outperforms several methods using LLMs as planners in complex task domains. See our project website https://yongchao98.github.io/MIT-REALM-AutoTAMP/ for prompts, videos, and code. △ Less

Submitted 21 March, 2024; v1 submitted 10 June, 2023; originally announced June 2023.

Comments: 8 pages, 4 figures

Journal ref: The 2024 International Conference on Robotics and Automation

Shield Model Predictive Path Integral: A Computationally Efficient Robust MPC Approach Using Control Barrier Functions

Authors: Ji Yin, Charles Dawson, Chuchu Fan, Panagiotis Tsiotras

Abstract: Model Predictive Path Integral (MPPI) control is a type of sampling-based model predictive control that simulates thousands of trajectories and uses these trajectories to synthesize optimal controls on-the-fly. In practice, however, MPPI encounters problems limiting its application. For instance, it has been observed that MPPI tends to make poor decisions if unmodeled dynamics or environmental dis… ▽ More Model Predictive Path Integral (MPPI) control is a type of sampling-based model predictive control that simulates thousands of trajectories and uses these trajectories to synthesize optimal controls on-the-fly. In practice, however, MPPI encounters problems limiting its application. For instance, it has been observed that MPPI tends to make poor decisions if unmodeled dynamics or environmental disturbances exist, preventing its use in safety-critical applications. Moreover, the multi-threaded simulations used by MPPI require significant onboard computational resources, making the algorithm inaccessible to robots without modern GPUs. To alleviate these issues, we propose a novel (Shield-MPPI) algorithm that provides robustness against unpredicted disturbances and achieves real-time planning using a much smaller number of parallel simulations on regular CPUs. The novel Shield-MPPI algorithm is tested on an aggressive autonomous racing platform both in simulation and using experiments. The results show that the proposed controller greatly reduces the number of constraint violations compared to state-of-the-art robust MPPI variants and stochastic MPC methods. △ Less

Submitted 22 February, 2023; originally announced February 2023.

Comments: 8 pages, 7 figures. Submitted to RA-L for review

Chance-Constrained Trajectory Optimization for High-DOF Robots in Uncertain Environments

Authors: Charles Dawson, Ashkan Jasour, Andreas Hofmann, Brian Williams

Abstract: Many practical applications of robotics require systems that can operate safely despite uncertainty. In the context of motion planning, two types of uncertainty are particularly important when planning safe robot trajectories. The first is environmental uncertainty -- uncertainty in the locations of nearby obstacles, stemming from sensor noise or (in the case of obstacles' future locations) predic… ▽ More Many practical applications of robotics require systems that can operate safely despite uncertainty. In the context of motion planning, two types of uncertainty are particularly important when planning safe robot trajectories. The first is environmental uncertainty -- uncertainty in the locations of nearby obstacles, stemming from sensor noise or (in the case of obstacles' future locations) prediction error. The second class of uncertainty is uncertainty in the robots own state, typically caused by tracking or estimation error. To achieve high levels of safety, it is necessary for robots to consider both of these sources of uncertainty. In this paper, we propose a risk-bounded trajectory optimization algorithm, known as Sequential Convex Optimization with Risk Optimization (SCORA), to solve chance-constrained motion planning problems despite both environmental uncertainty and tracking error. Through experiments in simulation, we demonstrate that SCORA significantly outperforms state-of-the-art risk-aware motion planners both in planning time and in the safety of the resulting trajectories. △ Less

Submitted 31 January, 2023; originally announced February 2023.

Barrier functions enable safety-conscious force-feedback control

Authors: Charles Dawson, Austin Garrett, Falk Pollok, Yang Zhang, Chuchu Fan

Abstract: In order to be effective partners for humans, robots must become increasingly comfortable with making contact with their environment. Unfortunately, it is hard for robots to distinguish between ``just enough'' and ``too much'' force: some force is required to accomplish the task but too much might damage equipment or injure humans. Traditional approaches to designing compliant force-feedback contr… ▽ More In order to be effective partners for humans, robots must become increasingly comfortable with making contact with their environment. Unfortunately, it is hard for robots to distinguish between ``just enough'' and ``too much'' force: some force is required to accomplish the task but too much might damage equipment or injure humans. Traditional approaches to designing compliant force-feedback controllers, such as stiffness control, require difficult hand-tuning of control parameters and make it difficult to build safe, effective robot collaborators. In this paper, we propose a novel yet easy-to-implement force feedback controller that uses control barrier functions (CBFs) to derive a compliant controller directly from users' specifications of the maximum allowable forces and torques. We compare our approach to traditional stiffness control to demonstrate potential advantages of our control architecture, and we demonstrate the effectiveness of our controller on an example human-robot collaboration task: cooperative manipulation of a bulky object. △ Less

Submitted 25 September, 2022; originally announced September 2022.

Enforcing safety for vision-based controllers via Control Barrier Functions and Neural Radiance Fields

Authors: Mukun Tong, Charles Dawson, Chuchu Fan

Abstract: To navigate complex environments, robots must increasingly use high-dimensional visual feedback (e.g. images) for control. However, relying on high-dimensional image data to make control decisions raises important questions; particularly, how might we prove the safety of a visual-feedback controller? Control barrier functions (CBFs) are powerful tools for certifying the safety of feedback controll… ▽ More To navigate complex environments, robots must increasingly use high-dimensional visual feedback (e.g. images) for control. However, relying on high-dimensional image data to make control decisions raises important questions; particularly, how might we prove the safety of a visual-feedback controller? Control barrier functions (CBFs) are powerful tools for certifying the safety of feedback controllers in the state-feedback setting, but CBFs have traditionally been poorly-suited to visual feedback control due to the need to predict future observations in order to evaluate the barrier function. In this work, we solve this issue by leveraging recent advances in neural radiance fields (NeRFs), which learn implicit representations of 3D scenes and can render images from previously-unseen camera perspectives, to provide single-step visual foresight for a CBF-based controller. This novel combination is able to filter out unsafe actions and intervene to preserve safety. We demonstrate the effect of our controller in real-time simulation experiments where it successfully prevents the robot from taking dangerous actions. △ Less

Submitted 28 February, 2023; v1 submitted 25 September, 2022; originally announced September 2022.

Comments: Accepted to ICRA 2023

A Framework for Flexible Peak Storm Surge Prediction

Authors: Benjamin Pachev, Prateek Arora, Carlos del-Castillo-Negrete, Eirik Valseth, Clint Dawson

Abstract: Storm surge is a major natural hazard in coastal regions, responsible both for significant property damage and loss of life. Accurate, efficient models of storm surge are needed both to assess long-term risk and to guide emergency management decisions. While high-fidelity regional- and global-ocean circulation models such as the ADvanced CIRCulation (ADCIRC) model can accurately predict storm surg… ▽ More Storm surge is a major natural hazard in coastal regions, responsible both for significant property damage and loss of life. Accurate, efficient models of storm surge are needed both to assess long-term risk and to guide emergency management decisions. While high-fidelity regional- and global-ocean circulation models such as the ADvanced CIRCulation (ADCIRC) model can accurately predict storm surge, they are very computationally expensive. Here we develop a novel surrogate model for peak storm surge prediction based on a multi-stage approach. In the first stage, points are classified as inundated or not. In the second, the level of inundation is predicted . Additionally, we propose a new formulation of the surrogate problem in which storm surge is predicted independently for each point. This allows for predictions to be made directly for locations not present in the training data, and significantly reduces the number of model parameters. We demonstrate our modeling framework on two study areas: the Texas coast and the northern portion of the Alaskan coast. For Texas, the model is trained with a database of 446 synthetic hurricanes. The model is able to accurately match ADCIRC predictions on a test set of synthetic storms. We further present a test of the model on Hurricanes Ike (2008) and Harvey (2017). For Alaska, the model is trained on a dataset of 109 historical surge events. We test the surrogate model on actual surge events including the recent Typhoon Merbok (2022) that take place after the events in the training data. For both datasets, the surrogate model achieves similar performance to ADCIRC on real events when compared to observational data. In both cases, the surrogate models are many orders of magnitude faster than ADCIRC. △ Less

Submitted 31 March, 2023; v1 submitted 27 April, 2022; originally announced April 2022.

Certifiable Robot Design Optimization using Differentiable Programming

Authors: Charles Dawson, Chuchu Fan

Abstract: There is a growing need for computational tools to automatically design and verify autonomous systems, especially complex robotic systems involving perception, planning, control, and hardware in the autonomy stack. Differentiable programming has recently emerged as powerful tool for modeling and optimization. However, very few studies have been done to understand how differentiable programming can… ▽ More There is a growing need for computational tools to automatically design and verify autonomous systems, especially complex robotic systems involving perception, planning, control, and hardware in the autonomy stack. Differentiable programming has recently emerged as powerful tool for modeling and optimization. However, very few studies have been done to understand how differentiable programming can be used for robust, certifiable end-to-end design optimization. In this paper, we fill this gap by combining differentiable programming for robot design optimization with a novel statistical framework for certifying the robustness of optimized designs. Our framework can conduct end-to-end optimization and robustness certification for robotics systems, enabling simultaneous optimization of navigation, perception, planning, control, and hardware subsystems. Using simulation and hardware experiments, we show how our tool can be used to solve practical problems in robotics. First, we optimize sensor placements for robot navigation (a design with 5 subsystems and 6 tunable parameters) in under 5 minutes to achieve an 8.4x performance improvement compared to the initial design. Second, we solve a multi-agent collaborative manipulation task (3 subsystems and 454 parameters) in under an hour to achieve a 44% performance improvement over the initial design. We find that differentiable programming enables much faster (32% and 20x, respectively for each example) optimization than approximate gradient methods. We certify the robustness of each design and successfully deploy the optimized designs in hardware. An open-source implementation is available at https://github.com/MIT-REALM/architect △ Less

Submitted 22 April, 2022; originally announced April 2022.

Comments: Accepted for RSS 2022

Robust Counterexample-guided Optimization for Planning from Differentiable Temporal Logic

Authors: Charles Dawson, Chuchu Fan

Abstract: Signal temporal logic (STL) provides a powerful, flexible framework for specifying complex autonomy tasks; however, existing methods for planning based on STL specifications have difficulty scaling to long-horizon tasks and are not robust to external disturbances. In this paper, we present an algorithm for finding robust plans that satisfy STL specifications. Our method alternates between local op… ▽ More Signal temporal logic (STL) provides a powerful, flexible framework for specifying complex autonomy tasks; however, existing methods for planning based on STL specifications have difficulty scaling to long-horizon tasks and are not robust to external disturbances. In this paper, we present an algorithm for finding robust plans that satisfy STL specifications. Our method alternates between local optimization and local falsification, using automatically differentiable temporal logic to iteratively optimize its plan in response to counterexamples found during the falsification process. We benchmark our counterexample-guided planning method against state-of-the-art planning methods on two long-horizon satellite rendezvous missions, showing that our method finds high-quality plans that satisfy STL specifications despite adversarial disturbances. We find that our method consistently finds plans that are robust to adversarial disturbances and requires less than half the time of competing methods. We provide an implementation of our planner at https://github.com/MIT-REALM/architect. △ Less

Submitted 3 March, 2022; originally announced March 2022.

Safe Control with Learned Certificates: A Survey of Neural Lyapunov, Barrier, and Contraction methods

Authors: Charles Dawson, Sicun Gao, Chuchu Fan

Abstract: Learning-enabled control systems have demonstrated impressive empirical performance on challenging control problems in robotics, but this performance comes at the cost of reduced transparency and lack of guarantees on the safety or stability of the learned controllers. In recent years, new techniques have emerged to provide these guarantees by learning certificates alongside control policies -- th… ▽ More Learning-enabled control systems have demonstrated impressive empirical performance on challenging control problems in robotics, but this performance comes at the cost of reduced transparency and lack of guarantees on the safety or stability of the learned controllers. In recent years, new techniques have emerged to provide these guarantees by learning certificates alongside control policies -- these certificates provide concise, data-driven proofs that guarantee the safety and stability of the learned control system. These methods not only allow the user to verify the safety of a learned controller but also provide supervision during training, allowing safety and stability requirements to influence the training process itself. In this paper, we provide a comprehensive survey of this rapidly developing field of certificate learning. We hope that this paper will serve as an accessible introduction to the theory and practice of certificate learning, both to those who wish to apply these tools to practical robotics problems and to those who wish to dive more deeply into the theory of learning for control. △ Less

Submitted 20 December, 2022; v1 submitted 23 February, 2022; originally announced February 2022.

Comments: Accepted at IEEE Transactions on Robotics. Supplementary code available at https://github.com/MIT-REALM/neural_clbf

Extending FEniCS to Work in Higher Dimensions Using Tensor Product Finite Elements

Authors: Mark Loveland, Eirik Valseth, Matt Lukac, Clint Dawson

Abstract: We present a method to extend the finite element library FEniCS to solve problems with domains in dimensions above three by constructing tensor product finite elements. This methodology only requires that the high dimensional domain is structured as a Cartesian product of two lower dimensional subdomains. In this study we consider Dirichlet problems for scalar linear partial differential equations… ▽ More We present a method to extend the finite element library FEniCS to solve problems with domains in dimensions above three by constructing tensor product finite elements. This methodology only requires that the high dimensional domain is structured as a Cartesian product of two lower dimensional subdomains. In this study we consider Dirichlet problems for scalar linear partial differential equations, though the methodology can be extended to non-linear problems. The utilization of tensor product finite elements allows us to construct a global system of linear algebraic equations that only relies on the finite element infrastructure of the lower dimensional subdomains contained in FEniCS. We demonstrate the effectiveness of our methodology in four distinctive test cases. The first test case is a Poisson equation posed in a four dimensional domain which is a Cartesian product of two unit squares solved using the classical Galerkin finite element method. The second test case is the wave equation in space-time, where the computational domain is a Cartesian product of a two dimensional space grid and a one dimensional time interval. In this second case we also employ the Galerkin method. The third test case is an advection dominated advection-diffusion equation where the global domain is a Cartesian product of two one dimensional intervals in which the streamline upwind Petrov-Galerkin method is applied to ensure discrete stability. The final test case uses the Galerkin approach to solve a Poisson problem on a Cartesian product of two intervals with a spatially varying, non-separable diffusivity term. In all cases, a p=1 basis is used and optimal L^2 convergence rates of order h^{p+1} of the errors are achieved with respect to h refinement △ Less

Submitted 25 May, 2022; v1 submitted 1 February, 2022; originally announced February 2022.

Learning Safe, Generalizable Perception-based Hybrid Control with Certificates

Authors: Charles Dawson, Bethany Lowenkamp, Dylan Goff, Chuchu Fan

Abstract: Many robotic tasks require high-dimensional sensors such as cameras and Lidar to navigate complex environments, but developing certifiably safe feedback controllers around these sensors remains a challenging open problem, particularly when learning is involved. Previous works have proved the safety of perception-feedback controllers by separating the perception and control subsystems and making st… ▽ More Many robotic tasks require high-dimensional sensors such as cameras and Lidar to navigate complex environments, but developing certifiably safe feedback controllers around these sensors remains a challenging open problem, particularly when learning is involved. Previous works have proved the safety of perception-feedback controllers by separating the perception and control subsystems and making strong assumptions on the abilities of the perception subsystem. In this work, we introduce a novel learning-enabled perception-feedback hybrid controller, where we use Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs) to show the safety and liveness of a full-stack perception-feedback controller. We use neural networks to learn a CBF and CLF for the full-stack system directly in the observation space of the robot, without the need to assume a separate perception-based state estimator. Our hybrid controller, called LOCUS (Learning-enabled Observation-feedback Control Using Switching), can safely navigate unknown environments, consistently reach its goal, and generalizes safely to environments outside of the training dataset. We demonstrate LOCUS in experiments both in simulation and in hardware, where it successfully navigates a changing environment using feedback from a Lidar sensor. △ Less

Submitted 3 January, 2022; originally announced January 2022.

Comments: Accepted for publication in RA-L

Safe Nonlinear Control Using Robust Neural Lyapunov-Barrier Functions

Authors: Charles Dawson, Zengyi Qin, Sicun Gao, Chuchu Fan

Abstract: Safety and stability are common requirements for robotic control systems; however, designing safe, stable controllers remains difficult for nonlinear and uncertain models. We develop a model-based learning approach to synthesize robust feedback controllers with safety and stability guarantees. We take inspiration from robust convex optimization and Lyapunov theory to define robust control Lyapunov… ▽ More Safety and stability are common requirements for robotic control systems; however, designing safe, stable controllers remains difficult for nonlinear and uncertain models. We develop a model-based learning approach to synthesize robust feedback controllers with safety and stability guarantees. We take inspiration from robust convex optimization and Lyapunov theory to define robust control Lyapunov barrier functions that generalize despite model uncertainty. We demonstrate our approach in simulation on problems including car trajectory tracking, nonlinear control with obstacle avoidance, satellite rendezvous with safety constraints, and flight control with a learned ground effect model. Simulation results show that our approach yields controllers that match or exceed the capabilities of robust MPC while reducing computational costs by an order of magnitude. △ Less

Submitted 6 October, 2021; v1 submitted 14 September, 2021; originally announced September 2021.

Comments: Accepted to the 5th Annual Conference on Robot Learning (CoRL 21)

SCA-Net: A Self-Correcting Two-Layer Autoencoder for Hyper-spectral Unmixing

Authors: Gurpreet Singh, Soumyajit Gupta, Clint Dawson

Abstract: Hyperspectral unmixing involves separating a pixel as a weighted combination of its constituent endmembers and corresponding fractional abundances, with the current state of the art results achieved by neural models on benchmark datasets. However, these networks are severely over-parameterized and consequently, the invariant endmember spectra extracted as decoder weights have a high variance over… ▽ More Hyperspectral unmixing involves separating a pixel as a weighted combination of its constituent endmembers and corresponding fractional abundances, with the current state of the art results achieved by neural models on benchmark datasets. However, these networks are severely over-parameterized and consequently, the invariant endmember spectra extracted as decoder weights have a high variance over multiple runs. These approaches perform substantial post-processing while requiring an exact specification of the number of endmembers and specialized initialization of weights from other algorithms like VCA. We show for the first time that a two-layer autoencoder (SCA), with $2FK$ parameters ($F$ features, $K$ endmembers), achieves error metrics that are scales apart ($10^{-5})$ from previously reported values $(10^{-2})$. SCA converges to this low error solution starting from a random initialization of weights. We also show that SCA, based upon a bi-orthogonal representation, performs a self-correction when the number of endmembers are over-specified. Numerical experiments on Samson, Jasper, and Urban datasets demonstrate that SCA outperforms previously reported error metrics for all the cases while being robust to noise and outliers. △ Less

Submitted 28 October, 2021; v1 submitted 10 February, 2021; originally announced February 2021.

A Hybrid 2-stage Neural Optimization for Pareto Front Extraction

Authors: Gurpreet Singh, Soumyajit Gupta, Matthew Lease, Clint Dawson

Abstract: Classification, recommendation, and ranking problems often involve competing goals with additional constraints (e.g., to satisfy fairness or diversity criteria). Such optimization problems are quite challenging, often involving non-convex functions along with considerations of user preferences in balancing trade-offs. Pareto solutions represent optimal frontiers for jointly optimizing multiple com… ▽ More Classification, recommendation, and ranking problems often involve competing goals with additional constraints (e.g., to satisfy fairness or diversity criteria). Such optimization problems are quite challenging, often involving non-convex functions along with considerations of user preferences in balancing trade-offs. Pareto solutions represent optimal frontiers for jointly optimizing multiple competing objectives. A major obstacle for frequently used linear-scalarization strategies is that the resulting optimization problem might not always converge to a global optimum. Furthermore, such methods only return one solution point per run. A Pareto solution set is a subset of all such global optima over multiple runs for different trade-off choices. Therefore, a Pareto front can only be guaranteed with multiple runs of the linear-scalarization problem, where all runs converge to their respective global optima. Consequently, extracting a Pareto front for practical problems is computationally intractable with substantial computational overheads, limited scalability, and reduced accuracy. We propose a robust, low cost, two-stage, hybrid neural Pareto optimization approach that is accurate and scales (compute space and time) with data dimensions, as well as number of functions and constraints. The first stage (neural network) efficiently extracts a weak Pareto front, using Fritz-John conditions as the discriminator, with no assumptions of convexity on the objectives or constraints. The second stage (efficient Pareto filter) extracts the strong Pareto optimal subset given the weak front from stage 1. Fritz-John conditions provide us with theoretical bounds on approximation error between the true and network extracted weak Pareto front. Numerical experiments demonstrates the accuracy and efficiency on a canonical set of benchmark problems and a fairness optimization task from prior works. △ Less

Submitted 13 February, 2021; v1 submitted 27 January, 2021; originally announced January 2021.

A Stable Mixed FE Method for Nearly Incompressible Linear Elastostatics

Authors: Eirik Valseth, Albert Romkes, Austin R. Kaul, Clint Dawson

Abstract: We present a new, stable, mixed finite element (FE) method for linear elastostatics of nearly incompressible solids. The method is the automatic variationally stable FE (AVS-FE) method of Calo, Romkes and Valseth, in which we consider a Petrov-Galerkin weak formulation where the stress and displacement variables are in the space H(div)xH1, respectively. This allows us to employ a fully conforming… ▽ More We present a new, stable, mixed finite element (FE) method for linear elastostatics of nearly incompressible solids. The method is the automatic variationally stable FE (AVS-FE) method of Calo, Romkes and Valseth, in which we consider a Petrov-Galerkin weak formulation where the stress and displacement variables are in the space H(div)xH1, respectively. This allows us to employ a fully conforming FE discretization for any elastic solid using classical FE subspaces of H(div) and H1. Hence, the resulting FE approximation yields both continuous stresses and displacements. To ensure stability of the method, we employ the philosophy of the discontinuous Petrov-Galerkin (DPG) method of Demkowicz and Gopalakrishnan and use optimal test spaces. Thus, the resulting FE discretization is stable even as the Poisson ratio approaches 0.5, and the system of linear algebraic equations is symmetric and positive definite. Our method also comes with a built-in a posteriori error estimator as well as well as indicators which are used to drive mesh adaptive refinements. We present several numerical verifications of our method including comparisons to existing FE technologies. △ Less

Submitted 7 May, 2021; v1 submitted 15 January, 2021; originally announced January 2021.

Range-Net: A High Precision Streaming SVD for Big Data Applications

Authors: Gurpreet Singh, Soumyajit Gupta, Matthew Lease, Clint Dawson

Abstract: In a Big Data setting computing the dominant SVD factors is restrictive due to the main memory requirements. Recently introduced streaming Randomized SVD schemes work under the restrictive assumption that the singular value spectrum of the data has exponential decay. This is seldom true for any practical data. Although these methods are claimed to be applicable to scientific computations due to as… ▽ More In a Big Data setting computing the dominant SVD factors is restrictive due to the main memory requirements. Recently introduced streaming Randomized SVD schemes work under the restrictive assumption that the singular value spectrum of the data has exponential decay. This is seldom true for any practical data. Although these methods are claimed to be applicable to scientific computations due to associated tail-energy error bounds, the approximation errors in the singular vectors and values are high when the aforementioned assumption does not hold. Furthermore from a practical perspective, oversampling can still be memory intensive or worse can exceed the feature dimension of the data. To address these issues, we present Range-Net as an alternative to randomized SVD that satisfies the tail-energy lower bound given by Eckart-Young-Mirsky (EYM) theorem. Range-Net is a deterministic two-stage neural optimization approach with random initialization, where the main memory requirement depends explicitly on the feature dimension and desired rank, independent of the sample dimension. The data samples are read in a streaming setting with the network minimization problem converging to the desired rank-r approximation. Range-Net is fully interpretable where all the network outputs and weights have a specific meaning. We provide theoretical guarantees that Range-Net extracted SVD factors satisfy EYM tail-energy lower bound at machine precision. Our numerical experiments on real data at various scales confirms this bound. A comparison against the state of the art streaming Randomized SVD shows that Range-Net accuracy is better by six orders of magnitude while being memory efficient. △ Less

Submitted 18 March, 2021; v1 submitted 27 October, 2020; originally announced October 2020.

Measuring the Credibility of Student Attendance Data in Higher Education for Data Mining

Authors: Mohammed Alsuwaiket, Christian Dawson, Firat Batmaz

Abstract: Educational Data Mining (EDM) is a developing discipline, concerned with expanding the classical Data Mining (DM) methods and developing new methods for discovering the data that originate from educational systems. Student attendance in higher education has always been dealt with in a classical way, educators rely on counting the occurrence of attendance or absence building their knowledge about s… ▽ More Educational Data Mining (EDM) is a developing discipline, concerned with expanding the classical Data Mining (DM) methods and developing new methods for discovering the data that originate from educational systems. Student attendance in higher education has always been dealt with in a classical way, educators rely on counting the occurrence of attendance or absence building their knowledge about students as well as modules based on this count. This method is neither credible nor does it necessarily provide a real indication of a student performance. This study tries to formulate the extracted knowledge in a way that guarantees achieving accurate and credible results. Student attendance data, gathered from the educational system, were first cleaned in order to remove any randomness and noise, then various attributes were studied so as to highlight the most significant ones that affect the real attendance of students. The next step was to derive an equation that measures the Student Attendance Credibility (SAC) considering the attributes chosen in the previous step. The reliability of the newly developed measure was then evaluated in order to examine its consistency. Finally, the J48 DM classification technique was utilized in order to classify modules based on the strength of their SAC values. Results of this study were promising, and credibility values achieved using the newly derived formula gave accurate, credible, and real indicators of student attendance, as well as accurate classification of modules based on the credibility of student attendance on those modules. △ Less

Submitted 1 September, 2020; originally announced September 2020.

Comments: 7 pages, 2 figures, 6 tables

Prevention is Better than Cure: Handling Basis Collapse and Transparency in Dense Networks

Authors: Gurpreet Singh, Soumyajit Gupta, Clint N. Dawson

Abstract: Dense nets are an integral part of any classification and regression problem. Recently, these networks have found a new application as solvers for known representations in various domains. However, one crucial issue with dense nets is it's feature interpretation and lack of reproducibility over multiple training runs. In this work, we identify a basis collapse issue as a primary cause and propose… ▽ More Dense nets are an integral part of any classification and regression problem. Recently, these networks have found a new application as solvers for known representations in various domains. However, one crucial issue with dense nets is it's feature interpretation and lack of reproducibility over multiple training runs. In this work, we identify a basis collapse issue as a primary cause and propose a modified loss function that circumvents this problem. We also provide a few general guidelines relating the choice of activations to loss surface roughness and appropriate scaling for designing low-weight dense nets. We demonstrate through carefully chosen numerical experiments that the basis collapse issue leads to the design of massively redundant networks. Our approach results in substantially concise nets, having $100 \times$ fewer parameters, while achieving a much lower $(10\times)$ MSE loss at scale than reported in prior works. Further, we show that the width of a dense net is acutely dependent on the feature complexity. This is in contrast to the dimension dependent width choice reported in prior theoretical works. To the best of our knowledge, this is the first time these issues and contradictions have been reported and experimentally verified. With our design guidelines we render transparency in terms of a low-weight network design. We share our codes for full reproducibility available at https://github.com/smjtgupta/Dense_Net_Regress. △ Less

Submitted 22 August, 2020; originally announced August 2020.

Provably Safe Trajectory Optimization in the Presence of Uncertain Convex Obstacles

Authors: Charles Dawson, Ashkan Jasour, Andreas Hofmann, Brian Williams

Abstract: Real-world environments are inherently uncertain, and to operate safely in these environments robots must be able to plan around this uncertainty. In the context of motion planning, we desire systems that can maintain an acceptable level of safety as the robot moves, even when the exact locations of nearby obstacles are not known. In this paper, we solve this chance-constrained motion planning pro… ▽ More Real-world environments are inherently uncertain, and to operate safely in these environments robots must be able to plan around this uncertainty. In the context of motion planning, we desire systems that can maintain an acceptable level of safety as the robot moves, even when the exact locations of nearby obstacles are not known. In this paper, we solve this chance-constrained motion planning problem using a sequential convex optimization framework. To constrain the risk of collision incurred by planned movements, we employ geometric objects called $ε$-shadows to compute upper bounds on the risk of collision between the robot and uncertain obstacles. We use these $ε$-shadow-based estimates as constraints in a nonlinear trajectory optimization problem, which we then solve by iteratively linearizing the non-convex risk constraints. This sequential optimization approach quickly finds trajectories that accomplish the desired motion while maintaining a user-specified limit on collision risk. Our method can be applied to robots and environments with arbitrary convex geometry; even in complex environments, it runs in less than a second and provides provable guarantees on the safety of planned trajectories, enabling fast, reactive, and safe robot motion in realistic environments. △ Less

Submitted 17 March, 2020; originally announced March 2020.

Comments: Submitted to IROS 2020

Fast Certification of Collision Probability Bounds with Uncertain Convex Obstacles

Authors: Charles Dawson, Andreas Hofmann, Brian Williams

Abstract: To operate reactively in uncertain environments, robots need to be able to quickly estimate the risk that they will collide with their environment. This ability is important for both planning (to ensure that plans maintain acceptable levels of safety) and execution (to provide real-time warnings when risk exceeds some threshold). Existing methods for estimating this risk are often limited to model… ▽ More To operate reactively in uncertain environments, robots need to be able to quickly estimate the risk that they will collide with their environment. This ability is important for both planning (to ensure that plans maintain acceptable levels of safety) and execution (to provide real-time warnings when risk exceeds some threshold). Existing methods for estimating this risk are often limited to models with simplified geometry (e.g. point robots); others handle complex geometry but are too slow for many applications. In this paper, we present two algorithms for quickly computing upper bounds on the risk of collision between a robot and uncertain obstacles by searching for certificate regions that capture collision probability mass while avoiding the robot. These algorithms come with strong theoretical guarantees that the true risk does not exceed the estimated value, support arbitrary geometry via convex decomposition, and provide fast query times ($<200μ$s) in representative scenarios. We characterize the performance of these algorithms in environments of varying complexity, demonstrating at least an order of magnitude speedup over existing techniques. △ Less

Submitted 17 March, 2020; originally announced March 2020.

Comments: Submitted to RSS 2020

TIME: A Transparent, Interpretable, Model-Adaptive and Explainable Neural Network for Dynamic Physical Processes

Authors: Gurpreet Singh, Soumyajit Gupta, Matt Lease, Clint N. Dawson

Abstract: Partial Differential Equations are infinite dimensional encoded representations of physical processes. However, imbibing multiple observation data towards a coupled representation presents significant challenges. We present a fully convolutional architecture that captures the invariant structure of the domain to reconstruct the observable system. The proposed architecture is significantly low-weig… ▽ More Partial Differential Equations are infinite dimensional encoded representations of physical processes. However, imbibing multiple observation data towards a coupled representation presents significant challenges. We present a fully convolutional architecture that captures the invariant structure of the domain to reconstruct the observable system. The proposed architecture is significantly low-weight compared to other networks for such problems. Our intent is to learn coupled dynamic processes interpreted as deviations from true kernels representing isolated processes for model-adaptivity. Experimental analysis shows that our architecture is robust and transparent in capturing process kernels and system anomalies. We also show that high weights representation is not only redundant but also impacts network interpretability. Our design is guided by domain knowledge, with isolated process representations serving as ground truths for verification. These allow us to identify redundant kernels and their manifestations in activation maps to guide better designs that are both interpretable and explainable unlike traditional deep-nets. △ Less

Submitted 5 March, 2020; v1 submitted 4 March, 2020; originally announced March 2020.

High Temporal Resolution Rainfall Runoff Modelling Using Long-Short-Term-Memory (LSTM) Networks

Authors: Wei Li, Amin Kiaghadi, Clint N. Dawson

Abstract: Accurate and efficient models for rainfall runoff (RR) simulations are crucial for flood risk management. Most rainfall models in use today are process-driven; i.e. they solve either simplified empirical formulas or some variation of the St. Venant (shallow water) equations. With the development of machine-learning techniques, we may now be able to emulate rainfall models using, for example, neura… ▽ More Accurate and efficient models for rainfall runoff (RR) simulations are crucial for flood risk management. Most rainfall models in use today are process-driven; i.e. they solve either simplified empirical formulas or some variation of the St. Venant (shallow water) equations. With the development of machine-learning techniques, we may now be able to emulate rainfall models using, for example, neural networks. In this study, a data-driven RR model using a sequence-to-sequence Long-short-Term-Memory (LSTM) network was constructed. The model was tested for a watershed in Houston, TX, known for severe flood events. The LSTM network's capability in learning long-term dependencies between the input and output of the network allowed modeling RR with high resolution in time (15 minutes). Using 10-years precipitation from 153 rainfall gages and river channel discharge data (more than 5.3 million data points), and by designing several numerical tests the developed model performance in predicting river discharge was tested. The model results were also compared with the output of a process-driven model Gridded Surface Subsurface Hydrologic Analysis (GSSHA). Moreover, physical consistency of the LSTM model was explored. The model results showed that the LSTM model was able to efficiently predict discharge and achieve good model performance. When compared to GSSHA, the data-driven model was more efficient and robust in terms of prediction and calibration. Interestingly, the performance of the LSTM model improved (test Nash-Sutcliffe model efficiency from 0.666 to 0.942) when a selected subset of rainfall gages based on the model performance, were used as input instead of all rainfall gages. △ Less

Submitted 11 June, 2020; v1 submitted 6 February, 2020; originally announced February 2020.

An Infinite Hidden Markov Model With Similarity-Biased Transitions

Authors: Colin Reimer Dawson, Chaofan Huang, Clayton T. Morrison

Abstract: We describe a generalization of the Hierarchical Dirichlet Process Hidden Markov Model (HDP-HMM) which is able to encode prior information that state transitions are more likely between "nearby" states. This is accomplished by defining a similarity function on the state space and scaling transition probabilities by pair-wise similarities, thereby inducing correlations among the transition distribu… ▽ More We describe a generalization of the Hierarchical Dirichlet Process Hidden Markov Model (HDP-HMM) which is able to encode prior information that state transitions are more likely between "nearby" states. This is accomplished by defining a similarity function on the state space and scaling transition probabilities by pair-wise similarities, thereby inducing correlations among the transition distributions. We present an augmented data representation of the model as a Markov Jump Process in which: (1) some jump attempts fail, and (2) the probability of success is proportional to the similarity between the source and destination states. This augmentation restores conditional conjugacy and admits a simple Gibbs sampler. We evaluate the model and inference method on a speaker diarization task and a "harmonic parsing" task using four-part chorale data, as well as on several synthetic datasets, achieving favorable comparisons to existing models. △ Less

Submitted 21 July, 2017; originally announced July 2017.

Comments: 16 pages, 4 figures, accepted to ICML 2017, includes supplemental appendix

ACM Class: G.3; I.2.6

Bayesian Inference of Recursive Sequences of Group Activities from Tracks

Authors: Ernesto Brau, Colin Dawson, Alfredo Carrillo, David Sidi, Clayton T. Morrison

Abstract: We present a probabilistic generative model for inferring a description of coordinated, recursively structured group activities at multiple levels of temporal granularity based on observations of individuals' trajectories. The model accommodates: (1) hierarchically structured groups, (2) activities that are temporally and compositionally recursive, (3) component roles assigning different subactivi… ▽ More We present a probabilistic generative model for inferring a description of coordinated, recursively structured group activities at multiple levels of temporal granularity based on observations of individuals' trajectories. The model accommodates: (1) hierarchically structured groups, (2) activities that are temporally and compositionally recursive, (3) component roles assigning different subactivity dynamics to subgroups of participants, and (4) a nonparametric Gaussian Process model of trajectories. We present an MCMC sampling framework for performing joint inference over recursive activity descriptions and assignment of trajectories to groups, integrating out continuous parameters. We demonstrate the model's expressive power in several simulated and complex real-world scenarios from the VIRAT and UCLA Aerial Event video data sets. △ Less

Submitted 23 April, 2016; originally announced April 2016.

Comments: 10 pages, 6 figures, in Proceedings of the 30th AAAI Conference on Artificial Intelligence (AAAI'16), Phoenix, AZ, 2016