-
Validating Drone Trust Testing in Navigation Deviation Cases in Simulation
Authors:
Zahra Rezaei Khavas,
Edwin Meriaux,
Amin Majdi,
Paul Robinette
Abstract:
Developing videos for trust testing is very time-consuming, expensive and potentially dangerous. For trust tests, it requires a person to be flying the drone while another might be filming. The drones can be very expensive and if something goes wrong the costs might be very high. In previous work, we have looked at how collisions and basic communication loss can be accurately modeled in simulation…
▽ More
Developing videos for trust testing is very time-consuming, expensive and potentially dangerous. For trust tests, it requires a person to be flying the drone while another might be filming. The drones can be very expensive and if something goes wrong the costs might be very high. In previous work, we have looked at how collisions and basic communication loss can be accurately modeled in simulation and to be able to generate the same trust results from users. That work looked at two specific cases using two drones, but to expand upon this in other cases more testing is required. This paper looks to propose how to test and evaluate the change in user's trust of a drone when it is experiencing path deviation in simulation. If the environment is very realistic can simulations be a good alternative to real life videos for trust testing when there is path deviation? This deviation can occur due to the physical conditions of the space, faulty piloting, or communication loss.
△ Less
Submitted 7 October, 2023;
originally announced October 2023.
-
DECISIVE Benchmarking Data Report: sUAS Performance Results from Phase I
Authors:
Adam Norton,
Reza Ahmadzadeh,
Kshitij Jerath,
Paul Robinette,
Jay Weitzen,
Thanuka Wickramarathne,
Holly Yanco,
Minseop Choi,
Ryan Donald,
Brendan Donoghue,
Christian Dumas,
Peter Gavriel,
Alden Giedraitis,
Brendan Hertel,
Jack Houle,
Nathan Letteri,
Edwin Meriaux,
Zahra Rezaei Khavas,
Rakshith Singh,
Gregg Willcox,
Naye Yoni
Abstract:
This report reviews all results derived from performance benchmarking conducted during Phase I of the Development and Execution of Comprehensive and Integrated Subterranean Intelligent Vehicle Evaluations (DECISIVE) project by the University of Massachusetts Lowell, using the test methods specified in the DECISIVE Test Methods Handbook v1.1 for evaluating small unmanned aerial systems (sUAS) perfo…
▽ More
This report reviews all results derived from performance benchmarking conducted during Phase I of the Development and Execution of Comprehensive and Integrated Subterranean Intelligent Vehicle Evaluations (DECISIVE) project by the University of Massachusetts Lowell, using the test methods specified in the DECISIVE Test Methods Handbook v1.1 for evaluating small unmanned aerial systems (sUAS) performance in subterranean and constrained indoor environments, spanning communications, field readiness, interface, obstacle avoidance, navigation, mapping, autonomy, trust, and situation awareness. Using those 20 test methods, over 230 tests were conducted across 8 sUAS platforms: Cleo Robotics Dronut X1P (P = prototype), FLIR Black Hornet PRS, Flyability Elios 2 GOV, Lumenier Nighthawk V3, Parrot ANAFI USA GOV, Skydio X2D, Teal Golden Eagle, and Vantage Robotics Vesper. Best in class criteria is specified for each applicable test method and the sUAS that match this criteria are named for each test method, including a high-level executive summary of their performance.
△ Less
Submitted 20 January, 2023; v1 submitted 18 January, 2023;
originally announced January 2023.
-
DECISIVE Test Methods Handbook: Test Methods for Evaluating sUAS in Subterranean and Constrained Indoor Environments, Version 1.1
Authors:
Adam Norton,
Reza Ahmadzadeh,
Kshitij Jerath,
Paul Robinette,
Jay Weitzen,
Thanuka Wickramarathne,
Holly Yanco,
Minseop Choi,
Ryan Donald,
Brendan Donoghue,
Christian Dumas,
Peter Gavriel,
Alden Giedraitis,
Brendan Hertel,
Jack Houle,
Nathan Letteri,
Edwin Meriaux,
Zahra Rezaei Khavas,
Rakshith Singh,
Gregg Willcox,
Naye Yoni
Abstract:
This handbook outlines all test methods developed under the Development and Execution of Comprehensive and Integrated Subterranean Intelligent Vehicle Evaluations (DECISIVE) project by the University of Massachusetts Lowell for evaluating small unmanned aerial systems (sUAS) performance in subterranean and constrained indoor environments, spanning communications, field readiness, interface, obstac…
▽ More
This handbook outlines all test methods developed under the Development and Execution of Comprehensive and Integrated Subterranean Intelligent Vehicle Evaluations (DECISIVE) project by the University of Massachusetts Lowell for evaluating small unmanned aerial systems (sUAS) performance in subterranean and constrained indoor environments, spanning communications, field readiness, interface, obstacle avoidance, navigation, mapping, autonomy, trust, and situation awareness. For sUAS deployment in subterranean and constrained indoor environments, this puts forth two assumptions about applicable sUAS to be evaluated using these test methods: (1) able to operate without access to GPS signal, and (2) width from prop top to prop tip does not exceed 91 cm (36 in) wide (i.e., can physically fit through a typical doorway, although successful navigation through is not guaranteed). All test methods are specified using a common format: Purpose, Summary of Test Method, Apparatus and Artifacts, Equipment, Metrics, Procedure, and Example Data. All test methods are designed to be run in real-world environments (e.g., MOUT sites) or using fabricated apparatuses (e.g., test bays built from wood, or contained inside of one or more shipping containers).
△ Less
Submitted 20 January, 2023; v1 submitted 1 November, 2022;
originally announced November 2022.
