-
GAP9Shield: A 150GOPS AI-capable Ultra-low Power Module for Vision and Ranging Applications on Nano-drones
Authors:
Hanna Müller,
Victor Kartsch,
Luca Benini
Abstract:
The evolution of AI and digital signal processing technologies, combined with affordable energy-efficient processors, has propelled the development of both hardware and software for drone applications. Nano-drones, which fit into the palm of the hand, are suitable for indoor environments and safe for human interaction; however, they often fail to deliver the required performance for complex tasks…
▽ More
The evolution of AI and digital signal processing technologies, combined with affordable energy-efficient processors, has propelled the development of both hardware and software for drone applications. Nano-drones, which fit into the palm of the hand, are suitable for indoor environments and safe for human interaction; however, they often fail to deliver the required performance for complex tasks due to the lack of hardware providing sufficient sensing and computing performance. Addressing this gap, we present the GAP9Shield, a nano-drone-compatible module powered by the GAP9, a 150GOPS-capable SoC. The system also includes a 5MP OV5647 camera for high-definition imaging, a WiFi-BLE NINA module, and a 5D VL53L1-based ranging subsystem, which enhances obstacle avoidance capabilities. In comparison with similarly targeted state-of-the-art systems, GAP9Shield provides a 20% higher sample rate (RGB images) while offering a 20% weight reduction. In this paper, we also highlight the energy efficiency and processing power capabilities of GAP9 for object detection (YOLO), localization, and mapping, which can run within a power envelope of below 100 mW and at low latency (as 17 ms for object detection), highlighting the transformative potential of GAP9 for the new generation of nano-drone applications.
△ Less
Submitted 27 June, 2024;
originally announced July 2024.
-
BatDeck: Advancing Nano-drone Navigation with Low-power Ultrasound-based Obstacle Avoidance
Authors:
Hanna Müller,
Victor Kartsch,
Michele Magno,
Luca Benini
Abstract:
Nano-drones, distinguished by their agility, minimal weight, and cost-effectiveness, are particularly well-suited for exploration in confined, cluttered and narrow spaces. Recognizing transparent, highly reflective or absorbing materials, such as glass and metallic surfaces is challenging, as classical sensors, such as cameras or laser rangers, often do not detect them. Inspired by bats, which can…
▽ More
Nano-drones, distinguished by their agility, minimal weight, and cost-effectiveness, are particularly well-suited for exploration in confined, cluttered and narrow spaces. Recognizing transparent, highly reflective or absorbing materials, such as glass and metallic surfaces is challenging, as classical sensors, such as cameras or laser rangers, often do not detect them. Inspired by bats, which can fly at high speeds in complete darkness with the help of ultrasound, this paper introduces \textit{BatDeck}, a pioneering sensor-deck employing a lightweight and low-power ultrasonic sensor for nano-drone autonomous navigation. This paper first provides insights about sensor characteristics, highlighting the influence of motor noise on the ultrasound readings, then it introduces the results of extensive experimental tests for obstacle avoidance (OA) in a diverse environment. Results show that \textit{BatDeck} allows exploration for a flight time of 8 minutes while covering 136m on average before crash in a challenging environment with transparent and reflective obstacles, proving the effectiveness of ultrasonic sensors for OA on nano-drones.
△ Less
Submitted 25 March, 2024;
originally announced March 2024.
-
BioGAP: a 10-Core FP-capable Ultra-Low Power IoT Processor, with Medical-Grade AFE and BLE Connectivity for Wearable Biosignal Processing
Authors:
Sebastian Frey,
Marco Guermandi,
Simone Benatti,
Victor Kartsch,
Andrea Cossettini,
Luca Benini
Abstract:
Wearable biosignal processing applications are driving significant progress toward miniaturized, energy-efficient Internet-of-Things solutions for both clinical and consumer applications. However, scaling toward high-density multi-channel front-ends is only feasible by performing data processing and machine Learning (ML) near-sensor through energy-efficient edge processing. To tackle these challen…
▽ More
Wearable biosignal processing applications are driving significant progress toward miniaturized, energy-efficient Internet-of-Things solutions for both clinical and consumer applications. However, scaling toward high-density multi-channel front-ends is only feasible by performing data processing and machine Learning (ML) near-sensor through energy-efficient edge processing. To tackle these challenges, we introduce BioGAP, a novel, compact, modular, and lightweight (6g) medical-grade biosignal acquisition and processing platform powered by GAP9, a ten-core ultra-low-power SoC designed for efficient multi-precision (from FP to aggressively quantized integer) processing, as required for advanced ML and DSP. BioGAPs form factor is 16x21x14 mm$^3$ and comprises two stacked PCBs: a baseboard integrating the GAP9 SoC, a wireless Bluetooth Low Energy (BLE) capable SoC, a power management circuit, and an accelerometer; and a shield including an analog front-end (AFE) for ExG acquisition. Finally, the system also includes a flexibly placeable photoplethysmogram (PPG) PCB with a size of 9x7x3 mm$^3$ and a rechargeable battery ($φ$ 12x5 mm$^2$). We demonstrate BioGAP on a Steady State Visually Evoked Potential (SSVEP)-based Brain-Computer Interface (BCI) application. We achieve 3.6 uJ/sample in streaming and 2.2 uJ/sample in onboard processing mode, thanks to an efficiency on the FFT computation task of 16.7 Mflops/s/mW with wireless bandwidth reduction of 97%, within a power budget of just 18.2 mW allowing for an operation time of 15 h.
△ Less
Submitted 4 July, 2023;
originally announced July 2023.
-
Bio-inspired Autonomous Exploration Policies with CNN-based Object Detection on Nano-drones
Authors:
Lorenzo Lamberti,
Luca Bompani,
Victor Javier Kartsch,
Manuele Rusci,
Daniele Palossi,
Luca Benini
Abstract:
Nano-sized drones, with palm-sized form factor, are gaining relevance in the Internet-of-Things ecosystem. Achieving a high degree of autonomy for complex multi-objective missions (e.g., safe flight, exploration, object detection) is extremely challenging for the onboard chip-set due to tight size, payload (<10g), and power envelope constraints, which strictly limit both memory and computation. Ou…
▽ More
Nano-sized drones, with palm-sized form factor, are gaining relevance in the Internet-of-Things ecosystem. Achieving a high degree of autonomy for complex multi-objective missions (e.g., safe flight, exploration, object detection) is extremely challenging for the onboard chip-set due to tight size, payload (<10g), and power envelope constraints, which strictly limit both memory and computation. Our work addresses this complex problem by combining bio-inspired navigation policies, which rely on time-of-flight distance sensor data, with a vision-based convolutional neural network (CNN) for object detection. Our field-proven nano-drone is equipped with two microcontroller units (MCUs), a single-core ARM Cortex-M4 (STM32) for safe navigation and exploration policies, and a parallel ultra-low power octa-core RISC-V (GAP8) for onboard CNN inference, with a power envelope of just 134mW, including image sensors and external memories. The object detection task achieves a mean average precision of 50% (at 1.6 frame/s) on an in-field collected dataset. We compare four bio-inspired exploration policies and identify a pseudo-random policy to achieve the highest coverage area of 83% in a ~36m^2 unknown room in a 3 minutes flight. By combining the detection CNN and the exploration policy, we show an average detection rate of 90% on six target objects in a never-seen-before environment.
△ Less
Submitted 1 February, 2023; v1 submitted 28 January, 2023;
originally announced January 2023.
-
Towards a Wearable Interface for Food Quality Grading through ERP Analysis
Authors:
M. Guermandi,
S. Benatti,
D. Brunelli,
V. Kartsch,
L. Benini
Abstract:
Sensory evaluation is used to assess the consumer acceptance of foods or other consumer products, so as to improve industrial processes and marketing strategies. The procedures currently involved are time-consuming because they require a statistical approach from measurements and feedback reports from a wide set of evaluators under a well-established measurement setup. In this paper, we propose to…
▽ More
Sensory evaluation is used to assess the consumer acceptance of foods or other consumer products, so as to improve industrial processes and marketing strategies. The procedures currently involved are time-consuming because they require a statistical approach from measurements and feedback reports from a wide set of evaluators under a well-established measurement setup. In this paper, we propose to collect directly the signal of the perceived quality of the food from Event-related potentials (ERPs) that are the outcome of the processing of visual stimuli. This permits to narrow the number of evaluators since errors related to psychological factors are by-passed. We present the design of a wearable system for ERP measurement and we present preliminary results on the use of ERP to give a quantitative measure to the appearance of a food product. The system is developed to be wearable and our experiments demonstrate that is possible to use it to identify and classify the grade of acceptance of the food.
△ Less
Submitted 28 May, 2019;
originally announced May 2019.
