Showing 1–25 of 25 results for author: Van Nguyen, H

Multimodal Learning and Cognitive Processes in Radiology: MedGaze for Chest X-ray Scanpath Prediction

Authors: Akash Awasthi, Ngan Le, Zhigang Deng, Rishi Agrawal, Carol C. Wu, Hien Van Nguyen

Abstract: Predicting human gaze behavior within computer vision is integral for developing interactive systems that can anticipate user attention, address fundamental questions in cognitive science, and hold implications for fields like human-computer interaction (HCI) and augmented/virtual reality (AR/VR) systems. Despite methodologies introduced for modeling human eye gaze behavior, applying these models… ▽ More Predicting human gaze behavior within computer vision is integral for developing interactive systems that can anticipate user attention, address fundamental questions in cognitive science, and hold implications for fields like human-computer interaction (HCI) and augmented/virtual reality (AR/VR) systems. Despite methodologies introduced for modeling human eye gaze behavior, applying these models to medical imaging for scanpath prediction remains unexplored. Our proposed system aims to predict eye gaze sequences from radiology reports and CXR images, potentially streamlining data collection and enhancing AI systems using larger datasets. However, predicting human scanpaths on medical images presents unique challenges due to the diverse nature of abnormal regions. Our model predicts fixation coordinates and durations critical for medical scanpath prediction, outperforming existing models in the computer vision community. Utilizing a two-stage training process and large publicly available datasets, our approach generates static heatmaps and eye gaze videos aligned with radiology reports, facilitating comprehensive analysis. We validate our approach by comparing its performance with state-of-the-art methods and assessing its generalizability among different radiologists, introducing novel strategies to model radiologists' search patterns during CXR image diagnosis. Based on the radiologist's evaluation, MedGaze can generate human-like gaze sequences with a high focus on relevant regions over the CXR images. It sometimes also outperforms humans in terms of redundancy and randomness in the scanpaths. △ Less

Submitted 28 June, 2024; originally announced July 2024.

Comments: Submitted to the Journal

Enhancing Radiological Diagnosis: A Collaborative Approach Integrating AI and Human Expertise for Visual Miss Correction

Authors: Akash Awasthi, Ngan Le, Zhigang Deng, Carol C. Wu, Hien Van Nguyen

Abstract: Human-AI collaboration to identify and correct perceptual errors in chest radiographs has not been previously explored. This study aimed to develop a collaborative AI system, CoRaX, which integrates eye gaze data and radiology reports to enhance diagnostic accuracy in chest radiology by pinpointing perceptual errors and refining the decision-making process. Using public datasets REFLACX and EGD-CX… ▽ More Human-AI collaboration to identify and correct perceptual errors in chest radiographs has not been previously explored. This study aimed to develop a collaborative AI system, CoRaX, which integrates eye gaze data and radiology reports to enhance diagnostic accuracy in chest radiology by pinpointing perceptual errors and refining the decision-making process. Using public datasets REFLACX and EGD-CXR, the study retrospectively developed CoRaX, employing a large multimodal model to analyze image embeddings, eye gaze data, and radiology reports. The system's effectiveness was evaluated based on its referral-making process, the quality of referrals, and performance in collaborative diagnostic settings. CoRaX was tested on a simulated error dataset of 271 samples with 28% (93 of 332) missed abnormalities. The system corrected 21% (71 of 332) of these errors, leaving 7% (22 of 312) unresolved. The Referral-Usefulness score, indicating the accuracy of predicted regions for all true referrals, was 0.63 (95% CI 0.59, 0.68). The Total-Usefulness score, reflecting the diagnostic accuracy of CoRaX's interactions with radiologists, showed that 84% (237 of 280) of these interactions had a score above 0.40. In conclusion, CoRaX efficiently collaborates with radiologists to address perceptual errors across various abnormalities, with potential applications in the education and training of novice radiologists. △ Less

Submitted 28 June, 2024; originally announced June 2024.

Comments: Under Review in Journal

$\textit{UniSaT}$: Unified-Objective Belief Model and Planner to Search for and Track Multiple Objects

Authors: Leonardo Santos, Brady Moon, Sebastian Scherer, Hoa Van Nguyen

Abstract: The problem of path planning for autonomously searching and tracking multiple objects is important to reconnaissance, surveillance, and many other data-gathering applications. Due to the inherent competing objectives of searching for new objects while maintaining tracks for found objects, most current approaches rely on multi-objective planning methods, leaving it up to the user to tune parameters… ▽ More The problem of path planning for autonomously searching and tracking multiple objects is important to reconnaissance, surveillance, and many other data-gathering applications. Due to the inherent competing objectives of searching for new objects while maintaining tracks for found objects, most current approaches rely on multi-objective planning methods, leaving it up to the user to tune parameters to balance between the two objectives, usually based on heuristics or trial and error. In this paper, we introduce $\textit{UniSaT}$ ($\textit{Unified Search and Track}$), a unified-objective formulation for the search and track problem based on Random Finite Sets (RFS). This is done by modeling both the unknown and known objects through a combined generalized labeled multi-Bernoulli (GLMB) filter. For the unseen objects, we can leverage both cardinality and spatial prior distributions, which means $\textit{UniSaT}$ does not rely on knowing the exact count of the expected number of objects in the space. The planner maximizes the mutual information of this unified belief model, creating balanced search and tracking behaviors. We demonstrate our work in a simulated environment and show both qualitative results as well as quantitative improvements over a multi-objective method. △ Less

Submitted 24 May, 2024; originally announced May 2024.

Comments: 13 pages, 5 figures, 1 table

Decoding Radiologists' Intentions: A Novel System for Accurate Region Identification in Chest X-ray Image Analysis

Authors: Akash Awasthi, Safwan Ahmad, Bryant Le, Hien Van Nguyen

Abstract: In the realm of chest X-ray (CXR) image analysis, radiologists meticulously examine various regions, documenting their observations in reports. The prevalence of errors in CXR diagnoses, particularly among inexperienced radiologists and hospital residents, underscores the importance of understanding radiologists' intentions and the corresponding regions of interest. This understanding is crucial f… ▽ More In the realm of chest X-ray (CXR) image analysis, radiologists meticulously examine various regions, documenting their observations in reports. The prevalence of errors in CXR diagnoses, particularly among inexperienced radiologists and hospital residents, underscores the importance of understanding radiologists' intentions and the corresponding regions of interest. This understanding is crucial for correcting mistakes by guiding radiologists to the accurate regions of interest, especially in the diagnosis of chest radiograph abnormalities. In response to this imperative, we propose a novel system designed to identify the primary intentions articulated by radiologists in their reports and the corresponding regions of interest in CXR images. This system seeks to elucidate the visual context underlying radiologists' textual findings, with the potential to rectify errors made by less experienced practitioners and direct them to precise regions of interest. Importantly, the proposed system can be instrumental in providing constructive feedback to inexperienced radiologists or junior residents in the hospital, bridging the gap in face-to-face communication. The system represents a valuable tool for enhancing diagnostic accuracy and fostering continuous learning within the medical community. △ Less

Submitted 29 April, 2024; originally announced April 2024.

Comments: Accepted in ISBI 2024

Distributed Multi-Object Tracking Under Limited Field of View Heterogeneous Sensors with Density Clustering

Authors: Fei Chen, Hoa Van Nguyen, Alex S. Leong, Sabita Panicker, Robin Baker, Damith C. Ranasinghe

Abstract: We consider the problem of tracking multiple, unknown, and time-varying numbers of objects using a distributed network of heterogeneous sensors. In an effort to derive a formulation for practical settings, we consider limited and unknown sensor field-of-views (FoVs), sensors with limited local computational resources and communication channel capacity. The resulting distributed multi-object tracki… ▽ More We consider the problem of tracking multiple, unknown, and time-varying numbers of objects using a distributed network of heterogeneous sensors. In an effort to derive a formulation for practical settings, we consider limited and unknown sensor field-of-views (FoVs), sensors with limited local computational resources and communication channel capacity. The resulting distributed multi-object tracking algorithm involves solving an NP-hard multidimensional assignment problem either optimally for small-size problems or sub-optimally for general practical problems. For general problems, we propose an efficient distributed multi-object tracking algorithm that performs track-to-track fusion using a clustering-based analysis of the state space transformed into a density space to mitigate the complexity of the assignment problem. The proposed algorithm can more efficiently group local track estimates for fusion than existing approaches. To ensure we achieve globally consistent identities for tracks across a network of nodes as objects move between FoVs, we develop a graph-based algorithm to achieve label consensus and minimise track segmentation. Numerical experiments with a synthetic and a real-world trajectory dataset demonstrate that our proposed method is significantly more computationally efficient than state-of-the-art solutions, achieving similar tracking accuracy and bandwidth requirements but with improved label consistency. △ Less

Submitted 31 December, 2023; originally announced January 2024.

Label Space Partition Selection for Multi-Object Tracking Using Two-Layer Partitioning

Authors: Ji Youn Lee, Changbeom Shim, Hoa Van Nguyen, Tran Thien Dat Nguyen, Hyunjin Choi, Youngho Kim

Abstract: Estimating the trajectories of multi-objects poses a significant challenge due to data association ambiguity, which leads to a substantial increase in computational requirements. To address such problems, a divide-and-conquer manner has been employed with parallel computation. In this strategy, distinguished objects that have unique labels are grouped based on their statistical dependencies, the i… ▽ More Estimating the trajectories of multi-objects poses a significant challenge due to data association ambiguity, which leads to a substantial increase in computational requirements. To address such problems, a divide-and-conquer manner has been employed with parallel computation. In this strategy, distinguished objects that have unique labels are grouped based on their statistical dependencies, the intersection of predicted measurements. Several geometry approaches have been used for label grouping since finding all intersected label pairs is clearly infeasible for large-scale tracking problems. This paper proposes an efficient implementation of label grouping for label-partitioned generalized labeled multi-Bernoulli filter framework using a secondary partitioning technique. This allows for parallel computation in the label graph indexing step, avoiding generating and eliminating duplicate comparisons. Additionally, we compare the performance of the proposed technique with several efficient spatial searching algorithms. The results demonstrate the superior performance of the proposed approach on large-scale data sets, enabling scalable trajectory estimation. △ Less

Submitted 22 October, 2023; originally announced October 2023.

Comments: 6 pages, 4 figures

ConservationBots: Autonomous Aerial Robot for Fast Robust Wildlife Tracking in Complex Terrains

Authors: Fei Chen, Hoa Van Nguyen, David A. Taggart, Katrina Falkner, S. Hamid Rezatofighi, Damith C. Ranasinghe

Abstract: Today, the most widespread, widely applicable technology for gathering data relies on experienced scientists armed with handheld radio telemetry equipment to locate low-power radio transmitters attached to wildlife from the ground. Although aerial robots can transform labor-intensive conservation tasks, the realization of autonomous systems for tackling task complexities under real-world condition… ▽ More Today, the most widespread, widely applicable technology for gathering data relies on experienced scientists armed with handheld radio telemetry equipment to locate low-power radio transmitters attached to wildlife from the ground. Although aerial robots can transform labor-intensive conservation tasks, the realization of autonomous systems for tackling task complexities under real-world conditions remains a challenge. We developed ConservationBots-small aerial robots for tracking multiple, dynamic, radio-tagged wildlife. The aerial robot achieves robust localization performance and fast task completion times -- significant for energy-limited aerial systems while avoiding close encounters with potential, counter-productive disturbances to wildlife. Our approach overcomes the technical and practical problems posed by combining a lightweight sensor with new concepts: i) planning to determine both trajectory and measurement actions guided by an information-theoretic objective, which allows the robot to strategically select near-instantaneous range-only measurements to achieve faster localization, and time-consuming sensor rotation actions to acquire bearing measurements and achieve robust tracking performance; ii) a bearing detector more robust to noise and iii) a tracking algorithm formulation robust to missed and false detections experienced in real-world conditions. We conducted extensive studies: simulations built upon complex signal propagation over high-resolution elevation data on diverse geographical terrains; field testing; studies with wombats (Lasiorhinus latifrons; nocturnal, vulnerable species dwelling in underground warrens) and tracking comparisons with a highly experienced biologist to validate the effectiveness of our aerial robot and demonstrate the significant advantages over the manual method. △ Less

Submitted 12 November, 2023; v1 submitted 15 August, 2023; originally announced August 2023.

Comments: Accepted to The Journal of Field Robotics

Fairness Enhancement of UAV Systems with Hybrid Active-Passive RIS

Authors: Nhan Thanh Nguyen, Van-Dinh Nguyen, Hieu Van Nguyen, Qingqing Wu, Antti Tolli, Symeon Chatzinotas, Markku Juntti

Abstract: We consider unmanned aerial vehicle (UAV)-enabled wireless systems where downlink communications between a multi-antenna UAV and multiple users are assisted by a hybrid active-passive reconfigurable intelligent surface (RIS). We aim at a fairness design of two typical UAV-enabled networks, namely the static-UAV network where the UAV is deployed at a fixed location to serve all users at the same ti… ▽ More We consider unmanned aerial vehicle (UAV)-enabled wireless systems where downlink communications between a multi-antenna UAV and multiple users are assisted by a hybrid active-passive reconfigurable intelligent surface (RIS). We aim at a fairness design of two typical UAV-enabled networks, namely the static-UAV network where the UAV is deployed at a fixed location to serve all users at the same time, and the mobile-UAV network which employs the time division multiple access protocol. In both networks, our goal is to maximize the minimum rate among users through jointly optimizing the UAV's location/trajectory, transmit beamformer, and RIS coefficients. The resulting problems are highly nonconvex due to a strong coupling between the involved variables. We develop efficient algorithms based on block coordinate ascend and successive convex approximation to effectively solve these problems in an iterative manner. In particular, in the optimization of the mobile-UAV network, closed-form solutions to the transmit beamformer and RIS passive coefficients are derived. Numerical results show that a hybrid RIS equipped with only 4 active elements and a power budget of 0 dBm offers an improvement of 38%-63% in minimum rate, while that achieved by a passive RIS is only about 15%, with the same total number of elements. △ Less

Submitted 20 September, 2023; v1 submitted 24 June, 2023; originally announced June 2023.

Comments: This paper has been submitted to IEEE Transaction on Wireless Communications

Anomaly Detection in Satellite Videos using Diffusion Models

Authors: Akash Awasthi, Son Ly, Jaer Nizam, Samira Zare, Videet Mehta, Safwan Ahmed, Keshav Shah, Ramakrishna Nemani, Saurabh Prasad, Hien Van Nguyen

Abstract: The definition of anomaly detection is the identification of an unexpected event. Real-time detection of extreme events such as wildfires, cyclones, or floods using satellite data has become crucial for disaster management. Although several earth-observing satellites provide information about disasters, satellites in the geostationary orbit provide data at intervals as frequent as every minute, ef… ▽ More The definition of anomaly detection is the identification of an unexpected event. Real-time detection of extreme events such as wildfires, cyclones, or floods using satellite data has become crucial for disaster management. Although several earth-observing satellites provide information about disasters, satellites in the geostationary orbit provide data at intervals as frequent as every minute, effectively creating a video from space. There are many techniques that have been proposed to identify anomalies in surveillance videos; however, the available datasets do not have dynamic behavior, so we discuss an anomaly framework that can work on very high-frequency datasets to find very fast-moving anomalies. In this work, we present a diffusion model which does not need any motion component to capture the fast-moving anomalies and outperforms the other baseline methods. △ Less

Submitted 25 May, 2023; originally announced June 2023.

Deep Learning-Derived Optimal Aviation Strategies to Control Pandemics

Authors: Syed Rizvi, Akash Awasthi, Maria J. Peláez, Zhihui Wang, Vittorio Cristini, Hien Van Nguyen, Prashant Dogra

Abstract: The COVID-19 pandemic has affected countries across the world, demanding drastic public health policies to mitigate the spread of infection, leading to economic crisis as a collateral damage. In this work, we investigated the impact of human mobility (described via international commercial flights) on COVID-19 infection dynamics at the global scale. For this, we developed a graph neural network-ba… ▽ More The COVID-19 pandemic has affected countries across the world, demanding drastic public health policies to mitigate the spread of infection, leading to economic crisis as a collateral damage. In this work, we investigated the impact of human mobility (described via international commercial flights) on COVID-19 infection dynamics at the global scale. For this, we developed a graph neural network-based framework referred to as Dynamic Connectivity GraphSAGE (DCSAGE), which operates over spatiotemporal graphs and is well-suited for dynamically changing adjacency information. To obtain insights on the relative impact of different geographical locations, due to their associated air traffic, on the evolution of the pandemic, we conducted local sensitivity analysis on our model through node perturbation experiments. From our analyses, we identified Western Europe, North America, and Middle East as the leading geographical locations fueling the pandemic, attributed to the enormity of air traffic originating or transiting through these regions. We used these observations to identify tangible air traffic reduction strategies that can have a high impact on controlling the pandemic, with minimal interference to human mobility. Our work provides a robust deep learning-based tool to study global pandemics and is of key relevance to policy makers to take informed decisions regarding air traffic restrictions during future outbreaks. △ Less

Submitted 12 October, 2022; originally announced October 2022.

Comments: 32 pages, 6 figures (with additional 9 supplementary figures)

Multi-Objective Multi-Agent Planning for Discovering and Tracking Multiple Mobile Objects

Authors: Hoa Van Nguyen, Ba-Ngu Vo, Ba-Tuong Vo, Hamid Rezatofighi, Damith C. Ranasinghe

Abstract: We consider the online planning problem for a team of agents to discover and track an unknown and time-varying number of moving objects from onboard sensor measurements with uncertain measurement-object origins. Since the onboard sensors have limited field-of-views, the usual planning strategy based solely on either tracking detected objects or discovering unseen objects is inadequate. To address… ▽ More We consider the online planning problem for a team of agents to discover and track an unknown and time-varying number of moving objects from onboard sensor measurements with uncertain measurement-object origins. Since the onboard sensors have limited field-of-views, the usual planning strategy based solely on either tracking detected objects or discovering unseen objects is inadequate. To address this, we formulate a new information-based multi-objective multi-agent control problem, cast as a partially observable Markov decision process (POMDP). The resulting multi-agent planning problem is exponentially complex due to the unknown data association between objects and multi-sensor measurements; hence, computing an optimal control action is intractable. We prove that the proposed multi-objective value function is a monotone submodular set function, which admits low-cost suboptimal solutions via greedy search with a tight optimality bound. The resulting planning algorithm has a linear complexity in the number of objects and measurements across the sensors, and quadratic in the number of agents. We demonstrate the proposed solution via a series of numerical experiments with a real-world dataset. △ Less

Submitted 3 July, 2024; v1 submitted 9 March, 2022; originally announced March 2022.

Comments: Accepted to IEEE Transactions on Signal Processing. 16 pages, 10 Figures

Controlling Smart Propagation Environments: Long-Term versus Short-Term Phase Shift Optimization

Authors: Trinh Van Chien, Lam Thanh Tu, Dinh-Hieu Tran, Hieu Van Nguyen, Symeon Chatzinotas, Marco Di Renzo, Björn Ottersten

Abstract: Reconfigurable intelligent surfaces (RISs) have recently gained significant interest as an emerging technology for future wireless networks. This paper studies an RIS-assisted propagation environment, where a single-antenna source transmits data to a single-antenna destination in the presence of a weak direct link. We analyze and compare RIS designs based on long-term and short-term channel statis… ▽ More Reconfigurable intelligent surfaces (RISs) have recently gained significant interest as an emerging technology for future wireless networks. This paper studies an RIS-assisted propagation environment, where a single-antenna source transmits data to a single-antenna destination in the presence of a weak direct link. We analyze and compare RIS designs based on long-term and short-term channel statistics in terms of coverage probability and ergodic rate. For the considered optimization designs, closed-form expressions for the coverage probability and ergodic rate are derived. We use numerical simulations to analyze and compare against analytic results in finite samples. Also, we show that the considered optimal phase shift designs outperform several heuristic benchmarks. △ Less

Submitted 25 October, 2021; originally announced October 2021.

Comments: 5 pages, 1 figure. Submitted for publication

The Megopolis Resampler: Memory Coalesced Resampling on GPUs

Authors: Joshua A. Chesser, Hoa Van Nguyen, Damith C. Ranasinghe

Abstract: The resampling process employed in widely used methods such as Importance Sampling (IS), with its adaptive extension (AIS), are used to solve challenging problems requiring approximate inference; for example, non-linear, non-Gaussian state estimation problems. However, the re-sampling process can be computationally prohibitive for practical problems with real-time requirements. We consider the pro… ▽ More The resampling process employed in widely used methods such as Importance Sampling (IS), with its adaptive extension (AIS), are used to solve challenging problems requiring approximate inference; for example, non-linear, non-Gaussian state estimation problems. However, the re-sampling process can be computationally prohibitive for practical problems with real-time requirements. We consider the problem of developing highly parallelisable resampling algorithms for massively parallel hardware architectures of modern graphics processing units (GPUs) to accomplish real-time performance. We develop a new variant of the Metropolis algorithm -- Megopolis -- that improves performance without requiring a tuning parameter or reducing resampling quality. The Megopolis algorithm is built upon exploiting the memory access patterns of modern GPU units to reduce the number of memory transactions without the need for tuning parameters. Extensive numerical experiments on GPU hardware demonstrate that the proposed Megopolis algorithm is numerically stable and outperforms the original Metropolis algorithm and its variants -- Metropolis-C1 and Metropolis-C2 -- in speed and quality metrics. Further, given the absence of open tools in this domain and facilitating fair comparisons in the future and supporting the signal processing community, we also open source the complete project, including a repository of source code with Megopolis and all other comparison methods. △ Less

Submitted 28 September, 2021; originally announced September 2021.

Comments: Accepted to the Digital Signal Processing - an Elsevier Journal with 23 pages and 10 figures

PICASO: Permutation-Invariant Cascaded Attentional Set Operator

Authors: Samira Zare, Hien Van Nguyen

Abstract: Set-input deep networks have recently drawn much interest in computer vision and machine learning. This is in part due to the increasing number of important tasks such as meta-learning, clustering, and anomaly detection that are defined on set inputs. These networks must take an arbitrary number of input samples and produce the output invariant to the input set permutation. Several algorithms have… ▽ More Set-input deep networks have recently drawn much interest in computer vision and machine learning. This is in part due to the increasing number of important tasks such as meta-learning, clustering, and anomaly detection that are defined on set inputs. These networks must take an arbitrary number of input samples and produce the output invariant to the input set permutation. Several algorithms have been recently developed to address this urgent need. Our paper analyzes these algorithms using both synthetic and real-world datasets, and shows that they are not effective in dealing with common data variations such as image translation or viewpoint change. To address this limitation, we propose a permutation-invariant cascaded attentional set operator (PICASO). The gist of PICASO is a cascade of multihead attention blocks with dynamic templates. The proposed operator is a stand-alone module that can be adapted and extended to serve different machine learning tasks. We demonstrate the utilities of PICASO in four diverse scenarios: (i) clustering, (ii) image classification under novel viewpoints, (iii) image anomaly detection, and (iv) state prediction. PICASO increases the SmallNORB image classification accuracy with novel viewpoints by about 10% points. For set anomaly detection on CelebA dataset, our model improves the areas under ROC and PR curves dataset by about 22% and 10%, respectively. For the state prediction on CLEVR dataset, it improves the AP by about 40%. △ Less

Submitted 17 July, 2021; originally announced July 2021.

First arrival picking using U-net with Lovasz loss and nearest point picking method

Authors: Pengyu Yuan, Wenyi Hu, Xuqing Wu, Jiefu Chen, Hien Van Nguyen

Abstract: We proposed a robust segmentation and picking workflow to solve the first arrival picking problem for seismic signal processing. Unlike traditional classification algorithm, image segmentation method can utilize the location information by outputting a prediction map which has the same size of the input image. A parameter-free nearest point picking algorithm is proposed to further improve the accu… ▽ More We proposed a robust segmentation and picking workflow to solve the first arrival picking problem for seismic signal processing. Unlike traditional classification algorithm, image segmentation method can utilize the location information by outputting a prediction map which has the same size of the input image. A parameter-free nearest point picking algorithm is proposed to further improve the accuracy of the first arrival picking. The algorithm is test on synthetic clean data, synthetic noisy data, synthetic picking-disconnected data and field data. It performs well on all of them and the picking deviation reaches as low as 4.8ms per receiver. The first arrival picking problem is formulated as the contour detection problem. Similar to \cite{wu2019semi}, we use U-net to perform the segmentation as it is proven to be state-of-the-art in many image segmentation tasks. Particularly, a Lovasz loss instead of the traditional cross-entropy loss is used to train the network for a better segmentation performance. Lovasz loss is a surrogate loss for Jaccard index or the so-called intersection-over-union (IoU) score, which is often one of the most used metrics for segmentation tasks. In the picking part, we use a novel nearest point picking (NPP) method to take the advantage of the coherence of the first arrival picking among adjacent receivers. Our model is tested and validated on both synthetic and field data with harmonic noises. The main contributions of this paper are as follows: 1. Used Lovasz loss to directly optimize the IoU for segmentation task. Improvement over the cross-entropy loss with regard to the segmentation accuracy is verified by the test result. 2. Proposed a nearest point picking post processing method to overcome any defects left by the segmentation output. 3. Conducted noise analysis and verified the model with both noisy synthetic and field datasets. △ Less

Submitted 6 April, 2021; originally announced April 2021.

Distributed Multi-object Tracking under Limited Field of View Sensors

Authors: Hoa Van Nguyen, Hamid Rezatofighi, Ba-Ngu Vo, Damith C. Ranasinghe

Abstract: We consider the challenging problem of tracking multiple objects using a distributed network of sensors. In the practical setting of nodes with limited field of views (FoVs), computing power and communication resources, we develop a novel distributed multi-object tracking algorithm. To accomplish this, we first formalise the concept of label consistency, determine a sufficient condition to achieve… ▽ More We consider the challenging problem of tracking multiple objects using a distributed network of sensors. In the practical setting of nodes with limited field of views (FoVs), computing power and communication resources, we develop a novel distributed multi-object tracking algorithm. To accomplish this, we first formalise the concept of label consistency, determine a sufficient condition to achieve it and develop a novel \textit{label consensus approach} that reduces label inconsistency caused by objects' movements from one node's limited FoV to another. Second, we develop a distributed multi-object fusion algorithm that fuses local multi-object state estimates instead of local multi-object densities. This algorithm: i) requires significantly less processing time than multi-object density fusion methods; ii) achieves better tracking accuracy by considering Optimal Sub-Pattern Assignment (OSPA) tracking errors over several scans rather than a single scan; iii) is agnostic to local multi-object tracking techniques, and only requires each node to provide a set of estimated tracks. Thus, it is not necessary to assume that the nodes maintain multi-object densities, and hence the fusion outcomes do not modify local multi-object densities. Numerical experiments demonstrate our proposed solution's real-time computational efficiency and accuracy compared to state-of-the-art solutions in challenging scenarios. We also release source code at https://github.com/AdelaideAuto-IDLab/Distributed-limitedFoV-MOT for our fusion method to foster developments in DMOT algorithms. △ Less

Submitted 31 July, 2021; v1 submitted 23 December, 2020; originally announced December 2020.

Comments: Accepted to The IEEE Transactions on Signal Processing (TSP). 15 pages, 11 figures

LAVAPilot: Lightweight UAV Trajectory Planner with Situational Awareness for Embedded Autonomy to Track and Locate Radio-tags

Authors: Hoa Van Nguyen, Fei Chen, Joshua Chesser, Hamid Rezatofighi, Damith Ranasinghe

Abstract: Tracking and locating radio-tagged wildlife is a labor-intensive and time-consuming task necessary in wildlife conservation. In this article, we focus on the problem of achieving embedded autonomy for a resource-limited aerial robot for the task capable of avoiding undesirable disturbances to wildlife. We employ a lightweight sensor system capable of simultaneous (noisy) measurements of radio sign… ▽ More Tracking and locating radio-tagged wildlife is a labor-intensive and time-consuming task necessary in wildlife conservation. In this article, we focus on the problem of achieving embedded autonomy for a resource-limited aerial robot for the task capable of avoiding undesirable disturbances to wildlife. We employ a lightweight sensor system capable of simultaneous (noisy) measurements of radio signal strength information from multiple tags for estimating object locations. We formulate a new lightweight task-based trajectory planning method-LAVAPilot-with a greedy evaluation strategy and a void functional formulation to achieve situational awareness to maintain a safe distance from objects of interest. Conceptually, we embed our intuition of moving closer to reduce the uncertainty of measurements into LAVAPilot instead of employing a computationally intensive information gain based planning strategy. We employ LAVAPilot and the sensor to build a lightweight aerial robot platform with fully embedded autonomy for jointly tracking and planning to track and locate multiple VHF radio collar tags used by conservation biologists. Using extensive Monte Carlo simulation-based experiments, implementations on a single board compute module, and field experiments using an aerial robot platform with multiple VHF radio collar tags, we evaluate our joint planning and tracking algorithms. Further, we compare our method with other information-based planning methods with and without situational awareness to demonstrate the effectiveness of our robot executing LAVAPilot. Our experiments demonstrate that LAVAPilot significantly reduces (by 98.5%) the computational cost of planning to enable real-time planning decisions whilst achieving similar localization accuracy of objects compared to information gain based planning methods, albeit taking a slightly longer time to complete a mission. △ Less

Submitted 31 July, 2020; originally announced July 2020.

Comments: Accepted to 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)

DECAPS: Detail-Oriented Capsule Networks

Authors: Aryan Mobiny, Pengyu Yuan, Pietro Antonio Cicalese, Hien Van Nguyen

Abstract: Capsule Networks (CapsNets) have demonstrated to be a promising alternative to Convolutional Neural Networks (CNNs). However, they often fall short of state-of-the-art accuracies on large-scale high-dimensional datasets. We propose a Detail-Oriented Capsule Network (DECAPS) that combines the strength of CapsNets with several novel techniques to boost its classification accuracies. First, DECAPS us… ▽ More Capsule Networks (CapsNets) have demonstrated to be a promising alternative to Convolutional Neural Networks (CNNs). However, they often fall short of state-of-the-art accuracies on large-scale high-dimensional datasets. We propose a Detail-Oriented Capsule Network (DECAPS) that combines the strength of CapsNets with several novel techniques to boost its classification accuracies. First, DECAPS uses an Inverted Dynamic Routing (IDR) mechanism to group lower-level capsules into heads before sending them to higher-level capsules. This strategy enables capsules to selectively attend to small but informative details within the data which may be lost during pooling operations in CNNs. Second, DECAPS employs a Peekaboo training procedure, which encourages the network to focus on fine-grained information through a second-level attention scheme. Finally, the distillation process improves the robustness of DECAPS by averaging over the original and attended image region predictions. We provide extensive experiments on the CheXpert and RSNA Pneumonia datasets to validate the effectiveness of DECAPS. Our networks achieve state-of-the-art accuracies not only in classification (increasing the average area under ROC curves from 87.24% to 92.82% on the CheXpert dataset) but also in the weakly-supervised localization of diseased areas (increasing average precision from 41.7% to 80% for the RSNA Pneumonia detection dataset). △ Less

Submitted 8 July, 2020; originally announced July 2020.

Comments: arXiv admin note: text overlap with arXiv:2004.07407

Few Is Enough: Task-Augmented Active Meta-Learning for Brain Cell Classification

Authors: Pengyu Yuan, Aryan Mobiny, Jahandar Jahanipour, Xiaoyang Li, Pietro Antonio Cicalese, Badrinath Roysam, Vishal Patel, Maric Dragan, Hien Van Nguyen

Abstract: Deep Neural Networks (or DNNs) must constantly cope with distribution changes in the input data when the task of interest or the data collection protocol changes. Retraining a network from scratch to combat this issue poses a significant cost. Meta-learning aims to deliver an adaptive model that is sensitive to these underlying distribution changes, but requires many tasks during the meta-training… ▽ More Deep Neural Networks (or DNNs) must constantly cope with distribution changes in the input data when the task of interest or the data collection protocol changes. Retraining a network from scratch to combat this issue poses a significant cost. Meta-learning aims to deliver an adaptive model that is sensitive to these underlying distribution changes, but requires many tasks during the meta-training process. In this paper, we propose a tAsk-auGmented actIve meta-LEarning (AGILE) method to efficiently adapt DNNs to new tasks by using a small number of training examples. AGILE combines a meta-learning algorithm with a novel task augmentation technique which we use to generate an initial adaptive model. It then uses Bayesian dropout uncertainty estimates to actively select the most difficult samples when updating the model to a new task. This allows AGILE to learn with fewer tasks and a few informative samples, achieving high performance with a limited dataset. We perform our experiments using the brain cell classification task and compare the results to a plain meta-learning model trained from scratch. We show that the proposed task-augmented meta-learning framework can learn to classify new cell types after a single gradient step with a limited number of training samples. We show that active learning with Bayesian uncertainty can further improve the performance when the number of training samples is extremely small. Using only 1% of the training data and a single update step, we achieved 90% accuracy on the new cell type classification task, a 50% points improvement over a state-of-the-art meta-learning algorithm. △ Less

Submitted 9 July, 2020; originally announced July 2020.

StyPath: Style-Transfer Data Augmentation For Robust Histology Image Classification

Authors: Pietro Antonio Cicalese, Aryan Mobiny, Pengyu Yuan, Jan Becker, Chandra Mohan, Hien Van Nguyen

Abstract: The classification of Antibody Mediated Rejection (AMR) in kidney transplant remains challenging even for experienced nephropathologists; this is partly because histological tissue stain analysis is often characterized by low inter-observer agreement and poor reproducibility. One of the implicated causes for inter-observer disagreement is the variability of tissue stain quality between (and within… ▽ More The classification of Antibody Mediated Rejection (AMR) in kidney transplant remains challenging even for experienced nephropathologists; this is partly because histological tissue stain analysis is often characterized by low inter-observer agreement and poor reproducibility. One of the implicated causes for inter-observer disagreement is the variability of tissue stain quality between (and within) pathology labs, coupled with the gradual fading of archival sections. Variations in stain colors and intensities can make tissue evaluation difficult for pathologists, ultimately affecting their ability to describe relevant morphological features. Being able to accurately predict the AMR status based on kidney histology images is crucial for improving patient treatment and care. We propose a novel pipeline to build robust deep neural networks for AMR classification based on StyPath, a histological data augmentation technique that leverages a light weight style-transfer algorithm as a means to reduce sample-specific bias. Each image was generated in 1.84 +- 0.03 seconds using a single GTX TITAN V gpu and pytorch, making it faster than other popular histological data augmentation techniques. We evaluated our model using a Monte Carlo (MC) estimate of Bayesian performance and generate an epistemic measure of uncertainty to compare both the baseline and StyPath augmented models. We also generated Grad-CAM representations of the results which were assessed by an experienced nephropathologist; we used this qualitative analysis to elucidate on the assumptions being made by each model. Our results imply that our style-transfer augmentation technique improves histological classification performance (reducing error from 14.8% to 11.5%) and generalization ability. △ Less

Submitted 9 July, 2020; originally announced July 2020.

Radiologist-Level COVID-19 Detection Using CT Scans with Detail-Oriented Capsule Networks

Authors: Aryan Mobiny, Pietro Antonio Cicalese, Samira Zare, Pengyu Yuan, Mohammadsajad Abavisani, Carol C. Wu, Jitesh Ahuja, Patricia M. de Groot, Hien Van Nguyen

Abstract: Radiographic images offer an alternative method for the rapid screening and monitoring of Coronavirus Disease 2019 (COVID-19) patients. This approach is limited by the shortage of radiology experts who can provide a timely interpretation of these images. Motivated by this challenge, our paper proposes a novel learning architecture, called Detail-Oriented Capsule Networks (DECAPS), for the automati… ▽ More Radiographic images offer an alternative method for the rapid screening and monitoring of Coronavirus Disease 2019 (COVID-19) patients. This approach is limited by the shortage of radiology experts who can provide a timely interpretation of these images. Motivated by this challenge, our paper proposes a novel learning architecture, called Detail-Oriented Capsule Networks (DECAPS), for the automatic diagnosis of COVID-19 from Computed Tomography (CT) scans. Our network combines the strength of Capsule Networks with several architecture improvements meant to boost classification accuracies. First, DECAPS uses an Inverted Dynamic Routing mechanism which increases model stability by preventing the passage of information from non-descriptive regions. Second, DECAPS employs a Peekaboo training procedure which uses a two-stage patch crop and drop strategy to encourage the network to generate activation maps for every target concept. The network then uses the activation maps to focus on regions of interest and combines both coarse and fine-grained representations of the data. Finally, we use a data augmentation method based on conditional generative adversarial networks to deal with the issue of data scarcity. Our model achieves 84.3% precision, 91.5% recall, and 96.1% area under the ROC curve, significantly outperforming state-of-the-art methods. We compare the performance of the DECAPS model with three experienced, well-trained thoracic radiologists and show that the architecture significantly outperforms them. While further studies on larger datasets are required to confirm this finding, our results imply that architectures like DECAPS can be used to assist radiologists in the CT scan mediated diagnosis of COVID-19. △ Less

Submitted 15 April, 2020; originally announced April 2020.

Multi-Objective Multi-Agent Planning for Jointly Discovering and Tracking Mobile Object

Authors: Hoa Van Nguyen, Hamid Rezatofighi, Ba-Ngu Vo, Damith C. Ranasinghe

Abstract: We consider the challenging problem of online planning for a team of agents to autonomously search and track a time-varying number of mobile objects under the practical constraint of detection range limited onboard sensors. A standard POMDP with a value function that either encourages discovery or accurate tracking of mobile objects is inadequate to simultaneously meet the conflicting goals of sea… ▽ More We consider the challenging problem of online planning for a team of agents to autonomously search and track a time-varying number of mobile objects under the practical constraint of detection range limited onboard sensors. A standard POMDP with a value function that either encourages discovery or accurate tracking of mobile objects is inadequate to simultaneously meet the conflicting goals of searching for undiscovered mobile objects whilst keeping track of discovered objects. The planning problem is further complicated by misdetections or false detections of objects caused by range limited sensors and noise inherent to sensor measurements. We formulate a novel multi-objective POMDP based on information theoretic criteria, and an online multi-object tracking filter for the problem. Since controlling multi-agent is a well known combinatorial optimization problem, assigning control actions to agents necessitates a greedy algorithm. We prove that our proposed multi-objective value function is a monotone submodular set function; consequently, the greedy algorithm can achieve a (1-1/e) approximation for maximizing the submodular multi-objective function. △ Less

Submitted 21 November, 2019; originally announced November 2019.

Comments: Accepted for publication to the Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI-20). Added algorithm 1, background on MPOMDP and OSPA

Fast CapsNet for Lung Cancer Screening

Authors: Aryan Mobiny, Hien Van Nguyen

Abstract: Lung cancer is the leading cause of cancer-related deaths in the past several years. A major challenge in lung cancer screening is the detection of lung nodules from computed tomography (CT) scans. State-of-the-art approaches in automated lung nodule classification use deep convolutional neural networks (CNNs). However, these networks require a large number of training samples to generalize well.… ▽ More Lung cancer is the leading cause of cancer-related deaths in the past several years. A major challenge in lung cancer screening is the detection of lung nodules from computed tomography (CT) scans. State-of-the-art approaches in automated lung nodule classification use deep convolutional neural networks (CNNs). However, these networks require a large number of training samples to generalize well. This paper investigates the use of capsule networks (CapsNets) as an alternative to CNNs. We show that CapsNets significantly outperforms CNNs when the number of training samples is small. To increase the computational efficiency, our paper proposes a consistent dynamic routing mechanism that results in $3\times$ speedup of CapsNet. Finally, we show that the original image reconstruction method of CapNets performs poorly on lung nodule data. We propose an efficient alternative, called convolutional decoder, that yields lower reconstruction error and higher classification accuracy. △ Less

Submitted 19 June, 2018; originally announced June 2018.

TrackerBots: Autonomous Unmanned Aerial Vehicle for Real-Time Localization and Tracking of Multiple Radio-Tagged Animals

Authors: Hoa Van Nguyen, Michael Chesser, Lian Pin Koh, S. Hamid Rezatofighi, Damith C. Ranasinghe

Abstract: Autonomous aerial robots provide new possibilities to study the habitats and behaviors of endangered species through the efficient gathering of location information at temporal and spatial granularities not possible with traditional manual survey methods. We present a novel autonomous aerial vehicle system-TrackerBots-to track and localize multiple radio-tagged animals. The simplicity of measuring… ▽ More Autonomous aerial robots provide new possibilities to study the habitats and behaviors of endangered species through the efficient gathering of location information at temporal and spatial granularities not possible with traditional manual survey methods. We present a novel autonomous aerial vehicle system-TrackerBots-to track and localize multiple radio-tagged animals. The simplicity of measuring the received signal strength indicator (RSSI) values of very high frequency (VHF) radio-collars commonly used in the field is exploited to realize a low cost and lightweight tracking platform suitable for integration with unmanned aerial vehicles (UAVs). Due to uncertainty and the nonlinearity of the system based on RSSI measurements, our tracking and planning approaches integrate a particle filter for tracking and localizing; a partially observable Markov decision process (POMDP) for dynamic path planning. This approach allows autonomous navigation of a UAV in a direction of maximum information gain to locate multiple mobile animals and reduce exploration time; and, consequently, conserve onboard battery power. We also employ the concept of a search termination criteria to maximize the number of located animals within power constraints of the aerial system. We validated our real-time and online approach through both extensive simulations and field experiments with two mobile VHF radio-tags. △ Less

Submitted 19 March, 2020; v1 submitted 5 December, 2017; originally announced December 2017.

Comments: The accepted version to the Journal of Field Robotics, published after the embargo period (12 months)

Journal ref: Journal of Field Robotics. 2019; 36: 617 - 635

Lung Cancer Screening Using Adaptive Memory-Augmented Recurrent Networks

Authors: Aryan Mobiny, Supratik Moulik, Hien Van Nguyen

Abstract: In this paper, we investigate the effectiveness of deep learning techniques for lung nodule classification in computed tomography scans. Using less than 10,000 training examples, our deep networks perform two times better than a standard radiology software. Visualization of the networks' neurons reveals semantically meaningful features that are consistent with the clinical knowledge and radiologis… ▽ More In this paper, we investigate the effectiveness of deep learning techniques for lung nodule classification in computed tomography scans. Using less than 10,000 training examples, our deep networks perform two times better than a standard radiology software. Visualization of the networks' neurons reveals semantically meaningful features that are consistent with the clinical knowledge and radiologists' perception. Our paper also proposes a novel framework for rapidly adapting deep networks to the radiologists' feedback, or change in the data due to the shift in sensor's resolution or patient population. The classification accuracy of our approach remains above 80% while popular deep networks' accuracy is around chance. Finally, we provide in-depth analysis of our framework by asking a radiologist to examine important networks' features and perform blind re-labeling of networks' mistakes. △ Less

Submitted 7 September, 2018; v1 submitted 11 October, 2017; originally announced October 2017.