Showing 1–37 of 37 results for author: Hao, Q

Batch-FPM: Random batch-update multi-parameter physical Fourier ptychography neural network

Authors: Ruiqing Sun, Delong Yang, Yiyan Su, Shaohui Zhang, Qun Hao

Abstract: Fourier Ptychographic Microscopy (FPM) is a computational imaging technique that enables high-resolution imaging over a large field of view. However, its application in the biomedical field has been limited due to the long image reconstruction time and poor noise robustness. In this paper, we propose a fast and robust FPM reconstruction method based on physical neural networks with batch update st… ▽ More Fourier Ptychographic Microscopy (FPM) is a computational imaging technique that enables high-resolution imaging over a large field of view. However, its application in the biomedical field has been limited due to the long image reconstruction time and poor noise robustness. In this paper, we propose a fast and robust FPM reconstruction method based on physical neural networks with batch update stochastic gradient descent (SGD) optimization strategy, capable of achieving attractive results with low single-to-noise ratio and correcting multiple system parameters simultaneously. Our method leverages a random batch optimization approach, breaks away from the fixed sequential iterative order and gives greater attention to high-frequency information. The proposed method has better convergence performance even for low signal-to-noise ratio data sets, such as low exposure time dark-field images. As a result, it can greatly increase the image recording and result reconstruction speed without any additional hardware modifications. By utilizing advanced deep learning optimizers and perform parallel computational scheme, our method enhances GPU computational efficiency, significantly reducing reconstruction costs. Experimental results demonstrate that our method achieves near real-time digital refocusing of a 1024 x 1024 pixels region of interest on consumer-grade GPUs. This approach significantly improves temporal resolution (by reducing the exposure time of dark-field images), noise resistance, and reconstruction speed, and therefore can efficiently promote the practical application of FPM in clinical diagnostics, digital pathology, and biomedical research, etc. In addition, we believe our algorithm scheme can help researchers quickly validate and implement FPM-related ideas. We invite requests for the full code via email. △ Less

Submitted 25 August, 2024; originally announced August 2024.

Panoramic single-pixel imaging with megapixel resolution based on rotational subdivision

Authors: Huan Cui, Jie Cao, Haoyu Zhang, Chang Zhou, Haifeng Yao, Yingbo Wang, Qun Hao

Abstract: Single-pixel imaging (SPI) using a single-pixel detector is an unconventional imaging method, which has great application prospects in many fields to realize high-performance imaging. In especial, the recent proposed catadioptric panoramic ghost imaging (CPGI) extends the application potential of SPI to high-performance imaging at a wide field of view (FOV) with recent growing demands. However, th… ▽ More Single-pixel imaging (SPI) using a single-pixel detector is an unconventional imaging method, which has great application prospects in many fields to realize high-performance imaging. In especial, the recent proposed catadioptric panoramic ghost imaging (CPGI) extends the application potential of SPI to high-performance imaging at a wide field of view (FOV) with recent growing demands. However, the resolution of CPGI is limited by hardware parameters of the digital micromirror device (DMD), which may not meet ultrahigh-resolution panoramic imaging needs that require detailed information. Therefore, to overcome the resolution limitation of CPGI, we propose a panoramic SPI based on rotational subdivision (RSPSI). The key of the proposed RSPSI is to obtain the entire panoramic scene by the rotation-scanning with a rotating mirror tilted 45°, so that one single pattern that only covers one sub-Fov with a small FOV can complete a uninterrupted modulation on the entire panoramic FOV during a once-through pattern projection. Then, based on temporal resolution subdivision, images sequence of sub-Fovs subdivided from the entire panoramic FOV can be reconstructed with pixels-level or even subpixels-level horizontal shifting adjacently. Experimental results using a proof-of-concept setup show that the panoramic image can be obtained with 10428*543 of 5,662,404 pixels, which is more than 9.6 times higher than the resolution limit of the CPGI using the same DMD. To our best knowledge, the RSPSI is the first to achieve a megapixel resolution via SPI, which can provide potential applications in fields requiring the imaging with ultrahigh-resolution and wide FOV. △ Less

Submitted 26 July, 2024; originally announced July 2024.

CityLight: A Universal Model for Coordinated Traffic Signal Control in City-scale Heterogeneous Intersections

Authors: Jinwei Zeng, Chao Yu, Xinyi Yang, Wenxuan Ao, Qianyue Hao, Jian Yuan, Yong Li, Yu Wang, Huazhong Yang

Abstract: The increasingly severe congestion problem in modern cities strengthens the significance of developing city-scale traffic signal control (TSC) methods for traffic efficiency enhancement. While reinforcement learning has been widely explored in TSC, most of them still target small-scale optimization and cannot directly scale to the city level due to unbearable resource demand. Only a few of them ma… ▽ More The increasingly severe congestion problem in modern cities strengthens the significance of developing city-scale traffic signal control (TSC) methods for traffic efficiency enhancement. While reinforcement learning has been widely explored in TSC, most of them still target small-scale optimization and cannot directly scale to the city level due to unbearable resource demand. Only a few of them manage to tackle city-level optimization, namely a thousand-scale optimization, by incorporating parameter-sharing mechanisms, but hardly have they fully tackled the heterogeneity of intersections and intricate between-intersection interactions inherent in real-world city road networks. To fill in the gap, we target at the two important challenges in adopting parameter-sharing paradigms to solve TSC: inconsistency of inner state representations for intersections heterogeneous in configuration, scale, and orders of available traffic phases; intricacy of impacts from neighborhood intersections that have various relative traffic relationships due to inconsistent phase orders and diverse relative positioning. Our method, CityLight, features a universal representation module that not only aligns the state representations of intersections by reindexing their phases based on their semantics and designing heterogeneity-preserving observations, but also encodes the narrowed relative traffic relation types to project the neighborhood intersections onto a uniform relative traffic impact space. We further attentively fuse neighborhood representations based on their competing relations and incorporate neighborhood-integrated rewards to boost coordination. Extensive experiments with hundreds to tens of thousands of intersections validate the surprising effectiveness and generalizability of CityLight, with an overall performance gain of 11.68% and a 22.59% improvement in transfer scenarios in throughput. △ Less

Submitted 28 August, 2024; v1 submitted 4 June, 2024; originally announced June 2024.

Natural-artificial hybrid swarm: Cyborg-insect group navigation in unknown obstructed soft terrain

Authors: Yang Bai, Phuoc Thanh Tran Ngoc, Huu Duoc Nguyen, Duc Long Le, Quang Huy Ha, Kazuki Kai, Yu Xiang See To, Yaosheng Deng, Jie Song, Naoki Wakamiya, Hirotaka Sato, Masaki Ogura

Abstract: Navigating multi-robot systems in complex terrains has always been a challenging task. This is due to the inherent limitations of traditional robots in collision avoidance, adaptation to unknown environments, and sustained energy efficiency. In order to overcome these limitations, this research proposes a solution by integrating living insects with miniature electronic controllers to enable roboti… ▽ More Navigating multi-robot systems in complex terrains has always been a challenging task. This is due to the inherent limitations of traditional robots in collision avoidance, adaptation to unknown environments, and sustained energy efficiency. In order to overcome these limitations, this research proposes a solution by integrating living insects with miniature electronic controllers to enable robotic-like programmable control, and proposing a novel control algorithm for swarming. Although these creatures, called cyborg insects, have the ability to instinctively avoid collisions with neighbors and obstacles while adapting to complex terrains, there is a lack of literature on the control of multi-cyborg systems. This research gap is due to the difficulty in coordinating the movements of a cyborg system under the presence of insects' inherent individual variability in their reactions to control input. In response to this issue, we propose a novel swarm navigation algorithm addressing these challenges. The effectiveness of the algorithm is demonstrated through an experimental validation in which a cyborg swarm was successfully navigated through an unknown sandy field with obstacles and hills. This research contributes to the domain of swarm robotics and showcases the potential of integrating biological organisms with robotics and control theory to create more intelligent autonomous systems with real-world applications. △ Less

Submitted 27 March, 2024; v1 submitted 26 March, 2024; originally announced March 2024.

Hybrid deep learning and physics-based neural network for programmable illumination computational microscopy

Authors: Ruiqing Sun, Delong Yang, Shaohui Zhang, Qun Hao

Abstract: Relying on either deep models or physical models are two mainstream approaches for solving inverse sample reconstruction problems in programmable illumination computational microscopy. Solutions based on physical models possess strong generalization capabilities while struggling with global optimization of inverse problems due to a lack of insufficient physical constraints. In contrast, deep learn… ▽ More Relying on either deep models or physical models are two mainstream approaches for solving inverse sample reconstruction problems in programmable illumination computational microscopy. Solutions based on physical models possess strong generalization capabilities while struggling with global optimization of inverse problems due to a lack of insufficient physical constraints. In contrast, deep learning methods have strong problem-solving abilities, but their generalization ability is often questioned because of the unclear physical principles. Besides, conventional deep models are difficult to apply to some specific scenes because of the difficulty in acquiring high-quality training data and their limited capacity to generalize across different scenarios. In this paper, to combine the advantages of deep models and physical models together, we propose a hybrid framework consisting of three sub-neural networks (two deep learning networks and one physics-based network). We first obtain a result with rich semantic information through a light deep learning neural network and then use it as the initial value of the physical network to make its output comply with physical process constraints. These two results are then used as the input of a fusion deep learning neural work which utilizes the paired features between the reconstruction results of two different models to further enhance imaging quality. The final result integrates the advantages of both deep models and physical models and can quickly solve the computational reconstruction inverse problem in programmable illumination computational microscopy and achieve better results. We verified the feasibility and effectiveness of the proposed hybrid framework with theoretical analysis and actual experiments on resolution targets and biological samples. △ Less

Submitted 17 January, 2024; originally announced March 2024.

Flexible uniform-sampling foveated Fourier single-pixel imaging

Authors: Huan Cui, Jie Cao, Qun Hao, Haoyu Zhang, Chang Zhou

Abstract: Fourier single-pixel imaging (FSI) is a data-efficient single-pixel imaging (SPI). However, there is still a serious challenge to obtain higher imaging quality using fewer measurements, which limits the development of real-time SPI. In this work, a uniform-sampling foveated FSI (UFFSI) is proposed with three features, uniform sampling, effective sampling and flexible fovea, to achieve under-sampli… ▽ More Fourier single-pixel imaging (FSI) is a data-efficient single-pixel imaging (SPI). However, there is still a serious challenge to obtain higher imaging quality using fewer measurements, which limits the development of real-time SPI. In this work, a uniform-sampling foveated FSI (UFFSI) is proposed with three features, uniform sampling, effective sampling and flexible fovea, to achieve under-sampling high-efficiency and high-quality SPI, even in a large-scale scene. First, by flexibly using the three proposed foveated pattern structures, data redundancy is reduced significantly to only require high resolution (HR) on regions of interest (ROIs), which radically reduces the need of total data number. Next, by the non-uniform weight distribution processing, non-uniform spatial sampling is transformed into uniform sampling, then the fast Fourier transform is used accurately and directly to obtain under-sampling high imaging quality with further reduced measurements. At a sampling ratio of 0.0084 referring to HR FSI with 1024*768 pixels, experimentally, by UFFSI with 255*341 cells of 89% reduction in data redundancy, the ROI has a significantly better imaging quality to meet imaging needs. We hope this work can provide a breakthrough for future real-time SPI. △ Less

Submitted 5 November, 2023; originally announced November 2023.

Comments: 7 pages,5 figures

Adaptive coded illumination Fourier ptychography microscopy based on physical neural network

Authors: Ruiqing Sun, Delong Yang, Yao Hu, Qun Hao, Xin Li, Shaohui Zhang

Abstract: Fourier Ptychographic Microscopy (FPM) is a computational technique that achieves a large space-bandwidth product imaging. It addresses the challenge of balancing a large field of view and high resolution by fusing information from multiple images taken with varying illumination angles. Nevertheless, conventional FPM framework always suffers from long acquisition time and a heavy computational bur… ▽ More Fourier Ptychographic Microscopy (FPM) is a computational technique that achieves a large space-bandwidth product imaging. It addresses the challenge of balancing a large field of view and high resolution by fusing information from multiple images taken with varying illumination angles. Nevertheless, conventional FPM framework always suffers from long acquisition time and a heavy computational burden. In this paper, we propose a novel physical neural network that generates an adaptive illumination mode by incorporating temporally-encoded illumination modes as a distinct layer, aiming to improve the acquisition and calculation efficiency. Both simulations and experiments have been conducted to validate the feasibility and effectiveness of the proposed method. It is worth mentioning that, unlike previous works that obtain the intensity of a multiplexed illumination by post-combination of each sequentially illuminated and obtained low-resolution images, our experimental data is captured directly by turning on multiple LEDs with a coded illumination pattern. Our method has exhibited state-of-the-art performance in terms of both detail fidelity and imaging velocity when assessed through a multitude of evaluative aspects. △ Less

Submitted 20 April, 2023; originally announced April 2023.

Anti-scattering medium computational ghost imaging with modified Hadamard patterns

Authors: Li-Xing Lin, Jie Cao, Qun Hao

Abstract: Illumination patterns of computational ghost imaging (CGI) systems suffer from reduced contrast when passing through a scattering medium, which causes the effective information in the reconstruction result to be drowned out by noise. A two-dimensional (2D) Gaussian filter performs linear smoothing operation on the whole image for image denoising. It can be combined with linear reconstruction algor… ▽ More Illumination patterns of computational ghost imaging (CGI) systems suffer from reduced contrast when passing through a scattering medium, which causes the effective information in the reconstruction result to be drowned out by noise. A two-dimensional (2D) Gaussian filter performs linear smoothing operation on the whole image for image denoising. It can be combined with linear reconstruction algorithms of CGI to obtain the noise-reduced results directly, without post-processing. However, it results in blurred image edges while performing denoising and, in addition, a suitable standard deviation is difficult to choose in advance, especially in an unknown scattering environment. In this work, we subtly exploit the characteristics of CGI to solve these two problems very well. A kind of modified Hadamard pattern based on the 2D Gaussian filter and the differential operation features of Hadamard-based CGI is developed. We analyze and demonstrate that using Hadamard patterns for illumination but using our developed modified Hadamard patterns for reconstruction (MHCGI) can enhance the robustness of CGI against turbid scattering medium. Our method not only helps directly obtain noise-reduced results without blurred edges but also requires only an approximate standard deviation, i.e., it can be set in advance. The experimental results on transmitted and reflected targets demonstrate the feasibility of our method. Our method helps to promote the practical application of CGI in the scattering environment. △ Less

Submitted 15 April, 2023; originally announced April 2023.

Comments: 14 pages, 7 figures

Single-shot quantitative differential phase contrast imaging combined with programmable polarization multiplexing illumination

Authors: Siying Liu, Chuanjian Zheng, Qun Hao, Xin Li, Shaohui Zhang

Abstract: We propose a single-shot quantitative differential phase contrast (DPC) method with polarization multiplexing illumination. In the illumination module of our system, the programmable LED array is divided into four quadrants and covered with polarizing films of four different polarization angles. We use a polarization camera with polarizers before the pixels in the imaging module. By matching the p… ▽ More We propose a single-shot quantitative differential phase contrast (DPC) method with polarization multiplexing illumination. In the illumination module of our system, the programmable LED array is divided into four quadrants and covered with polarizing films of four different polarization angles. We use a polarization camera with polarizers before the pixels in the imaging module. By matching the polarization angle between the polarizing films over the custom LED array and the polarizers in the camera, two sets of asymmetric illumination acquisition images can be calculated from a single-shot acquisition image. Combined with the phase transfer function, we can calculate the quantitative phase of the sample. We present the design, implementation, and experimental image data demonstrating the ability of our method to obtain quantitative phase images of the phase resolution target, as well as Hela cells. △ Less

Submitted 13 April, 2023; originally announced April 2023.

Comments: 5 pages,4figures

Resolution Improvement for OpticalCoherence Tomography based on Sparse Continuous Deconvolution

Authors: Zhengyu Qiao, Yong Huang, Qun Hao

Abstract: We propose an image resolution improvement method for optical coherence tomography (OCT) based on sparse continuous deconvolution. Traditional deconvolution techniques such as Lucy-Richardson deconvolution suffers from the artifact convergence problem after a small number of iterations, which brings limitation to practical applications. In this work, we take advantage of the prior knowledge about… ▽ More We propose an image resolution improvement method for optical coherence tomography (OCT) based on sparse continuous deconvolution. Traditional deconvolution techniques such as Lucy-Richardson deconvolution suffers from the artifact convergence problem after a small number of iterations, which brings limitation to practical applications. In this work, we take advantage of the prior knowledge about the sample sparsity and continuity to constrain the deconvolution iteration. Sparsity is used to achieve the resolution improvement through the resolution preserving regularization term. And the continuity based on the correlation of the grayscale values in different directions is introduced to mitigate excessive image sparsity and noise reduction through the continuity regularization term. The Bregman splitting technique is then used to solve the resulting optimization problem. Both the numerical simulation study and experimental study on phantoms and biological samples show that our method can suppress artefacts of traditional deconvolution techniques effectively. Meanwhile, clear resolution improvement is demonstrated. It achieved nearly twofold resolution improvement for phantom beads image that can be quantitatively evaluated △ Less

Submitted 10 April, 2023; originally announced April 2023.

Robust full-pose-parameter estimation for the LED array in Fourier ptychographic microscopy

Authors: Chuanjian Zheng, Shaohui Zhang, Delong Yang, Guocheng Zhou, Yao Hu, Qun Hao

Abstract: Fourier ptychographic microscopy (FPM) can achieve quantitative phase imaging with a large space-bandwidth product by synthesizing a set of low-resolution intensity images captured under angularly varying illuminations. Determining accurate illumination angles is critical because the consistency between actual systematic parameters and those used in the recovery algorithm is essential for high-qua… ▽ More Fourier ptychographic microscopy (FPM) can achieve quantitative phase imaging with a large space-bandwidth product by synthesizing a set of low-resolution intensity images captured under angularly varying illuminations. Determining accurate illumination angles is critical because the consistency between actual systematic parameters and those used in the recovery algorithm is essential for high-quality imaging. This paper presents a full-pose-parameter and physics-based method for calibrating illumination angles. Using a physics-based model constructed with general knowledge of the employed microscope and the brightfield-to-darkfield boundaries inside captured images, we can solve for the full-pose parameters of misplaced LED array, which consist of the distance between the sample and the LED array, two orthogonal lateral shifts, one in-plane rotation angle, and two tilt angles, to correct illumination angles precisely. The feasibility and effectiveness of the proposed method for recovering random or remarkable pose parameters have been demonstrated by both qualitative and quantitative experiments. Due to the completeness of the pose parameters, the clarity of the physical model, and the high robustness for arbitrary misalignments, our method can significantly facilitate the design, implementation, and application of concise and robust FPM platforms. △ Less

Submitted 11 June, 2022; originally announced June 2022.

Comments: 14 pages, 6 figures

Temporally and Spatially variant-resolution illumination patterns in computational ghost imaging

Authors: Dong Zhou, Jie Cao, Huan Cui, Li-Xing Lin, Haoyu Zhang, Yingqiang Zhang, Qun Hao

Abstract: Conventional computational ghost imaging (CGI) uses light carrying a sequence of patterns with uniform-resolution to illuminate the object, then performs correlation calculation based on the light intensity value reflected by the target and the preset patterns to obtain object image. It requires a large number of measurements to obtain high-quality images, especially if high-resolution images are… ▽ More Conventional computational ghost imaging (CGI) uses light carrying a sequence of patterns with uniform-resolution to illuminate the object, then performs correlation calculation based on the light intensity value reflected by the target and the preset patterns to obtain object image. It requires a large number of measurements to obtain high-quality images, especially if high-resolution images are to be obtained. To solve this problem, we developed temporally variable-resolution illumination patterns, replacing the conventional uniform-resolution illumination patterns with a sequence of patterns of different imaging resolutions. In addition, we propose to combine temporally variable-resolution illumination patterns and spatially variable-resolution structure to develop temporally and spatially variable-resolution (TSV) illumination patterns, which not only improve the imaging quality of the region of interest (ROI) but also improve the robustness to noise. The methods using proposed illumination patterns are verified by simulations and experiments compared with CGI. For the same number of measurements, the method using temporally variable-resolution illumination patterns has better imaging quality than CGI, but it is less robust to noise. The method using TSV illumination patterns has better imaging quality in ROI than the method using temporally variable-resolution illumination patterns and CGI under the same number of measurements. We also experimentally verify that the method using TSV patterns have better imaging performance when applied to higher resolution imaging. The proposed methods are expected to solve the current computational ghost imaging that is difficult to achieve high-resolution and high-quality imaging. △ Less

Submitted 14 May, 2022; v1 submitted 5 May, 2022; originally announced May 2022.

Pose correction scheme for camera-scanning Fourier ptychography based on camera calibration and homography transform

Authors: Baiqi Cui, Shaohui Zhang, Yechao Wang, Yao Hu, Qun Hao

Abstract: Fourier ptychography (FP), as a computational imaging method, is a powerful tool to improve imaging resolution. Camera-scanning Fourier ptychography extends the application of FP from micro to macro creatively. Due to the non-ideal scanning of the camera driven by the mechanical translation stage, the pose error of the camera occurs, greatly degrading the reconstruction quality, while a precise tr… ▽ More Fourier ptychography (FP), as a computational imaging method, is a powerful tool to improve imaging resolution. Camera-scanning Fourier ptychography extends the application of FP from micro to macro creatively. Due to the non-ideal scanning of the camera driven by the mechanical translation stage, the pose error of the camera occurs, greatly degrading the reconstruction quality, while a precise translation stage is expensive and not suitable for wide-range imaging. Here, to improve the imaging performance of camera-scanning Fourier ptychography, we propose a pose correction scheme based on camera calibration and homography transform approaches. The scheme realizes the accurate alignment of data set and location error correction in the frequency domain. Simulation and experimental results demonstrate this method can optimize the reconstruction results and realize high-quality imaging effectively. Combined with the feature recognition algorithm, the scheme provides the possibility for applying FP in remote sensing imaging and space imaging. △ Less

Submitted 31 March, 2022; originally announced April 2022.

Window Filtering Algorithm for Pulsed Light Coherent Combining of Low Repetition Frequency

Authors: Jiali Zhang, Jie Cao, Qun Hao, Yang Cheng, Liquan Dong, Bin Han, Xuesheng Liu

Abstract: The multi-dithering method has been well verified in phase locking of polarization coherent combination experiment. However, it is hard to apply to low repetition frequency pulsed lasers, since there exists an overlap frequency domain between pulse laser and the amplitude phase noise and traditional filters cannot effectively separate phase noise. Aiming to solve the problem in this paper, we prop… ▽ More The multi-dithering method has been well verified in phase locking of polarization coherent combination experiment. However, it is hard to apply to low repetition frequency pulsed lasers, since there exists an overlap frequency domain between pulse laser and the amplitude phase noise and traditional filters cannot effectively separate phase noise. Aiming to solve the problem in this paper, we propose a novel method of pulse noise detection, identification, and filtering based on the autocorrelation characteristics between noise signals. In the proposed algorithm, a self-designed window algorithm is used to identify the pulse, and then the pulse signal group in the window is replaced by interpolation, which effectively filter the pulse signal doped in the phase noise within 0.1 ms. After filtering the pulses in the phase noise, the phase difference of two pulsed beams (10 kHz) is successfully compensated to zero in 1 ms, and the coherent combination of closed-loop phase lock is realized. At the same time, the phase correction times are few, the phase lock effect is stable, and the final light intensity increases to the ideal value (0.9 Imax). △ Less

Submitted 3 March, 2022; originally announced March 2022.

Parallel Fourier Ptychography reconstruction

Authors: Guocheng Zhou, Shaohui Zhang, Yao Hu, Lei Cao, Yong Huang, Qun Hao

Abstract: Fourier ptychography has attracted a wide range of focus for its ability of large space-bandwidth-produce, and quantative phase measurement. It is a typical computational imaging technique which refers to optimizing both the imaging hardware and reconstruction algorithms simultaneously. The data redundancy and inverse problem algorithms are the sources of FPM's excellent performance. But at the sa… ▽ More Fourier ptychography has attracted a wide range of focus for its ability of large space-bandwidth-produce, and quantative phase measurement. It is a typical computational imaging technique which refers to optimizing both the imaging hardware and reconstruction algorithms simultaneously. The data redundancy and inverse problem algorithms are the sources of FPM's excellent performance. But at the same time, this large amount of data processing and complex algorithms also greatly reduce the imaging speed. In this article, we propose a parallel Fourier ptychography reconstruction framework consisting of three levels of parallel computing parts and implemented it with both central processing unit (CPU) and compute unified device architecture (CUDA) platform. In the conventional FPM reconstruction framework, the sample image is divided into multiple sub-regions for separately processing because the illumination angles for different subregions are varied for the same LED and different subregions contain different defocus distances due to the non-planar distribution or non-ideal posture of biological sample. We first build a parallel computing sub-framework in spatial domain based on the above-mentioned characteristics. And then, by utilizing the sequential characteristics of different spectrum regions to update, a parallel computing sub-framework in the spectrum domain is carried out in our scheme. The feasibility of the proposed parallel FPM reconstruction framework is verified with different experimental results acquired with the system we built. △ Less

Submitted 3 March, 2022; originally announced March 2022.

Comments: 12 pages with 11 figures

Fourier ptychography multi-parameter neural network with composite physical priori optimization

Authors: Delong Yang, Shaohui Zhang, Chuanjian Zheng, Guocheng Zhou, Lei Cao, Yao Hu, Qun Hao

Abstract: Fourier ptychography microscopy(FP) is a recently developed computational imaging approach for microscopic super-resolution imaging. By turning on each light-emitting-diode (LED) located on different position on the LED array sequentially and acquiring the corresponding images that contain different spatial frequency components, high spatial resolution and quantitative phase imaging can be achieve… ▽ More Fourier ptychography microscopy(FP) is a recently developed computational imaging approach for microscopic super-resolution imaging. By turning on each light-emitting-diode (LED) located on different position on the LED array sequentially and acquiring the corresponding images that contain different spatial frequency components, high spatial resolution and quantitative phase imaging can be achieved in the case of large field-of-view. Nevertheless, FPM has high requirements for the system construction and data acquisition processes, such as precise LEDs position, accurate focusing and appropriate exposure time, which brings many limitations to its practical applications. In this paper, inspired by artificial neural network, we propose a Fourier ptychography multi-parameter neural network (FPMN) with composite physical prior optimization. A hybrid parameter determination strategy combining physical imaging model and data-driven network training is proposed to recover the multi layers of the network corresponding to different physical parameters, including sample complex function, system pupil function, defocus distance, LED array position deviation and illumination intensity fluctuation, etc. Among these parameters, LED array position deviation is recovered based on the features of brightfield to darkfield transition low-resolution images while the others are recovered in the process of training of the neural network. The feasibility and effectiveness of FPMN are verified through simulations and actual experiments. Therefore FPMN can evidently reduce the requirement for practical applications of FPM. △ Less

Submitted 17 February, 2022; originally announced February 2022.

Comments: 13 pages, 12 figures, solving inverse problem of computational imaging by neural network

Robotic Wireless Energy Transfer in Dynamic Environments: System Design and Experimental Validation

Authors: Shuai Wang, Ruihua Han, Yuncong Hong, Qi Hao, Miaowen Wen, Leila Musavian, Shahid Mumtaz, Derrick Wing Kwan Ng

Abstract: Wireless energy transfer (WET) is a ground-breaking technology for cutting the last wire between mobile sensors and power grids in smart cities. Yet, WET only offers effective transmission of energy over a short distance. Robotic WET is an emerging paradigm that mounts the energy transmitter on a mobile robot and navigates the robot through different regions in a large area to charge remote energy… ▽ More Wireless energy transfer (WET) is a ground-breaking technology for cutting the last wire between mobile sensors and power grids in smart cities. Yet, WET only offers effective transmission of energy over a short distance. Robotic WET is an emerging paradigm that mounts the energy transmitter on a mobile robot and navigates the robot through different regions in a large area to charge remote energy harvesters. However, it is challenging to determine the robotic charging strategy in an unknown and dynamic environment due to the uncertainty of obstacles. This paper proposes a hardware-in-the-loop joint optimization framework that offers three distinctive features: 1) efficient model updates and re-optimization based on the last-round experimental data; 2) iterative refinement of the anchor list for adaptation to different environments; 3) verification of algorithms in a high-fidelity Gazebo simulator and a multi-robot testbed. Experimental results show that the proposed framework significantly saves the WET mission completion time while satisfying collision avoidance and energy harvesting constraints. △ Less

Submitted 10 February, 2022; v1 submitted 29 January, 2022; originally announced January 2022.

Comments: single column, 18 pages, 6 figures, to appear in IEEE Communications Magazine

Journal ref: IEEE Communications Magazine, Mar. 2022

Runtime Safety Assurance for Learning-enabled Control of Autonomous Driving Vehicles

Authors: Shengduo Chen, Yaowei Sun, Dachuan Li, Qiang Wang, Qi Hao, Joseph Sifakis

Abstract: Providing safety guarantees for Autonomous Vehicle (AV) systems with machine-learning-based controllers remains a challenging issue. In this work, we propose Simplex-Drive, a framework that can achieve runtime safety assurance for machine-learning enabled controllers of AVs. The proposed Simplex-Drive consists of an unverified Deep Reinforcement Learning (DRL)-based advanced controller (AC) that a… ▽ More Providing safety guarantees for Autonomous Vehicle (AV) systems with machine-learning-based controllers remains a challenging issue. In this work, we propose Simplex-Drive, a framework that can achieve runtime safety assurance for machine-learning enabled controllers of AVs. The proposed Simplex-Drive consists of an unverified Deep Reinforcement Learning (DRL)-based advanced controller (AC) that achieves desirable performance in complex scenarios, a Velocity-Obstacle (VO) based baseline safe controller (BC) with provably safety guarantees, and a verified mode management unit that monitors the operation status and switches the control authority between AC and BC based on safety-related conditions. We provide a formal correctness proof of Simplex-Drive and conduct a lane-changing case study in dense traffic scenarios. The simulation experiment results demonstrate that Simplex-Drive can always ensure operation safety without sacrificing control performance, even if the DRL policy may lead to deviations from the safe status. △ Less

Submitted 27 September, 2021; originally announced September 2021.

MSC Class: 68T40 ACM Class: I.2.9

Unit-Modulus Wireless Federated Learning Via Penalty Alternating Minimization

Authors: Shuai Wang, Dachuan Li, Rui Wang, Qi Hao, Yik-Chung Wu, Derrick Wing Kwan Ng

Abstract: Wireless federated learning (FL) is an emerging machine learning paradigm that trains a global parametric model from distributed datasets via wireless communications. This paper proposes a unit-modulus wireless FL (UMWFL) framework, which simultaneously uploads local model parameters and computes global model parameters via optimized phase shifting. The proposed framework avoids sophisticated base… ▽ More Wireless federated learning (FL) is an emerging machine learning paradigm that trains a global parametric model from distributed datasets via wireless communications. This paper proposes a unit-modulus wireless FL (UMWFL) framework, which simultaneously uploads local model parameters and computes global model parameters via optimized phase shifting. The proposed framework avoids sophisticated baseband signal processing, leading to both low communication delays and implementation costs. A training loss bound is derived and a penalty alternating minimization (PAM) algorithm is proposed to minimize the nonconvex nonsmooth loss bound. Experimental results in the Car Learning to Act (CARLA) platform show that the proposed UMWFL framework with PAM algorithm achieves smaller training losses and testing errors than those of the benchmark scheme. △ Less

Submitted 31 August, 2021; originally announced August 2021.

Comments: IEEE Global Communications Conference 2021. arXiv admin note: substantial text overlap with arXiv:2101.12051

Capture Uncertainties in Deep Neural Networks for Safe Operation of Autonomous Driving Vehicles

Authors: Liuhui Ding, Dachuan Li, Bowen Liu, Wenxing Lan, Bing Bai, Qi Hao, Weipeng Cao, Ke Pei

Abstract: Uncertainties in Deep Neural Network (DNN)-based perception and vehicle's motion pose challenges to the development of safe autonomous driving vehicles. In this paper, we propose a safe motion planning framework featuring the quantification and propagation of DNN-based perception uncertainties and motion uncertainties. Contributions of this work are twofold: (1) A Bayesian Deep Neural network mode… ▽ More Uncertainties in Deep Neural Network (DNN)-based perception and vehicle's motion pose challenges to the development of safe autonomous driving vehicles. In this paper, we propose a safe motion planning framework featuring the quantification and propagation of DNN-based perception uncertainties and motion uncertainties. Contributions of this work are twofold: (1) A Bayesian Deep Neural network model which detects 3D objects and quantitatively captures the associated aleatoric and epistemic uncertainties of DNNs; (2) An uncertainty-aware motion planning algorithm (PU-RRT) that accounts for uncertainties in object detection and ego-vehicle's motion. The proposed approaches are validated via simulated complex scenarios built in CARLA. Experimental results show that the proposed motion planning scheme can cope with uncertainties of DNN-based perception and vehicle motion, and improve the operational safety of autonomous vehicles while still achieving desirable efficiency. △ Less

Submitted 11 August, 2021; originally announced August 2021.

Comments: To appear in the 19th IEEE International Symposium on Parallel and Distributed Processing with Applications (IEEE ISPA 2021)

MSC Class: 68T40 ACM Class: I.2.9

Omnidirectional ghost imaging system and unwrapping-free panoramic ghost imaging

Authors: Huan Cui, Jie Cao, Qun Hao, Dong Zhou, Mingyuan Tang, Kaiyu Zhang, Yingqiang Zhang

Abstract: Ghost imaging (GI) is a novel imaging method, which can reconstruct the object information by the light intensity correlation measurements. However, at present, the field of view (FOV) is limited to the illuminating range of the light patterns. To enlarge FOV of GI efficiently, here we proposed the omnidirectional ghost imaging system (OGIS), which can achieve a 360° omnidirectional FOV at one sho… ▽ More Ghost imaging (GI) is a novel imaging method, which can reconstruct the object information by the light intensity correlation measurements. However, at present, the field of view (FOV) is limited to the illuminating range of the light patterns. To enlarge FOV of GI efficiently, here we proposed the omnidirectional ghost imaging system (OGIS), which can achieve a 360° omnidirectional FOV at one shot only by adding a curved mirror. Moreover, by designing the retina-like annular patterns with log-polar patterns, OGIS can obtain unwrapping-free undistorted panoramic images with uniform resolution, which opens up a new way for the application of GI. △ Less

Submitted 11 August, 2021; originally announced August 2021.

Optimization of retina-like illumination patterns in ghost imaging

Authors: Jie Cao, Dong Zhou, Ying-Qiang Zhang, Huan Cui, Fang-Hua Zhang, Qun Hao

Abstract: Ghost imaging (GI) reconstructs images using a single-pixel or bucket detector, which has the advantages of scattering robustness, wide spectrum and beyond-visual-field imaging. However, this technique needs large amount of measurements to obtain a sharp image. There have been a lot of methods proposed to overcome this disadvantage. Retina-like patterns, as one of the compressive sensing approache… ▽ More Ghost imaging (GI) reconstructs images using a single-pixel or bucket detector, which has the advantages of scattering robustness, wide spectrum and beyond-visual-field imaging. However, this technique needs large amount of measurements to obtain a sharp image. There have been a lot of methods proposed to overcome this disadvantage. Retina-like patterns, as one of the compressive sensing approaches, enhance the imaging quality of region of interest (ROI) while not increase measurements. The design of the retina-like patterns determines the performance of the ROI in the reconstructed image. Unlike the conventional method to fill in ROI with random patterns, we propose to optimize retina-like patterns by filling in the ROI with the patterns containing the sparsity prior of objects. This proposed method is verified by simulations and experiments compared with conventional GI, retina-like GI and GI using patterns optimized by principal component analysis. The method using optimized retina-like patterns obtain the best imaging quality in ROI than other methods. Meanwhile, the good generalization ability of the optimized retina-like pattern is also verified. While designing the size and position of the ROI of retina-like pattern, the feature information of the target can be obtained to optimize the pattern of ROI. This proposed method paves the way for realizing high-quality GI. △ Less

Submitted 2 August, 2021; originally announced August 2021.

Complementary Fourier single-pixel imaging

Authors: Dong Zhou, Jie Cao, Huan Cui, Qun Hao, Bing-Kun Chen, Kai Lin

Abstract: Single-pixel imaging, with the advantages of a wide spectrum, beyond-visual-field imaging, and robustness to light scattering, has attracted increasing attention in recent years. Fourier single-pixel imaging (FSI) can reconstruct sharp images under sub-Nyquist sampling. However, the conventional FSI has difficulty with balancing the imaging quality and efficiency. To overcome this issue, we propos… ▽ More Single-pixel imaging, with the advantages of a wide spectrum, beyond-visual-field imaging, and robustness to light scattering, has attracted increasing attention in recent years. Fourier single-pixel imaging (FSI) can reconstruct sharp images under sub-Nyquist sampling. However, the conventional FSI has difficulty with balancing the imaging quality and efficiency. To overcome this issue, we proposed a novel approach called complementary Fourier single-pixel imaging (CFSI) to reduce measurements while retaining its robustness. The complementary nature of Fourier patterns based on a four-step phase-shift algorithm is combined with the complementary nature of a digital micromirror device. CFSI only requires two phase-shifted patterns to obtain one Fourier spectral value. Four light intensity values are obtained by load the two patterns, and the spectral value is calculated through differential measurement, which has good robustness to noise. The proposed method is verified by simulations and experiments compared with FSI based on two-, three-, and four-step phase shift algorithms. CFSI performed better than the other methods under the condition that the best imaging quality of CFSI is not reached. The reported technique provides an alternative approach to realize real-time and high-quality imaging. △ Less

Submitted 2 August, 2021; originally announced August 2021.

Safety-enhanced UAV Path Planning with Spherical Vector-based Particle Swarm Optimization

Authors: Manh Duong Phung, Quang Phuc Ha

Abstract: This paper presents a new algorithm named spherical vector-based particle swarm optimization (SPSO) to deal with the problem of path planning for unmanned aerial vehicles (UAVs) in complicated environments subjected to multiple threats. A cost function is first formulated to convert the path planning into an optimization problem that incorporates requirements and constraints for the feasible and s… ▽ More This paper presents a new algorithm named spherical vector-based particle swarm optimization (SPSO) to deal with the problem of path planning for unmanned aerial vehicles (UAVs) in complicated environments subjected to multiple threats. A cost function is first formulated to convert the path planning into an optimization problem that incorporates requirements and constraints for the feasible and safe operation of the UAV. SPSO is then used to find the optimal path that minimizes the cost function by efficiently searching the configuration space of the UAV via the correspondence between the particle position and the speed, turn angle and climb/dive angle of the UAV. To evaluate the performance of SPSO, eight benchmarking scenarios have been generated from real digital elevation model maps. The results show that the proposed SPSO outperforms not only other particle swarm optimization (PSO) variants including the classic PSO, phase angle-encoded PSO and quantum-behave PSO but also other state-of-the-art metaheuristic optimization algorithms including the genetic algorithm (GA), artificial bee colony (ABC), and differential evolution (DE) in most scenarios. In addition, experiments have been conducted to demonstrate the validity of the generated paths for real UAV operations. Source code of the algorithm can be found at https://github.com/duongpm/SPSO. △ Less

Submitted 13 April, 2021; originally announced April 2021.

Journal ref: Applied Soft Computing, Volume 107, August 2021, 107376

Fast digital refocusing and depth of field extended Fourier ptychography microscopy

Authors: G. Zhou, S. Zhang, C. Zheng, T. Li, Y. Hu, Q. Hao

Abstract: Fourier ptychography microscopy (FPM), sharing its roots with synthetic aperture technique and phase retrieval method, is a recently developed computational microscopic super-resolution technique. By turning on the light-emitting diode (LED) elements sequentially and acquiring the corresponding images that contain different spatial frequencies, FPM can achieve a wide field-of-view (FOV), high-spat… ▽ More Fourier ptychography microscopy (FPM), sharing its roots with synthetic aperture technique and phase retrieval method, is a recently developed computational microscopic super-resolution technique. By turning on the light-emitting diode (LED) elements sequentially and acquiring the corresponding images that contain different spatial frequencies, FPM can achieve a wide field-of-view (FOV), high-spatial-resolution imaging, and phase recovery simultaneously. Conventional FPM assumes that the sample is sufficiently thin and strictly in focus. Nevertheless, even for a relatively thin sample, the non-planar distribution characteristics and the non-ideal position/posture of the sample will cause all or part of FOV to be defocused. In this paper, we proposed a fast digital refocusing and depth-of-field (DOF) extended FPM strategy by taking the advantages of image lateral shift caused by sample defocusing and varied-angle illuminations. The lateral shift amount is proportional to the defocus distance and the tangent of the illumination angle. Instead of searching the optimal defocus distance in optimization strategy, which is time-consuming, the defocus distance of each subregion of the sample can be precisely and quickly obtained by calculating the relative lateral shift amounts corresponding to different oblique illuminations. And then, the digital refocusing strategy rooting in the Fresnel propagator is integrated into the FPM framework to achieve the high-resolution and phase information reconstruction for each part of the sample, which means the DOF the FPM is effectively extended. The feasibility of the proposed method in fast digital refocusing and FOV extending is verified in the actual experiments with the USAF chart and biological samples. △ Less

Submitted 5 May, 2021; v1 submitted 13 April, 2021; originally announced April 2021.

Comments: 13 pages, 9 figures

Learning Centric Power Allocation for Edge Intelligence

Authors: Shuai Wang, Rui Wang, Qi Hao, Yik-Chung Wu, H. Vincent Poor

Abstract: While machine-type communication (MTC) devices generate massive data, they often cannot process this data due to limited energy and computation power. To this end, edge intelligence has been proposed, which collects distributed data and performs machine learning at the edge. However, this paradigm needs to maximize the learning performance instead of the communication throughput, for which the cel… ▽ More While machine-type communication (MTC) devices generate massive data, they often cannot process this data due to limited energy and computation power. To this end, edge intelligence has been proposed, which collects distributed data and performs machine learning at the edge. However, this paradigm needs to maximize the learning performance instead of the communication throughput, for which the celebrated water-filling and max-min fairness algorithms become inefficient since they allocate resources merely according to the quality of wireless channels. This paper proposes a learning centric power allocation (LCPA) method, which allocates radio resources based on an empirical classification error model. To get insights into LCPA, an asymptotic optimal solution is derived. The solution shows that the transmit powers are inversely proportional to the channel gain, and scale exponentially with the learning parameters. Experimental results show that the proposed LCPA algorithm significantly outperforms other power allocation algorithms. △ Less

Submitted 21 July, 2020; originally announced July 2020.

Comments: 6 pages, 7 figures, in Proc. IEEE ICC 2020, Wireless Communications Symposium, Jun. 2020. arXiv admin note: substantial text overlap with arXiv:1911.04922

Sensing Data Fusion for Enhanced Indoor Air Quality Monitoring

Authors: Q. P. Ha, S. Metia, M. D. Phung

Abstract: Multisensor fusion of air pollutant data in smart buildings remains an important input to address the well-being and comfort perceived by their inhabitants. An integrated sensing system is part of a smart building where real-time indoor air quality data are monitored round the clock using sensors and operating in the Internet-of-Things (IoT) environment. In this work, we propose an air quality man… ▽ More Multisensor fusion of air pollutant data in smart buildings remains an important input to address the well-being and comfort perceived by their inhabitants. An integrated sensing system is part of a smart building where real-time indoor air quality data are monitored round the clock using sensors and operating in the Internet-of-Things (IoT) environment. In this work, we propose an air quality management system merging indoor air quality index (IAQI) and humidex into an enhanced indoor air quality index (EIAQI) by using sensor data on a real-time basis. Here, indoor air pollutant levels are measured by a network of waspmote sensors while IAQI and humidex data are fused together using an extended fractional-order Kalman filter (EFKF). According to the obtained EIAQI, overall air quality alerts are provided in a timely fashion for accurate prediction with enhanced performance against measurement noise and nonlinearity. The estimation scheme is implemented by using the fractional-order modeling and control (FOMCON) toolbox. A case study is analysed to prove the effectiveness and validity of the proposed approach. △ Less

Submitted 7 January, 2020; originally announced January 2020.

Comments: Published in IEEE Sensors Journal (Early Access)

Journal ref: IEEE Sensors Journal (2020)

Crack Detection Using Enhanced Hierarchical Convolutional Neural Networks

Authors: Qiuchen Zhu, Manh Duong Phung, Quang Ha

Abstract: Unmanned aerial vehicles (UAV) are expected to replace human in hazardous tasks of surface inspection due to their flexibility in operating space and capability of collecting high quality visual data. In this study, we propose enhanced hierarchical convolutional neural networks (HCNN) to detect cracks from image data collected by UAVs. Unlike traditional HCNN, here a set of branch networks is util… ▽ More Unmanned aerial vehicles (UAV) are expected to replace human in hazardous tasks of surface inspection due to their flexibility in operating space and capability of collecting high quality visual data. In this study, we propose enhanced hierarchical convolutional neural networks (HCNN) to detect cracks from image data collected by UAVs. Unlike traditional HCNN, here a set of branch networks is utilised to reduce the obscuration in the down-sampling process. Moreover, the feature preserving blocks combine the current and previous terms from the convolutional blocks to provide input to the loss functions. As a result, the weights of resized images can be reduced to minimise the information loss. Experiments on images of different crack datasets have been carried out to demonstrate the effectiveness of proposed HCNN. △ Less

Submitted 13 December, 2019; originally announced December 2019.

Comments: In Proceedings of Australasian Conference on Robotics and Automation 2019 (ACRA), Adelaide, Australia

High signal-to-noise ratio reconstruction of low bit-depth optical coherence tomography using deep learning

Authors: Qiangjiang Hao, Kang Zhou, Jianlong Yang, Liyang Fang, Zhengjie Chai, Yuhui Ma, Yan Hu, Shenghua Gao, Jiang Liu

Abstract: Reducing the bit-depth is an effective approach to lower the cost of optical coherence tomography (OCT) systems and increase the transmission efficiency in data acquisition and telemedicine. However, a low bit-depth will lead to the degeneration of the detection sensitivity thus reduce the signal-to-noise ratio (SNR) of OCT images. In this paper, we propose to use deep learning for the reconstruct… ▽ More Reducing the bit-depth is an effective approach to lower the cost of optical coherence tomography (OCT) systems and increase the transmission efficiency in data acquisition and telemedicine. However, a low bit-depth will lead to the degeneration of the detection sensitivity thus reduce the signal-to-noise ratio (SNR) of OCT images. In this paper, we propose to use deep learning for the reconstruction of the high SNR OCT images from the low bit-depth acquisition. Its feasibility was preliminarily evaluated by applying the proposed method to the quantized $3\sim8$-bit data from native 12-bit interference fringes. We employed a pixel-to-pixel generative adversarial network architecture in the low to high bit-depth OCT image transition. Retinal OCT data of a healthy subject from a homemade spectral-domain OCT system was used in the study. Extensively qualitative and quantitative results show this deep-learning-based approach could significantly improve the SNR of the low bit-depth OCT images especially at the choroidal region. Superior similarity and SNR between the reconstructed images and the original 12-bit OCT images could be derived when the bit-depth $\geq 5$. This work demonstrates the proper integration of OCT and deep learning could benefit the development of healthcare in low-resource settings. △ Less

Submitted 11 February, 2020; v1 submitted 12 October, 2019; originally announced October 2019.

Comments: submitted

Reconfigurable Multi-UAV Formation Using Angle-Encoded PSO

Authors: V. T. Hoang, M. D. Phung, T. H. Dinh, Q. Zhu, Q. P. Ha

Abstract: In this paper, we propose an algorithm for the formation of multiple UAVs used in vision-based inspection of infrastructure. A path planning algorithm is first developed by using a variant of the particle swarm optimisation, named theta-PSO, to generate a feasible path for the overall formation configuration taken into account the constraints for visual inspection. Here, we introduced a cost funct… ▽ More In this paper, we propose an algorithm for the formation of multiple UAVs used in vision-based inspection of infrastructure. A path planning algorithm is first developed by using a variant of the particle swarm optimisation, named theta-PSO, to generate a feasible path for the overall formation configuration taken into account the constraints for visual inspection. Here, we introduced a cost function that includes various constraints on flight safety and visual inspection. A reconfigurable topology is then added based on the use of intermediate waypoints to allow the formation to avoid collision with obstacles during operation. The planned path and formation are then combined to derive the trajectory and velocity profiles for each UAV. Experiments have been conducted for the task of inspecting a light rail bridge. The results confirmed the validity and effectiveness of the proposed algorithm. △ Less

Submitted 7 September, 2019; originally announced September 2019.

Comments: Pages 1670 - 1675

Journal ref: 2019 IEEE 15th International Conference on Automation Science and Engineering (CASE)

System Architecture for Real-time Surface Inspection Using Multiple UAVs

Authors: Van Truong Hoang, Manh Duong Phung, Tran Hiep Dinh, Quang P. Ha

Abstract: This paper presents a real-time control system for surface inspection using multiple unmanned aerial vehicles (UAVs). The UAVs are coordinated in a specific formation to collect data of the inspecting objects. The communication platform for data transmission is based on the Internet of Things (IoT). In the proposed architecture, the UAV formation is established via using the angle-encoded particle… ▽ More This paper presents a real-time control system for surface inspection using multiple unmanned aerial vehicles (UAVs). The UAVs are coordinated in a specific formation to collect data of the inspecting objects. The communication platform for data transmission is based on the Internet of Things (IoT). In the proposed architecture, the UAV formation is established via using the angle-encoded particle swarm optimisation to generate an inspecting path and redistribute it to each UAV where communication links are embedded with an IoT board for network and data processing capabilities. Data collected are transmitted in real time through the network to remote computational units. To detect potential damage or defects, an online image processing technique is proposed and implemented based on histograms. Extensive simulation, experiments and comparisons have been conducted to verify the validity and performance of the proposed system. △ Less

Submitted 7 July, 2019; originally announced July 2019.

Journal ref: IEEE Systems Journal, pp.1-12, 2019

Modelling and Fast Terminal Sliding Mode Control for Mirror-based Pointing Systems

Authors: Ansu Man Singh, Manh Duong Phung, Quang Ha

Abstract: In this paper, we present a new discrete-time Fast Terminal Sliding Mode (FTSM) controller for mirror-based pointing systems. We first derive the decoupled model of those systems and then estimate the parameters using a nonlinear least-square identification method. Based on the derived model, we design a FTSM sliding manifold in the continuous domain. We then exploit the Euler discretization on th… ▽ More In this paper, we present a new discrete-time Fast Terminal Sliding Mode (FTSM) controller for mirror-based pointing systems. We first derive the decoupled model of those systems and then estimate the parameters using a nonlinear least-square identification method. Based on the derived model, we design a FTSM sliding manifold in the continuous domain. We then exploit the Euler discretization on the designed FTSM sliding surfaces to synthesize a discrete-time controller. Furthermore, we improve the transient dynamics of the sliding surface by adding a linear term. Finally, we prove the stability of the proposed controller based on the Sarpturk reaching condition. Extensive simulations, followed by comparisons with the Terminal Sliding Mode (TSM) and Model Predictive Control (MPC) have been carried out to evaluate the effectiveness of the proposed approach. A comparative study with data obtained from a real-time experiment was also conducted. The results indicate the advantage of the proposed method over the other techniques. △ Less

Submitted 19 December, 2018; originally announced December 2018.

Comments: In Proceedings of the 15th International Conference on Control, Automation, Robotics and Vision (ICARCV 2018)

Adaptive twisting sliding mode control for quadrotor unmanned aerial vehicles

Authors: V. T. Hoang, M. D. Phung, Q. P. Ha

Abstract: This work addresses the problem of robust attitude control of quadcopters. First, the mathematical model of the quadcopter is derived considering factors such as nonlinearity, external disturbances, uncertain dynamics and strong coupling. An adaptive twisting sliding mode control algorithm is then developed with the objective of controlling the quadcopter to track desired attitudes under various c… ▽ More This work addresses the problem of robust attitude control of quadcopters. First, the mathematical model of the quadcopter is derived considering factors such as nonlinearity, external disturbances, uncertain dynamics and strong coupling. An adaptive twisting sliding mode control algorithm is then developed with the objective of controlling the quadcopter to track desired attitudes under various conditions. For this, the twisting sliding mode control law is modified with a proposed gain adaptation scheme to improve the control transient and tracking performance. Extensive simulation studies and comparisons with experimental data have been carried out for a Solo quadcopter. The results show that the proposed control scheme can achieve strong robustness against disturbances while is adaptable to parametric variations. △ Less

Submitted 5 June, 2018; originally announced June 2018.

Comments: 2017 11th Asian Control Conference (ASCC)

Management of solar energy in microgrids using IoT-based dependable control

Authors: Manh Duong Phung, Michel De La Villefromoy, Quang Ha

Abstract: Solar energy generation requires efficient monitoring and management in moving towards technologies for net-zero energy buildings. This paper presents a dependable control system based on the Internet of Things (IoT) to control and manage the energy flow of renewable energy collected by solar panels within a microgrid. Data for optimal control include not only measurements from local sensors but a… ▽ More Solar energy generation requires efficient monitoring and management in moving towards technologies for net-zero energy buildings. This paper presents a dependable control system based on the Internet of Things (IoT) to control and manage the energy flow of renewable energy collected by solar panels within a microgrid. Data for optimal control include not only measurements from local sensors but also meteorological information retrieved in real-time from online sources. For system fault tolerance across the whole distributed control system featuring multiple controllers, dependable controllers are developed to control and optimise the tracking performance of photovoltaic arrays to maximally capture solar radiation and maintain system resilience and reliability in real time despite failures of one or more redundant controllers due to a problem with communication, hardware or cybersecurity. Experimental results have been obtained to evaluate the validity of the proposed approach. △ Less

Submitted 10 October, 2017; originally announced October 2017.

Comments: 2017 20th International Conference on Electrical Machines and Systems (ICEMS)

Adaptive Second-order Sliding Mode Control of UAVs for Civil Applications

Authors: Van Truong Hoang, Ansu Man Singh, Manh Duong Phung, Quang Ha

Abstract: Quadcopters, as unmanned aerial vehicles (UAVs), have great potential in civil applications such as surveying, building monitoring, and infrastructure condition assessment. Quadcopters, however, are relatively sensitive to noises and disturbances so that their performance may be quickly downgraded in the case of inadequate control, system uncertainties and/or external disturbances. In this study,… ▽ More Quadcopters, as unmanned aerial vehicles (UAVs), have great potential in civil applications such as surveying, building monitoring, and infrastructure condition assessment. Quadcopters, however, are relatively sensitive to noises and disturbances so that their performance may be quickly downgraded in the case of inadequate control, system uncertainties and/or external disturbances. In this study, we deal with the quadrotor low-level control by proposing a robust scheme named the adaptive second-order quasi-continuous sliding mode control (adaptive 2-QCSM). The ultimate objective is for robust attitude control of the UAV in monitoring and inspection of built infrastructure. First, the mathematical model of the quadcopter is derived considering nonlinearity, strong coupling, uncertain dynamics and external disturbances. The control design includes the selection of the sliding manifold and the development of quasi-continuous second-order sliding mode controller with an adaptive gain. Stability of the overall control system is analysed by using a global Lyapunov function for convergence of both the sliding dynamics and adaptation scheme. Extensive simulations have been carried out for evaluation. Results show that the proposed controller can achieve robustness against disturbances or parameter variations and has better tracking performance in comparison with experimental responses of a UAV in a real-time monitoring task. △ Less

Submitted 31 July, 2017; originally announced July 2017.

Comments: in Proceeding of The 34th International Symposium on Automation and Robotics in Construction (ISARC), pp. 823-829, Taipei, Taiwan, 2017

Automatic Crack Detection in Built Infrastructure Using Unmanned Aerial Vehicles

Authors: Manh Duong Phung, Van Truong Hoang, Tran Hiep Dinh, Quang Ha

Abstract: This paper addresses the problem of crack detection which is essential for health monitoring of built infrastructure. Our approach includes two stages, data collection using unmanned aerial vehicles (UAVs) and crack detection using histogram analysis. For the data collection, a 3D model of the structure is first created by using laser scanners. Based on the model, geometric properties are extracte… ▽ More This paper addresses the problem of crack detection which is essential for health monitoring of built infrastructure. Our approach includes two stages, data collection using unmanned aerial vehicles (UAVs) and crack detection using histogram analysis. For the data collection, a 3D model of the structure is first created by using laser scanners. Based on the model, geometric properties are extracted to generate way points necessary for navigating the UAV to take images of the structure. Then, our next step is to stick together those obtained images from the overlapped field of view. The resulting image is then clustered by histogram analysis and peak detection. Potential cracks are finally identified by using locally adaptive thresholds. The whole process is automatically carried out so that the inspection time is significantly improved while safety hazards can be minimised. A prototypical system has been developed for evaluation and experimental results are included. △ Less

Submitted 31 July, 2017; originally announced July 2017.

Comments: In proceeding of The 34th International Symposium on Automation and Robotics in Construction (ISARC), pp. 823-829, Taipei, Taiwan, 2017

Automatic Interpretation of Unordered Point Cloud Data for UAV Navigation in Construction

Authors: M. D. Phung, C. H. Quach, D. T. Chu, N. Q. Nguyen, T. H. Dinh, Q. P. Ha

Abstract: The objective of this work is to develop a data processing system that can automatically generate waypoints for navigation of an unmanned aerial vehicle (UAV) to inspect surfaces of structures like buildings and bridges. The input includes data recorded by two 2D laser scanners, orthogonally mounted on the UAV, and an inertial measurement unit (IMU). To achieve the goal, algorithms are developed t… ▽ More The objective of this work is to develop a data processing system that can automatically generate waypoints for navigation of an unmanned aerial vehicle (UAV) to inspect surfaces of structures like buildings and bridges. The input includes data recorded by two 2D laser scanners, orthogonally mounted on the UAV, and an inertial measurement unit (IMU). To achieve the goal, algorithms are developed to process the data collected. They are separated into three major groups: (i) the data registration and filtering to generate a 3D model of the structure and control the density of point clouds for data completeness enhancement; (ii) the surface and obstacle detection to assist the UAV in monitoring tasks; and (iii) the waypoint generation to set the flight path. Experiments on different data sets show that the developed system is able to reconstruct a 3D point cloud of the structure, extract its surfaces and objects, and generate waypoints for the UAV to accomplish inspection tasks. △ Less

Submitted 12 February, 2017; v1 submitted 22 December, 2016; originally announced December 2016.

Comments: In The 14th International Conference on Control, Automation, Robotics and Vision, ICARCV 2016