Showing 1–27 of 27 results for author: Khan, M U

Pragmatic Formal Verification Methodology for Clock Domain Crossing (CDC)

Authors: Aman Kumar, Muhammad Ul Haque Khan, Bijitendra Mittra

Abstract: Modern System-on-Chip (SoC) designs are becoming more and more complex due to the technology upscaling. SoC designs often operate on multiple asynchronous clock domains, further adding to the complexity of the overall design. To make the devices power efficient, designers take a Globally-Asynchronous Locally-Synchronous (GALS) approach that creates multiple asynchronous domains. These Clock Domain… ▽ More Modern System-on-Chip (SoC) designs are becoming more and more complex due to the technology upscaling. SoC designs often operate on multiple asynchronous clock domains, further adding to the complexity of the overall design. To make the devices power efficient, designers take a Globally-Asynchronous Locally-Synchronous (GALS) approach that creates multiple asynchronous domains. These Clock Domain Crossings (CDC) are prone to metastability effects, and functional verification of such CDC is very important to ensure that no bug escapes. Conventional verification methods, such as register transfer level (RTL) simulations and static timing analysis, are not enough to address these CDC issues, which may lead to verification gaps. Additionally, identifying these CDC-related bugs is very time-consuming and is one of the most common reasons for costly silicon re-spins. This paper is focused on the development of a pragmatic formal verification methodology to minimize the CDC issues by exercising Metastability Injection (MSI) in different CDC paths. △ Less

Submitted 20 April, 2024; originally announced June 2024.

Comments: Published in DVCon Europe 2023

Leveraging Pre-trained CNNs for Efficient Feature Extraction in Rice Leaf Disease Classification

Authors: Md. Shohanur Islam Sobuj, Md. Imran Hossen, Md. Foysal Mahmud, Mahbub Ul Islam Khan

Abstract: Rice disease classification is a critical task in agricultural research, and in this study, we rigorously evaluate the impact of integrating feature extraction methodologies within pre-trained convolutional neural networks (CNNs). Initial investigations into baseline models, devoid of feature extraction, revealed commendable performance with ResNet-50 and ResNet-101 achieving accuracies of 91% and… ▽ More Rice disease classification is a critical task in agricultural research, and in this study, we rigorously evaluate the impact of integrating feature extraction methodologies within pre-trained convolutional neural networks (CNNs). Initial investigations into baseline models, devoid of feature extraction, revealed commendable performance with ResNet-50 and ResNet-101 achieving accuracies of 91% and 92%, respectively. Subsequent integration of Histogram of Oriented Gradients (HOG) yielded substantial improvements across architectures, notably propelling the accuracy of EfficientNet-B7 from 92\% to an impressive 97%. Conversely, the application of Local Binary Patterns (LBP) demonstrated more conservative performance enhancements. Moreover, employing Gradient-weighted Class Activation Mapping (Grad-CAM) unveiled that HOG integration resulted in heightened attention to disease-specific features, corroborating the performance enhancements observed. Visual representations further validated HOG's notable influence, showcasing a discernible surge in accuracy across epochs due to focused attention on disease-affected regions. These results underscore the pivotal role of feature extraction, particularly HOG, in refining representations and bolstering classification accuracy. The study's significant highlight was the achievement of 97% accuracy with EfficientNet-B7 employing HOG and Grad-CAM, a noteworthy advancement in optimizing pre-trained CNN-based rice disease identification systems. The findings advocate for the strategic integration of advanced feature extraction techniques with cutting-edge pre-trained CNN architectures, presenting a promising avenue for substantially augmenting the precision and effectiveness of image-based disease classification systems in agricultural contexts. △ Less

Submitted 26 February, 2024; originally announced May 2024.

Automated System-level Testing of Unmanned Aerial Systems

Authors: Hassan Sartaj, Asmar Muqeet, Muhammad Zohaib Iqbal, Muhammad Uzair Khan

Abstract: Unmanned aerial systems (UAS) rely on various avionics systems that are safety-critical and mission-critical. A major requirement of international safety standards is to perform rigorous system-level testing of avionics software systems. The current industrial practice is to manually create test scenarios, manually/automatically execute these scenarios using simulators, and manually evaluate outco… ▽ More Unmanned aerial systems (UAS) rely on various avionics systems that are safety-critical and mission-critical. A major requirement of international safety standards is to perform rigorous system-level testing of avionics software systems. The current industrial practice is to manually create test scenarios, manually/automatically execute these scenarios using simulators, and manually evaluate outcomes. The test scenarios typically consist of setting certain flight or environment conditions and testing the system under test in these settings. The state-of-the-art approaches for this purpose also require manual test scenario development and evaluation. In this paper, we propose a novel approach to automate the system-level testing of the UAS. The proposed approach (AITester) utilizes model-based testing and artificial intelligence (AI) techniques to automatically generate, execute, and evaluate various test scenarios. The test scenarios are generated on the fly, i.e., during test execution based on the environmental context at runtime. The approach is supported by a toolset. We empirically evaluate the proposed approach on two core components of UAS, an autopilot system of an unmanned aerial vehicle (UAV) and cockpit display systems (CDS) of the ground control station (GCS). The results show that the AITester effectively generates test scenarios causing deviations from the expected behavior of the UAV autopilot and reveals potential flaws in the GCS-CDS. △ Less

Submitted 2 August, 2024; v1 submitted 23 March, 2024; originally announced March 2024.

Comments: Published in Automated Software Engineering

Journal ref: Autom Softw Eng 31, 64 (2024)

Efficient Test Data Generation for MC/DC with OCL and Search

Authors: Hassan Sartaj, Muhammad Zohaib Iqbal, Atif Aftab Ahmed Jilani, Muhammad Uzair Khan

Abstract: System-level testing of avionics software systems requires compliance with different international safety standards such as DO-178C. An important consideration of the avionics industry is automated test data generation according to the criteria suggested by safety standards. One of the recommended criteria by DO-178C is the modified condition/decision coverage (MC/DC) criterion. The current model-… ▽ More System-level testing of avionics software systems requires compliance with different international safety standards such as DO-178C. An important consideration of the avionics industry is automated test data generation according to the criteria suggested by safety standards. One of the recommended criteria by DO-178C is the modified condition/decision coverage (MC/DC) criterion. The current model-based test data generation approaches use constraints written in Object Constraint Language (OCL), and apply search techniques to generate test data. These approaches either do not support MC/DC criterion or suffer from performance issues while generating test data for large-scale avionics systems. In this paper, we propose an effective way to automate MC/DC test data generation during model-based testing. We develop a strategy that utilizes case-based reasoning (CBR) and range reduction heuristics designed to solve MC/DC-tailored OCL constraints. We performed an empirical study to compare our proposed strategy for MC/DC test data generation using CBR, range reduction, both CBR and range reduction, with an original search algorithm, and random search. We also empirically compared our strategy with existing constraint-solving approaches. The results show that both CBR and range reduction for MC/DC test data generation outperform the baseline approach. Moreover, the combination of both CBR and range reduction for MC/DC test data generation is an effective approach compared to existing constraint solvers. △ Less

Submitted 2 August, 2024; v1 submitted 7 January, 2024; originally announced January 2024.

Transformer Network for Multi-Person Tracking and Re-Identification in Unconstrained Environment

Authors: Hamza Mukhtar, Muhammad Usman Ghani Khan

Abstract: Multi-object tracking (MOT) has profound applications in a variety of fields, including surveillance, sports analytics, self-driving, and cooperative robotics. Despite considerable advancements, existing MOT methodologies tend to falter when faced with non-uniform movements, occlusions, and appearance-reappearance scenarios of the objects. Recognizing this inadequacy, we put forward an integrated… ▽ More Multi-object tracking (MOT) has profound applications in a variety of fields, including surveillance, sports analytics, self-driving, and cooperative robotics. Despite considerable advancements, existing MOT methodologies tend to falter when faced with non-uniform movements, occlusions, and appearance-reappearance scenarios of the objects. Recognizing this inadequacy, we put forward an integrated MOT method that not only marries object detection and identity linkage within a singular, end-to-end trainable framework but also equips the model with the ability to maintain object identity links over long periods of time. Our proposed model, named STMMOT, is built around four key modules: 1) candidate proposal generation, which generates object proposals via a vision-transformer encoder-decoder architecture that detects the object from each frame in the video; 2) scale variant pyramid, a progressive pyramid structure to learn the self-scale and cross-scale similarities in multi-scale feature maps; 3) spatio-temporal memory encoder, extracting the essential information from the memory associated with each object under tracking; and 4) spatio-temporal memory decoder, simultaneously resolving the tasks of object detection and identity association for MOT. Our system leverages a robust spatio-temporal memory module that retains extensive historical observations and effectively encodes them using an attention-based aggregator. The uniqueness of STMMOT lies in representing objects as dynamic query embeddings that are updated continuously, which enables the prediction of object states with attention mechanisms and eradicates the need for post-processing. △ Less

Submitted 19 December, 2023; originally announced December 2023.

Deriving and Evaluating a Detailed Taxonomy of Game Bugs

Authors: Nigar Azhar Butt, Salman Sherin, Muhammad Uzair Khan, Atif Aftab Jilani, Muhammad Zohaib Iqbal

Abstract: Game development has become an extremely competitive multi-billion-dollar industry. Many games fail even after years of development efforts because of game-breaking bugs that disrupt the game-play and ruin the player experience. The goal of this work is to provide a bug taxonomy for games that will help game developers in developing bug-resistant games, game testers in designing and executing faul… ▽ More Game development has become an extremely competitive multi-billion-dollar industry. Many games fail even after years of development efforts because of game-breaking bugs that disrupt the game-play and ruin the player experience. The goal of this work is to provide a bug taxonomy for games that will help game developers in developing bug-resistant games, game testers in designing and executing fault-finding test cases, and researchers in evaluating game testing approaches. For this purpose, we performed a Multivocal Literature Review (MLR) by analyzing 436 sources, out of which 189 (78 academic and 111 grey) sources reporting bugs encountered in the game development industry were selected for analysis. We validate the proposed taxonomy by conducting a survey involving different game industry practitioners. The MLR allowed us to finalize a detailed taxonomy of 63 game bug categories in end-user perspective including eight first-tier categories: Gaming Balance, Implementation Response, Network, Sound, Temporal, Unexpected Crash, Navigational, and Non-Temporal faults. We observed that manual approaches towards game testing are still widely used. Only one of the approaches targets sound bugs whereas game balancing and how to incorporate machine learning in game testing is trending in the recent literature. Most of the game testing techniques are specialized and dependent on specific platforms. △ Less

Submitted 28 November, 2023; originally announced November 2023.

Perceptual Tone Mapping Model for High Dynamic Range Imaging

Authors: Imran Mehmood, Xinye Shi, M. Usman Khan, Ming Ronnier Luo

Abstract: One of the key challenges in tone mapping is to preserve the perceptual quality of high dynamic range (HDR) images when mapping them to standard dynamic range (SDR) displays. Traditional tone mapping operators (TMOs) compress the luminance of HDR images without considering the surround and display conditions emanating into suboptimal results. Current research addresses this challenge by incorporat… ▽ More One of the key challenges in tone mapping is to preserve the perceptual quality of high dynamic range (HDR) images when mapping them to standard dynamic range (SDR) displays. Traditional tone mapping operators (TMOs) compress the luminance of HDR images without considering the surround and display conditions emanating into suboptimal results. Current research addresses this challenge by incorporating perceptual color appearance attributes. In this work, we propose a TMO (TMOz) that leverages CIECAM16 perceptual attributes, i.e., brightness, colorfulness, and hue. TMOz accounts for the effects of both the surround and the display conditions to achieve more optimal colorfulness reproduction. The perceptual brightness is compressed, and the perceptual color scales, i.e., colorfulness and hue are derived from HDR images by employing CIECAM16 color adaptation equations. A psychophysical experiment was conducted to automate the brightness compression parameter. The model employs fully automatic and adaptive approach, obviating the requirement for manual parameter selection. TMOz was evaluated in terms of contrast, colorfulness and overall image quality. The objective and subjective evaluation methods revealed that the proposed model outperformed the state-of-the-art TMOs. △ Less

Submitted 29 September, 2023; originally announced September 2023.

GAANet: Ghost Auto Anchor Network for Detecting Varying Size Drones in Dark

Authors: Misha Urooj Khan, Maham Misbah, Zeeshan Kaleem, Yansha Deng, Abbas Jamalipour

Abstract: The usage of drones has tremendously increased in different sectors spanning from military to industrial applications. Despite all the benefits they offer, their misuse can lead to mishaps, and tackling them becomes more challenging particularly at night due to their small size and low visibility conditions. To overcome those limitations and improve the detection accuracy at night, we propose an o… ▽ More The usage of drones has tremendously increased in different sectors spanning from military to industrial applications. Despite all the benefits they offer, their misuse can lead to mishaps, and tackling them becomes more challenging particularly at night due to their small size and low visibility conditions. To overcome those limitations and improve the detection accuracy at night, we propose an object detector called Ghost Auto Anchor Network (GAANet) for infrared (IR) images. The detector uses a YOLOv5 core to address challenges in object detection for IR images, such as poor accuracy and a high false alarm rate caused by extended altitudes, poor lighting, and low image resolution. To improve performance, we implemented auto anchor calculation, modified the conventional convolution block to ghost-convolution, adjusted the input channel size, and used the AdamW optimizer. To enhance the precision of multiscale tiny object recognition, we also introduced an additional extra-small object feature extractor and detector. Experimental results in a custom IR dataset with multiple classes (birds, drones, planes, and helicopters) demonstrate that GAANet shows improvement compared to state-of-the-art detectors. In comparison to GhostNet-YOLOv5, GAANet has higher overall mean average precision (mAP@50), recall, and precision around 2.5\%, 2.3\%, and 1.4\%, respectively. The dataset and code for this paper are available as open source at https://github.com/ZeeshanKaleem/GhostAutoAnchorNet. △ Less

Submitted 5 May, 2023; originally announced May 2023.

Comments: Accepted @ IEEE VTC2023-Spring, Florence, Italy

SafeSpace MFNet: Precise and Efficient MultiFeature Drone Detection Network

Authors: Misha Urooj Khan, Mahnoor Dil, Muhammad Zeshan Alam, Farooq Alam Orakazi, Abdullah M. Almasoud, Zeeshan Kaleem, Chau Yuen

Abstract: The increasing prevalence of unmanned aerial vehicles (UAVs), commonly known as drones, has generated a demand for reliable detection systems. The inappropriate use of drones presents potential security and privacy hazards, particularly concerning sensitive facilities. To overcome those obstacles, we proposed the concept of MultiFeatureNet (MFNet), a solution that enhances feature representation b… ▽ More The increasing prevalence of unmanned aerial vehicles (UAVs), commonly known as drones, has generated a demand for reliable detection systems. The inappropriate use of drones presents potential security and privacy hazards, particularly concerning sensitive facilities. To overcome those obstacles, we proposed the concept of MultiFeatureNet (MFNet), a solution that enhances feature representation by capturing the most concentrated feature maps. Additionally, we present MultiFeatureNet-Feature Attention (MFNet-FA), a technique that adaptively weights different channels of the input feature maps. To meet the requirements of multi-scale detection, we presented the versions of MFNet and MFNet-FA, namely the small (S), medium (M), and large (L). The outcomes reveal notable performance enhancements. For optimal bird detection, MFNet-M (Ablation study 2) achieves an impressive precision of 99.8\%, while for UAV detection, MFNet-L (Ablation study 2) achieves a precision score of 97.2\%. Among the options, MFNet-FA-S (Ablation study 3) emerges as the most resource-efficient alternative, considering its small feature map size, computational demands (GFLOPs), and operational efficiency (in frame per second). This makes it particularly suitable for deployment on hardware with limited capabilities. Additionally, MFNet-FA-S (Ablation study 3) stands out for its swift real-time inference and multiple-object detection due to the incorporation of the FA module. The proposed MFNet-L with the focus module (Ablation study 2) demonstrates the most remarkable classification outcomes, boasting an average precision of 98.4\%, average recall of 96.6\%, average mean average precision (mAP) of 98.3\%, and average intersection over union (IoU) of 72.8\%. To encourage reproducible research, the dataset, and code for MFNet are freely available as an open-source project: github.com/ZeeshanKaleem/MultiFeatureNet. △ Less

Submitted 6 October, 2023; v1 submitted 30 November, 2022; originally announced November 2022.

Comments: Paper accepted in IEEE TVT

TF-Net: Deep Learning Empowered Tiny Feature Network for Night-time UAV Detection

Authors: Maham Misbah, Misha Urooj Khan, Zhaohui Yang, Zeeshan Kaleem

Abstract: Technological advancements have normalized the usage of unmanned aerial vehicles (UAVs) in every sector, spanning from military to commercial but they also pose serious security concerns due to their enhanced functionalities and easy access to private and highly secured areas. Several instances related to UAVs have raised security concerns, leading to UAV detection research studies. Visual techniq… ▽ More Technological advancements have normalized the usage of unmanned aerial vehicles (UAVs) in every sector, spanning from military to commercial but they also pose serious security concerns due to their enhanced functionalities and easy access to private and highly secured areas. Several instances related to UAVs have raised security concerns, leading to UAV detection research studies. Visual techniques are widely adopted for UAV detection, but they perform poorly at night, in complex backgrounds, and in adverse weather conditions. Therefore, a robust night vision-based drone detection system is required to that could efficiently tackle this problem. Infrared cameras are increasingly used for nighttime surveillance due to their wide applications in night vision equipment. This paper uses a deep learning-based TinyFeatureNet (TF-Net), which is an improved version of YOLOv5s, to accurately detect UAVs during the night using infrared (IR) images. In the proposed TF-Net, we introduce architectural changes in the neck and backbone of the YOLOv5s. We also simulated four different YOLOv5 models (s,m,n,l) and proposed TF-Net for a fair comparison. The results showed better performance for the proposed TF-Net in terms of precision, IoU, GFLOPS, model size, and FPS compared to the YOLOv5s. TF-Net yielded the best results with 95.7\% precision, 84\% mAp, and 44.8\% $IoU$. △ Less

Submitted 29 November, 2022; originally announced November 2022.

A Prospective Approach for Human-to-Human Interaction Recognition from Wi-Fi Channel Data using Attention Bidirectional Gated Recurrent Neural Network with GUI Application Implementation

Authors: Md. Mohi Uddin Khan, Abdullah Bin Shams, Md. Mohsin Sarker Raihan

Abstract: Human Activity Recognition (HAR) research has gained significant momentum due to recent technological advancements, artificial intelligence algorithms, the need for smart cities, and socioeconomic transformation. However, existing computer vision and sensor-based HAR solutions have limitations such as privacy issues, memory and power consumption, and discomfort in wearing sensors for which researc… ▽ More Human Activity Recognition (HAR) research has gained significant momentum due to recent technological advancements, artificial intelligence algorithms, the need for smart cities, and socioeconomic transformation. However, existing computer vision and sensor-based HAR solutions have limitations such as privacy issues, memory and power consumption, and discomfort in wearing sensors for which researchers are observing a paradigm shift in HAR research. In response, WiFi-based HAR is gaining popularity due to the availability of more coarse-grained Channel State Information. However, existing WiFi-based HAR approaches are limited to classifying independent and non-concurrent human activities performed within equal time duration. Recent research commonly utilizes a Single Input Multiple Output communication link with a WiFi signal of 5 GHz channel frequency, using two WiFi routers or two Intel 5300 NICs as transmitter-receiver. Our study, on the other hand, utilizes a Multiple Input Multiple Output radio link between a WiFi router and an Intel 5300 NIC, with the time-series Wi-Fi channel state information based on 2.4 GHz channel frequency for mutual human-to-human concurrent interaction recognition. The proposed Self-Attention guided Bidirectional Gated Recurrent Neural Network (Attention-BiGRU) deep learning model can classify 13 mutual interactions with a maximum benchmark accuracy of 94% for a single subject-pair. This has been expanded for ten subject pairs, which secured a benchmark accuracy of 88% with improved classification around the interaction-transition region. An executable graphical user interface (GUI) software has also been developed in this study using the PyQt5 python module to classify, save, and display the overall mutual concurrent human interactions performed within a given time duration. ... △ Less

Submitted 9 May, 2023; v1 submitted 16 February, 2022; originally announced February 2022.

Comments: 48 Pages. This is the pre-print version article submitted for peer-review to a prestigious journal

Telehealthcare and Telepathology in Pandemic: A Noninvasive, Low-Cost Micro-Invasive and Multimodal Real-Time Online Application for Early Diagnosis of COVID-19 Infection

Authors: Abdullah Bin Shams, Md. Mohsin Sarker Raihan, Md. Mohi Uddin Khan, Ocean Monjur, Rahat Bin Preo

Abstract: To contain the spread of the virus and stop the overcrowding of hospitalized patients, the coronavirus pandemic crippled healthcare facilities, mandating lockdowns and promoting remote work. As a result, telehealth has become increasingly popular for offering low-risk care to patients. However, the difficulty of preventing the next potential waves of infection has increased by constant virus mutat… ▽ More To contain the spread of the virus and stop the overcrowding of hospitalized patients, the coronavirus pandemic crippled healthcare facilities, mandating lockdowns and promoting remote work. As a result, telehealth has become increasingly popular for offering low-risk care to patients. However, the difficulty of preventing the next potential waves of infection has increased by constant virus mutation into new forms and a general lack of test kits, particularly in developing nations. In this research, a unique cloud-based application for the early identification of individuals who may have COVID-19 infection is proposed. The application provides five modes of diagnosis from possible symptoms (f1), cough sound (f2), specific blood biomarkers (f3), Raman spectral data of blood specimens (f4), and ECG signal paper-based image (f5). When a user selects an option and enters the information, the data is sent to the cloud server. The deployed machine learning (ML) and deep learning (DL) models classify the data in real time and inform the user of the likelihood of COVID-19 infection. Our deployed models can classify with an accuracy of 100%, 99.80%, 99.55%, 95.65%, and 77.59% from f3, f4, f5, f2, and f1 respectively. Moreover, the sensitivity for f2, f3, and f4 is 100%, which indicates the correct identification of COVID positive patients. This is significant in limiting the spread of the virus. Additionally, another ML model, as seen to offer 92% accuracy serves to identify patients who, out of a large group of patients admitted to the hospital cohort, need immediate critical care support by estimating the mortality risk of patients from blood parameters. The instantaneous multimodal nature of our technique offers multiplex and accurate diagnostic methods, highlighting the effectiveness of telehealth as a simple, widely available, and low-cost diagnostic solution, even for future pandemics. △ Less

Submitted 15 October, 2022; v1 submitted 16 September, 2021; originally announced September 2021.

Comments: 32 Pages. This article has been submitted for review to a prestigious journal

Development of a Risk-Free COVID-19 Screening Algorithm from Routine Blood Tests Using Ensemble Machine Learning

Authors: Md. Mohsin Sarker Raihan, Md. Mohi Uddin Khan, Laboni Akter, Abdullah Bin Shams

Abstract: The Reverse Transcription Polymerase Chain Reaction (RTPCR)} test is the silver bullet diagnostic test to discern COVID infection. Rapid antigen detection is a screening test to identify COVID positive patients in little as 15 minutes, but has a lower sensitivity than the PCR tests. Besides having multiple standardized test kits, many people are getting infected and either recovering or dying even… ▽ More The Reverse Transcription Polymerase Chain Reaction (RTPCR)} test is the silver bullet diagnostic test to discern COVID infection. Rapid antigen detection is a screening test to identify COVID positive patients in little as 15 minutes, but has a lower sensitivity than the PCR tests. Besides having multiple standardized test kits, many people are getting infected and either recovering or dying even before the test due to the shortage and cost of kits, lack of indispensable specialists and labs, time-consuming result compared to bulk population especially in developing and underdeveloped countries. Intrigued by the parametric deviations in immunological and hematological profile of a COVID patient, this research work leveraged the concept of COVID-19 detection by proposing a risk-free and highly accurate Stacked Ensemble Machine Learning model to identify a COVID patient from communally available-widespread-cheap routine blood tests which gives a promising accuracy, precision, recall and F1-score of 100%. Analysis from R-curve also shows the preciseness of the risk-free model to be implemented. The proposed method has the potential for large scale ubiquitous low-cost screening application. This can add an extra layer of protection in keeping the number of infected cases to a minimum and control the pandemic by identifying asymptomatic or pre-symptomatic people early. △ Less

Submitted 9 May, 2023; v1 submitted 12 August, 2021; originally announced August 2021.

Comments: Please read the (most updated) published version from here: https://doi.org/10.1201/9781003256083 and cite our article (Chapter-11). Video and BibTex citation format can be found in the description: https://youtu.be/Ci8dznDadJ4

Journal ref: Applied Intelligence for Industry 4.0. Chapman and Hall/CRC. 2023

Towards Continual, Online, Self-Supervised Depth

Authors: Muhammad Umar Karim Khan

Abstract: Although depth extraction with passive sensors has seen remarkable improvement with deep learning, these approaches may fail to obtain correct depth if they are exposed to environments not observed during training. Online adaptation, where the neural network trains while deployed, with self-supervised learning provides a convenient solution as the network can learn from the scene where it is deplo… ▽ More Although depth extraction with passive sensors has seen remarkable improvement with deep learning, these approaches may fail to obtain correct depth if they are exposed to environments not observed during training. Online adaptation, where the neural network trains while deployed, with self-supervised learning provides a convenient solution as the network can learn from the scene where it is deployed without external supervision. However, online adaptation causes a neural network to forget the past. Thus, past training is wasted and the network is not able to provide good results if it observes past scenes. This work deals with practical online-adaptation where the input is online and temporally-correlated, and training is completely self-supervised. Regularization and replay-based methods without task boundaries are proposed to avoid catastrophic forgetting while adapting to online data. Effort has been made to make the proposed approach suitable for practical use. We apply our method to both structure-from-motion and stereo depth estimation. We evaluate our method on diverse public datasets that include outdoor, indoor and synthetic scenes. Qualitative and quantitative results with both structure-from-motion and stereo show superior forgetting as well as adaptation performance compared to recent methods. Furthermore, the proposed method incurs negligible overhead compared to fine-tuning for online adaptation, proving to be an adequate choice in terms of plasticity, stability and applicability. The proposed approach is more inline with the artificial general intelligence paradigm as the neural network learns continually with no supervision. Source code is available at https://github.com/umarKarim/cou_sfm and https://github.com/umarKarim/cou_stereo. △ Less

Submitted 19 June, 2022; v1 submitted 27 February, 2021; originally announced March 2021.

Weight Training Analysis of Sportsmen with Kinect Bioinformatics for Form Improvement

Authors: Muhammad Umair Khan, Khawar Saeed, Sidra Qadeer

Abstract: Sports franchises invest a lot in training their athletes. use of latest technology for this purpose is also very common. We propose a system of capturing motion of athletes during weight training and analyzing that data to find out any shortcomings and imperfections. Our system uses Kinect depth image to compute different parameters of athlete's selected joints. These parameters are passed throug… ▽ More Sports franchises invest a lot in training their athletes. use of latest technology for this purpose is also very common. We propose a system of capturing motion of athletes during weight training and analyzing that data to find out any shortcomings and imperfections. Our system uses Kinect depth image to compute different parameters of athlete's selected joints. These parameters are passed through certain algorithms to process them and formulate results on their basis. Some parameters like range of motion, speed and balance can be analyzed in real time. But for comparison to be performed between motions, data is first recorded and stored and then processed for accurate results. Our results depict that this system can be easily deployed and implemented to provide a very valuable insight to dynamics of a work out and help an athlete in improving his form. △ Less

Submitted 13 August, 2020; originally announced September 2020.

Mobile Robot Path Planning in Static Environments using Particle Swarm Optimization

Authors: M. Shahab Alam, M. Usman Rafique, M. Umer Khan

Abstract: Motion planning is a key element of robotics since it empowers a robot to navigate autonomously. Particle Swarm Optimization is a simple, yet a very powerful optimization technique which has been effectively used in many complex multi-dimensional optimization problems. This paper proposes a path planning algorithm based on particle swarm optimization for computing a shortest collision-free path fo… ▽ More Motion planning is a key element of robotics since it empowers a robot to navigate autonomously. Particle Swarm Optimization is a simple, yet a very powerful optimization technique which has been effectively used in many complex multi-dimensional optimization problems. This paper proposes a path planning algorithm based on particle swarm optimization for computing a shortest collision-free path for a mobile robot in environments populated with static convex obstacles. The proposed algorithm finds the optimal path by performing random sampling on grid lines generated between the robot start and goal positions. Functionality of the proposed algorithm is illustrated via simulation results for different scenarios. △ Less

Submitted 23 August, 2020; originally announced August 2020.

Increased-Range Unsupervised Monocular Depth Estimation

Authors: Saad Imran, Muhammad Umar Karim Khan, Sikander Bin Mukarram, Chong-Min Kyung

Abstract: Unsupervised deep learning methods have shown promising performance for single-image depth estimation. Since most of these methods use binocular stereo pairs for self-supervision, the depth range is generally limited. Small-baseline stereo pairs provide small depth range but handle occlusions well. On the other hand, stereo images acquired with a wide-baseline rig cause occlusions-related errors i… ▽ More Unsupervised deep learning methods have shown promising performance for single-image depth estimation. Since most of these methods use binocular stereo pairs for self-supervision, the depth range is generally limited. Small-baseline stereo pairs provide small depth range but handle occlusions well. On the other hand, stereo images acquired with a wide-baseline rig cause occlusions-related errors in the near range but estimate depth well in the far range. In this work, we propose to integrate the advantages of the small and wide baselines. By training the network using three horizontally aligned views, we obtain accurate depth predictions for both close and far ranges. Our strategy allows to infer multi-baseline depth from a single image. This is unlike previous multi-baseline systems which employ more than two cameras. The qualitative and quantitative results show the superior performance of multi-baseline approach over previous stereo-based monocular methods. For 0.1 to 80 meters depth range, our approach decreases the absolute relative error of depth by 24% compared to Monodepth2. Our approach provides 21 frames per second on a single Nvidia1080 GPU, making it useful for practical applications. △ Less

Submitted 23 June, 2020; originally announced June 2020.

Self-Supervised Representation Learning for Visual Anomaly Detection

Authors: Rabia Ali, Muhammad Umar Karim Khan, Chong Min Kyung

Abstract: Self-supervised learning allows for better utilization of unlabelled data. The feature representation obtained by self-supervision can be used in downstream tasks such as classification, object detection, segmentation, and anomaly detection. While classification, object detection, and segmentation have been investigated with self-supervised learning, anomaly detection needs more attention. We cons… ▽ More Self-supervised learning allows for better utilization of unlabelled data. The feature representation obtained by self-supervision can be used in downstream tasks such as classification, object detection, segmentation, and anomaly detection. While classification, object detection, and segmentation have been investigated with self-supervised learning, anomaly detection needs more attention. We consider the problem of anomaly detection in images and videos, and present a new visual anomaly detection technique for videos. Numerous seminal and state-of-the-art self-supervised methods are evaluated for anomaly detection on a variety of image datasets. The best performing image-based self-supervised representation learning method is then used for video anomaly detection to see the importance of spatial features in visual anomaly detection in videos. We also propose a simple self-supervision approach for learning temporal coherence across video frames without the use of any optical flow information. At its core, our method identifies the frame indices of a jumbled video sequence allowing it to learn the spatiotemporal features of the video. This intuitive approach shows superior performance of visual anomaly detection compared to numerous methods for images and videos on UCF101 and ILSVRC2015 video datasets. △ Less

Submitted 17 June, 2020; originally announced June 2020.

Fine-Tuning DARTS for Image Classification

Authors: Muhammad Suhaib Tanveer, Muhammad Umar Karim Khan, Chong-Min Kyung

Abstract: Neural Architecture Search (NAS) has gained attraction due to superior classification performance. Differential Architecture Search (DARTS) is a computationally light method. To limit computational resources DARTS makes numerous approximations. These approximations result in inferior performance. We propose to fine-tune DARTS using fixed operations as they are independent of these approximations.… ▽ More Neural Architecture Search (NAS) has gained attraction due to superior classification performance. Differential Architecture Search (DARTS) is a computationally light method. To limit computational resources DARTS makes numerous approximations. These approximations result in inferior performance. We propose to fine-tune DARTS using fixed operations as they are independent of these approximations. Our method offers a good trade-off between the number of parameters and classification accuracy. Our approach improves the top-1 accuracy on Fashion-MNIST, CompCars, and MIO-TCD datasets by 0.56%, 0.50%, and 0.39%, respectively compared to the state-of-the-art approaches. Our approach performs better than DARTS, improving the accuracy by 0.28%, 1.64%, 0.34%, 4.5%, and 3.27% compared to DARTS, on CIFAR-10, CIFAR-100, Fashion-MNIST, CompCars, and MIO-TCD datasets, respectively. △ Less

Submitted 16 June, 2020; originally announced June 2020.

Comments: 8 pages, 6 figures

Global Feature Aggregation for Accident Anticipation

Authors: Mishal Fatima, Muhammad Umar Karim Khan, Chong Min Kyung

Abstract: Anticipation of accidents ahead of time in autonomous and non-autonomous vehicles aids in accident avoidance. In order to recognize abnormal events such as traffic accidents in a video sequence, it is important that the network takes into account interactions of objects in a given frame. We propose a novel Feature Aggregation (FA) block that refines each object's features by computing a weighted s… ▽ More Anticipation of accidents ahead of time in autonomous and non-autonomous vehicles aids in accident avoidance. In order to recognize abnormal events such as traffic accidents in a video sequence, it is important that the network takes into account interactions of objects in a given frame. We propose a novel Feature Aggregation (FA) block that refines each object's features by computing a weighted sum of the features of all objects in a frame. We use FA block along with Long Short Term Memory (LSTM) network to anticipate accidents in the video sequences. We report mean Average Precision (mAP) and Average Time-to-Accident (ATTA) on Street Accident (SA) dataset. Our proposed method achieves the highest score for risk anticipation by predicting accidents 0.32 sec and 0.75 sec earlier compared to the best results with Adaptive Loss and dynamic parameter prediction based methods respectively. △ Less

Submitted 16 June, 2020; originally announced June 2020.

Plug-and-Play Anomaly Detection with Expectation Maximization Filtering

Authors: Muhammad Umar Karim Khan, Mishal Fatima, Chong-Min Kyung

Abstract: Anomaly detection in crowds enables early rescue response. A plug-and-play smart camera for crowd surveillance has numerous constraints different from typical anomaly detection: the training data cannot be used iteratively; there are no training labels; and training and classification needs to be performed simultaneously. We tackle all these constraints with our approach in this paper. We propose… ▽ More Anomaly detection in crowds enables early rescue response. A plug-and-play smart camera for crowd surveillance has numerous constraints different from typical anomaly detection: the training data cannot be used iteratively; there are no training labels; and training and classification needs to be performed simultaneously. We tackle all these constraints with our approach in this paper. We propose a Core Anomaly-Detection (CAD) neural network which learns the motion behavior of objects in the scene with an unsupervised method. On average over standard datasets, CAD with a single epoch of training shows a percentage increase in Area Under the Curve (AUC) of 4.66% and 4.9% compared to the best results with convolutional autoencoders and convolutional LSTM-based methods, respectively. With a single epoch of training, our method improves the AUC by 8.03% compared to the convolutional LSTM-based approach. We also propose an Expectation Maximization filter which chooses samples for training the core anomaly-detection network. The overall framework improves the AUC compared to future frame prediction-based approach by 24.87% when crowd anomaly detection is performed on a video stream. We believe our work is the first step towards using deep learning methods with autonomous plug-and-play smart cameras for crowd anomaly detection. △ Less

Submitted 16 June, 2020; originally announced June 2020.

CDST: A Toolkit for Testing Cockpit Display Systems of Avionics

Authors: Hassan Sartaj, Muhammad Zohaib Iqbal, Muhammad Uzair Khan

Abstract: Avionics are highly critical systems that require extensive testing governed by international safety standards. Cockpit Display Systems (CDS) are an essential component of modern aircraft cockpits and display information from the user application using various widgets. A significant step in the testing of avionics is to evaluate whether these CDS are displaying the correct information. A common in… ▽ More Avionics are highly critical systems that require extensive testing governed by international safety standards. Cockpit Display Systems (CDS) are an essential component of modern aircraft cockpits and display information from the user application using various widgets. A significant step in the testing of avionics is to evaluate whether these CDS are displaying the correct information. A common industrial practice is to manually test the information on these CDS by taking the aircraft into different scenarios during the simulation. Given the large number of scenarios to test, manual testing of such behavior is a laborious activity. In this paper, we present a CDST toolkit that automates the testing of Cockpit Display Systems (CDS). We discuss the workflow and architecture of the tool and also demonstrates the tool on an industrial case study. The results show that the tool is able to generate, execute, and evaluate the test cases and identify 3 bugs in the case study. △ Less

Submitted 21 January, 2020; originally announced January 2020.

Landscaping Systematic Mapping Studies in Software Engineering: A Tertiary Study

Authors: Muhammad Uzair khan, Salman Sherin, Muhammad Zohaib Iqbal, Rubab Zahid

Abstract: Context: A number of Systematic Mapping Studies (SMSs) that cover Software Engineering (SE) are reported in literature. Tertiary studies synthesize the secondary studies to provide a holistic view of an area. Objectives: We synthesize SMSs in SE to provide insights into existing SE areas and to investigate the trends and quality of SMSs. Methodology: We use Systematic Literature Review protocol to… ▽ More Context: A number of Systematic Mapping Studies (SMSs) that cover Software Engineering (SE) are reported in literature. Tertiary studies synthesize the secondary studies to provide a holistic view of an area. Objectives: We synthesize SMSs in SE to provide insights into existing SE areas and to investigate the trends and quality of SMSs. Methodology: We use Systematic Literature Review protocol to analyze and map the SMSs in SE, till August 2017, to SE Body of Knowledge (SWEBOK). Results: We analyze 210 SMSs and results show that: (1) Software design and construction are most active areas in SE; (2) Some areas lack SMSs, including mathematical foundations, software configuration management, and SE tools; (3) The quality of SMSs is improving with time; (4) SMSs in journals have higher quality than SMSs in conferences and are cited more often; (5) Low quality in SMSs can be attributed to a lack of quality assessment in SMSs and not reporting information about the primary studies. Conclusion: There is a potential for more SMSs in some SE areas. A number of SMSs do not provide the required information for an SMS, which leads to a low quality score. △ Less

Submitted 24 September, 2019; originally announced September 2019.

Journal ref: Journal of Systems and Software 149 (2019): 396-436

A Systematic Literature Review of Test Breakage Prevention and Repair Techniques

Authors: Javaria Imtiaz, Salman Sherin, Muhammad Uzair khan, Muhammad Zohaib Iqbal

Abstract: Context: When an application evolves, some of the developed test cases break. Discarding broken test cases causes a significant waste of effort and leads to test suites that are less effective and have lower coverage. Test repair approaches evolve test suites along with applications by repairing the broken test cases. Objective: Numerous studies are published on test repair approaches every year.… ▽ More Context: When an application evolves, some of the developed test cases break. Discarding broken test cases causes a significant waste of effort and leads to test suites that are less effective and have lower coverage. Test repair approaches evolve test suites along with applications by repairing the broken test cases. Objective: Numerous studies are published on test repair approaches every year. It is important to summarise and consolidate the existing knowledge in the area to provide directions to researchers and practitioners. This research work provides a systematic literature review in the area of test case repair and breakage prevention, aiming to guide researchers and practitioners in the field of software testing. Method: We followed the standard protocol for conducting a systematic literature review. First, research goals were defined using the Goal Question Metric (GQM). Then we formulate research questions corresponding to each goal. Finally, metrics are extracted from the included papers. Based on the defined selection criteria a final set of 41 primary studies are included for analysis. Results: The selection process resulted in 5 journal papers, and 36 conference papers. We present a taxonomy that lists the causes of test case breakages extracted from the literature. We found that only four proposed test repair tools are publicly available. Most studies evaluated their approaches on open-source case studies. Conclusion: There is significant room for future research on test repair techniques. Despite the positive trend of evaluating approaches on large scale open-source studies, there is a clear lack of results from studies done in a real industrial context. Few tools are publicly available which lowers the potential of adaption by industry practitioners. △ Less

Submitted 24 September, 2019; originally announced September 2019.

Journal ref: Information and Software Technology 113 (2019): 1-19

A Robust Hybrid Approach for Textual Document Classification

Authors: Muhammad Nabeel Asim, Muhammad Usman Ghani Khan, Muhammad Imran Malik, Andreas Dengel, Sheraz Ahmed

Abstract: Text document classification is an important task for diverse natural language processing based applications. Traditional machine learning approaches mainly focused on reducing dimensionality of textual data to perform classification. This although improved the overall classification accuracy, the classifiers still faced sparsity problem due to lack of better data representation techniques. Deep l… ▽ More Text document classification is an important task for diverse natural language processing based applications. Traditional machine learning approaches mainly focused on reducing dimensionality of textual data to perform classification. This although improved the overall classification accuracy, the classifiers still faced sparsity problem due to lack of better data representation techniques. Deep learning based text document classification, on the other hand, benefitted greatly from the invention of word embeddings that have solved the sparsity problem and researchers focus mainly remained on the development of deep architectures. Deeper architectures, however, learn some redundant features that limit the performance of deep learning based solutions. In this paper, we propose a two stage text document classification methodology which combines traditional feature engineering with automatic feature engineering (using deep learning). The proposed methodology comprises a filter based feature selection (FSE) algorithm followed by a deep convolutional neural network. This methodology is evaluated on the two most commonly used public datasets, i.e., 20 Newsgroups data and BBC news data. Evaluation results reveal that the proposed methodology outperforms the state-of-the-art of both the (traditional) machine learning and deep learning based text document classification methodologies with a significant margin of 7.7% on 20 Newsgroups and 6.6% on BBC news datasets. △ Less

Submitted 12 September, 2019; originally announced September 2019.

Comments: ICDAR Conference

A Systematic Mapping Study on Testing of Machine Learning Programs

Authors: Salman Sherin, Muhammad Uzair khan, Muhammad Zohaib Iqbal

Abstract: We aim to conduct a systematic mapping in the area of testing ML programs. We identify, analyze and classify the existing literature to provide an overview of the area. We followed well-established guidelines of systematic mapping to develop a systematic protocol to identify and review the existing literature. We formulate three sets of research questions, define inclusion and exclusion criteria a… ▽ More We aim to conduct a systematic mapping in the area of testing ML programs. We identify, analyze and classify the existing literature to provide an overview of the area. We followed well-established guidelines of systematic mapping to develop a systematic protocol to identify and review the existing literature. We formulate three sets of research questions, define inclusion and exclusion criteria and systematically identify themes for the classification of existing techniques. We also report the quality of the published works using established assessment criteria. we finally selected 37 papers out of 1654 based on our selection criteria up to January 2019. We analyze trends such as contribution facet, research facet, test approach, type of ML and the kind of testing with several other attributes. We also discuss the empirical evidence and reporting quality of selected papers. The data from the study is made publicly available for other researchers and practitioners. We present an overview of the area by answering several research questions. The area is growing rapidly, however, there is lack of enough empirical evidence to compare and assess the effectiveness of the techniques. More publicly available tools are required for use of practitioners and researchers. Further attention is needed on non-functional testing and testing of ML programs using reinforcement learning. We believe that this study can help researchers and practitioners to obtain an overview of the area and identify several sub-areas where more research is required △ Less

Submitted 11 July, 2019; originally announced July 2019.

Efficient Neural Network Compression

Authors: Hyeji Kim, Muhammad Umar Karim Khan, Chong-Min Kyung

Abstract: Network compression reduces the computational complexity and memory consumption of deep neural networks by reducing the number of parameters. In SVD-based network compression, the right rank needs to be decided for every layer of the network. In this paper, we propose an efficient method for obtaining the rank configuration of the whole network. Unlike previous methods which consider each layer se… ▽ More Network compression reduces the computational complexity and memory consumption of deep neural networks by reducing the number of parameters. In SVD-based network compression, the right rank needs to be decided for every layer of the network. In this paper, we propose an efficient method for obtaining the rank configuration of the whole network. Unlike previous methods which consider each layer separately, our method considers the whole network to choose the right rank configuration. We propose novel accuracy metrics to represent the accuracy and complexity relationship for a given neural network. We use these metrics in a non-iterative fashion to obtain the right rank configuration which satisfies the constraints on FLOPs and memory while maintaining sufficient accuracy. Experiments show that our method provides better compromise between accuracy and computational complexity/memory consumption while performing compression at much higher speed. For VGG-16 our network can reduce the FLOPs by 25% and improve accuracy by 0.7% compared to the baseline, while requiring only 3 minutes on a CPU to search for the right rank configuration. Previously, similar results were achieved in 4 hours with 8 GPUs. The proposed method can be used for lossless compression of a neural network as well. The better accuracy and complexity compromise, as well as the extremely fast speed of our method makes it suitable for neural network compression. △ Less

Submitted 12 April, 2019; v1 submitted 30 November, 2018; originally announced November 2018.