Showing 1–19 of 19 results for author: Mishra, P K

Depth-Weighted Detection of Behaviours of Risk in People with Dementia using Cameras

Authors: Pratik K. Mishra, Irene Ballester, Andrea Iaboni, Bing Ye, Kristine Newman, Alex Mihailidis, Shehroz S. Khan

Abstract: The behavioural and psychological symptoms of dementia, such as agitation and aggression, present a significant health and safety risk in residential care settings. Many care facilities have video cameras in place for digital monitoring of public spaces, which can be leveraged to develop an automated behaviours of risk detection system that can alert the staff to enable timely intervention and pre… ▽ More The behavioural and psychological symptoms of dementia, such as agitation and aggression, present a significant health and safety risk in residential care settings. Many care facilities have video cameras in place for digital monitoring of public spaces, which can be leveraged to develop an automated behaviours of risk detection system that can alert the staff to enable timely intervention and prevent the situation from escalating. However, one of the challenges in our previous study was the presence of false alarms due to obstruction of view by activities happening close to the camera. To address this issue, we proposed a novel depth-weighted loss function to train a customized convolutional autoencoder to enforce equivalent importance to the events happening both near and far from the cameras; thus, helping to reduce false alarms and making the method more suitable for real-world deployment. The proposed method was trained using data from nine participants with dementia across three cameras situated in a specialized dementia unit and achieved an area under the curve of receiver operating characteristic of $0.852$, $0.81$ and $0.768$ for the three cameras. Ablation analysis was conducted for the individual components of the proposed method and the performance of the proposed method was investigated for participant-specific and sex-specific behaviours of risk detection. The proposed method performed reasonably well in detecting behaviours of risk in people with dementia motivating further research toward the development of a behaviours of risk detection system suitable for deployment in video surveillance systems in care facilities. △ Less

Submitted 28 August, 2024; originally announced August 2024.

Unmatched uncertainty mitigation through neural network supported model predictive control

Authors: Mateus V. Gasparino, Prabhat K. Mishra, Girish Chowdhary

Abstract: This paper presents a deep learning based model predictive control (MPC) algorithm for systems with unmatched and bounded state-action dependent uncertainties of unknown structure. We utilize a deep neural network (DNN) as an oracle in the underlying optimization problem of learning based MPC (LBMPC) to estimate unmatched uncertainties. Generally, non-parametric oracles such as DNN are considered… ▽ More This paper presents a deep learning based model predictive control (MPC) algorithm for systems with unmatched and bounded state-action dependent uncertainties of unknown structure. We utilize a deep neural network (DNN) as an oracle in the underlying optimization problem of learning based MPC (LBMPC) to estimate unmatched uncertainties. Generally, non-parametric oracles such as DNN are considered difficult to employ with LBMPC due to the technical difficulties associated with estimation of their coefficients in real time. We employ a dual-timescale adaptation mechanism, where the weights of the last layer of the neural network are updated in real time while the inner layers are trained on a slower timescale using the training data collected online and selectively stored in a buffer. Our results are validated through a numerical experiment on the compression system model of jet engine. These results indicate that the proposed approach is implementable in real time and carries the theoretical guarantees of LBMPC. △ Less

Submitted 22 April, 2023; originally announced April 2023.

Deep Model Predictive Control

Authors: Prabhat K. Mishra, Mateus V. Gasparino, Andres E. B. Velasquez, Girish Chowdhary

Abstract: This paper presents a deep learning based model predictive control algorithm for control affine nonlinear discrete time systems with matched and bounded state-dependent uncertainties of unknown structure. Since the structure of uncertainties is not known, a deep neural network (DNN) is employed to approximate the disturbances. In order to avoid any unwanted behavior during the learning phase, a tu… ▽ More This paper presents a deep learning based model predictive control algorithm for control affine nonlinear discrete time systems with matched and bounded state-dependent uncertainties of unknown structure. Since the structure of uncertainties is not known, a deep neural network (DNN) is employed to approximate the disturbances. In order to avoid any unwanted behavior during the learning phase, a tube based model predictive controller is employed, which ensures satisfaction of constraints and input-to-state stability of the closed-loop states. △ Less

Submitted 27 February, 2023; originally announced February 2023.

Comments: arXiv admin note: text overlap with arXiv:2104.07171

Journal ref: Conference on Robot Learning (CoRL'22): Workshop on Learning to Adapt and Improve in the Real World, 2022

Skeletal Video Anomaly Detection using Deep Learning: Survey, Challenges and Future Directions

Authors: Pratik K. Mishra, Alex Mihailidis, Shehroz S. Khan

Abstract: The existing methods for video anomaly detection mostly utilize videos containing identifiable facial and appearance-based features. The use of videos with identifiable faces raises privacy concerns, especially when used in a hospital or community-based setting. Appearance-based features can also be sensitive to pixel-based noise, straining the anomaly detection methods to model the changes in the… ▽ More The existing methods for video anomaly detection mostly utilize videos containing identifiable facial and appearance-based features. The use of videos with identifiable faces raises privacy concerns, especially when used in a hospital or community-based setting. Appearance-based features can also be sensitive to pixel-based noise, straining the anomaly detection methods to model the changes in the background and making it difficult to focus on the actions of humans in the foreground. Structural information in the form of skeletons describing the human motion in the videos is privacy-protecting and can overcome some of the problems posed by appearance-based features. In this paper, we present a survey of privacy-protecting deep learning anomaly detection methods using skeletons extracted from videos. We present a novel taxonomy of algorithms based on the various learning approaches. We conclude that skeleton-based approaches for anomaly detection can be a plausible privacy-protecting alternative for video anomaly detection. Lastly, we identify major open research questions and provide guidelines to address them. △ Less

Submitted 17 January, 2024; v1 submitted 30 December, 2022; originally announced January 2023.

Comments: This work has been accepted by IEEE Transactions on Emerging Topics in Computational Intelligence

Privacy-Protecting Behaviours of Risk Detection in People with Dementia using Videos

Authors: Pratik K. Mishra, Andrea Iaboni, Bing Ye, Kristine Newman, Alex Mihailidis, Shehroz S. Khan

Abstract: People living with dementia often exhibit behavioural and psychological symptoms of dementia that can put their and others' safety at risk. Existing video surveillance systems in long-term care facilities can be used to monitor such behaviours of risk to alert the staff to prevent potential injuries or death in some cases. However, these behaviours of risk events are heterogeneous and infrequent i… ▽ More People living with dementia often exhibit behavioural and psychological symptoms of dementia that can put their and others' safety at risk. Existing video surveillance systems in long-term care facilities can be used to monitor such behaviours of risk to alert the staff to prevent potential injuries or death in some cases. However, these behaviours of risk events are heterogeneous and infrequent in comparison to normal events. Moreover, analyzing raw videos can also raise privacy concerns. In this paper, we present two novel privacy-protecting video-based anomaly detection approaches to detect behaviours of risks in people with dementia. We either extracted body pose information as skeletons or used semantic segmentation masks to replace multiple humans in the scene with their semantic boundaries. Our work differs from most existing approaches for video anomaly detection that focus on appearance-based features, which can put the privacy of a person at risk and is also susceptible to pixel-based noise, including illumination and viewing direction. We used anonymized videos of normal activities to train customized spatio-temporal convolutional autoencoders and identify behaviours of risk as anomalies. We showed our results on a real-world study conducted in a dementia care unit with patients with dementia, containing approximately 21 hours of normal activities data for training and 9 hours of data containing normal and behaviours of risk events for testing. We compared our approaches with the original RGB videos and obtained a similar area under the receiver operating characteristic curve performance of 0.807 for the skeleton-based approach and 0.823 for the segmentation mask-based approach. △ Less

Submitted 17 January, 2023; v1 submitted 20 December, 2022; originally announced December 2022.

RDD-Eclat: Approaches to Parallelize Eclat Algorithm on Spark RDD Framework (Extended Version)

Authors: Pankaj Singh, Sudhakar Singh, P K Mishra, Rakhi Garg

Abstract: Frequent itemset mining (FIM) is a highly computational and data intensive algorithm. Therefore, parallel and distributed FIM algorithms have been designed to process large volume of data in a reduced time. Recently, a number of FIM algorithms have been designed on Hadoop MapReduce, a distributed big data processing framework. But, due to heavy disk I/O, MapReduce is found to be inefficient for th… ▽ More Frequent itemset mining (FIM) is a highly computational and data intensive algorithm. Therefore, parallel and distributed FIM algorithms have been designed to process large volume of data in a reduced time. Recently, a number of FIM algorithms have been designed on Hadoop MapReduce, a distributed big data processing framework. But, due to heavy disk I/O, MapReduce is found to be inefficient for the highly iterative FIM algorithms. Therefore, Spark, a more efficient distributed data processing framework, has been developed with in-memory computation and resilient distributed dataset (RDD) features to support the iterative algorithms. On this framework, Apriori and FP-Growth based FIM algorithms have been designed on the Spark RDD framework, but Eclat-based algorithm has not been explored yet. In this paper, RDD-Eclat, a parallel Eclat algorithm on the Spark RDD framework is proposed with its five variants. The proposed algorithms are evaluated on the various benchmark datasets, and the experimental results show that RDD-Eclat outperforms the Spark-based Apriori by many times. Also, the experimental results show the scalability of the proposed algorithms on increasing the number of cores and size of the dataset. △ Less

Submitted 22 October, 2021; originally announced October 2021.

Comments: This version is not published or communicated anywhere. arXiv admin note: substantial text overlap with arXiv:1912.06415

Approximation Algorithms For The Dispersion Problems in a Metric Space

Authors: Pawan K. Mishra, Gautam K. Das

Abstract: In this article, we consider the $c$-dispersion problem in a metric space $(X,d)$. Let $P=\{p_{1}, p_{2}, \ldots, p_{n}\}$ be a set of $n$ points in a metric space $(X,d)$. For each point $p \in P$ and $S \subseteq P$, we define $cost_{c}(p,S)$ as the sum of distances from $p$ to the nearest $c $ points in $S \setminus \{p\}$, where $c\geq 1$ is a fixed integer. We define… ▽ More In this article, we consider the $c$-dispersion problem in a metric space $(X,d)$. Let $P=\{p_{1}, p_{2}, \ldots, p_{n}\}$ be a set of $n$ points in a metric space $(X,d)$. For each point $p \in P$ and $S \subseteq P$, we define $cost_{c}(p,S)$ as the sum of distances from $p$ to the nearest $c $ points in $S \setminus \{p\}$, where $c\geq 1$ is a fixed integer. We define $cost_{c}(S)=\min_{p \in S}\{cost_{c}(p,S)\}$ for $S \subseteq P$. In the $c$-dispersion problem, a set $P$ of $n$ points in a metric space $(X,d)$ and a positive integer $k \in [c+1,n]$ are given. The objective is to find a subset $S\subseteq P$ of size $k$ such that $cost_{c}(S)$ is maximized. We propose a simple polynomial time greedy algorithm that produces a $2c$-factor approximation result for the $c$-dispersion problem in a metric space. The best known result for the $c$-dispersion problem in the Euclidean metric space $(X,d)$ is $2c^2$, where $P \subseteq \mathbb{R}^2$ and the distance function is Euclidean distance [ Amano, K. and Nakano, S. I., Away from Rivals, CCCG, pp.68-71, 2018 ]. We also prove that the $c$-dispersion problem in a metric space is $W[1]$-hard. △ Less

Submitted 9 June, 2021; v1 submitted 19 May, 2021; originally announced May 2021.

Comments: 9. arXiv admin note: text overlap with arXiv:2105.09217

Approximation Algorithms For The Euclidean Dispersion Problems

Authors: Pawan K. Mishra, Gautam K. Das

Abstract: In this article, we consider the Euclidean dispersion problems. Let $P=\{p_{1}, p_{2}, \ldots, p_{n}\}$ be a set of $n$ points in $\mathbb{R}^2$. For each point $p \in P$ and $S \subseteq P$, we define $cost_γ(p,S)$ as the sum of Euclidean distance from $p$ to the nearest $γ$ point in $S \setminus \{p\}$. We define $cost_γ(S)=\min_{p \in S}\{cost_γ(p,S)\}$ for $S \subseteq P$. In the $γ$-dispersio… ▽ More In this article, we consider the Euclidean dispersion problems. Let $P=\{p_{1}, p_{2}, \ldots, p_{n}\}$ be a set of $n$ points in $\mathbb{R}^2$. For each point $p \in P$ and $S \subseteq P$, we define $cost_γ(p,S)$ as the sum of Euclidean distance from $p$ to the nearest $γ$ point in $S \setminus \{p\}$. We define $cost_γ(S)=\min_{p \in S}\{cost_γ(p,S)\}$ for $S \subseteq P$. In the $γ$-dispersion problem, a set $P$ of $n$ points in $\mathbb{R}^2$ and a positive integer $k \in [γ+1,n]$ are given. The objective is to find a subset $S\subseteq P$ of size $k$ such that $cost_γ(S)$ is maximized. We consider both $2$-dispersion and $1$-dispersion problem in $\mathbb{R}^2$. Along with these, we also consider $2$-dispersion problem when points are placed on a line. In this paper, we propose a simple polynomial time $(2\sqrt 3 + ε)$-factor approximation algorithm for the $2$-dispersion problem, for any $ε> 0$, which is an improvement over the best known approximation factor $4\sqrt3$ [Amano, K. and Nakano, S. I., An approximation algorithm for the $2$-dispersion problem, IEICE Transactions on Information and Systems, Vol. 103(3), pp. 506-508, 2020]. Next, we develop a common framework for designing an approximation algorithm for the Euclidean dispersion problem. With this common framework, we improve the approximation factor to $2\sqrt 3$ for the $2$-dispersion problem in $\mathbb{R}^2$. Using the same framework, we propose a polynomial time algorithm, which returns an optimal solution for the $2$-dispersion problem when points are placed on a line. Moreover, to show the effectiveness of the framework, we also propose a $2$-factor approximation algorithm for the $1$-dispersion problem in $\mathbb{R}^2$. △ Less

Submitted 19 May, 2021; originally announced May 2021.

Comments: 17

Minimum Variance Embedded Auto-associative Kernel Extreme Learning Machine for One-class Classification

Authors: Pratik K. Mishra, Chandan Gautam, Aruna Tiwari

Abstract: One-class classification (OCC) needs samples from only a single class to train the classifier. Recently, an auto-associative kernel extreme learning machine was developed for the OCC task. This paper introduces a novel extension of this classifier by embedding minimum variance information within its architecture and is referred to as VAAKELM. The minimum variance embedding forces the network outpu… ▽ More One-class classification (OCC) needs samples from only a single class to train the classifier. Recently, an auto-associative kernel extreme learning machine was developed for the OCC task. This paper introduces a novel extension of this classifier by embedding minimum variance information within its architecture and is referred to as VAAKELM. The minimum variance embedding forces the network output weights to focus in regions of low variance and reduces the intra-class variance. This leads to a better separation of target samples and outliers, resulting in an improvement in the generalization performance of the classifier. The proposed classifier follows a reconstruction-based approach to OCC and minimizes the reconstruction error by using the kernel extreme learning machine as the base classifier. It uses the deviation in reconstruction error to identify the outliers. We perform experiments on 15 small-size and 10 medium-size one-class benchmark datasets to demonstrate the efficiency of the proposed classifier. We compare the results with 13 existing one-class classifiers by considering the mean F1 score as the comparison metric. The experimental results show that VAAKELM consistently performs better than the existing classifiers, making it a viable alternative for the OCC task. △ Less

Submitted 24 November, 2020; originally announced November 2020.

RBF-FD analysis of 2D time-domain acoustic wave propagation in heterogeneous media

Authors: Jure Močnik - Berljavac, Pankaj K Mishra, Jure Slak, Gregor Kosec

Abstract: Radial Basis Function-generated Finite Differences (RBF-FD) is a popular variant of local strong-form meshless methods that do not require a predefined connection between the nodes, making it easier to adapt node-distribution to the problem under consideration. This paper investigates an RBF-FD solution of time-domain acoustic wave propagation in the context of seismic modeling in the Earth's subs… ▽ More Radial Basis Function-generated Finite Differences (RBF-FD) is a popular variant of local strong-form meshless methods that do not require a predefined connection between the nodes, making it easier to adapt node-distribution to the problem under consideration. This paper investigates an RBF-FD solution of time-domain acoustic wave propagation in the context of seismic modeling in the Earth's subsurface. Through a number of numerical tests, ranging from homogeneous to highly-heterogeneous velocity models including non-smooth irregular topography, we demonstrate that the present approach can be further generalized to solve large-scale seismic modeling and full waveform inversion problems in arbitrarily complex models enabling more robust interpretations of geophysical observations △ Less

Submitted 10 May, 2021; v1 submitted 2 January, 2020; originally announced January 2020.

Comments: To reproduce the numerical tests in this paper, please see the project repository \url{https://gitlab.com/e62Lab/2019_p_wavepropagation_code}

Journal ref: Computers & Geosciences 153 (2021)

RDD-Eclat: Approaches to Parallelize Eclat Algorithm on Spark RDD Framework

Authors: Pankaj Singh, Sudhakar Singh, P. K. Mishra, Rakhi Garg

Abstract: Initially, a number of frequent itemset mining (FIM) algorithms have been designed on the Hadoop MapReduce, a distributed big data processing framework. But, due to heavy disk I/O, MapReduce is found to be inefficient for such highly iterative algorithms. Therefore, Spark, a more efficient distributed data processing framework, has been developed with in-memory computation and resilient distribute… ▽ More Initially, a number of frequent itemset mining (FIM) algorithms have been designed on the Hadoop MapReduce, a distributed big data processing framework. But, due to heavy disk I/O, MapReduce is found to be inefficient for such highly iterative algorithms. Therefore, Spark, a more efficient distributed data processing framework, has been developed with in-memory computation and resilient distributed dataset (RDD) features to support the iterative algorithms. On the Spark RDD framework, Apriori and FP-Growth based FIM algorithms have been designed, but Eclat-based algorithm has not been explored yet. In this paper, RDD-Eclat, a parallel Eclat algorithm on the Spark RDD framework is proposed with its five variants. The proposed algorithms are evaluated on the various benchmark datasets, which shows that RDD-Eclat outperforms the Spark-based Apriori by many times. Also, the experimental results show the scalability of the proposed algorithms on increasing the number of cores and size of the dataset. △ Less

Submitted 13 December, 2019; originally announced December 2019.

Comments: 16 pages, 6 figures, ICCNCT 2019

Report number: ICCNCT-171

Journal ref: ICCNCT 2019, LNDECT 44

Mining Association Rules in Various Computing Environments: A Survey

Authors: Sudhakar Singh, Pankaj Singh, Rakhi Garg, P. K. Mishra

Abstract: Association Rule Mining (ARM) is one of the well know and most researched technique of data mining. There are so many ARM algorithms have been designed that their counting is a large number. In this paper we have surveyed the various ARM algorithms in four computing environments. The considered computing environments are sequential computing, parallel and distributed computing, grid computing and… ▽ More Association Rule Mining (ARM) is one of the well know and most researched technique of data mining. There are so many ARM algorithms have been designed that their counting is a large number. In this paper we have surveyed the various ARM algorithms in four computing environments. The considered computing environments are sequential computing, parallel and distributed computing, grid computing and cloud computing. With the emergence of new computing paradigm, ARM algorithms have been designed by many researchers to improve the efficiency by utilizing the new paradigm. This paper represents the journey of ARM algorithms started from sequential algorithms, and through parallel and distributed, and grid based algorithms to the current state-of-the-art, along with the motives for adopting new machinery. △ Less

Submitted 30 June, 2019; originally announced August 2019.

Comments: 14 pages

Journal ref: International Journal of Applied Engineering Research 2016; 11(8): 5629-5640

A Data Structure Perspective to the RDD-based Apriori Algorithm on Spark

Authors: Pankaj Singh, Sudhakar Singh, P. K. Mishra, Rakhi Garg

Abstract: During the recent years, a number of efficient and scalable frequent itemset mining algorithms for big data analytics have been proposed by many researchers. Initially, MapReduce-based frequent itemset mining algorithms on Hadoop cluster were proposed. Although, Hadoop has been developed as a cluster computing system for handling and processing big data, but the performance of Hadoop does not meet… ▽ More During the recent years, a number of efficient and scalable frequent itemset mining algorithms for big data analytics have been proposed by many researchers. Initially, MapReduce-based frequent itemset mining algorithms on Hadoop cluster were proposed. Although, Hadoop has been developed as a cluster computing system for handling and processing big data, but the performance of Hadoop does not meet the expectation for the iterative algorithms of data mining, due to its high I/O, and writing and then reading intermediate results in the disk. Consequently, Spark has been developed as another cluster computing infrastructure which is much faster than Hadoop due to its in-memory computation. It is highly suitable for iterative algorithms and supports batch, interactive, iterative, and stream processing of data. Many frequent itemset mining algorithms have been re-designed on the Spark, and most of them are Apriori-based. All these Spark-based Apriori algorithms use Hash Tree as the underlying data structure. This paper investigates the efficiency of various data structures for the Spark-based Apriori. Although, the data structure perspective has been investigated previously, but for MapReduce-based Apriori, and it must be re-investigated in the distributed computing environment of Spark. The considered underlying data structures are Hash Tree, Trie, and Hash Table Trie. The experimental results on the benchmark datasets show that the performance of Spark-based Apriori with Trie and Hash Table Trie are almost similar but both perform many times better than Hash Tree in the distributed computing environment of Spark. △ Less

Submitted 4 August, 2019; originally announced August 2019.

Comments: 14 pages

Report number: BJIT-D-18-00717

Journal ref: International Journal of Information Technology, Springer, 2019

Performance Optimization of MapReduce-based Apriori Algorithm on Hadoop Cluster

Authors: Sudhakar Singh, Rakhi Garg, P K Mishra

Abstract: Many techniques have been proposed to implement the Apriori algorithm on MapReduce framework but only a few have focused on performance improvement. FPC (Fixed Passes Combined-counting) and DPC (Dynamic Passes Combined-counting) algorithms combine multiple passes of Apriori in a single MapReduce phase to reduce the execution time. In this paper, we propose improved MapReduce based Apriori algorith… ▽ More Many techniques have been proposed to implement the Apriori algorithm on MapReduce framework but only a few have focused on performance improvement. FPC (Fixed Passes Combined-counting) and DPC (Dynamic Passes Combined-counting) algorithms combine multiple passes of Apriori in a single MapReduce phase to reduce the execution time. In this paper, we propose improved MapReduce based Apriori algorithms VFPC (Variable Size based Fixed Passes Combined-counting) and ETDPC (Elapsed Time based Dynamic Passes Combined-counting) over FPC and DPC. Further, we optimize the multi-pass phases of these algorithms by skipping pruning step in some passes, and propose Optimized-VFPC and Optimized-ETDPC algorithms. Quantitative analysis reveals that counting cost of additional un-pruned candidates produced due to skipped-pruning is less significant than reduction in computation cost due to the same. Experimental results show that VFPC and ETDPC are more robust and flexible than FPC and DPC whereas their optimized versions are more efficient in terms of execution time. △ Less

Submitted 16 July, 2018; originally announced July 2018.

Comments: 24 pages, 5 figures, 12 tables, 5 algorithms

Journal ref: Computers & Electrical Engineering 2018; 67: 348-364

A Quantitative Analysis of WCAG 2.0 Compliance For Some Indian Web Portals

Authors: Manas Ranjan Patra, Amar Ranjan Dash, Prasanna Kumar Mishra

Abstract: Web portals have served as an excellent medium to facilitate user centric services for organizations irrespective of the type, size, and domain of operation. The objective of these portals has been to deliver a plethora of services such as information dissemination, transactional services, and customer feedback. Therefore, the design of a web portal is crucial in order that it is accessible to a w… ▽ More Web portals have served as an excellent medium to facilitate user centric services for organizations irrespective of the type, size, and domain of operation. The objective of these portals has been to deliver a plethora of services such as information dissemination, transactional services, and customer feedback. Therefore, the design of a web portal is crucial in order that it is accessible to a wide range of user community irrespective of age group, physical abilities, and level of literacy. In this paper, we have studied the compliance of WCAG 2.0 by three different categories of Indian web sites which are most frequently accessed by a large section of user community. We have provided a quantitative evaluation of different aspects of accessibility which we believe can pave the way for better design of web sites by taking care of the deficiencies inherent in the web portals. △ Less

Submitted 24 October, 2017; originally announced October 2017.

Comments: 15 Pages, 15 figure, 2 Table

Journal ref: International Journal of Computer Science, Engineering and Applications (IJCSEA), AIRCC, Vol. 4(1), pp. 9-24, DOI: 10.5121/ijcsea.2014.4102

A Comparative Study of Association Rule Mining Algorithms on Grid and Cloud Platform

Authors: Sudhakar Singh, Rakhi Garg, P. K. Mishra

Abstract: Association rule mining is a time consuming process due to involving both data intensive and computation intensive nature. In order to mine large volume of data and to enhance the scalability and performance of existing sequential association rule mining algorithms, parallel and distributed algorithms are developed. These traditional parallel and distributed algorithms are based on homogeneous pla… ▽ More Association rule mining is a time consuming process due to involving both data intensive and computation intensive nature. In order to mine large volume of data and to enhance the scalability and performance of existing sequential association rule mining algorithms, parallel and distributed algorithms are developed. These traditional parallel and distributed algorithms are based on homogeneous platform and are not lucrative for heterogeneous platform such as grid and cloud. This requires design of new algorithms which address the issues of good data set partition and distribution, load balancing strategy, optimization of communication and synchronization technique among processors in such heterogeneous system. Grid and cloud are the emerging platform for distributed data processing and various association rule mining algorithms have been proposed on such platforms. This survey article integrates the brief architectural aspect of distributed system, various recent approaches of grid based and cloud based association rule mining algorithms with comparative perception. We differentiate between approaches of association rule mining algorithms developed on these architectures on the basis of data locality, programming paradigm, fault tolerance, communication cost, partition and distribution of data sets. Although it is not complete in order to cover all algorithms, yet it can be very useful for the new researchers working in the direction of distributed association rule mining algorithms. △ Less

Submitted 22 September, 2017; originally announced September 2017.

Comments: 8 pages, preprint

Journal ref: International Journal of Emerging Technologies in Computational and Applied Sciences, 2014

Review of Apriori Based Algorithms on MapReduce Framework

Authors: Sudhakar Singh, Rakhi Garg, P. K. Mishra

Abstract: The Apriori algorithm that mines frequent itemsets is one of the most popular and widely used data mining algorithms. Now days many algorithms have been proposed on parallel and distributed platforms to enhance the performance of Apriori algorithm. They differ from each other on the basis of load balancing technique, memory system, data decomposition technique and data layout used to implement the… ▽ More The Apriori algorithm that mines frequent itemsets is one of the most popular and widely used data mining algorithms. Now days many algorithms have been proposed on parallel and distributed platforms to enhance the performance of Apriori algorithm. They differ from each other on the basis of load balancing technique, memory system, data decomposition technique and data layout used to implement them. The problems with most of the distributed framework are overheads of managing distributed system and lack of high level parallel programming language. Also with grid computing there is always potential chances of node failures which cause multiple re-executions of tasks. These problems can be overcome by the MapReduce framework introduced by Google. MapReduce is an efficient, scalable and simplified programming model for large scale distributed data processing on a large cluster of commodity computers and also used in cloud computing. In this paper, we present the overview of parallel Apriori algorithm implemented on MapReduce framework. They are categorized on the basis of Map and Reduce functions used to implement them e.g. 1-phase vs. k-phase, I/O of Mapper, Combiner and Reducer, using functionality of Combiner inside Mapper etc. This survey discusses and analyzes the various implementations of Apriori on MapReduce framework on the basis of their distinguishing characteristics. Moreover, it also includes the advantages and limitations of MapReduce framework. △ Less

Submitted 21 February, 2017; originally announced February 2017.

Comments: 12 pages, 3 figures, ICC-2014

Report number: 115

Observations on Factors Affecting Performance of MapReduce based Apriori on Hadoop Cluster

Authors: Sudhakar Singh, Rakhi Garg, P. K. Mishra

Abstract: Designing fast and scalable algorithm for mining frequent itemsets is always being a most eminent and promising problem of data mining. Apriori is one of the most broadly used and popular algorithm of frequent itemset mining. Designing efficient algorithms on MapReduce framework to process and analyze big datasets is contemporary research nowadays. In this paper, we have focused on the performance… ▽ More Designing fast and scalable algorithm for mining frequent itemsets is always being a most eminent and promising problem of data mining. Apriori is one of the most broadly used and popular algorithm of frequent itemset mining. Designing efficient algorithms on MapReduce framework to process and analyze big datasets is contemporary research nowadays. In this paper, we have focused on the performance of MapReduce based Apriori on homogeneous as well as on heterogeneous Hadoop cluster. We have investigated a number of factors that significantly affects the execution time of MapReduce based Apriori running on homogeneous and heterogeneous Hadoop Cluster. Factors are specific to both algorithmic and non-algorithmic improvements. Considered factors specific to algorithmic improvements are filtered transactions and data structures. Experimental results show that how an appropriate data structure and filtered transactions technique drastically reduce the execution time. The non-algorithmic factors include speculative execution, nodes with poor performance, data locality & distribution of data blocks, and parallelism control with input split size. We have applied strategies against these factors and fine tuned the relevant parameters in our particular application. Experimental results show that if cluster specific parameters are taken care of then there is a significant reduction in execution time. Also we have discussed the issues regarding MapReduce implementation of Apriori which may significantly influence the performance. △ Less

Submitted 21 January, 2017; originally announced January 2017.

Comments: 8 pages, 8 figures, International Conference on Computing, Communication and Automation (ICCCA2016)

Report number: 466

Journal ref: 2016 International Conference on Computing, Communication and Automation (ICCCA), Greater Noida, India, 2016, pp. 87-94

Performance Analysis of Apriori Algorithm with Different Data Structures on Hadoop Cluster

Authors: Sudhakar Singh, Rakhi Garg, P. K. Mishra

Abstract: Mining frequent itemsets from massive datasets is always being a most important problem of data mining. Apriori is the most popular and simplest algorithm for frequent itemset mining. To enhance the efficiency and scalability of Apriori, a number of algorithms have been proposed addressing the design of efficient data structures, minimizing database scan and parallel and distributed processing. Ma… ▽ More Mining frequent itemsets from massive datasets is always being a most important problem of data mining. Apriori is the most popular and simplest algorithm for frequent itemset mining. To enhance the efficiency and scalability of Apriori, a number of algorithms have been proposed addressing the design of efficient data structures, minimizing database scan and parallel and distributed processing. MapReduce is the emerging parallel and distributed technology to process big datasets on Hadoop Cluster. To mine big datasets it is essential to re-design the data mining algorithm on this new paradigm. In this paper, we implement three variations of Apriori algorithm using data structures hash tree, trie and hash table trie i.e. trie with hash technique on MapReduce paradigm. We emphasize and investigate the significance of these three data structures for Apriori algorithm on Hadoop cluster, which has not been given attention yet. Experiments are carried out on both real life and synthetic datasets which shows that hash table trie data structures performs far better than trie and hash tree in terms of execution time. Moreover the performance in case of hash tree becomes worst. △ Less

Submitted 22 November, 2015; originally announced November 2015.

Comments: 2009-2015 International Journal of Computer Applications, FCS(Foundation of Computer Science)