-
Joint Modeling of Search and Recommendations Via an Unified Contextual Recommender (UniCoRn)
Authors:
Moumita Bhattacharya,
Vito Ostuni,
Sudarshan Lamkhede
Abstract:
Search and recommendation systems are essential in many services, and they are often developed separately, leading to complex maintenance and technical debt. In this paper, we present a unified deep learning model that efficiently handles key aspects of both tasks.
Search and recommendation systems are essential in many services, and they are often developed separately, leading to complex maintenance and technical debt. In this paper, we present a unified deep learning model that efficiently handles key aspects of both tasks.
△ Less
Submitted 19 August, 2024;
originally announced August 2024.
-
IntentRec: Predicting User Session Intent with Hierarchical Multi-Task Learning
Authors:
Sejoon Oh,
Moumita Bhattacharya,
Yesu Feng,
Sudarshan Lamkhede
Abstract:
Recommender systems have played a critical role in diverse digital services such as e-commerce, streaming media, social networks, etc. If we know what a user's intent is in a given session (e.g. do they want to watch short videos or a movie or play games; are they shopping for a camping trip), it becomes easier to provide high-quality recommendations. In this paper, we introduce IntentRec, a novel…
▽ More
Recommender systems have played a critical role in diverse digital services such as e-commerce, streaming media, social networks, etc. If we know what a user's intent is in a given session (e.g. do they want to watch short videos or a movie or play games; are they shopping for a camping trip), it becomes easier to provide high-quality recommendations. In this paper, we introduce IntentRec, a novel recommendation framework based on hierarchical multi-task neural network architecture that tries to estimate a user's latent intent using their short- and long-term implicit signals as proxies and uses the intent prediction to predict the next item user is likely to engage with. By directly leveraging the intent prediction, we can offer accurate and personalized recommendations to users. Our comprehensive experiments on Netflix user engagement data show that IntentRec outperforms the state-of-the-art next-item and next-intent predictors. We also share several findings and downstream applications of IntentRec.
△ Less
Submitted 25 July, 2024;
originally announced August 2024.
-
Time Series Anomaly Detection with CNN for Environmental Sensors in Healthcare-IoT
Authors:
Mirza Akhi Khatun,
Mangolika Bhattacharya,
Ciarán Eising,
Lubna Luxmi Dhirani
Abstract:
This research develops a new method to detect anomalies in time series data using Convolutional Neural Networks (CNNs) in healthcare-IoT. The proposed method creates a Distributed Denial of Service (DDoS) attack using an IoT network simulator, Cooja, which emulates environmental sensors such as temperature and humidity. CNNs detect anomalies in time series data, resulting in a 92\% accuracy in ide…
▽ More
This research develops a new method to detect anomalies in time series data using Convolutional Neural Networks (CNNs) in healthcare-IoT. The proposed method creates a Distributed Denial of Service (DDoS) attack using an IoT network simulator, Cooja, which emulates environmental sensors such as temperature and humidity. CNNs detect anomalies in time series data, resulting in a 92\% accuracy in identifying possible attacks.
△ Less
Submitted 30 July, 2024;
originally announced July 2024.
-
Malware Resistant Data Protection in Hyper-connected Networks: A survey
Authors:
Jannatul Ferdous,
Rafiqul Islam,
Maumita Bhattacharya,
Md Zahidul Islam
Abstract:
Data protection is the process of securing sensitive information from being corrupted, compromised, or lost. A hyperconnected network, on the other hand, is a computer networking trend in which communication occurs over a network. However, what about malware. Malware is malicious software meant to penetrate private data, threaten a computer system, or gain unauthorised network access without the u…
▽ More
Data protection is the process of securing sensitive information from being corrupted, compromised, or lost. A hyperconnected network, on the other hand, is a computer networking trend in which communication occurs over a network. However, what about malware. Malware is malicious software meant to penetrate private data, threaten a computer system, or gain unauthorised network access without the users consent. Due to the increasing applications of computers and dependency on electronically saved private data, malware attacks on sensitive information have become a dangerous issue for individuals and organizations across the world. Hence, malware defense is critical for keeping our computer systems and data protected. Many recent survey articles have focused on either malware detection systems or single attacking strategies variously. To the best of our knowledge, no survey paper demonstrates malware attack patterns and defense strategies combinedly. Through this survey, this paper aims to address this issue by merging diverse malicious attack patterns and machine learning (ML) based detection models for modern and sophisticated malware. In doing so, we focus on the taxonomy of malware attack patterns based on four fundamental dimensions the primary goal of the attack, method of attack, targeted exposure and execution process, and types of malware that perform each attack. Detailed information on malware analysis approaches is also investigated. In addition, existing malware detection techniques employing feature extraction and ML algorithms are discussed extensively. Finally, it discusses research difficulties and unsolved problems, including future research directions.
△ Less
Submitted 24 July, 2023;
originally announced July 2023.
-
Augmenting Netflix Search with In-Session Adapted Recommendations
Authors:
Moumita Bhattacharya,
Sudarshan Lamkhede
Abstract:
We motivate the need for recommendation systems that can cater to the members in-the-moment intent by leveraging their interactions from the current session. We provide an overview of an end-to-end in-session adaptive recommendations system in the context of Netflix Search. We discuss the challenges and potential solutions when developing such a system at production scale.
We motivate the need for recommendation systems that can cater to the members in-the-moment intent by leveraging their interactions from the current session. We provide an overview of an end-to-end in-session adaptive recommendations system in the context of Netflix Search. We discuss the challenges and potential solutions when developing such a system at production scale.
△ Less
Submitted 5 June, 2022;
originally announced June 2022.
-
RadioTransformer: A Cascaded Global-Focal Transformer for Visual Attention-guided Disease Classification
Authors:
Moinak Bhattacharya,
Shubham Jain,
Prateek Prasanna
Abstract:
In this work, we present RadioTransformer, a novel visual attention-driven transformer framework, that leverages radiologists' gaze patterns and models their visuo-cognitive behavior for disease diagnosis on chest radiographs. Domain experts, such as radiologists, rely on visual information for medical image interpretation. On the other hand, deep neural networks have demonstrated significant prom…
▽ More
In this work, we present RadioTransformer, a novel visual attention-driven transformer framework, that leverages radiologists' gaze patterns and models their visuo-cognitive behavior for disease diagnosis on chest radiographs. Domain experts, such as radiologists, rely on visual information for medical image interpretation. On the other hand, deep neural networks have demonstrated significant promise in similar tasks even where visual interpretation is challenging. Eye-gaze tracking has been used to capture the viewing behavior of domain experts, lending insights into the complexity of visual search. However, deep learning frameworks, even those that rely on attention mechanisms, do not leverage this rich domain information. RadioTransformer fills this critical gap by learning from radiologists' visual search patterns, encoded as 'human visual attention regions' in a cascaded global-focal transformer framework. The overall 'global' image characteristics and the more detailed 'local' features are captured by the proposed global and focal modules, respectively. We experimentally validate the efficacy of our student-teacher approach for 8 datasets involving different disease classification tasks where eye-gaze data is not available during the inference phase. Code: https://github.com/bmi-imaginelab/radiotransformer.
△ Less
Submitted 21 July, 2022; v1 submitted 23 February, 2022;
originally announced February 2022.
-
BYOD Security: A Study of Human Dimensions
Authors:
Kathleen Downer,
Maumita Bhattacharya
Abstract:
The prevalence and maturity of Bring Your Own Device (BYOD) security along with subsequent frameworks and security mechanisms in Australian organisations is a growing phenomenon somewhat similar to other developed nations. During the COVID 19 pandemic, even organisations that were previously reluctant to embrace BYOD have been forced to accept it to facilitate remote work. The aim of this paper is…
▽ More
The prevalence and maturity of Bring Your Own Device (BYOD) security along with subsequent frameworks and security mechanisms in Australian organisations is a growing phenomenon somewhat similar to other developed nations. During the COVID 19 pandemic, even organisations that were previously reluctant to embrace BYOD have been forced to accept it to facilitate remote work. The aim of this paper is to discover, through a study conducted using a survey questionnaire instrument, how employees practice and perceive the BYOD security mechanisms deployed by Australian businesses which can help guide the development of future BYOD security frameworks. Three research questions are answered by this study - What levels of awareness do Australian businesses have for BYOD security aspects? How are employees currently responding to the security mechanisms applied by their organisations for mobile devices? What are the potential weaknesses in businesses IT networks that have a direct effect on BYOD security? Overall, the aim of this research is to illuminate the findings of these research objectives so that they can be used as a basis for building new and strengthening existing BYOD security frameworks in order to enhance their effectiveness against an ever-growing list of attacks and threats targeting mobile devices in a virtually driven work force.
△ Less
Submitted 23 February, 2022;
originally announced February 2022.
-
Distributed Anomaly Detection in Edge Streams using Frequency based Sketch Datastructures
Authors:
Prateek Chanda,
Malay Bhattacharya
Abstract:
Often logs hosted in large data centers represent network traffic data over a long period of time. For instance, such network traffic data logged via a TCP dump packet sniffer (as considered in the 1998 DARPA intrusion attack) included network packets being transmitted between computers. While an online framework is necessary for detecting any anomalous or suspicious network activities like denial…
▽ More
Often logs hosted in large data centers represent network traffic data over a long period of time. For instance, such network traffic data logged via a TCP dump packet sniffer (as considered in the 1998 DARPA intrusion attack) included network packets being transmitted between computers. While an online framework is necessary for detecting any anomalous or suspicious network activities like denial of service attacks or unauthorized usage in real time, often such large data centers log data over long periods of time (e.g., TCP dump) and hence an offline framework is much more suitable in such scenarios. Given a network log history of edges from a dynamic graph, how can we assign anomaly scores to individual edges indicating suspicious events with high accuracy using only constant memory and within limited time than state-of-the-art methods? We propose MDistrib and its variants which provides (a) faster detection of anomalous events via distributed processing with GPU support compared to other approaches, (b) better false positive guarantees than state of the art methods considering fixed space and (c) with collision aware based anomaly scoring for better accuracy results than state-of-the-art approaches. We describe experiments confirming that MDistrib is more efficient than prior work.
△ Less
Submitted 27 November, 2021;
originally announced November 2021.
-
Is Attention always needed? A Case Study on Language Identification from Speech
Authors:
Atanu Mandal,
Santanu Pal,
Indranil Dutta,
Mahidas Bhattacharya,
Sudip Kumar Naskar
Abstract:
Language Identification (LID) is a crucial preliminary process in the field of Automatic Speech Recognition (ASR) that involves the identification of a spoken language from audio samples. Contemporary systems that can process speech in multiple languages require users to expressly designate one or more languages prior to utilization. The LID task assumes a significant role in scenarios where ASR s…
▽ More
Language Identification (LID) is a crucial preliminary process in the field of Automatic Speech Recognition (ASR) that involves the identification of a spoken language from audio samples. Contemporary systems that can process speech in multiple languages require users to expressly designate one or more languages prior to utilization. The LID task assumes a significant role in scenarios where ASR systems are unable to comprehend the spoken language in multilingual settings, leading to unsuccessful speech recognition outcomes. The present study introduces convolutional recurrent neural network (CRNN) based LID, designed to operate on the Mel-frequency Cepstral Coefficient (MFCC) characteristics of audio samples. Furthermore, we replicate certain state-of-the-art methodologies, specifically the Convolutional Neural Network (CNN) and Attention-based Convolutional Recurrent Neural Network (CRNN with attention), and conduct a comparative analysis with our CRNN-based approach. We conducted comprehensive evaluations on thirteen distinct Indian languages and our model resulted in over 98\% classification accuracy. The LID model exhibits high-performance levels ranging from 97% to 100% for languages that are linguistically similar. The proposed LID model exhibits a high degree of extensibility to additional languages and demonstrates a strong resistance to noise, achieving 91.2% accuracy in a noisy setting when applied to a European Language (EU) dataset.
△ Less
Submitted 25 October, 2023; v1 submitted 5 October, 2021;
originally announced October 2021.
-
Identifying Ventricular Arrhythmias and Their Predictors by Applying Machine Learning Methods to Electronic Health Records in Patients With Hypertrophic Cardiomyopathy(HCM-VAr-Risk Model)
Authors:
Moumita Bhattacharya,
Dai-Yin Lu,
Shibani M Kudchadkar,
Gabriela Villarreal Greenland,
Prasanth Lingamaneni,
Celia P Corona-Villalobos,
Yufan Guan,
Joseph E Marine,
Jeffrey E Olgin,
Stefan Zimmerman,
Theodore P Abraham,
Hagit Shatkay,
Maria Roselle Abraham
Abstract:
Clinical risk stratification for sudden cardiac death (SCD) in hypertrophic cardiomyopathy (HC) employs rules derived from American College of Cardiology Foundation/American Heart Association (ACCF/AHA) guidelines or the HCM Risk-SCD model (C-index of 0.69), which utilize a few clinical variables. We assessed whether data-driven machine learning methods that consider a wider range of variables can…
▽ More
Clinical risk stratification for sudden cardiac death (SCD) in hypertrophic cardiomyopathy (HC) employs rules derived from American College of Cardiology Foundation/American Heart Association (ACCF/AHA) guidelines or the HCM Risk-SCD model (C-index of 0.69), which utilize a few clinical variables. We assessed whether data-driven machine learning methods that consider a wider range of variables can effectively identify HC patients with ventricular arrhythmias (VAr) that lead to SCD. We scanned the electronic health records of 711 HC patients for sustained ventricular tachycardia or ventricular fibrillation. Patients with ventricular tachycardia or ventricular fibrillation (n = 61) were tagged as VAr cases and the remaining (n = 650) as non-VAr. The 2-sample t test and information gain criterion were used to identify the most informative clinical variables that distinguish VAr from non-VAr; patient records were reduced to include only these variables. Data imbalance stemming from low number of VAr cases was addressed by applying a combination of over- and under-sampling strategies.We trained and tested multiple classifiers under this sampling approach, showing effective classification. We evaluated 93 clinical variables, of which 22 proved predictive of VAr. The ensemble of logistic regression and naive Bayes classifiers, trained based on these 22 variables and corrected for data imbalance, was most effective in separating VAr from non-VAr cases (sensitivity = 0.73, specificity = 0.76, C-index = 0.83). Our method (HCM-VAr-Risk Model) identified 12 new predictors of VAr, in addition to 10 established SCD predictors. In conclusion, this is the first application of machine learning for identifying HC patients with VAr, using clinical attributes.
△ Less
Submitted 19 September, 2021;
originally announced September 2021.
-
Machine Learning Methods for Identifying Atrial Fibrillation Cases and Their Predictors in Patients With Hypertrophic Cardiomyopathy: The HCM-AF-Risk Model
Authors:
Moumita Bhattacharya,
Dai-Yin Lu,
Ioannis Ventoulis,
Gabriela V. Greenland,
Hulya Yalcin,
Yufan Guan,
Joseph E. Marine,
Jeffrey E. Olgin,
Stefan L. Zimmerman,
Theodore P. Abraham,
M. Roselle Abraham,
Hagit Shatkay
Abstract:
Hypertrophic cardiomyopathy (HCM) patients have a high incidence of atrial fibrillation (AF) and increased stroke risk, even with low risk of congestive heart failure, hypertension, age, diabetes, previous stroke/transient ischemic attack scores. Hence, there is a need to understand the pathophysiology of AF and stroke in HCM. In this retrospective study, we develop and apply a data-driven, machin…
▽ More
Hypertrophic cardiomyopathy (HCM) patients have a high incidence of atrial fibrillation (AF) and increased stroke risk, even with low risk of congestive heart failure, hypertension, age, diabetes, previous stroke/transient ischemic attack scores. Hence, there is a need to understand the pathophysiology of AF and stroke in HCM. In this retrospective study, we develop and apply a data-driven, machine learning based method to identify AF cases, and clinical and imaging features associated with AF, using electronic health record data. HCM patients with documented paroxysmal/persistent/permanent AF (n = 191) were considered AF cases, and the remaining patients in sinus rhythm (n = 640) were tagged as No-AF. We evaluated 93 clinical variables and the most informative variables useful for distinguishing AF from No-AF cases were selected based on the 2-sample t test and the information gain criterion. We identified 18 highly informative variables that are positively (n = 11) and negatively (n = 7) correlated with AF in HCM. Next, patient records were represented via these 18 variables. Data imbalance resulting from the relatively low number of AF cases was addressed via a combination of oversampling and under-sampling strategies. We trained and tested multiple classifiers under this sampling approach, showing effective classification. Specifically, an ensemble of logistic regression and naive Bayes classifiers, trained based on the 18 variables and corrected for data imbalance, proved most effective for separating AF from No-AF cases (sensitivity = 0.74, specificity = 0.70, C-index = 0.80). Our model is the first machine learning based method for identification of AF cases in HCM. This model demonstrates good performance, addresses data imbalance, and suggests that AF is associated with a more severe cardiac HCM phenotype.
△ Less
Submitted 19 September, 2021;
originally announced September 2021.
-
Co-occurrence of medical conditions: Exposing patterns through probabilistic topic modeling of SNOMED codes
Authors:
Moumita Bhattacharya,
Claudine Jurkovitz,
Hagit Shatkay
Abstract:
Patients associated with multiple co-occurring health conditions often face aggravated complications and less favorable outcomes. Co-occurring conditions are especially prevalent among individuals suffering from kidney disease, an increasingly widespread condition affecting 13% of the general population in the US. This study aims to identify and characterize patterns of co-occurring medical condit…
▽ More
Patients associated with multiple co-occurring health conditions often face aggravated complications and less favorable outcomes. Co-occurring conditions are especially prevalent among individuals suffering from kidney disease, an increasingly widespread condition affecting 13% of the general population in the US. This study aims to identify and characterize patterns of co-occurring medical conditions in patients employing a probabilistic framework. Specifically, we apply topic modeling in a non-traditional way to find associations across SNOMEDCT codes assigned and recorded in the EHRs of>13,000 patients diagnosed with kidney disease. Unlike most prior work on topic modeling, we apply the method to codes rather than to natural language. Moreover, we quantitatively evaluate the topics, assessing their tightness and distinctiveness, and also assess the medical validity of our results. Our experiments show that each topic is succinctly characterized by a few highly probable and unique disease codes, indicating that the topics are tight. Furthermore, inter-topic distance between each pair of topics is typically high, illustrating distinctiveness. Last, most coded conditions grouped together within a topic, are indeed reported to co-occur in the medical literature. Notably, our results uncover a few indirect associations among conditions that have hitherto not been reported as correlated in the medical literature.
△ Less
Submitted 19 September, 2021;
originally announced September 2021.
-
A Human Dimension of Hacking: Social Engineering through Social Media
Authors:
Heidi Wilcox,
Maumita Bhattacharya
Abstract:
Social engineering through social media channels targeting organizational employees is emerging as one of the most challenging information security threats. Social engineering defies traditional security efforts due to the method of attack relying on human naiveté or error. The vast amount of information now made available to social engineers through online social networks is facilitating methods…
▽ More
Social engineering through social media channels targeting organizational employees is emerging as one of the most challenging information security threats. Social engineering defies traditional security efforts due to the method of attack relying on human naiveté or error. The vast amount of information now made available to social engineers through online social networks is facilitating methods of attack which rely on some form of human error to enable infiltration into company networks. While, paramount to organisational information security objectives is the introduction of relevant comprehensive policy and guideline, perspectives and practices vary from global region to region. This paper identifies such regional variations and then presents a detailed investigation on information security outlooks and practices, surrounding social media, in Australian organisations (both public and private). Results detected disparate views and practices, suggesting further work is needed to achieve effective protection against security threats arsing due to social media adoption.
△ Less
Submitted 8 May, 2020;
originally announced May 2020.
-
Convergence of IT and SCADA: Associated Security Threats and Vulnerabilities
Authors:
Michael Smurthwaite,
Maumita Bhattacharya
Abstract:
As many industries shift towards centralised controlled information systems for monitoring and control, more importance is being placed upon technologies such as Supervisory Control and Data Acquisitions industrial systems (SCADA). This focus on integration and interoperability presents numerous challenges for security personnel and organisational management alike. It becomes paramount therefore t…
▽ More
As many industries shift towards centralised controlled information systems for monitoring and control, more importance is being placed upon technologies such as Supervisory Control and Data Acquisitions industrial systems (SCADA). This focus on integration and interoperability presents numerous challenges for security personnel and organisational management alike. It becomes paramount therefore to reciprocate this new direction within an organisation with adequate plans and frameworks that ensure protection and security of its SCADA architecture. A clear understanding of the relevant threats and vulnerabilities is critical for adopting/developing appropriate policy and frameworks. To this end, in this research we identify and analyse relevant SCADA security threats and vulnerabilities and present a simple scheme to classify them for better understanding.
△ Less
Submitted 8 May, 2020;
originally announced May 2020.
-
Identifying Patterns of Associated-Conditions through Topic Models of Electronic Medical Records
Authors:
Moumita Bhattacharya,
Claudine Jurkovitz,
Hagit Shatkay
Abstract:
Multiple adverse health conditions co-occurring in a patient are typically associated with poor prognosis and increased office or hospital visits. Developing methods to identify patterns of co-occurring conditions can assist in diagnosis. Thus identifying patterns of associations among co-occurring conditions is of growing interest. In this paper, we report preliminary results from a data-driven s…
▽ More
Multiple adverse health conditions co-occurring in a patient are typically associated with poor prognosis and increased office or hospital visits. Developing methods to identify patterns of co-occurring conditions can assist in diagnosis. Thus identifying patterns of associations among co-occurring conditions is of growing interest. In this paper, we report preliminary results from a data-driven study, in which we apply a machine learning method, namely, topic modeling, to electronic medical records, aiming to identify patterns of associated conditions. Specifically, we use the well established latent dirichlet allocation, a method based on the idea that documents can be modeled as a mixture of latent topics, where each topic is a distribution over words. In our study, we adapt the LDA model to identify latent topics in patients' EMRs. We evaluate the performance of our method both qualitatively, and show that the obtained topics indeed align well with distinct medical phenomena characterized by co-occurring conditions.
△ Less
Submitted 17 November, 2017;
originally announced November 2017.
-
Identifying Growth-Patterns in Children by Applying Cluster analysis to Electronic Medical Records
Authors:
Moumita Bhattacharya,
Deborah Ehrenthal,
Hagit Shatkay
Abstract:
Obesity is one of the leading health concerns in the United States. Researchers and health care providers are interested in understanding factors affecting obesity and detecting the likelihood of obesity as early as possible. In this paper, we set out to recognize children who have higher risk of obesity by identifying distinct growth patterns in them. This is done by using clustering methods, whi…
▽ More
Obesity is one of the leading health concerns in the United States. Researchers and health care providers are interested in understanding factors affecting obesity and detecting the likelihood of obesity as early as possible. In this paper, we set out to recognize children who have higher risk of obesity by identifying distinct growth patterns in them. This is done by using clustering methods, which group together children who share similar body measurements over a period of time. The measurements characterizing children within the same cluster are plotted as a function of age. We refer to these plots as growthpattern curves. We show that distinct growth-pattern curves are associated with different clusters and thus can be used to separate children into the topmost (heaviest), middle, or bottom-most cluster based on early growth measurements.
△ Less
Submitted 16 August, 2017;
originally announced August 2017.
-
BYOD Security: A New Business Challenge
Authors:
K. Downer,
Maumita Bhattacharya
Abstract:
Bring Your Own Device (BYOD) is a rapidly growing trend in businesses concerned with information technology. BYOD presents a unique list of security concerns for businesses implementing BYOD policies. Recent publications indicate a definite awareness of risks involved in incorporating BYOD into business, however it is still an underrated issue compared to other IT security concerns. This paper foc…
▽ More
Bring Your Own Device (BYOD) is a rapidly growing trend in businesses concerned with information technology. BYOD presents a unique list of security concerns for businesses implementing BYOD policies. Recent publications indicate a definite awareness of risks involved in incorporating BYOD into business, however it is still an underrated issue compared to other IT security concerns. This paper focuses on two key BYOD security issues: security challenges and available frameworks. A taxonomy specifically classifying BYOD security challenges is introduced alongside comprehensive frameworks and solutions which are also analysed to gauge their limitations.
△ Less
Submitted 6 January, 2016;
originally announced January 2016.
-
Some Experimental Issues in Financial Fraud Detection: An Investigation
Authors:
J. West,
Maumita Bhattacharya
Abstract:
Financial fraud detection is an important problem with a number of design aspects to consider. Issues such as algorithm selection and performance analysis will affect the perceived ability of proposed solutions, so for auditors and re-searchers to be able to sufficiently detect financial fraud it is necessary that these issues be thoroughly explored. In this paper we will revisit the key performan…
▽ More
Financial fraud detection is an important problem with a number of design aspects to consider. Issues such as algorithm selection and performance analysis will affect the perceived ability of proposed solutions, so for auditors and re-searchers to be able to sufficiently detect financial fraud it is necessary that these issues be thoroughly explored. In this paper we will revisit the key performance metrics used for financial fraud detection with a focus on credit card fraud, critiquing the prevailing ideas and offering our own understandings. There are many different performance metrics that have been employed in prior financial fraud detection research. We will analyse several of the popular metrics and compare their effectiveness at measuring the ability of detection mechanisms. We further investigated the performance of a range of computational intelligence techniques when applied to this problem domain, and explored the efficacy of several binary classification methods.
△ Less
Submitted 6 January, 2016;
originally announced January 2016.
-
Countering Social Engineering through Social Media: An Enterprise Security Perspective
Authors:
H. Wilcox,
Maumita Bhattacharya
Abstract:
The increasing threat of social engineers targeting social media channels to advance their attack effectiveness on company data has seen many organizations introducing initiatives to better understand these vulnerabilities. This paper examines concerns of social engineering through social media within the enterprise and explores countermeasures undertaken to stem ensuing risk. Also included is an…
▽ More
The increasing threat of social engineers targeting social media channels to advance their attack effectiveness on company data has seen many organizations introducing initiatives to better understand these vulnerabilities. This paper examines concerns of social engineering through social media within the enterprise and explores countermeasures undertaken to stem ensuing risk. Also included is an analysis of existing social media security policies and guidelines within the public and private sectors.
△ Less
Submitted 21 November, 2015;
originally announced November 2015.
-
Mining Financial Statement Fraud: An Analysis of Some Experimental Issues
Authors:
J. West,
Maumita Bhattacharya
Abstract:
Financial statement fraud detection is an important problem with a number of design aspects to consider. Issues such as (i) problem representation, (ii) feature selection, and (iii) choice of performance metrics all influence the perceived performance of detection algorithms. Efficient implementation of financial fraud detection methods relies on a clear understanding of these issues. In this pape…
▽ More
Financial statement fraud detection is an important problem with a number of design aspects to consider. Issues such as (i) problem representation, (ii) feature selection, and (iii) choice of performance metrics all influence the perceived performance of detection algorithms. Efficient implementation of financial fraud detection methods relies on a clear understanding of these issues. In this paper we present an analysis of the three key experimental issues associated with financial statement fraud detection, critiquing the prevailing ideas and providing new understandings.
△ Less
Submitted 24 October, 2015;
originally announced October 2015.
-
Intelligent Financial Fraud Detection Practices: An Investigation
Authors:
J. West,
Maumita Bhattacharya,
R. Islam
Abstract:
Financial fraud is an issue with far reaching consequences in the finance industry, government, corporate sectors, and for ordinary consumers. Increasing dependence on new technologies such as cloud and mobile computing in recent years has compounded the problem. Traditional methods of detection involve extensive use of auditing, where a trained individual manually observes reports or transactions…
▽ More
Financial fraud is an issue with far reaching consequences in the finance industry, government, corporate sectors, and for ordinary consumers. Increasing dependence on new technologies such as cloud and mobile computing in recent years has compounded the problem. Traditional methods of detection involve extensive use of auditing, where a trained individual manually observes reports or transactions in an attempt to discover fraudulent behaviour. This method is not only time consuming, expensive and inaccurate, but in the age of big data it is also impractical. Not surprisingly, financial institutions have turned to automated processes using statistical and computational methods. This paper presents a comprehensive investigation on financial fraud detection practices using such data mining methods, with a particular focus on computational intelligence-based techniques. Classification of the practices based on key aspects such as detection algorithm used, fraud type investigated, and success rate have been covered. Issues and challenges associated with the current practices and potential future direction of research have also been identified.
△ Less
Submitted 24 October, 2015;
originally announced October 2015.
-
Evolutionary Landscape and Management of Population Diversity
Authors:
Maumita Bhattacharya
Abstract:
The search ability of an Evolutionary Algorithm (EA) depends on the variation among the individuals in the population [3, 4, 8]. Maintaining an optimal level of diversity in the EA population is imperative to ensure that progress of the EA search is unhindered by premature convergence to suboptimal solutions. Clearer understanding of the concept of population diversity, in the context of evolution…
▽ More
The search ability of an Evolutionary Algorithm (EA) depends on the variation among the individuals in the population [3, 4, 8]. Maintaining an optimal level of diversity in the EA population is imperative to ensure that progress of the EA search is unhindered by premature convergence to suboptimal solutions. Clearer understanding of the concept of population diversity, in the context of evolutionary search and premature convergence in particular, is the key to designing efficient EAs. To this end, this paper first presents a brief analysis of the EA population diversity issues. Next we present an investigation on a counter-niching EA technique [4] that introduces and maintains constructive diversity in the population. The proposed approach uses informed genetic operations to reach promising, but unexplored or under-explored areas of the search space, while discouraging premature local convergence. Simulation runs on a suite of standard benchmark test functions with Genetic Algorithm (GA) implementation shows promising results.
△ Less
Submitted 24 October, 2015;
originally announced October 2015.
-
Diversity Handling In Evolutionary Landscape
Authors:
Maumita Bhattacharya
Abstract:
The search ability of an Evolutionary Algorithm (EA) depends on the variation among the individuals in the population. Maintaining an optimal level of diversity in the EA population is imperative to ensure that progress of the EA search is unhindered by premature convergence to suboptimal solutions. Clearer understanding of the concept of population diversity, in the context of evolutionary search…
▽ More
The search ability of an Evolutionary Algorithm (EA) depends on the variation among the individuals in the population. Maintaining an optimal level of diversity in the EA population is imperative to ensure that progress of the EA search is unhindered by premature convergence to suboptimal solutions. Clearer understanding of the concept of population diversity, in the context of evolutionary search and premature convergence in particular, is the key to designing efficient EAs. To this end, this paper first presents a comprehensive analysis of the EA population diversity issues. Next we present an investigation on a counter-niching EA technique that introduces and maintains constructive diversity in the population. The proposed approach uses informed genetic operations to reach promising, but un-explored or under-explored areas of the search space, while discouraging premature local convergence. Simulation runs on a number of standard benchmark test functions with Genetic Algorithm (GA) implementation shows promising results.
△ Less
Submitted 15 November, 2014;
originally announced November 2014.
-
Uncertainty And Evolutionary Optimization: A Novel Approach
Authors:
Maumita Bhattacharya,
R. Islam,
A. Mahmood
Abstract:
Evolutionary algorithms (EA) have been widely accepted as efficient solvers for complex real world optimization problems, including engineering optimization. However, real world optimization problems often involve uncertain environment including noisy and/or dynamic environments, which pose major challenges to EA-based optimization. The presence of noise interferes with the evaluation and the sele…
▽ More
Evolutionary algorithms (EA) have been widely accepted as efficient solvers for complex real world optimization problems, including engineering optimization. However, real world optimization problems often involve uncertain environment including noisy and/or dynamic environments, which pose major challenges to EA-based optimization. The presence of noise interferes with the evaluation and the selection process of EA, and thus adversely affects its performance. In addition, as presence of noise poses challenges to the evaluation of the fitness function, it may need to be estimated instead of being evaluated. Several existing approaches attempt to address this problem, such as introduction of diversity (hyper mutation, random immigrants, special operators) or incorporation of memory of the past (diploidy, case based memory). However, these approaches fail to adequately address the problem. In this paper we propose a Distributed Population Switching Evolutionary Algorithm (DPSEA) method that addresses optimization of functions with noisy fitness using a distributed population switching architecture, to simulate a distributed self-adaptive memory of the solution space. Local regression is used in the pseudo-populations to estimate the fitness. Successful applications to benchmark test problems ascertain the proposed method's superior performance in terms of both robustness and accuracy.
△ Less
Submitted 21 November, 2014; v1 submitted 15 July, 2014;
originally announced July 2014.
-
Bioclimating Modelling: A Machine Learning Perspective
Authors:
Maumita Bhattacharya
Abstract:
Many machine learning (ML) approaches are widely used to generate bioclimatic models for prediction of geographic range of organism as a function of climate. Applications such as prediction of range shift in organism, range of invasive species influenced by climate change are important parameters in understanding the impact of climate change. However, success of machine learning-based approaches d…
▽ More
Many machine learning (ML) approaches are widely used to generate bioclimatic models for prediction of geographic range of organism as a function of climate. Applications such as prediction of range shift in organism, range of invasive species influenced by climate change are important parameters in understanding the impact of climate change. However, success of machine learning-based approaches depends on a number of factors. While it can be safely said that no particular ML technique can be effective in all applications and success of a technique is predominantly dependent on the application or the type of the problem, it is useful to understand their behaviour to ensure informed choice of techniques. This paper presents a comprehensive review of machine learning-based bioclimatic model generation and analyses the factors influencing success of such models. Considering the wide use of statistical techniques, in our discussion we also include conventional statistical techniques used in bioclimatic modelling.
△ Less
Submitted 18 June, 2013;
originally announced June 2013.
-
Evolutionary Approaches to Expensive Optimisation
Authors:
Maumita Bhattacharya
Abstract:
Surrogate assisted evolutionary algorithms (EA) are rapidly gaining popularity where applications of EA in complex real world problem domains are concerned. Although EAs are powerful global optimizers, finding optimal solution to complex high dimensional, multimodal problems often require very expensive fitness function evaluations. Needless to say, this could brand any population-based iterative…
▽ More
Surrogate assisted evolutionary algorithms (EA) are rapidly gaining popularity where applications of EA in complex real world problem domains are concerned. Although EAs are powerful global optimizers, finding optimal solution to complex high dimensional, multimodal problems often require very expensive fitness function evaluations. Needless to say, this could brand any population-based iterative optimization technique to be the most crippling choice to handle such problems. Use of approximate model or surrogates provides a much cheaper option. However, naturally this cheaper option comes with its own price. This paper discusses some of the key issues involved with use of approximation in evolutionary algorithm, possible best practices and solutions. Answers to the following questions have been sought: what type of fitness approximation to be used; which approximation model to use; how to integrate the approximation model in EA; how much approximation to use; and how to ensure reliable approximation.
△ Less
Submitted 11 March, 2013;
originally announced March 2013.
-
Machine Learning for Bioclimatic Modelling
Authors:
Maumita Bhattacharya
Abstract:
Many machine learning (ML) approaches are widely used to generate bioclimatic models for prediction of geographic range of organism as a function of climate. Applications such as prediction of range shift in organism, range of invasive species influenced by climate change are important parameters in understanding the impact of climate change. However, success of machine learning-based approaches d…
▽ More
Many machine learning (ML) approaches are widely used to generate bioclimatic models for prediction of geographic range of organism as a function of climate. Applications such as prediction of range shift in organism, range of invasive species influenced by climate change are important parameters in understanding the impact of climate change. However, success of machine learning-based approaches depends on a number of factors. While it can be safely said that no particular ML technique can be effective in all applications and success of a technique is predominantly dependent on the application or the type of the problem, it is useful to understand their behavior to ensure informed choice of techniques. This paper presents a comprehensive review of machine learning-based bioclimatic model generation and analyses the factors influencing success of such models. Considering the wide use of statistical techniques, in our discussion we also include conventional statistical techniques used in bioclimatic modelling.
△ Less
Submitted 11 March, 2013;
originally announced March 2013.
-
Expensive Optimisation: A Metaheuristics Perspective
Authors:
Maumita Bhattacharya
Abstract:
Stochastic, iterative search methods such as Evolutionary Algorithms (EAs) are proven to be efficient optimizers. However, they require evaluation of the candidate solutions which may be prohibitively expensive in many real world optimization problems. Use of approximate models or surrogates is being explored as a way to reduce the number of such evaluations. In this paper we investigated three su…
▽ More
Stochastic, iterative search methods such as Evolutionary Algorithms (EAs) are proven to be efficient optimizers. However, they require evaluation of the candidate solutions which may be prohibitively expensive in many real world optimization problems. Use of approximate models or surrogates is being explored as a way to reduce the number of such evaluations. In this paper we investigated three such methods. The first method (DAFHEA) partially replaces an expensive function evaluation by its approximate model. The approximation is realized with support vector machine (SVM) regression models. The second method (DAFHEA II) is an enhancement on DAFHEA to accommodate for uncertain environments. The third one uses surrogate ranking with preference learning or ordinal regression. The fitness of the candidates is estimated by modeling their rank. The techniques' performances on some of the benchmark numerical optimization problems have been reported. The comparative benefits and shortcomings of both techniques have been identified.
△ Less
Submitted 9 March, 2013;
originally announced March 2013.
