Showing 1–22 of 22 results for author: Arun, A

DOLOS: Tricking the Wi-Fi APs with Incorrect User Locations

Authors: Aditya Arun, Vaibhav Anand, Wei Sun, Roshan Ayyalasomayajula, Dinesh Bharadia

Abstract: Wi-Fi-based indoor localization has been extensively studied for context-aware services. As a result, the accurate Wi-Fi-based indoor localization introduces a great location privacy threat. However, the existing solutions for location privacy protection are hard to implement on current devices. They require extra hardware deployment in the environment or hardware modifications at the transmitter… ▽ More Wi-Fi-based indoor localization has been extensively studied for context-aware services. As a result, the accurate Wi-Fi-based indoor localization introduces a great location privacy threat. However, the existing solutions for location privacy protection are hard to implement on current devices. They require extra hardware deployment in the environment or hardware modifications at the transmitter or receiver side. To this end, we propose DOLOS, a system that can protect the location privacy of the Wi-Fi user with a novel signal obfuscation approach. DOLOSis a software-only solution that can be deployed on existing protocol-compliant Wi-Fi user devices. We provide this obfuscation by invalidating a simple assumption made by most localization systems -- "direct path signal arrives earlier than all the reflections to distinguish this direct path prior to estimating the location". However, DOLOS creates a novel software fix that allows the user to transmit the signal wherein this direct path arrives later, creating ambiguity in the location estimates. Our experimental results demonstrate DOLOS can degrade the localization accuracy of state-of-art systems by 6x for a single AP and 2.5x for multiple AP scenarios, thereby protecting the Wi-Fi user's location privacy without compromising the Wi-Fi communication performance. △ Less

Submitted 22 July, 2024; originally announced July 2024.

Next Generation of Phishing Attacks using AI powered Browsers

Authors: Akshaya Arun, Nasr Abosata

Abstract: The increase in the number of phishing demands innovative solutions to safeguard users from phishing attacks. This study explores the development and utilization of a real-time browser extension integrated with machine learning model to improve the detection of phishing websites. The results showed that the model had an accuracy of 98.32%, precision of 98.62%, recall of 97.86%, and an F1-score of… ▽ More The increase in the number of phishing demands innovative solutions to safeguard users from phishing attacks. This study explores the development and utilization of a real-time browser extension integrated with machine learning model to improve the detection of phishing websites. The results showed that the model had an accuracy of 98.32%, precision of 98.62%, recall of 97.86%, and an F1-score of 98.24%. When compared to other algorithms like Support Vector Machine, Naïve Bayes, Decision Tree, XGBoost, and K Nearest Neighbor, the Random Forest algorithm stood out for its effectiveness in detecting phishing attacks. The zero-day phishing attack detection testing over a 15-day period revealed the model's capability to identify previously unseen threats and thus achieving an overall accuracy rate of 99.11%. Furthermore, the model showed better performance when compared to conventional security measures like Google Safe Browsing. The model had successfully detected phishing URLs that evaded detection by Google safe browsing. This research shows how using machine learning in real-time browser extensions can defend against phishing attacks. It gives useful information about cybersecurity and helps make the internet safer for everyone. △ Less

Submitted 18 June, 2024; originally announced June 2024.

Comments: 6 pages, 10 Figures

X-posing Free Speech: Examining the Impact of Moderation Relaxation on Online Social Networks

Authors: Arvindh Arun, Saurav Chhatani, Jisun An, Ponnurangam Kumaraguru

Abstract: We investigate the impact of free speech and the relaxation of moderation on online social media platforms using Elon Musk's takeover of Twitter as a case study. By curating a dataset of over 10 million tweets, our study employs a novel framework combining content and network analysis. Our findings reveal a significant increase in the distribution of certain forms of hate content, particularly tar… ▽ More We investigate the impact of free speech and the relaxation of moderation on online social media platforms using Elon Musk's takeover of Twitter as a case study. By curating a dataset of over 10 million tweets, our study employs a novel framework combining content and network analysis. Our findings reveal a significant increase in the distribution of certain forms of hate content, particularly targeting the LGBTQ+ community and liberals. Network analysis reveals the formation of cohesive hate communities facilitated by influential bridge users, with substantial growth in interactions hinting at increased hate production and diffusion. By tracking the temporal evolution of PageRank, we identify key influencers, primarily self-identified far-right supporters disseminating hate against liberals and woke culture. Ironically, embracing free speech principles appears to have enabled hate speech against the very concept of freedom of expression and free speech itself. Our findings underscore the delicate balance platforms must strike between open expression and robust moderation to curb the proliferation of hate online. △ Less

Submitted 23 May, 2024; v1 submitted 17 April, 2024; originally announced April 2024.

EXPLORE -- Explainable Song Recommendation

Authors: Abhinav Arun, Mehul Soni, Palash Choudhary, Saksham Arora

Abstract: This study explores the development of an explainable music recommendation system with enhanced user control. Leveraging a hybrid of collaborative filtering and content-based filtering, we address the challenges of opaque recommendation logic and lack of user influence on results. We present a novel approach combining advanced algorithms and an interactive user interface. Our methodology integrate… ▽ More This study explores the development of an explainable music recommendation system with enhanced user control. Leveraging a hybrid of collaborative filtering and content-based filtering, we address the challenges of opaque recommendation logic and lack of user influence on results. We present a novel approach combining advanced algorithms and an interactive user interface. Our methodology integrates Spotify data with user preference analytics to tailor music suggestions. Evaluation through RMSE and user studies underscores the efficacy and user satisfaction with our system. The paper concludes with potential directions for future enhancements in group recommendations and dynamic feedback integration. △ Less

Submitted 30 December, 2023; originally announced January 2024.

Comments: 6 pages, 7 figures

Towards a Unified Multimodal Reasoning Framework

Authors: Abhinav Arun, Dipendra Singh Mal, Mehul Soni, Tomohiro Sawada

Abstract: Recent advancements in deep learning have led to the development of powerful language models (LMs) that excel in various tasks. Despite these achievements, there is still room for improvement, particularly in enhancing reasoning abilities and incorporating multimodal data. This report investigates the potential impact of combining Chain-of-Thought (CoT) reasoning and Visual Question Answering (VQA… ▽ More Recent advancements in deep learning have led to the development of powerful language models (LMs) that excel in various tasks. Despite these achievements, there is still room for improvement, particularly in enhancing reasoning abilities and incorporating multimodal data. This report investigates the potential impact of combining Chain-of-Thought (CoT) reasoning and Visual Question Answering (VQA) techniques to improve LM's accuracy in solving multiple-choice questions. By employing TextVQA and ScienceQA datasets, we assessed the effectiveness of three text embedding methods and three visual embedding approaches. Our experiments aimed to fill the gap in current research by investigating the combined impact of CoT and VQA, contributing to the understanding of how these techniques can improve the reasoning capabilities of state-of-the-art models like GPT-4. Results from our experiments demonstrated the potential of these approaches in enhancing LM's reasoning and question-answering capabilities, providing insights for further research and development in the field, and paving the way for more accurate and reliable AI systems that can handle complex reasoning tasks across multiple modalities. △ Less

Submitted 22 December, 2023; originally announced December 2023.

Comments: 6 pages, 11 figures

Numerical Reasoning for Financial Reports

Authors: Abhinav Arun, Ashish Dhiman, Mehul Soni, Yibei Hu

Abstract: Financial reports offer critical insights into a company's operations, yet their extensive length typically spanning 30 40 pages poses challenges for swift decision making in dynamic markets. To address this, we leveraged finetuned Large Language Models (LLMs) to distill key indicators and operational metrics from these reports basis questions from the user. We devised a method to locate critical… ▽ More Financial reports offer critical insights into a company's operations, yet their extensive length typically spanning 30 40 pages poses challenges for swift decision making in dynamic markets. To address this, we leveraged finetuned Large Language Models (LLMs) to distill key indicators and operational metrics from these reports basis questions from the user. We devised a method to locate critical data, and leverage the FinQA dataset to fine-tune both Llama-2 7B and T5 models for customized question answering. We achieved results comparable to baseline on the final numerical answer, a competitive accuracy in numerical reasoning and calculation. △ Less

Submitted 22 December, 2023; originally announced December 2023.

Comments: 10 pages, 11 figures, 6 tables

Efficient Profit Maximization in Reliability Concerned Static Vehicular Cloud System

Authors: Suvarthi Sarkar, Akshat Arun, Harshit Surekha, Aryabartta Sahu

Abstract: Modern electric VUs are equipped with a variety of increasingly potent computing, communication, and storage resources, and with this tremendous computation power in their arsenal can be used to enhance the computing power of regular cloud systems, which is termed as vehicular cloud. Unlike in the traditional cloud computing resources, these vehicular cloud resource moves around and participates i… ▽ More Modern electric VUs are equipped with a variety of increasingly potent computing, communication, and storage resources, and with this tremendous computation power in their arsenal can be used to enhance the computing power of regular cloud systems, which is termed as vehicular cloud. Unlike in the traditional cloud computing resources, these vehicular cloud resource moves around and participates in the vehicular cloud for a sporadic duration at parking places, shopping malls, etc. This introduces the dynamic nature of vehicular resource participation in the vehicular cloud. As the user-submitted task gets allocated on these vehicular units for execution and the dynamic stay nature of vehicular units, enforce the system to ensure the reliability of task execution by allocating multiple redundant vehicular units for the task. In this work, we are maximizing the profit of vehicular cloud by ensuring the reliability of task execution where user tasks come online manner with different revenue, execution, and deadline. We propose an efficient approach to solve this problem by considering (a) task classification based on the deadline and laxity of the task, (b) ordering of tasks for task admission based on the expected profit of the task, (c) classification of vehicular units based in expected residency time and reliability concerned redundant allocation of tasks of vehicular units considering this classification and (d) handing dynamic scenario of the vehicular unit leaving the cloud system by copying the maximum percentage of executed virtual machine of the task to the substitute unit. We compared our proposed profit maximization approach with the state of art approach and showed that our approach outperforms the state of art approach with an extra 10\% to 20\% profit margin. △ Less

Submitted 17 November, 2023; originally announced November 2023.

Ensemble of Task-Specific Language Models for Brain Encoding

Authors: Arvindh Arun, Jerrin John, Sanjai Kumaran

Abstract: Language models have been shown to be rich enough to encode fMRI activations of certain Regions of Interest in our Brains. Previous works have explored transfer learning from representations learned for popular natural language processing tasks for predicting brain responses. In our work, we improve the performance of such encoders by creating an ensemble model out of 10 popular Language Models (2… ▽ More Language models have been shown to be rich enough to encode fMRI activations of certain Regions of Interest in our Brains. Previous works have explored transfer learning from representations learned for popular natural language processing tasks for predicting brain responses. In our work, we improve the performance of such encoders by creating an ensemble model out of 10 popular Language Models (2 syntactic and 8 semantic). We beat the current baselines by 10% on average across all ROIs through our ensembling methods. △ Less

Submitted 9 November, 2023; v1 submitted 24 October, 2023; originally announced October 2023.

XRLoc: Accurate UWB Localization to Realize XR Deployments

Authors: Aditya Arun, Shunsuke Saruwatari, Sureel Shah, Dinesh Bharadia

Abstract: Understanding the location of ultra-wideband (UWB) tag-attached objects and people in the real world is vital to enabling a smooth cyber-physical transition. However, most UWB localization systems today require multiple anchors in the environment, which can be very cumbersome to set up. In this work, we develop XRLoc, providing an accuracy of a few centimeters in many real-world scenarios. This pa… ▽ More Understanding the location of ultra-wideband (UWB) tag-attached objects and people in the real world is vital to enabling a smooth cyber-physical transition. However, most UWB localization systems today require multiple anchors in the environment, which can be very cumbersome to set up. In this work, we develop XRLoc, providing an accuracy of a few centimeters in many real-world scenarios. This paper will delineate the key ideas which allow us to overcome the fundamental restrictions that plague a single anchor point from localization of a device to within an error of a few centimeters. We deploy a VR chess game using everyday objects as a demo and find that our system achieves $2.4$ cm median accuracy and $5.3$ cm $90^\mathrm{th}$ percentile accuracy in dynamic scenarios, performing at least $8\times$ better than state-of-art localization systems. Additionally, we implement a MAC protocol to furnish these locations for over $10$ tags at update rates of $100$ Hz, with a localization latency of $\sim 1$ ms. △ Less

Submitted 2 May, 2024; v1 submitted 24 July, 2023; originally announced July 2023.

Comments: This paper is accepted by ACM SenSys 2023. The published version is https://doi.org/10.1145/3625687.3625810 in ACM Digital Library

Journal ref: Proceedings of ACM Conference on Embedded Networked Sensor Systems (ACM SenSys'23), pp.459-473, 2023

WiROS: WiFi sensing toolbox for robotics

Authors: William Hunter, Aditya Arun, Dinesh Bharadia

Abstract: Many recent works have explored using WiFi-based sensing to improve SLAM, robot manipulation, or exploration. Moreover, widespread availability makes WiFi the most advantageous RF signal to leverage. But WiFi sensors lack an accurate, tractable, and versatile toolbox, which hinders their widespread adoption with robot's sensor stacks. We develop WiROS to address this immediate need, furnishing m… ▽ More Many recent works have explored using WiFi-based sensing to improve SLAM, robot manipulation, or exploration. Moreover, widespread availability makes WiFi the most advantageous RF signal to leverage. But WiFi sensors lack an accurate, tractable, and versatile toolbox, which hinders their widespread adoption with robot's sensor stacks. We develop WiROS to address this immediate need, furnishing many WiFi-related measurements as easy-to-consume ROS topics. Specifically, WiROS is a plug-and-play WiFi sensing toolbox providing access to coarse-grained WiFi signal strength (RSSI), fine-grained WiFi channel state information (CSI), and other MAC-layer information (device address, packet id's or frequency-channel information). Additionally, WiROS open-sources state-of-art algorithms to calibrate and process WiFi measurements to furnish accurate bearing information for received WiFi signals. The open-sourced repository is: https://github.com/ucsdwcsng/WiROS △ Less

Submitted 22 May, 2023; originally announced May 2023.

CAFIN: Centrality Aware Fairness inducing IN-processing for Unsupervised Representation Learning on Graphs

Authors: Arvindh Arun, Aakash Aanegola, Amul Agrawal, Ramasuri Narayanam, Ponnurangam Kumaraguru

Abstract: Unsupervised Representation Learning on graphs is gaining traction due to the increasing abundance of unlabelled network data and the compactness, richness, and usefulness of the representations generated. In this context, the need to consider fairness and bias constraints while generating the representations has been well-motivated and studied to some extent in prior works. One major limitation o… ▽ More Unsupervised Representation Learning on graphs is gaining traction due to the increasing abundance of unlabelled network data and the compactness, richness, and usefulness of the representations generated. In this context, the need to consider fairness and bias constraints while generating the representations has been well-motivated and studied to some extent in prior works. One major limitation of most of the prior works in this setting is that they do not aim to address the bias generated due to connectivity patterns in the graphs, such as varied node centrality, which leads to a disproportionate performance across nodes. In our work, we aim to address this issue of mitigating bias due to inherent graph structure in an unsupervised setting. To this end, we propose CAFIN, a centrality-aware fairness-inducing framework that leverages the structural information of graphs to tune the representations generated by existing frameworks. We deploy it on GraphSAGE (a popular framework in this domain) and showcase its efficacy on two downstream tasks - Node Classification and Link Prediction. Empirically, CAFIN consistently reduces the performance disparity across popular datasets (varying from 18 to 80% reduction in performance disparity) from various domains while incurring only a minimal cost of fairness. △ Less

Submitted 20 April, 2024; v1 submitted 10 April, 2023; originally announced April 2023.

Improving Faithfulness of Abstractive Summarization by Controlling Confounding Effect of Irrelevant Sentences

Authors: Asish Ghoshal, Arash Einolghozati, Ankit Arun, Haoran Li, Lili Yu, Vera Gor, Yashar Mehdad, Scott Wen-tau Yih, Asli Celikyilmaz

Abstract: Lack of factual correctness is an issue that still plagues state-of-the-art summarization systems despite their impressive progress on generating seemingly fluent summaries. In this paper, we show that factual inconsistency can be caused by irrelevant parts of the input text, which act as confounders. To that end, we leverage information-theoretic measures of causal effects to quantify the amount… ▽ More Lack of factual correctness is an issue that still plagues state-of-the-art summarization systems despite their impressive progress on generating seemingly fluent summaries. In this paper, we show that factual inconsistency can be caused by irrelevant parts of the input text, which act as confounders. To that end, we leverage information-theoretic measures of causal effects to quantify the amount of confounding and precisely quantify how they affect the summarization performance. Based on insights derived from our theoretical results, we design a simple multi-task model to control such confounding by leveraging human-annotated relevant sentences when available. Crucially, we give a principled characterization of data distributions where such confounding can be large thereby necessitating the use of human annotated relevant sentences to generate factual summaries. Our approach improves faithfulness scores by 20\% over strong baselines on AnswerSumm \citep{fabbri2021answersumm}, a conversation summarization dataset where lack of faithfulness is a significant issue due to the subjective nature of the task. Our best method achieves the highest faithfulness score while also achieving state-of-the-art results on standard metrics like ROUGE and METEOR. We corroborate these improvements through human evaluation. △ Less

Submitted 18 January, 2024; v1 submitted 19 December, 2022; originally announced December 2022.

Users are Closer than they Appear: Protecting User Location from WiFi APs

Authors: Roshan Ayyalasomayajula, Aditya Arun, Wei Sun, Dinesh Bharadia

Abstract: WiFi-based indoor localization has now matured for over a decade. Most of the current localization algorithms rely on the WiFi access points (APs) in the enterprise network to localize the WiFi user accurately. Thus, the WiFi user's location information could be easily snooped by an attacker listening through a compromised WiFi AP. With indoor localization and navigation being the next step toward… ▽ More WiFi-based indoor localization has now matured for over a decade. Most of the current localization algorithms rely on the WiFi access points (APs) in the enterprise network to localize the WiFi user accurately. Thus, the WiFi user's location information could be easily snooped by an attacker listening through a compromised WiFi AP. With indoor localization and navigation being the next step towards automation, it is important to give users the capability to defend against such attacks. In this paper, we present MIRAGE, a system that can utilize the downlink physical layer information to create a defense against an attacker snooping on a WiFi user's location information. MIRAGE achieves this by utilizing the beamforming capability of the transmitter that is already part of the WiFi protocols. With this initial idea, we have demonstrated that the user can obfuscate his/her location from the WiFi AP always with no compromise to the throughput of the existing WiFi communication system and reduce the user location accuracy of the attacker from 2.3m to more than 10m. △ Less

Submitted 17 November, 2022; originally announced November 2022.

Comments: 6 pages, 6 figures, submitted to HotMobile 2023

ViWiD: Leveraging WiFi for Robust and Resource-Efficient SLAM

Authors: Aditya Arun, William Hunter, Roshan Ayyalasomayajula, Dinesh Bharadia

Abstract: Recent interest towards autonomous navigation and exploration robots for indoor applications has spurred research into indoor Simultaneous Localization and Mapping (SLAM) robot systems. While most of these SLAM systems use Visual and LiDAR sensors in tandem with an odometry sensor, these odometry sensors drift over time. To combat this drift, Visual SLAM systems deploy compute and memory intensive… ▽ More Recent interest towards autonomous navigation and exploration robots for indoor applications has spurred research into indoor Simultaneous Localization and Mapping (SLAM) robot systems. While most of these SLAM systems use Visual and LiDAR sensors in tandem with an odometry sensor, these odometry sensors drift over time. To combat this drift, Visual SLAM systems deploy compute and memory intensive search algorithms to detect `Loop Closures', which make the trajectory estimate globally consistent. To circumvent these resource (compute and memory) intensive algorithms, we present ViWiD, which integrates WiFi and Visual sensors in a dual-layered system. This dual-layered approach separates the tasks of local and global trajectory estimation making ViWiD resource efficient while achieving on-par or better performance to state-of-the-art Visual SLAM. We demonstrate ViWiD's performance on four datasets, covering over 1500 m of traversed path and show 4.3x and 4x reduction in compute and memory consumption respectively compared to state-of-the-art Visual and Lidar SLAM systems with on par SLAM performance. △ Less

Submitted 16 September, 2022; originally announced September 2022.

A Review on the Process of Automated Software Testing

Authors: Durga Shree N, Sree Dharinya S, Dasari Vijayasree, Nadendla Sai Roopa, Anugu Arun

Abstract: The requirements in automation, digitalization, and fast computations have loaded the IT sector with expectations of highly reliable, efficient, and cost-effective software. Given that the process of testing, verification, and validation of software products consumes 50-75% of the total revenue if the testing process is ineffective, "n" times the expenditure must be invested to mend the havoc caus… ▽ More The requirements in automation, digitalization, and fast computations have loaded the IT sector with expectations of highly reliable, efficient, and cost-effective software. Given that the process of testing, verification, and validation of software products consumes 50-75% of the total revenue if the testing process is ineffective, "n" times the expenditure must be invested to mend the havoc caused. A delay in project completion is often attributed to the testing phase because of the numerous cycles of debugging process. The software testing process determines the face of the product released to the user. It sets the standard and reliability of a company's outputs. As the complexity increases, testing gets intense so as to examine all the outliers and various branches of the processing flow. The testing process is automated using software tools to avoid the tedious manual process of test input generation and validation criteria, which certifies the program only to a certain confidence level in the presence of outliers. △ Less

Submitted 7 September, 2022; originally announced September 2022.

Comments: 7 pages, 2 figures, 2 tables

Erasing Labor with Labor: Dark Patterns and Lockstep Behaviors on Google Play

Authors: Ashwin Singh, Arvindh Arun, Ayushi Jain, Pooja Desur, Pulak Malhotra, Duen Horng Chau, Ponnurangam Kumaraguru

Abstract: Google Play's policy forbids the use of incentivized installs, ratings, and reviews to manipulate the placement of apps. However, there still exist apps that incentivize installs for other apps on the platform. To understand how install-incentivizing apps affect users, we examine their ecosystem through a socio-technical lens and perform a mixed-methods analysis of their reviews and permissions. O… ▽ More Google Play's policy forbids the use of incentivized installs, ratings, and reviews to manipulate the placement of apps. However, there still exist apps that incentivize installs for other apps on the platform. To understand how install-incentivizing apps affect users, we examine their ecosystem through a socio-technical lens and perform a mixed-methods analysis of their reviews and permissions. Our dataset contains 319K reviews collected daily over five months from 60 such apps that cumulatively account for over 160.5M installs. We perform qualitative analysis of reviews to reveal various types of dark patterns that developers incorporate in install-incentivizing apps, highlighting their normative concerns at both user and platform levels. Permissions requested by these apps validate our discovery of dark patterns, with over 92% apps accessing sensitive user information. We find evidence of fraudulent reviews on install-incentivizing apps, following which we model them as an edge stream in a dynamic bipartite graph of apps and reviewers. Our proposed reconfiguration of a state-of-the-art microcluster anomaly detection algorithm yields promising preliminary results in detecting this fraud. We discover highly significant lockstep behaviors exhibited by reviews that aim to boost the overall rating of an install-incentivizing app. Upon evaluating the 50 most suspicious clusters of boosting reviews detected by the algorithm, we find (i) near-identical pairs of reviews across 94% (47 clusters), and (ii) over 35% (1,687 of 4,717 reviews) present in the same form near-identical pairs within their cluster. Finally, we conclude with a discussion on how fraud is intertwined with labor and poses a threat to the trust and transparency of Google Play. △ Less

Submitted 17 May, 2022; v1 submitted 9 February, 2022; originally announced February 2022.

Best Practices for Data-Efficient Modeling in NLG:How to Train Production-Ready Neural Models with Less Data

Authors: Ankit Arun, Soumya Batra, Vikas Bhardwaj, Ashwini Challa, Pinar Donmez, Peyman Heidari, Hakan Inan, Shashank Jain, Anuj Kumar, Shawn Mei, Karthik Mohan, Michael White

Abstract: Natural language generation (NLG) is a critical component in conversational systems, owing to its role of formulating a correct and natural text response. Traditionally, NLG components have been deployed using template-based solutions. Although neural network solutions recently developed in the research community have been shown to provide several benefits, deployment of such model-based solutions… ▽ More Natural language generation (NLG) is a critical component in conversational systems, owing to its role of formulating a correct and natural text response. Traditionally, NLG components have been deployed using template-based solutions. Although neural network solutions recently developed in the research community have been shown to provide several benefits, deployment of such model-based solutions has been challenging due to high latency, correctness issues, and high data needs. In this paper, we present approaches that have helped us deploy data-efficient neural solutions for NLG in conversational systems to production. We describe a family of sampling and modeling techniques to attain production quality with light-weight neural network models using only a fraction of the data that would be necessary otherwise, and show a thorough comparison between each. Our results show that domain complexity dictates the appropriate approach to achieve high data efficiency. Finally, we distill the lessons from our experimental findings into a list of best practices for production-level NLG model development, and present them in a brief runbook. Importantly, the end products of all of the techniques are small sequence-to-sequence models (2Mb) that we can reliably deploy in production. △ Less

Submitted 7 November, 2020; originally announced November 2020.

Comments: Accepted for publication in COLING 2020

Weakly Supervised Instance Segmentation by Learning Annotation Consistent Instances

Authors: Aditya Arun, C. V. Jawahar, M. Pawan Kumar

Abstract: Recent approaches for weakly supervised instance segmentations depend on two components: (i) a pseudo label generation model that provides instances which are consistent with a given annotation; and (ii) an instance segmentation model, which is trained in a supervised manner using the pseudo labels as ground-truth. Unlike previous approaches, we explicitly model the uncertainty in the pseudo label… ▽ More Recent approaches for weakly supervised instance segmentations depend on two components: (i) a pseudo label generation model that provides instances which are consistent with a given annotation; and (ii) an instance segmentation model, which is trained in a supervised manner using the pseudo labels as ground-truth. Unlike previous approaches, we explicitly model the uncertainty in the pseudo label generation process using a conditional distribution. The samples drawn from our conditional distribution provide accurate pseudo labels due to the use of semantic class aware unary terms, boundary aware pairwise smoothness terms, and annotation aware higher order terms. Furthermore, we represent the instance segmentation model as an annotation agnostic prediction distribution. In contrast to previous methods, our representation allows us to define a joint probabilistic learning objective that minimizes the dissimilarity between the two distributions. Our approach achieves state of the art results on the PASCAL VOC 2012 data set, outperforming the best baseline by 4.2% [email protected] and 4.8% [email protected]. △ Less

Submitted 18 July, 2020; originally announced July 2020.

Comments: To appear at ECCV 2020

Dissimilarity Coefficient based Weakly Supervised Object Detection

Authors: Aditya Arun, C. V. Jawahar, M. Pawan Kumar

Abstract: We consider the problem of weakly supervised object detection, where the training samples are annotated using only image-level labels that indicate the presence or absence of an object category. In order to model the uncertainty in the location of the objects, we employ a dissimilarity coefficient based probabilistic learning objective. The learning objective minimizes the difference between an an… ▽ More We consider the problem of weakly supervised object detection, where the training samples are annotated using only image-level labels that indicate the presence or absence of an object category. In order to model the uncertainty in the location of the objects, we employ a dissimilarity coefficient based probabilistic learning objective. The learning objective minimizes the difference between an annotation agnostic prediction distribution and an annotation aware conditional distribution. The main computational challenge is the complex nature of the conditional distribution, which consists of terms over hundreds or thousands of variables. The complexity of the conditional distribution rules out the possibility of explicitly modeling it. Instead, we exploit the fact that deep learning frameworks rely on stochastic optimization. This allows us to use a state of the art discrete generative model that can provide annotation consistent samples from the conditional distribution. Extensive experiments on PASCAL VOC 2007 and 2012 data sets demonstrate the efficacy of our proposed approach. △ Less

Submitted 25 November, 2018; originally announced November 2018.

Comments: Preprint

Learning Human Poses from Actions

Authors: Aditya Arun, C. V. Jawahar, M. Pawan Kumar

Abstract: We consider the task of learning to estimate human pose in still images. In order to avoid the high cost of full supervision, we propose to use a diverse data set, which consists of two types of annotations: (i) a small number of images are labeled using the expensive ground-truth pose; and (ii) other images are labeled using the inexpensive action label. As action information helps narrow down th… ▽ More We consider the task of learning to estimate human pose in still images. In order to avoid the high cost of full supervision, we propose to use a diverse data set, which consists of two types of annotations: (i) a small number of images are labeled using the expensive ground-truth pose; and (ii) other images are labeled using the inexpensive action label. As action information helps narrow down the pose of a human, we argue that this approach can help reduce the cost of training without significantly affecting the accuracy. To demonstrate this we design a probabilistic framework that employs two distributions: (i) a conditional distribution to model the uncertainty over the human pose given the image and the action; and (ii) a prediction distribution, which provides the pose of an image without using any action information. We jointly estimate the parameters of the two aforementioned distributions by minimizing their dissimilarity coefficient, as measured by a task-specific loss function. During both training and testing, we only require an efficient sampling strategy for both the aforementioned distributions. This allows us to use deep probabilistic networks that are capable of providing accurate pose estimates for previously unseen images. Using the MPII data set, we show that our approach outperforms baseline methods that either do not use the diverse annotations or rely on pointwise estimates of the pose. △ Less

Submitted 24 July, 2018; originally announced July 2018.

Comments: Accepted at BMVC 2018

Hypertree Decompositions Revisited for PGMs

Authors: Aarthy Shivram Arun, Sai Vikneshwar Mani Jayaraman, Christopher Ré, Atri Rudra

Abstract: We revisit the classical problem of exact inference on probabilistic graphical models (PGMs). Our algorithm is based on recent \emph{worst-case optimal database join} algorithms, which can be asymptotically faster than traditional data processing methods. We present the first empirical evaluation of these algorithms via JoinInfer -- a new exact inference engine. We empirically explore the properti… ▽ More We revisit the classical problem of exact inference on probabilistic graphical models (PGMs). Our algorithm is based on recent \emph{worst-case optimal database join} algorithms, which can be asymptotically faster than traditional data processing methods. We present the first empirical evaluation of these algorithms via JoinInfer -- a new exact inference engine. We empirically explore the properties of the data for which our engine can be expected to outperform traditional inference engines, refining current theoretical notions. Further, JoinInfer outperforms existing state-of-the-art inference engines (ACE, IJGP and libDAI) on some standard benchmark datasets by up to a factor of 630x. Finally, we propose a promising data-driven heuristic that extends JoinInfer to automatically tailor its parameters and/or switch to the traditional inference algorithms. △ Less

Submitted 2 July, 2018; originally announced July 2018.

Comments: Accepted for StarAI Proceedings. Camera Ready Version of arXiv:1804.01640

Hypertree Decompositions Revisited for PGMs

Authors: Aarthy Shivram Arun, Sai Vikneshwar Mani Jayaraman, Christopher Ré, Atri Rudra

Abstract: We revisit the classical problem of exact inference on probabilistic graphical models (PGMs). Our algorithm is based on recent worst-case optimal database join algorithms, which can be asymptotically faster than traditional data processing methods. We present the first empirical evaluation of these new algorithms via JoinInfer, a new exact inference engine. We empirically explore the properties of… ▽ More We revisit the classical problem of exact inference on probabilistic graphical models (PGMs). Our algorithm is based on recent worst-case optimal database join algorithms, which can be asymptotically faster than traditional data processing methods. We present the first empirical evaluation of these new algorithms via JoinInfer, a new exact inference engine. We empirically explore the properties of the data for which our engine can be expected to outperform traditional inference engines refining current theoretical notions. Further, JoinInfer outperforms existing state-of-the-art inference engines (ACE, IJGP and libDAI) on some standard benchmark datasets by up to a factor of 630x. Finally, we propose a promising data-driven heuristic that extends JoinInfer to automatically tailor its parameters and/or switch to the traditional inference algorithms. △ Less

Submitted 4 April, 2018; originally announced April 2018.