Showing 1–14 of 14 results for author: Azad, A P

KITLM: Domain-Specific Knowledge InTegration into Language Models for Question Answering

Authors: Ankush Agarwal, Sakharam Gawade, Amar Prakash Azad, Pushpak Bhattacharyya

Abstract: Large language models (LLMs) have demonstrated remarkable performance in a wide range of natural language tasks. However, as these models continue to grow in size, they face significant challenges in terms of computational costs. Additionally, LLMs often lack efficient domain-specific understanding, which is particularly crucial in specialized fields such as aviation and healthcare. To boost the d… ▽ More Large language models (LLMs) have demonstrated remarkable performance in a wide range of natural language tasks. However, as these models continue to grow in size, they face significant challenges in terms of computational costs. Additionally, LLMs often lack efficient domain-specific understanding, which is particularly crucial in specialized fields such as aviation and healthcare. To boost the domain-specific understanding, we propose, KITLM, a novel knowledge base integration approach into language model through relevant information infusion. By integrating pertinent knowledge, not only the performance of the language model is greatly enhanced, but the model size requirement is also significantly reduced while achieving comparable performance. Our proposed knowledge-infused model surpasses the performance of both GPT-3.5-turbo and the state-of-the-art knowledge infusion method, SKILL, achieving over 1.5 times improvement in exact match scores on the MetaQA. KITLM showed a similar performance boost in the aviation domain with AeroQA. The drastic performance improvement of KITLM over the existing methods can be attributed to the infusion of relevant knowledge while mitigating noise. In addition, we release two curated datasets to accelerate knowledge infusion research in specialized fields: a) AeroQA, a new benchmark dataset designed for multi-hop question-answering within the aviation domain, and b) Aviation Corpus, a dataset constructed from unstructured text extracted from the National Transportation Safety Board reports. Our research contributes to advancing the field of domain-specific language understanding and showcases the potential of knowledge infusion techniques in improving the performance of language models on question-answering. △ Less

Submitted 7 August, 2023; originally announced August 2023.

Let the CAT out of the bag: Contrastive Attributed explanations for Text

Authors: Saneem Chemmengath, Amar Prakash Azad, Ronny Luss, Amit Dhurandhar

Abstract: Contrastive explanations for understanding the behavior of black box models has gained a lot of attention recently as they provide potential for recourse. In this paper, we propose a method Contrastive Attributed explanations for Text (CAT) which provides contrastive explanations for natural language text data with a novel twist as we build and exploit attribute classifiers leading to more semanti… ▽ More Contrastive explanations for understanding the behavior of black box models has gained a lot of attention recently as they provide potential for recourse. In this paper, we propose a method Contrastive Attributed explanations for Text (CAT) which provides contrastive explanations for natural language text data with a novel twist as we build and exploit attribute classifiers leading to more semantically meaningful explanations. To ensure that our contrastive generated text has the fewest possible edits with respect to the original text, while also being fluent and close to a human generated contrastive, we resort to a minimal perturbation approach regularized using a BERT language model and attribute classifiers trained on available attributes. We show through qualitative examples and a user study that our method not only conveys more insight because of these attributes, but also leads to better quality (contrastive) text. Quantitatively, we show that our method outperforms other state-of-the-art methods across four data sets on four benchmark metrics. △ Less

Submitted 1 November, 2022; v1 submitted 16 September, 2021; originally announced September 2021.

Journal ref: EMNLP 2022

Picking Pearl From Seabed: Extracting Artefacts from Noisy Issue Triaging Collaborative Conversations for Hybrid Cloud Services

Authors: Amar Prakash Azad, Supriyo Ghosh, Ajay Gupta, Harshit Kumar, Prateeti Mohapatra

Abstract: Site Reliability Engineers (SREs) play a key role in issue identification and resolution. After an issue is reported, SREs come together in a virtual room (collaboration platform) to triage the issue. While doing so, they leave behind a wealth of information which can be used later for triaging similar issues. However, usability of the conversations offer challenges due to them being i) noisy and… ▽ More Site Reliability Engineers (SREs) play a key role in issue identification and resolution. After an issue is reported, SREs come together in a virtual room (collaboration platform) to triage the issue. While doing so, they leave behind a wealth of information which can be used later for triaging similar issues. However, usability of the conversations offer challenges due to them being i) noisy and ii) unlabelled. This paper presents a novel approach for issue artefact extraction from the noisy conversations with minimal labelled data. We propose a combination of unsupervised and supervised model with minimum human intervention that leverages domain knowledge to predict artefacts for a small amount of conversation data and use that for fine-tuning an already pretrained language model for artefact prediction on a large amount of conversation data. Experimental results on our dataset show that the proposed ensemble of unsupervised and supervised model is better than using either one of them individually. △ Less

Submitted 31 May, 2021; originally announced May 2021.

No Rumours Please! A Multi-Indic-Lingual Approach for COVID Fake-Tweet Detection

Authors: Debanjana Kar, Mohit Bhardwaj, Suranjana Samanta, Amar Prakash Azad

Abstract: The sudden widespread menace created by the present global pandemic COVID-19 has had an unprecedented effect on our lives. Man-kind is going through humongous fear and dependence on social media like never before. Fear inevitably leads to panic, speculations, and the spread of misinformation. Many governments have taken measures to curb the spread of such misinformation for public well being. Besi… ▽ More The sudden widespread menace created by the present global pandemic COVID-19 has had an unprecedented effect on our lives. Man-kind is going through humongous fear and dependence on social media like never before. Fear inevitably leads to panic, speculations, and the spread of misinformation. Many governments have taken measures to curb the spread of such misinformation for public well being. Besides global measures, to have effective outreach, systems for demographically local languages have an important role to play in this effort. Towards this, we propose an approach to detect fake news about COVID-19 early on from social media, such as tweets, for multiple Indic-Languages besides English. In addition, we also create an annotated dataset of Hindi and Bengali tweet for fake news detection. We propose a BERT based model augmented with additional relevant features extracted from Twitter to identify fake tweets. To expand our approach to multiple Indic languages, we resort to mBERT based model which is fine-tuned over created dataset in Hindi and Bengali. We also propose a zero-shot learning approach to alleviate the data scarcity issue for such low resource languages. Through rigorous experiments, we show that our approach reaches around 89% F-Score in fake tweet detection which supercedes the state-of-the-art (SOTA) results. Moreover, we establish the first benchmark for two Indic-Languages, Hindi and Bengali. Using our annotated data, our model achieves about 79% F-Score in Hindi and 81% F-Score for Bengali Tweets. Our zero-shot model achieves about 81% F-Score in Hindi and 78% F-Score for Bengali Tweets without any annotated data, which clearly indicates the efficacy of our approach. △ Less

Submitted 14 October, 2020; originally announced October 2020.

Comments: 6 pages, 4 figures

Meta-Context Transformers for Domain-Specific Response Generation

Authors: Debanjana Kar, Suranjana Samanta, Amar Prakash Azad

Abstract: Despite the tremendous success of neural dialogue models in recent years, it suffers a lack of relevance, diversity, and some times coherence in generated responses. Lately, transformer-based models, such as GPT-2, have revolutionized the landscape of dialogue generation by capturing the long-range structures through language modeling. Though these models have exhibited excellent language coherenc… ▽ More Despite the tremendous success of neural dialogue models in recent years, it suffers a lack of relevance, diversity, and some times coherence in generated responses. Lately, transformer-based models, such as GPT-2, have revolutionized the landscape of dialogue generation by capturing the long-range structures through language modeling. Though these models have exhibited excellent language coherence, they often lack relevance and terms when used for domain-specific response generation. In this paper, we present DSRNet (Domain Specific Response Network), a transformer-based model for dialogue response generation by reinforcing domain-specific attributes. In particular, we extract meta attributes from context and infuse them with the context utterances for better attention over domain-specific key terms and relevance. We study the use of DSRNet in a multi-turn multi-interlocutor environment for domain-specific response generation. In our experiments, we evaluate DSRNet on Ubuntu dialogue datasets, which are mainly composed of various technical domain related dialogues for IT domain issue resolutions and also on CamRest676 dataset, which contains restaurant domain conversations. Trained with maximum likelihood objective, our model shows significant improvement over the state-of-the-art for multi-turn dialogue systems supported by better BLEU and semantic similarity (BertScore) scores. Besides, we also observe that the responses produced by our model carry higher relevance due to the presence of domain-specific key attributes that exhibit better overlap with the attributes of the context. Our analysis shows that the performance improvement is mostly due to the infusion of key terms along with dialogues which result in better attention over domain-relevant terms. Other contributing factors include joint modeling of dialogue context with the domain-specific meta attributes and topics. △ Less

Submitted 12 October, 2020; originally announced October 2020.

Comments: 7+2 pages, 6 figures, 4 tables

Carbon to Diamond: An Incident Remediation Assistant System From Site Reliability Engineers' Conversations in Hybrid Cloud Operations

Authors: Suranjana Samanta, Ajay Gupta, Prateeti Mohapatra, Amar Prakash Azad

Abstract: Conversational channels are changing the landscape of hybrid cloud service management. These channels are becoming important avenues for Site Reliability Engineers (SREs) %Subject Matter Experts (SME) to collaboratively work together to resolve an incident or issue. Identifying segmented conversations and extracting key insights or artefacts from them can help engineers to improve the efficiency o… ▽ More Conversational channels are changing the landscape of hybrid cloud service management. These channels are becoming important avenues for Site Reliability Engineers (SREs) %Subject Matter Experts (SME) to collaboratively work together to resolve an incident or issue. Identifying segmented conversations and extracting key insights or artefacts from them can help engineers to improve the efficiency of the incident remediation process by using information retrieval mechanisms for similar incidents. However, it has been empirically observed that due to the semi-formal behavior of such conversations (human language) they are very unique in nature and also contain lot of domain-specific terms. This makes it difficult to use the standard natural language processing frameworks directly, which are popularly used in standard NLP tasks. %It is important to identify the correct keywords and artefacts like symptoms, issue etc., present in the conversation chats. In this paper, we build a framework that taps into the conversational channels and uses various learning methods to (a) understand and extract key artefacts from conversations like diagnostic steps and resolution actions taken, and (b) present an approach to identify past conversations about similar issues. Experimental results on our dataset show the efficacy of our proposed method. △ Less

Submitted 12 October, 2020; originally announced October 2020.

Comments: 6 Pages, 5 figures, 2 tables

Collective Learning From Diverse Datasets for Entity Typing in the Wild

Authors: Abhishek Abhishek, Amar Prakash Azad, Balaji Ganesan, Ashish Anand, Amit Awekar

Abstract: Entity typing (ET) is the problem of assigning labels to given entity mentions in a sentence. Existing works for ET require knowledge about the domain and target label set for a given test instance. ET in the absence of such knowledge is a novel problem that we address as ET in the wild. We hypothesize that the solution to this problem is to build supervised models that generalize better on the ET… ▽ More Entity typing (ET) is the problem of assigning labels to given entity mentions in a sentence. Existing works for ET require knowledge about the domain and target label set for a given test instance. ET in the absence of such knowledge is a novel problem that we address as ET in the wild. We hypothesize that the solution to this problem is to build supervised models that generalize better on the ET task as a whole, rather than a specific dataset. In this direction, we propose a Collective Learning Framework (CLF), which enables learning from diverse datasets in a unified way. The CLF first creates a unified hierarchical label set (UHLS) and a label mapping by aggregating label information from all available datasets. Then it builds a single neural network classifier using UHLS, label mapping, and a partial loss function. The single classifier predicts the finest possible label across all available domains even though these labels may not be present in any domain-specific dataset. We also propose a set of evaluation schemes and metrics to evaluate the performance of models in this novel problem. Extensive experimentation on seven diverse real-world datasets demonstrates the efficacy of our CLF. △ Less

Submitted 16 September, 2019; v1 submitted 20 October, 2018; originally announced October 2018.

Comments: Accepted at EYRE'19 Workshop, CIKM 2019

Deep Domain Adaptation under Deep Label Scarcity

Authors: Amar Prakash Azad, Dinesh Garg, Priyanka Agrawal, Arun Kumar

Abstract: The goal behind Domain Adaptation (DA) is to leverage the labeled examples from a source domain so as to infer an accurate model in a target domain where labels are not available or in scarce at the best. A state-of-the-art approach for the DA is due to (Ganin et al. 2016), known as DANN, where they attempt to induce a common representation of source and target domains via adversarial training. Th… ▽ More The goal behind Domain Adaptation (DA) is to leverage the labeled examples from a source domain so as to infer an accurate model in a target domain where labels are not available or in scarce at the best. A state-of-the-art approach for the DA is due to (Ganin et al. 2016), known as DANN, where they attempt to induce a common representation of source and target domains via adversarial training. This approach requires a large number of labeled examples from the source domain to be able to infer a good model for the target domain. However, in many situations obtaining labels in the source domain is expensive which results in deteriorated performance of DANN and limits its applicability in such scenarios. In this paper, we propose a novel approach to overcome this limitation. In our work, we first establish that DANN reduces the original DA problem into a semi-supervised learning problem over the space of common representation. Next, we propose a learning approach, namely TransDANN, that amalgamates adversarial learning and transductive learning to mitigate the detrimental impact of limited source labels and yields improved performance. Experimental results (both on text and images) show a significant boost in the performance of TransDANN over DANN under such scenarios. We also provide theoretical justification for the performance boost. △ Less

Submitted 20 September, 2018; originally announced September 2018.

Unsupervised Controllable Text Formalization

Authors: Parag Jain, Abhijit Mishra, Amar Prakash Azad, Karthik Sankaranarayanan

Abstract: We propose a novel framework for controllable natural language transformation. Realizing that the requirement of parallel corpus is practically unsustainable for controllable generation tasks, an unsupervised training scheme is introduced. The crux of the framework is a deep neural encoder-decoder that is reinforced with text-transformation knowledge through auxiliary modules (called scorers). The… ▽ More We propose a novel framework for controllable natural language transformation. Realizing that the requirement of parallel corpus is practically unsustainable for controllable generation tasks, an unsupervised training scheme is introduced. The crux of the framework is a deep neural encoder-decoder that is reinforced with text-transformation knowledge through auxiliary modules (called scorers). The scorers, based on off-the-shelf language processing tools, decide the learning scheme of the encoder-decoder based on its actions. We apply this framework for the text-transformation task of formalizing an input text by improving its readability grade; the degree of required formalization can be controlled by the user at run-time. Experiments on public datasets demonstrate the efficacy of our model towards: (a) transforming a given text to a more formal style, and (b) introducing appropriate amount of formalness in the output text pertaining to the input control. Our code and datasets are released for academic use. △ Less

Submitted 20 February, 2019; v1 submitted 10 September, 2018; originally announced September 2018.

Comments: AAAI

DeepSolarEye: Power Loss Prediction and Weakly Supervised Soiling Localization via Fully Convolutional Networks for Solar Panels

Authors: Sachin Mehta, Amar P. Azad, Saneem A. Chemmengath, Vikas Raykar, Shivkumar Kalyanaraman

Abstract: The impact of soiling on solar panels is an important and well-studied problem in renewable energy sector. In this paper, we present the first convolutional neural network (CNN) based approach for solar panel soiling and defect analysis. Our approach takes an RGB image of solar panel and environmental factors as inputs to predict power loss, soiling localization, and soiling type. In computer visi… ▽ More The impact of soiling on solar panels is an important and well-studied problem in renewable energy sector. In this paper, we present the first convolutional neural network (CNN) based approach for solar panel soiling and defect analysis. Our approach takes an RGB image of solar panel and environmental factors as inputs to predict power loss, soiling localization, and soiling type. In computer vision, localization is a complex task which typically requires manually labeled training data such as bounding boxes or segmentation masks. Our proposed approach consists of specialized four stages which completely avoids localization ground truth and only needs panel images with power loss labels for training. The region of impact area obtained from the predicted localization masks are classified into soiling types using the webly supervised learning. For improving localization capabilities of CNNs, we introduce a novel bi-directional input-aware fusion (BiDIAF) block that reinforces the input at different levels of CNN to learn input-specific feature maps. Our empirical study shows that BiDIAF improves the power loss prediction accuracy by about 3% and localization accuracy by about 4%. Our end-to-end model yields further improvement of about 24% on localization when learned in a weakly supervised manner. Our approach is generalizable and showed promising results on web crawled solar panel images. Our system has a frame rate of 22 fps (including all steps) on a NVIDIA TitanX GPU. Additionally, we collected first of it's kind dataset for solar panel image analysis consisting 45,000+ images. △ Less

Submitted 18 March, 2018; v1 submitted 10 October, 2017; originally announced October 2017.

Comments: Accepted for publication at WACV 2018

Unilateral Altruism in Network Routing Games with Atomic Players

Authors: Amar Prakash Azad, John Musacchio

Abstract: We study a routing game in which one of the players unilaterally acts altruistically by taking into consideration the latency cost of other players as well as his own. By not playing selfishly, a player can not only improve the other players' equilibrium utility but also improve his own equilibrium utility. To quantify the effect, we define a metric called the Value of Unilateral Altruism (VoU) to… ▽ More We study a routing game in which one of the players unilaterally acts altruistically by taking into consideration the latency cost of other players as well as his own. By not playing selfishly, a player can not only improve the other players' equilibrium utility but also improve his own equilibrium utility. To quantify the effect, we define a metric called the Value of Unilateral Altruism (VoU) to be the ratio of the equilibrium utility of the altruistic user to the equilibrium utility he would have received in Nash equilibrium if he were selfish. We show by example that the VoU, in a game with nonlinear latency functions and atomic players, can be arbitrarily large. Since the Nash equilibrium social welfare of this example is arbitrarily far from social optimum, this example also has a Price of Anarchy (PoA) that is unbounded. The example is driven by there being a small number of players since the same example with non-atomic players yields a Nash equilibrium that is fully efficient. △ Less

Submitted 12 August, 2011; v1 submitted 4 August, 2011; originally announced August 2011.

Sleep Mode Analysis via Workload Decomposition

Authors: Amar Prakash Azad

Abstract: The goal of this paper is to establish a general approach for analyzing queueing models with repeated inhomogeneous vacations. The server goes on for a vacation if the inactivity prolongs more than the vacation trigger duration. Once the system enters in vacation mode, it may continue for several consecutive vacations. At the end of a vacation, the server goes on another vacation, possibly with a… ▽ More The goal of this paper is to establish a general approach for analyzing queueing models with repeated inhomogeneous vacations. The server goes on for a vacation if the inactivity prolongs more than the vacation trigger duration. Once the system enters in vacation mode, it may continue for several consecutive vacations. At the end of a vacation, the server goes on another vacation, possibly with a different probability distribution; if during the previous vacation there have been no arrivals. However the system enters in vacation mode only if the inactivity is persisted beyond defined trigger duration. In order to get an insight on the influence of parameters on the performance, we choose to study a simple M/G/1 queue (Poisson arrivals and general independent service times) which has the advantage of being tractable analytically. The theoretical model is applied to the problem of power saving for mobile devices in which the sleep durations of a device correspond to the vacations of the server. Various system performance metrics such as the frame response time and the economy of energy are derived. A constrained optimization problem is formulated to maximize the economy of energy achieved in power save mode, with constraints as QoS conditions to be met. An illustration of the proposed methods is shown with a WiMAX system scenario to obtain design parameters for better performance. Our analysis allows us not only to optimize the system parameters for a given traffic intensity but also to propose parameters that provide the best performance under worst case conditions. △ Less

Submitted 9 June, 2011; originally announced June 2011.

Stable Nash equilibria of medium access games under symmetric, socially altruistic behavior

Authors: G. Kesidis, Y. Jin, A. P. Azad, E. Altman

Abstract: We consider the effects of altruistic behavior on random medium access control (slotted ALOHA) for local area communication networks. For an idealized, synchronously iterative, two-player game with asymmetric player demands, we find a Hamiltonian governing the Jacobi dynamics under purely altruistic behavior. Though the positions of the interior Nash equilibrium points do not change in the presenc… ▽ More We consider the effects of altruistic behavior on random medium access control (slotted ALOHA) for local area communication networks. For an idealized, synchronously iterative, two-player game with asymmetric player demands, we find a Hamiltonian governing the Jacobi dynamics under purely altruistic behavior. Though the positions of the interior Nash equilibrium points do not change in the presence of altruistic behavior, the nature of their local asymptotic stability does. There is a region of partially altruistic behavior for which neither interior Nash equilibrium point is locally asymptotically stable. Also, for a power control game with a single Nash equilibrium, we show how its stability changes as a function of the altruism parameter. Variations of these altruistic game frameworks are discussed considering power (instead of throughput) based costs and linear utility functions. △ Less

Submitted 24 August, 2010; v1 submitted 27 March, 2010; originally announced March 2010.

From Altruism to Non-Cooperation in Routing Games

Authors: Amar Prakash Azad, Eitan Altman, R. El-Azouzi

Abstract: The paper studies the routing in the network shared by several users. Each user seeks to optimize either its own performance or some combination between its own performance and that of other users, by controlling the routing of its given flow demand. We parameterize the degree of cooperation which allows to cover the fully non-cooperative behavior, the fully cooperative behavior, and even more,… ▽ More The paper studies the routing in the network shared by several users. Each user seeks to optimize either its own performance or some combination between its own performance and that of other users, by controlling the routing of its given flow demand. We parameterize the degree of cooperation which allows to cover the fully non-cooperative behavior, the fully cooperative behavior, and even more, the fully altruistic behavior, all these as special cases of the parameter's choice. A large part of the work consists in exploring the impact of the degree of cooperation on the equilibrium. Our first finding is to identify multiple Nash equilibria with cooperative behavior that do not occur in the non-cooperative case under the same conditions (cost, demand and topology). We then identify Braess like paradox (in which adding capacity or adding a link to a network results in worse performance to all users) and study the impact of the degree of cooperation on it. We identify another type of paradox in cooperation scenario. We identify that when we increase the degree of cooperation of a user while other users keep unchanged their degree of cooperation, leads to an improvement in performance of that user. We then pursue the exploration and carry it on to the setting of Mixed equilibrium (i.e. some users are non atomic-they have infinitesimally small demand, and other have finite fixed demand). We finally obtain some theoretical results that show that for low degree of cooperation the equilibrium is unique, confirming the results of our numerical study. △ Less

Submitted 14 October, 2008; v1 submitted 29 August, 2008; originally announced August 2008.