-
SSP: Self-Supervised Prompting for Cross-Lingual Transfer to Low-Resource Languages using Large Language Models
Authors:
Vipul Rathore,
Aniruddha Deb,
Ankish Chandresh,
Parag Singla,
Mausam
Abstract:
Recently, very large language models (LLMs) have shown exceptional performance on several English NLP tasks with just in-context learning (ICL), but their utility in other languages is still underexplored. We investigate their effectiveness for NLP tasks in low-resource languages (LRLs), especially in the setting of zero-labelled cross-lingual transfer (0-CLT), where no labelled training data for…
▽ More
Recently, very large language models (LLMs) have shown exceptional performance on several English NLP tasks with just in-context learning (ICL), but their utility in other languages is still underexplored. We investigate their effectiveness for NLP tasks in low-resource languages (LRLs), especially in the setting of zero-labelled cross-lingual transfer (0-CLT), where no labelled training data for the target language is available -- however training data from one or more related medium-resource languages (MRLs) is utilized, alongside the available unlabeled test data for a target language. We introduce Self-Supervised Prompting (SSP), a novel ICL approach tailored for the 0-CLT setting.
SSP is based on the key observation that LLMs output more accurate labels if in-context exemplars are from the target language (even if their labels are slightly noisy). To operationalize this, since target language training data is not available in 0-CLT, SSP operates in two stages. In Stage I, using source MRL training data, target language's test data is noisily labeled. In Stage II, these noisy test data points are used as exemplars in ICL for further improved labelling. Additionally, our implementation of SSP uses a novel Integer Linear Programming (ILP)-based exemplar selection that balances similarity, prediction confidence (when available) and label coverage. Experiments on three tasks and eleven LRLs (from three regions) demonstrate that SSP strongly outperforms existing SOTA fine-tuned and prompting-based baselines in 0-CLT setup.
△ Less
Submitted 27 June, 2024;
originally announced June 2024.
-
Fill in the Blank: Exploring and Enhancing LLM Capabilities for Backward Reasoning in Math Word Problems
Authors:
Aniruddha Deb,
Neeva Oza,
Sarthak Singla,
Dinesh Khandelwal,
Dinesh Garg,
Parag Singla
Abstract:
While forward reasoning (i.e., find the answer given the question) has been explored extensively in recent literature, backward reasoning is relatively unexplored. We examine the backward reasoning capabilities of LLMs on Math Word Problems (MWPs): given a mathematical question and its answer, with some details omitted from the question, can LLMs effectively retrieve the missing information? On mo…
▽ More
While forward reasoning (i.e., find the answer given the question) has been explored extensively in recent literature, backward reasoning is relatively unexplored. We examine the backward reasoning capabilities of LLMs on Math Word Problems (MWPs): given a mathematical question and its answer, with some details omitted from the question, can LLMs effectively retrieve the missing information? On modifying three benchmark datasets for this task, to evaluate this task: GSM8k, SVAMP, and MultiArith, we find a significant drop in the accuracy of models on this task compared to forward reasoning across SOTA LLMs (GPT4, GPT3.5, PaLM-2, and LLaMa). Motivated by the fact backward reasoning can be seen as the ''inverse'' of forward reasoning, we propose variations of three different forward reasoning strategies to improve performance. Rephrase reformulates the given problem into a forward reasoning problem, PAL-Tools combines the idea of Program-Aided LLMs to produce a set of equations that can be solved by an external solver, and Check your Work exploits the availability of natural verifier of high accuracy in the forward direction, interleaving solving and verification steps. Finally, realizing that each of our base methods correctly solves a different set of problems, we propose a novel Bayesian formulation for creating an ensemble over the base methods to further boost the accuracy. Extensive experimentation demonstrates successive improvement in the performance of LLMs on the backward reasoning task, using our strategies, with our ensemble-based method resulting in significant performance gains compared to the SOTA forward reasoning strategies we adapt.
△ Less
Submitted 7 July, 2024; v1 submitted 3 October, 2023;
originally announced October 2023.
-
BeAts: Bengali Speech Acts Recognition using Multimodal Attention Fusion
Authors:
Ahana Deb,
Sayan Nag,
Ayan Mahapatra,
Soumitri Chattopadhyay,
Aritra Marik,
Pijush Kanti Gayen,
Shankha Sanyal,
Archi Banerjee,
Samir Karmakar
Abstract:
Spoken languages often utilise intonation, rhythm, intensity, and structure, to communicate intention, which can be interpreted differently depending on the rhythm of speech of their utterance. These speech acts provide the foundation of communication and are unique in expression to the language. Recent advancements in attention-based models, demonstrating their ability to learn powerful represent…
▽ More
Spoken languages often utilise intonation, rhythm, intensity, and structure, to communicate intention, which can be interpreted differently depending on the rhythm of speech of their utterance. These speech acts provide the foundation of communication and are unique in expression to the language. Recent advancements in attention-based models, demonstrating their ability to learn powerful representations from multilingual datasets, have performed well in speech tasks and are ideal to model specific tasks in low resource languages. Here, we develop a novel multimodal approach combining two models, wav2vec2.0 for audio and MarianMT for text translation, by using multimodal attention fusion to predict speech acts in our prepared Bengali speech corpus. We also show that our model BeAts ($\underline{\textbf{Be}}$ngali speech acts recognition using Multimodal $\underline{\textbf{At}}$tention Fu$\underline{\textbf{s}}$ion) significantly outperforms both the unimodal baseline using only speech data and a simpler bimodal fusion using both speech and text data. Project page: https://soumitri2001.github.io/BeAts
△ Less
Submitted 5 June, 2023;
originally announced June 2023.
-
Detecting Social Media Manipulation in Low-Resource Languages
Authors:
Samar Haider,
Luca Luceri,
Ashok Deb,
Adam Badawy,
Nanyun Peng,
Emilio Ferrara
Abstract:
Social media have been deliberately used for malicious purposes, including political manipulation and disinformation. Most research focuses on high-resource languages. However, malicious actors share content across countries and languages, including low-resource ones. Here, we investigate whether and to what extent malicious actors can be detected in low-resource language settings. We discovered t…
▽ More
Social media have been deliberately used for malicious purposes, including political manipulation and disinformation. Most research focuses on high-resource languages. However, malicious actors share content across countries and languages, including low-resource ones. Here, we investigate whether and to what extent malicious actors can be detected in low-resource language settings. We discovered that a high number of accounts posting in Tagalog were suspended as part of Twitter's crackdown on interference operations after the 2016 US Presidential election. By combining text embedding and transfer learning, our framework can detect, with promising accuracy, malicious users posting in Tagalog without any prior knowledge or training on malicious content in that language. We first learn an embedding model for each language, namely a high-resource language (English) and a low-resource one (Tagalog), independently. Then, we learn a mapping between the two latent spaces to transfer the detection model. We demonstrate that the proposed approach significantly outperforms state-of-the-art models, including BERT, and yields marked advantages in settings with very limited training data -- the norm when dealing with detecting malicious activity in online platforms.
△ Less
Submitted 19 February, 2023; v1 submitted 10 November, 2020;
originally announced November 2020.
-
#Election2020: The First Public Twitter Dataset on the 2020 US Presidential Election
Authors:
Emily Chen,
Ashok Deb,
Emilio Ferrara
Abstract:
The integrity of democratic political discourse is at the core to guarantee free and fair elections. With social media often dictating the tones and trends of politics-related discussion, it is of paramount important to be able to study online chatter, especially in the run up to important voting events, like in the case of the upcoming November 3, 2020 U.S. Presidential Election. Limited access t…
▽ More
The integrity of democratic political discourse is at the core to guarantee free and fair elections. With social media often dictating the tones and trends of politics-related discussion, it is of paramount important to be able to study online chatter, especially in the run up to important voting events, like in the case of the upcoming November 3, 2020 U.S. Presidential Election. Limited access to social media data is often the first barrier to impede, hinder, or slow down progress, and ultimately our understanding of online political discourse. To mitigate this issue and try to empower the Computational Social Science research community, we decided to publicly release a massive-scale, longitudinal dataset of U.S. politics- and election-related tweets. This multilingual dataset that we have been collecting for over one year encompasses hundreds of millions of tweets and tracks all salient U.S. politics trends, actors, and events between 2019 and 2020. It predates and spans the whole period of Republican and Democratic primaries, with real-time tracking of all presidential contenders of both sides of the isle. After that, it focuses on presidential and vice-presidential candidates. Our dataset release is curated, documented and will be constantly updated on a weekly-basis, until the November 3, 2020 election and beyond. We hope that the academic community, computational journalists, and research practitioners alike will all take advantage of our dataset to study relevant scientific and social issues, including problems like misinformation, information manipulation, interference, and distortion of online political discourse that have been prevalent in the context of recent election events in the United States and worldwide.
Our dataset is available at: https://github.com/echen102/us-pres-elections-2020
△ Less
Submitted 1 October, 2020;
originally announced October 2020.
-
Design of a sensor integrated adaptive gripper
Authors:
IA Sainul,
Sankha Deb,
AK Deb
Abstract:
In this paper, design and development of a sensor integrated adaptive gripper is presented. Adaptive grippers are useful for grasping objects of varied geometric shapes by wrapping fingers around the object. The finger closing sequence in adaptive grippers may lead to ejection of the object from the gripper due to any unbalanced grasping force and such grasp failure is common for lightweight objec…
▽ More
In this paper, design and development of a sensor integrated adaptive gripper is presented. Adaptive grippers are useful for grasping objects of varied geometric shapes by wrapping fingers around the object. The finger closing sequence in adaptive grippers may lead to ejection of the object from the gripper due to any unbalanced grasping force and such grasp failure is common for lightweight objects. Designing of the proposed gripper is focused on ensuring stable grasp on a wide variety of objects, especially, for lightweight objects (e.g., empty plastic bottles). The actuation mechanism is based on movable pulleys and tendon wires which ensure that once a link stops moving, the other links continue to move and wrap around the object. Further, optimisation is used to improve the design of the adaptive gripper. Finally, validation is done by executing object grasping in a simulated environment PyBullet for simple pick and place tasks on common household objects.
△ Less
Submitted 26 August, 2020;
originally announced August 2020.
-
Automatic pre-grasps generation for unknown 3D objects
Authors:
IA Sainul,
Sankha Deb,
AK Deb
Abstract:
In this paper, the problem of automating the pre-grasps generation for novel 3d objects has been discussed. The objects represented as cloud of 3D points are split into parts and organized in a tree structure, where parts are approximated by simple box primitives. Applying grasping only on the individual object parts may miss a good grasp which involves a combination of parts. The problem has been…
▽ More
In this paper, the problem of automating the pre-grasps generation for novel 3d objects has been discussed. The objects represented as cloud of 3D points are split into parts and organized in a tree structure, where parts are approximated by simple box primitives. Applying grasping only on the individual object parts may miss a good grasp which involves a combination of parts. The problem has been addressed by traversing the decomposition tree and checking each node of the tree for possible pre-grasps against a set of conditions. Further, a face mask has been introduced to encode the free and blocked faces of the box primitives. Pre-grasps are generated only for the free faces. Finally, the proposed method implemented on a set twenty-four household objects and toys, where a grasp planner based on object slicing method has been used to compute the contact-level grasp plan.
△ Less
Submitted 30 September, 2019; v1 submitted 1 August, 2019;
originally announced August 2019.
-
A novel object slicing based grasp planner for 3D object grasping using underactuated robot gripper
Authors:
IA Sainul,
Sankha Deb,
AK Deb
Abstract:
Robotic grasping of arbitrary objects even in completely known environments still remains a challenging problem. Most previously developed algorithms had focused on fingertip grasp, failing to solve the problem even for fully actuated hands/grippers during adaptive/wrapping type of grasps, where each finger makes contact with object at several points. Kinematic closed form solutions are not possib…
▽ More
Robotic grasping of arbitrary objects even in completely known environments still remains a challenging problem. Most previously developed algorithms had focused on fingertip grasp, failing to solve the problem even for fully actuated hands/grippers during adaptive/wrapping type of grasps, where each finger makes contact with object at several points. Kinematic closed form solutions are not possible for such an articulated finger which simultaneously reaches several given goal points. This paper, presents a framework for computing best grasp for an underactuated robotic gripper, based on a novel object slicing method. The proposed method quickly find contacts using an object slicing technique and use grasp quality measure to find the best grasp from a pool of grasps. To validate the proposed method, implementation has been done on twenty-four household objects and toys using a two finger underactuated robot gripper. Unlike the many other existing approaches, the proposed approach has several advantages: it can handle objects with complex shapes and sizes; it does not require simplifying the objects into primitive geometric shape; Most importantly, it can be applied on point clouds taken using depth sensor; it takes into account gripper kinematic constraints and generates feasible grasps for both adaptive/enveloping and fingertip types of grasps.
△ Less
Submitted 22 July, 2019;
originally announced July 2019.
-
Red Bots Do It Better: Comparative Analysis of Social Bot Partisan Behavior
Authors:
Luca Luceri,
Ashok Deb,
Adam Badawy,
Emilio Ferrara
Abstract:
Recent research brought awareness of the issue of bots on social media and the significant risks of mass manipulation of public opinion in the context of political discussion. In this work, we leverage Twitter to study the discourse during the 2018 US midterm elections and analyze social bot activity and interactions with humans. We collected 2.6 million tweets for 42 days around the election day…
▽ More
Recent research brought awareness of the issue of bots on social media and the significant risks of mass manipulation of public opinion in the context of political discussion. In this work, we leverage Twitter to study the discourse during the 2018 US midterm elections and analyze social bot activity and interactions with humans. We collected 2.6 million tweets for 42 days around the election day from nearly 1 million users. We use the collected tweets to answer three research questions: (i) Do social bots lean and behave according to a political ideology? (ii) Can we observe different strategies among liberal and conservative bots? (iii) How effective are bot strategies? We show that social bots can be accurately classified according to their political leaning and behave accordingly. Conservative bots share most of the topics of discussion with their human counterparts, while liberal bots show less overlap and a more inflammatory attitude. We studied bot interactions with humans and observed different strategies. Finally, we measured bots embeddedness in the social network and the effectiveness of their activities. Results show that conservative bots are more deeply embedded in the social network and more effective than liberal bots at exerting influence on humans.
△ Less
Submitted 7 February, 2019; v1 submitted 7 February, 2019;
originally announced February 2019.
-
Perils and Challenges of Social Media and Election Manipulation Analysis: The 2018 US Midterms
Authors:
Ashok Deb,
Luca Luceri,
Adam Badawy,
Emilio Ferrara
Abstract:
One of the hallmarks of a free and fair society is the ability to conduct a peaceful and seamless transfer of power from one leader to another. Democratically, this is measured in a citizen population's trust in the electoral system of choosing a representative government. In view of the well documented issues of the 2016 US Presidential election, we conducted an in-depth analysis of the 2018 US M…
▽ More
One of the hallmarks of a free and fair society is the ability to conduct a peaceful and seamless transfer of power from one leader to another. Democratically, this is measured in a citizen population's trust in the electoral system of choosing a representative government. In view of the well documented issues of the 2016 US Presidential election, we conducted an in-depth analysis of the 2018 US Midterm elections looking specifically for voter fraud or suppression. The Midterm election occurs in the middle of a 4 year presidential term. For the 2018 midterms, 35 senators and all the 435 seats in the House of Representatives were up for re-election, thus, every congressional district and practically every state had a federal election. In order to collect election related tweets, we analyzed Twitter during the month prior to, and the two weeks following, the November 6, 2018 election day. In a targeted analysis to detect statistical anomalies or election interference, we identified several biases that can lead to wrong conclusions. Specifically, we looked for divergence between actual voting outcomes and instances of the #ivoted hashtag on the election day. This analysis highlighted three states of concern: New York, California, and Texas. We repeated our analysis discarding malicious accounts, such as social bots. Upon further inspection and against a backdrop of collected general election-related tweets, we identified some confounding factors, such as population bias, or bot and political ideology inference, that can lead to false conclusions. We conclude by providing an in-depth discussion of the perils and challenges of using social media data to explore questions about election manipulation.
△ Less
Submitted 31 January, 2019;
originally announced February 2019.
-
DigiLock: User-controlled and Server-aware Digital Locker System
Authors:
Atrayee Deb,
Saloni Dalal,
Manik Lal Das
Abstract:
The growing popularity of digital systems have paved the way for digital locker that ensures security and safety of the digital documents in store. While facilitating this system to user and availing its services offered by service provider, non-repudiation of service offered and service consumed is an important security requirement in the digital locker system. In this paper, we present a digital…
▽ More
The growing popularity of digital systems have paved the way for digital locker that ensures security and safety of the digital documents in store. While facilitating this system to user and availing its services offered by service provider, non-repudiation of service offered and service consumed is an important security requirement in the digital locker system. In this paper, we present a digital locker system that addresses the aspect of confidentiality, integrity, and non-repudiation along with other security properties. The proposed protocol ensures the confirmed participation of the user as well as the service provider while accessing the digital locker. The protocol is analyzed against potential threats in the context of safety and security of the digital locker system.
△ Less
Submitted 19 October, 2018;
originally announced October 2018.
-
'Senator, We Sell Ads': Analysis of the 2016 Russian Facebook Ads Campaign
Authors:
Ritam Dutt,
Ashok Deb,
Emilio Ferrara
Abstract:
One of the key aspects of the United States democracy is free and fair elections that allow for a peaceful transfer of power from one President to the next. The 2016 US presidential election stands out due to suspected foreign influence before, during, and after the election. A significant portion of that suspected influence was carried out via social media. In this paper, we look specifically at…
▽ More
One of the key aspects of the United States democracy is free and fair elections that allow for a peaceful transfer of power from one President to the next. The 2016 US presidential election stands out due to suspected foreign influence before, during, and after the election. A significant portion of that suspected influence was carried out via social media. In this paper, we look specifically at 3,500 Facebook ads allegedly purchased by the Russian government. These ads were released on May 10, 2018 by the US Congress House Intelligence Committee. We analyzed the ads using natural language processing techniques to determine textual and semantic features associated with the most effective ones. We clustered the ads over time into the various campaigns and the labeled parties associated with them. We also studied the effectiveness of Ads on an individual, campaign and party basis. The most effective ads tend to have less positive sentiment, focus on past events and are more specific and personalized in nature. The more effective campaigns also show such similar characteristics. The campaigns' duration and promotion of the Ads suggest a desire to sow division rather than sway the election.
△ Less
Submitted 26 September, 2018;
originally announced September 2018.
-
Discovering Signals from Web Sources to Predict Cyber Attacks
Authors:
Palash Goyal,
KSM Tozammel Hossain,
Ashok Deb,
Nazgol Tavabi,
Nathan Bartley,
Andr'es Abeliuk,
Emilio Ferrara,
Kristina Lerman
Abstract:
Cyber attacks are growing in frequency and severity. Over the past year alone we have witnessed massive data breaches that stole personal information of millions of people and wide-scale ransomware attacks that paralyzed critical infrastructure of several countries. Combating the rising cyber threat calls for a multi-pronged strategy, which includes predicting when these attacks will occur. The in…
▽ More
Cyber attacks are growing in frequency and severity. Over the past year alone we have witnessed massive data breaches that stole personal information of millions of people and wide-scale ransomware attacks that paralyzed critical infrastructure of several countries. Combating the rising cyber threat calls for a multi-pronged strategy, which includes predicting when these attacks will occur. The intuition driving our approach is this: during the planning and preparation stages, hackers leave digital traces of their activities on both the surface web and dark web in the form of discussions on platforms like hacker forums, social media, blogs and the like. These data provide predictive signals that allow anticipating cyber attacks. In this paper, we describe machine learning techniques based on deep neural networks and autoregressive time series models that leverage external signals from publicly available Web sources to forecast cyber attacks. Performance of our framework across ground truth data over real-world forecasting tasks shows that our methods yield a significant lift or increase of F1 for the top signals on predicted cyber attacks. Our results suggest that, when deployed, our system will be able to provide an effective line of defense against various types of targeted cyber attacks.
△ Less
Submitted 8 June, 2018;
originally announced June 2018.
-
Social Bots for Online Public Health Interventions
Authors:
Ashok Deb,
Anuja Majmundar,
Sungyong Seo,
Akira Matsui,
Rajat Tandon,
Shen Yan,
Jon-Patrick Allem,
Emilio Ferrara
Abstract:
According to the Center for Disease Control and Prevention, in the United States hundreds of thousands initiate smoking each year, and millions live with smoking-related dis- eases. Many tobacco users discuss their habits and preferences on social media. This work conceptualizes a framework for targeted health interventions to inform tobacco users about the consequences of tobacco use. We designed…
▽ More
According to the Center for Disease Control and Prevention, in the United States hundreds of thousands initiate smoking each year, and millions live with smoking-related dis- eases. Many tobacco users discuss their habits and preferences on social media. This work conceptualizes a framework for targeted health interventions to inform tobacco users about the consequences of tobacco use. We designed a Twitter bot named Notobot (short for No-Tobacco Bot) that leverages machine learning to identify users posting pro-tobacco tweets and select individualized interventions to address their interest in tobacco use. We searched the Twitter feed for tobacco-related keywords and phrases, and trained a convolutional neural network using over 4,000 tweets dichotomously manually labeled as either pro- tobacco or not pro-tobacco. This model achieves a 90% recall rate on the training set and 74% on test data. Users posting pro- tobacco tweets are matched with former smokers with similar interests who posted anti-tobacco tweets. Algorithmic matching, based on the power of peer influence, allows for the systematic delivery of personalized interventions based on real anti-tobacco tweets from former smokers. Experimental evaluation suggests that our system would perform well if deployed. This research offers opportunities for public health researchers to increase health awareness at scale. Future work entails deploying the fully operational Notobot system in a controlled experiment within a public health campaign.
△ Less
Submitted 21 April, 2018;
originally announced April 2018.
-
Predicting Cyber Events by Leveraging Hacker Sentiment
Authors:
Ashok Deb,
Kristina Lerman,
Emilio Ferrara
Abstract:
Recent high-profile cyber attacks exemplify why organizations need better cyber defenses. Cyber threats are hard to accurately predict because attackers usually try to mask their traces. However, they often discuss exploits and techniques on hacking forums. The community behavior of the hackers may provide insights into groups' collective malicious activity. We propose a novel approach to predict…
▽ More
Recent high-profile cyber attacks exemplify why organizations need better cyber defenses. Cyber threats are hard to accurately predict because attackers usually try to mask their traces. However, they often discuss exploits and techniques on hacking forums. The community behavior of the hackers may provide insights into groups' collective malicious activity. We propose a novel approach to predict cyber events using sentiment analysis. We test our approach using cyber attack data from 2 major business organizations. We consider 3 types of events: malicious software installation, malicious destination visits, and malicious emails that surpassed the target organizations' defenses. We construct predictive signals by applying sentiment analysis on hacker forum posts to better understand hacker behavior. We analyze over 400K posts generated between January 2016 and January 2018 on over 100 hacking forums both on surface and Dark Web. We find that some forums have significantly more predictive power than others. Sentiment-based models that leverage specific forums can outperform state-of-the-art deep learning and time-series models on forecasting cyber attacks weeks ahead of the events.
△ Less
Submitted 14 April, 2018;
originally announced April 2018.
