Showing 1–28 of 28 results for author: Mathew, B

Modeling the Feedback of AI Price Estimations on Actual Market Values

Authors: Viorel Silaghi, Zobaida Alssadi, Ben Mathew, Majed Alotaibi, Ali Alqarni, Marius Silaghi

Abstract: Public availability of Artificial Intelligence generated information can change the markets forever, and its factoring into economical dynamics may take economists by surprise, out-dating models and schools of thought. Real estate hyper-inflation is not a new phenomenon but its consistent and almost monotonous persistence over 12 years, coinciding with prominence of public estimation information f… ▽ More Public availability of Artificial Intelligence generated information can change the markets forever, and its factoring into economical dynamics may take economists by surprise, out-dating models and schools of thought. Real estate hyper-inflation is not a new phenomenon but its consistent and almost monotonous persistence over 12 years, coinciding with prominence of public estimation information from Zillow, a successful Mass Real Estate Estimator (MREE), could not escape unobserved. What we model is a repetitive theoretical game between the MREE and the home owners, where each player has secret information and expertise. If the intention is to keep housing affordable and maintain old American lifestyle with broad home-ownership, new challenges are defined. Simulations show that a simple restriction of MREE-style price estimation availability to opt-in properties may help partially reduce feedback loop by acting on its likely causes, as suggested by experimental simulation models. The conjecture that the MREE pressure on real estate inflation rate is correlated with the absolute MREE estimation errors, which is logically explainable, is then validated in simulations. △ Less

Submitted 12 March, 2024; originally announced May 2024.

Comments: On February 15, 2022 we uploaded in overleaf the first draft of this paper under the name "Public AI on house price estimations through Zillow may influence a monotonic house price increase and inflation forever according to simulations", https://www.overleaf.com/read/yttcffkrhvjf\#7120e1

InfFeed: Influence Functions as a Feedback to Improve the Performance of Subjective Tasks

Authors: Somnath Banerjee, Maulindu Sarkar, Punyajoy Saha, Binny Mathew, Animesh Mukherjee

Abstract: Recently, influence functions present an apparatus for achieving explainability for deep neural models by quantifying the perturbation of individual train instances that might impact a test prediction. Our objectives in this paper are twofold. First we incorporate influence functions as a feedback into the model to improve its performance. Second, in a dataset extension exercise, using influence f… ▽ More Recently, influence functions present an apparatus for achieving explainability for deep neural models by quantifying the perturbation of individual train instances that might impact a test prediction. Our objectives in this paper are twofold. First we incorporate influence functions as a feedback into the model to improve its performance. Second, in a dataset extension exercise, using influence functions to automatically identify data points that have been initially `silver' annotated by some existing method and need to be cross-checked (and corrected) by annotators to improve the model performance. To meet these objectives, in this paper, we introduce InfFeed, which uses influence functions to compute the influential instances for a target instance. Toward the first objective, we adjust the label of the target instance based on its influencer(s) label. In doing this, InfFeed outperforms the state-of-the-art baselines (including LLMs) by a maximum macro F1-score margin of almost 4% for hate speech classification, 3.5% for stance classification, and 3% for irony and 2% for sarcasm detection. Toward the second objective we show that manually re-annotating only those silver annotated data points in the extension set that have a negative influence can immensely improve the model performance bringing it very close to the scenario where all the data points in the extension set have gold labels. This allows for huge reduction of the number of data points that need to be manually annotated since out of the silver annotated extension dataset, the influence function scheme picks up ~1/1000 points that need manual correction. △ Less

Submitted 9 March, 2024; v1 submitted 22 February, 2024; originally announced February 2024.

Comments: Accepted at LREC-COLING 2024 (Long Paper)

HateMM: A Multi-Modal Dataset for Hate Video Classification

Authors: Mithun Das, Rohit Raj, Punyajoy Saha, Binny Mathew, Manish Gupta, Animesh Mukherjee

Abstract: Hate speech has become one of the most significant issues in modern society, having implications in both the online and the offline world. Due to this, hate speech research has recently gained a lot of traction. However, most of the work has primarily focused on text media with relatively little work on images and even lesser on videos. Thus, early stage automated video moderation techniques are n… ▽ More Hate speech has become one of the most significant issues in modern society, having implications in both the online and the offline world. Due to this, hate speech research has recently gained a lot of traction. However, most of the work has primarily focused on text media with relatively little work on images and even lesser on videos. Thus, early stage automated video moderation techniques are needed to handle the videos that are being uploaded to keep the platform safe and healthy. With a view to detect and remove hateful content from the video sharing platforms, our work focuses on hate video detection using multi-modalities. To this end, we curate ~43 hours of videos from BitChute and manually annotate them as hate or non-hate, along with the frame spans which could explain the labelling decision. To collect the relevant videos we harnessed search keywords from hate lexicons. We observe various cues in images and audio of hateful videos. Further, we build deep learning multi-modal models to classify the hate videos and observe that using all the modalities of the videos improves the overall hate speech detection performance (accuracy=0.798, macro F1-score=0.790) by ~5.7% compared to the best uni-modal model in terms of macro F1 score. In summary, our work takes the first step toward understanding and modeling hateful videos on video hosting platforms such as BitChute. △ Less

Submitted 5 May, 2023; originally announced May 2023.

Comments: Accepted at ICWSM 2023(dataset track)

On the rise of fear speech in online social media

Authors: Punyajoy Saha, Kiran Garimella, Narla Komal Kalyan, Saurabh Kumar Pandey, Pauras Mangesh Meher, Binny Mathew, Animesh Mukherjee

Abstract: Recently, social media platforms are heavily moderated to prevent the spread of online hate speech, which is usually fertile in toxic words and is directed toward an individual or a community. Owing to such heavy moderation, newer and more subtle techniques are being deployed. One of the most striking among these is fear speech. Fear speech, as the name suggests, attempts to incite fear about a ta… ▽ More Recently, social media platforms are heavily moderated to prevent the spread of online hate speech, which is usually fertile in toxic words and is directed toward an individual or a community. Owing to such heavy moderation, newer and more subtle techniques are being deployed. One of the most striking among these is fear speech. Fear speech, as the name suggests, attempts to incite fear about a target community. Although subtle, it might be highly effective, often pushing communities toward a physical conflict. Therefore, understanding their prevalence in social media is of paramount importance. This article presents a large-scale study to understand the prevalence of 400K fear speech and over 700K hate speech posts collected from Gab.com. Remarkably, users posting a large number of fear speech accrue more followers and occupy more central positions in social networks than users posting a large number of hate speech. They can also reach out to benign users more effectively than hate speech users through replies, reposts, and mentions. This connects to the fact that, unlike hate speech, fear speech has almost zero toxic content, making it look plausible. Moreover, while fear speech topics mostly portray a community as a perpetrator using a (fake) chain of argumentation, hate speech topics hurl direct multitarget insults, thus pointing to why general users could be more gullible to fear speech. Our findings transcend even to other platforms (Twitter and Facebook) and thus necessitate using sophisticated moderation policies and mass awareness to combat fear speech. △ Less

Submitted 17 March, 2023; originally announced March 2023.

Comments: 16 pages, 9 tables, 15 figures, accepted in Proceedings of the National Academy of Sciences of the United States of America

HateProof: Are Hateful Meme Detection Systems really Robust?

Authors: Piush Aggarwal, Pranit Chawla, Mithun Das, Punyajoy Saha, Binny Mathew, Torsten Zesch, Animesh Mukherjee

Abstract: Exploiting social media to spread hate has tremendously increased over the years. Lately, multi-modal hateful content such as memes has drawn relatively more traction than uni-modal content. Moreover, the availability of implicit content payloads makes them fairly challenging to be detected by existing hateful meme detection systems. In this paper, we present a use case study to analyze such syste… ▽ More Exploiting social media to spread hate has tremendously increased over the years. Lately, multi-modal hateful content such as memes has drawn relatively more traction than uni-modal content. Moreover, the availability of implicit content payloads makes them fairly challenging to be detected by existing hateful meme detection systems. In this paper, we present a use case study to analyze such systems' vulnerabilities against external adversarial attacks. We find that even very simple perturbations in uni-modal and multi-modal settings performed by humans with little knowledge about the model can make the existing detection models highly vulnerable. Empirically, we find a noticeable performance drop of as high as 10% in the macro-F1 score for certain attacks. As a remedy, we attempt to boost the model's robustness using contrastive learning as well as an adversarial training-based method - VILLA. Using an ensemble of the above two approaches, in two of our high resolution datasets, we are able to (re)gain back the performance to a large extent for certain attacks. We believe that ours is a first step toward addressing this crucial problem in an adversarial setting and would inspire more such investigations in the future. △ Less

Submitted 11 February, 2023; originally announced February 2023.

Comments: Accepted at TheWebConf'2023 (WWW'2023)

Rationale-Guided Few-Shot Classification to Detect Abusive Language

Authors: Punyajoy Saha, Divyanshu Sheth, Kushal Kedia, Binny Mathew, Animesh Mukherjee

Abstract: Abusive language is a concerning problem in online social media. Past research on detecting abusive language covers different platforms, languages, demographies, etc. However, models trained using these datasets do not perform well in cross-domain evaluation settings. To overcome this, a common strategy is to use a few samples from the target domain to train models to get better performance in tha… ▽ More Abusive language is a concerning problem in online social media. Past research on detecting abusive language covers different platforms, languages, demographies, etc. However, models trained using these datasets do not perform well in cross-domain evaluation settings. To overcome this, a common strategy is to use a few samples from the target domain to train models to get better performance in that domain (cross-domain few-shot training). However, this might cause the models to overfit the artefacts of those samples. A compelling solution could be to guide the models toward rationales, i.e., spans of text that justify the text's label. This method has been found to improve model performance in the in-domain setting across various NLP tasks. In this paper, we propose RGFS (Rationale-Guided Few-Shot Classification) for abusive language detection. We first build a multitask learning setup to jointly learn rationales, targets, and labels, and find a significant improvement of 6% macro F1 on the rationale detection task over training solely rationale classifiers. We introduce two rationale-integrated BERT-based architectures (the RGFS models) and evaluate our systems over five different abusive language datasets, finding that in the few-shot classification setting, RGFS-based models outperform baseline models by about 7% in macro F1 scores and perform competitively to models finetuned on other source domains. Furthermore, RGFS-based models outperform LIME/SHAP-based approaches in terms of plausibility and are close in performance in terms of faithfulness. △ Less

Submitted 27 July, 2023; v1 submitted 30 November, 2022; originally announced November 2022.

Comments: 11 pages, 14 tables, 3 figures, The code repository is https://github.com/punyajoy/RGFS_ECAI

CounterGeDi: A controllable approach to generate polite, detoxified and emotional counterspeech

Authors: Punyajoy Saha, Kanishk Singh, Adarsh Kumar, Binny Mathew, Animesh Mukherjee

Abstract: Recently, many studies have tried to create generation models to assist counter speakers by providing counterspeech suggestions for combating the explosive proliferation of online hate. However, since these suggestions are from a vanilla generation model, they might not include the appropriate properties required to counter a particular hate speech instance. In this paper, we propose CounterGeDi -… ▽ More Recently, many studies have tried to create generation models to assist counter speakers by providing counterspeech suggestions for combating the explosive proliferation of online hate. However, since these suggestions are from a vanilla generation model, they might not include the appropriate properties required to counter a particular hate speech instance. In this paper, we propose CounterGeDi - an ensemble of generative discriminators (GeDi) to guide the generation of a DialoGPT model toward more polite, detoxified, and emotionally laden counterspeech. We generate counterspeech using three datasets and observe significant improvement across different attribute scores. The politeness and detoxification scores increased by around 15% and 6% respectively, while the emotion in the counterspeech increased by at least 10% across all the datasets. We also experiment with triple-attribute control and observe significant improvement over single attribute results when combining complementing attributes, e.g., politeness, joyfulness and detoxification. In all these experiments, the relevancy of the generated text does not deteriorate due to the application of these controls △ Less

Submitted 9 May, 2022; originally announced May 2022.

Comments: Accepted at IJCAI-ECAI 2022, 10 pages, 2 figures, 11 tables, Code is available at https://github.com/hate-alert/CounterGEDI

HateCheckHIn: Evaluating Hindi Hate Speech Detection Models

Authors: Mithun Das, Punyajoy Saha, Binny Mathew, Animesh Mukherjee

Abstract: Due to the sheer volume of online hate, the AI and NLP communities have started building models to detect such hateful content. Recently, multilingual hate is a major emerging challenge for automated detection where code-mixing or more than one language have been used for conversation in social media. Typically, hate speech detection models are evaluated by measuring their performance on the held-… ▽ More Due to the sheer volume of online hate, the AI and NLP communities have started building models to detect such hateful content. Recently, multilingual hate is a major emerging challenge for automated detection where code-mixing or more than one language have been used for conversation in social media. Typically, hate speech detection models are evaluated by measuring their performance on the held-out test data using metrics such as accuracy and F1-score. While these metrics are useful, it becomes difficult to identify using them where the model is failing, and how to resolve it. To enable more targeted diagnostic insights of such multilingual hate speech models, we introduce a set of functionalities for the purpose of evaluation. We have been inspired to design this kind of functionalities based on real-world conversation on social media. Considering Hindi as a base language, we craft test cases for each functionality. We name our evaluation dataset HateCheckHIn. To illustrate the utility of these functionalities , we test state-of-the-art transformer based m-BERT model and the Perspective API. △ Less

Submitted 30 April, 2022; originally announced May 2022.

Comments: Accepted at: 13th Edition of its Language Resources and Evaluation Conference. arXiv admin note: text overlap with arXiv:2012.15606 by other authors

You too Brutus! Trapping Hateful Users in Social Media: Challenges, Solutions & Insights

Authors: Mithun Das, Punyajoy Saha, Ritam Dutt, Pawan Goyal, Animesh Mukherjee, Binny Mathew

Abstract: Hate speech is regarded as one of the crucial issues plaguing the online social media. The current literature on hate speech detection leverages primarily the textual content to find hateful posts and subsequently identify hateful users. However, this methodology disregards the social connections between users. In this paper, we run a detailed exploration of the problem space and investigate an ar… ▽ More Hate speech is regarded as one of the crucial issues plaguing the online social media. The current literature on hate speech detection leverages primarily the textual content to find hateful posts and subsequently identify hateful users. However, this methodology disregards the social connections between users. In this paper, we run a detailed exploration of the problem space and investigate an array of models ranging from purely textual to graph based to finally semi-supervised techniques using Graph Neural Networks (GNN) that utilize both textual and graph-based features. We run exhaustive experiments on two datasets -- Gab, which is loosely moderated and Twitter, which is strictly moderated. Overall the AGNN model achieves 0.791 macro F1-score on the Gab dataset and 0.780 macro F1-score on the Twitter dataset using only 5% of the labeled instances, considerably outperforming all the other models including the fully supervised ones. We perform detailed error analysis on the best performing text and graph based models and observe that hateful users have unique network neighborhood signatures and the AGNN model benefits by paying attention to these signatures. This property, as we observe, also allows the model to generalize well across platforms in a zero-shot setting. Lastly, we utilize the best performing GNN model to analyze the evolution of hateful users and their targets over time in Gab. △ Less

Submitted 1 August, 2021; originally announced August 2021.

Comments: Extended Version of this paper has been accepted at ACM HT'21. Link to the Code: https://github.com/hate-alert/Hateful-users-detection

"Short is the Road that Leads from Fear to Hate": Fear Speech in Indian WhatsApp Groups

Authors: Punyajoy Saha, Binny Mathew, Kiran Garimella, Animesh Mukherjee

Abstract: WhatsApp is the most popular messaging app in the world. Due to its popularity, WhatsApp has become a powerful and cheap tool for political campaigning being widely used during the 2019 Indian general election, where it was used to connect to the voters on a large scale. Along with the campaigning, there have been reports that WhatsApp has also become a breeding ground for harmful speech against v… ▽ More WhatsApp is the most popular messaging app in the world. Due to its popularity, WhatsApp has become a powerful and cheap tool for political campaigning being widely used during the 2019 Indian general election, where it was used to connect to the voters on a large scale. Along with the campaigning, there have been reports that WhatsApp has also become a breeding ground for harmful speech against various protected groups and religious minorities. Many such messages attempt to instil fear among the population about a specific (minority) community. According to research on inter-group conflict, such `fear speech' messages could have a lasting impact and might lead to real offline violence. In this paper, we perform the first large scale study on fear speech across thousands of public WhatsApp groups discussing politics in India. We curate a new dataset and try to characterize fear speech from this dataset. We observe that users writing fear speech messages use various events and symbols to create the illusion of fear among the reader about a target community. We build models to classify fear speech and observe that current state-of-the-art NLP models do not perform well at this task. Fear speech messages tend to spread faster and could potentially go undetected by classifiers built to detect traditional toxic speech due to their low toxic nature. Finally, using a novel methodology to target users with Facebook ads, we conduct a survey among the users of these WhatsApp groups to understand the types of users who consume and share fear speech. We believe that this work opens up new research questions that are very different from tackling hate speech which the research community has been traditionally involved in. △ Less

Submitted 7 February, 2021; originally announced February 2021.

Comments: 13 pages, 9 figures, 8 tables, Accepted at The Web Conference 2021, code and dataset public at https://github.com/punyajoy/Fear-Speech-analysis

Mining the online infosphere: A survey

Authors: Sayantan Adak, Souvic Chakraborty, Paramtia Das, Mithun Das, Abhisek Dash, Rima Hazra, Binny Mathew, Punyajoy Saha, Soumya Sarkar, Animesh Mukherjee

Abstract: The evolution of AI-based system and applications had pervaded everyday life to make decisions that have momentous impact on individuals and society. With the staggering growth of online data, often termed as the Online Infosphere it has become paramount to monitor the infosphere to ensure social good as the AI-based decisions are severely dependent on it. The goal of this survey is to provide a c… ▽ More The evolution of AI-based system and applications had pervaded everyday life to make decisions that have momentous impact on individuals and society. With the staggering growth of online data, often termed as the Online Infosphere it has become paramount to monitor the infosphere to ensure social good as the AI-based decisions are severely dependent on it. The goal of this survey is to provide a comprehensive review of some of the most important research areas related to infosphere, focusing on the technical challenges and potential solutions. The survey also outlines some of the important future directions. We begin by discussions focused on the collaborative systems that have emerged within the infosphere with a special thrust on Wikipedia. In the follow up we demonstrate how the infosphere has been instrumental in the growth of scientific citations and collaborations thus fueling interdisciplinary research. Finally, we illustrate the issues related to the governance of the infosphere such as the tackling of the (a) rising hateful and abusive behavior and (b) bias and discrimination in different online platforms and news reporting. △ Less

Submitted 2 January, 2021; originally announced January 2021.

Comments: 29 pages

HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection

Authors: Binny Mathew, Punyajoy Saha, Seid Muhie Yimam, Chris Biemann, Pawan Goyal, Animesh Mukherjee

Abstract: Hate speech is a challenging issue plaguing the online social media. While better models for hate speech detection are continuously being developed, there is little research on the bias and interpretability aspects of hate speech. In this paper, we introduce HateXplain, the first benchmark hate speech dataset covering multiple aspects of the issue. Each post in our dataset is annotated from three… ▽ More Hate speech is a challenging issue plaguing the online social media. While better models for hate speech detection are continuously being developed, there is little research on the bias and interpretability aspects of hate speech. In this paper, we introduce HateXplain, the first benchmark hate speech dataset covering multiple aspects of the issue. Each post in our dataset is annotated from three different perspectives: the basic, commonly used 3-class classification (i.e., hate, offensive or normal), the target community (i.e., the community that has been the victim of hate speech/offensive speech in the post), and the rationales, i.e., the portions of the post on which their labelling decision (as hate, offensive or normal) is based. We utilize existing state-of-the-art models and observe that even models that perform very well in classification do not score high on explainability metrics like model plausibility and faithfulness. We also observe that models, which utilize the human rationales for training, perform better in reducing unintended bias towards target communities. We have made our code and dataset public at https://github.com/punyajoy/HateXplain △ Less

Submitted 12 April, 2022; v1 submitted 18 December, 2020; originally announced December 2020.

Comments: 12 pages, 7 figues, 8 tables. Accepted at AAAI 2021

Deep Learning Models for Multilingual Hate Speech Detection

Authors: Sai Saketh Aluru, Binny Mathew, Punyajoy Saha, Animesh Mukherjee

Abstract: Hate speech detection is a challenging problem with most of the datasets available in only one language: English. In this paper, we conduct a large scale analysis of multilingual hate speech in 9 languages from 16 different sources. We observe that in low resource setting, simple models such as LASER embedding with logistic regression performs the best, while in high resource setting BERT based mo… ▽ More Hate speech detection is a challenging problem with most of the datasets available in only one language: English. In this paper, we conduct a large scale analysis of multilingual hate speech in 9 languages from 16 different sources. We observe that in low resource setting, simple models such as LASER embedding with logistic regression performs the best, while in high resource setting BERT based models perform better. In case of zero-shot classification, languages such as Italian and Portuguese achieve good results. Our proposed framework could be used as an efficient solution for low-resource languages. These models could also act as good baselines for future multilingual hate speech detection tasks. We have made our code and experimental settings public for other researchers at https://github.com/punyajoy/DE-LIMIT. △ Less

Submitted 9 December, 2020; v1 submitted 14 April, 2020; originally announced April 2020.

Comments: 16 pages, Accepted at ECML-PKDD 2020

The POLAR Framework: Polar Opposites Enable Interpretability of Pre-Trained Word Embeddings

Authors: Binny Mathew, Sandipan Sikdar, Florian Lemmerich, Markus Strohmaier

Abstract: We introduce POLAR - a framework that adds interpretability to pre-trained word embeddings via the adoption of semantic differentials. Semantic differentials are a psychometric construct for measuring the semantics of a word by analysing its position on a scale between two polar opposites (e.g., cold -- hot, soft -- hard). The core idea of our approach is to transform existing, pre-trained word em… ▽ More We introduce POLAR - a framework that adds interpretability to pre-trained word embeddings via the adoption of semantic differentials. Semantic differentials are a psychometric construct for measuring the semantics of a word by analysing its position on a scale between two polar opposites (e.g., cold -- hot, soft -- hard). The core idea of our approach is to transform existing, pre-trained word embeddings via semantic differentials to a new "polar" space with interpretable dimensions defined by such polar opposites. Our framework also allows for selecting the most discriminative dimensions from a set of polar dimensions provided by an oracle, i.e., an external source. We demonstrate the effectiveness of our framework by deploying it to various downstream tasks, in which our interpretable word embeddings achieve a performance that is comparable to the original word embeddings. We also show that the interpretable dimensions selected by our framework align with human judgement. Together, these results demonstrate that interpretability can be added to word embeddings without compromising performance. Our work is relevant for researchers and engineers interested in interpreting pre-trained word embeddings. △ Less

Submitted 28 January, 2020; v1 submitted 27 January, 2020; originally announced January 2020.

Comments: Accepted at Web Conference (WWW) 2020

HateMonitors: Language Agnostic Abuse Detection in Social Media

Authors: Punyajoy Saha, Binny Mathew, Pawan Goyal, Animesh Mukherjee

Abstract: Reducing hateful and offensive content in online social media pose a dual problem for the moderators. On the one hand, rigid censorship on social media cannot be imposed. On the other, the free flow of such content cannot be allowed. Hence, we require efficient abusive language detection system to detect such harmful content in social media. In this paper, we present our machine learning model, Ha… ▽ More Reducing hateful and offensive content in online social media pose a dual problem for the moderators. On the one hand, rigid censorship on social media cannot be imposed. On the other, the free flow of such content cannot be allowed. Hence, we require efficient abusive language detection system to detect such harmful content in social media. In this paper, we present our machine learning model, HateMonitor, developed for Hate Speech and Offensive Content Identification in Indo-European Languages (HASOC), a shared task at FIRE 2019. We have used a Gradient Boosting model, along with BERT and LASER embeddings, to make the system language agnostic. Our model came at First position for the German sub-task A. We have also made our model public at https://github.com/punyajoy/HateMonitors-HASOC . △ Less

Submitted 27 September, 2019; originally announced September 2019.

Comments: 8 pages, 1 figure, 4 tables, models available at https://github.com/punyajoy/HateMonitors-HASOC

Hate begets Hate: A Temporal Study of Hate Speech

Authors: Binny Mathew, Anurag Illendula, Punyajoy Saha, Soumya Sarkar, Pawan Goyal, Animesh Mukherjee

Abstract: With the ongoing debate on 'freedom of speech' vs. 'hate speech' there is an urgent need to carefully understand the consequences of the inevitable culmination of the two, i.e., 'freedom of hate speech' over time. An ideal scenario to understand this would be to observe the effects of hate speech in an (almost) unrestricted environment. Hence, we perform the first temporal analysis of hate speech… ▽ More With the ongoing debate on 'freedom of speech' vs. 'hate speech' there is an urgent need to carefully understand the consequences of the inevitable culmination of the two, i.e., 'freedom of hate speech' over time. An ideal scenario to understand this would be to observe the effects of hate speech in an (almost) unrestricted environment. Hence, we perform the first temporal analysis of hate speech on Gab.com, a social media site with very loose moderation policy. We first generate temporal snapshots of Gab from millions of posts and users. Using these temporal snapshots, we compute an activity vector based on DeGroot model to identify hateful users. The amount of hate speech in Gab is steadily increasing and the new users are becoming hateful at an increased and faster rate. Further, our analysis analysis reveals that the hate users are occupying the prominent positions in the Gab network. Also, the language used by the community as a whole seem to correlate more with that of the hateful users as compared to the non-hateful ones. We discuss how, many crucial design questions in CSCW open up from our work. △ Less

Submitted 3 August, 2020; v1 submitted 24 September, 2019; originally announced September 2019.

Comments: 24 pages, 14 figures, 1 table. Accepted at CSCW 2020

Competing Topic Naming Conventions in Quora: Predicting Appropriate Topic Merges and Winning Topics from Millions of Topic Pairs

Authors: Binny Mathew, Suman Kalyan Maity, Pawan Goyal, Animesh Mukherjee

Abstract: Quora is a popular Q&A site which provides users with the ability to tag questions with multiple relevant topics which helps to attract quality answers. These topics are not predefined but user-defined conventions and it is not so rare to have multiple such conventions present in the Quora ecosystem describing exactly the same concept. In almost all such cases, users (or Quora moderators) manually… ▽ More Quora is a popular Q&A site which provides users with the ability to tag questions with multiple relevant topics which helps to attract quality answers. These topics are not predefined but user-defined conventions and it is not so rare to have multiple such conventions present in the Quora ecosystem describing exactly the same concept. In almost all such cases, users (or Quora moderators) manually merge the topic pair into one of the either topics, thus selecting one of the competing conventions. An important application for the site therefore is to identify such competing conventions early enough that should merge in future. In this paper, we propose a two-step approach that uniquely combines the anomaly detection and the supervised classification frameworks to predict whether two topics from among millions of topic pairs are indeed competing conventions, and should merge, achieving an F-score of 0.711. We also develop a model to predict the direction of the topic merge, i.e., the winning convention, achieving an F-score of 0.898. Our system is also able to predict ~ 25% of the correct case of merges within the first month of the merge and ~ 40% of the cases within a year. This is an encouraging result since Quora users on average take 936 days to identify such a correct merge. Human judgment experiments show that our system is able to predict almost all the correct cases that humans can predict plus 37.24% correct cases which the humans are not able to identify at all. △ Less

Submitted 10 September, 2019; originally announced September 2019.

Comments: 15 pages, 8 figures, 9 tables

Hateminers : Detecting Hate speech against Women

Authors: Punyajoy Saha, Binny Mathew, Pawan Goyal, Animesh Mukherjee

Abstract: With the online proliferation of hate speech, there is an urgent need for systems that can detect such harmful content. In this paper, We present the machine learning models developed for the Automatic Misogyny Identification (AMI) shared task at EVALITA 2018. We generate three types of features: Sentence Embeddings, TF-IDF Vectors, and BOW Vectors to represent each tweet. These features are then… ▽ More With the online proliferation of hate speech, there is an urgent need for systems that can detect such harmful content. In this paper, We present the machine learning models developed for the Automatic Misogyny Identification (AMI) shared task at EVALITA 2018. We generate three types of features: Sentence Embeddings, TF-IDF Vectors, and BOW Vectors to represent each tweet. These features are then concatenated and fed into the machine learning models. Our model came First for the English Subtask A and Fifth for the English Subtask B. We release our winning model for public use and it's available at https://github.com/punyajoy/Hateminers-EVALITA. △ Less

Submitted 17 December, 2018; originally announced December 2018.

Comments: 5 Pages, 2 Figures, 1 Table, Model Available at https://github.com/punyajoy/Hateminers-EVALITA

Analyzing the hate and counter speech accounts on Twitter

Authors: Binny Mathew, Navish Kumar, Ravina, Pawan Goyal, Animesh Mukherjee

Abstract: The online hate speech is proliferating with several organization and countries implementing laws to ban such harmful speech. While these restrictions might reduce the amount of such hateful content, it does so by restricting freedom of speech. Thus, an promising alternative supported by several organizations is to counter such hate speech with more speech. In this paper, We analyze hate speech an… ▽ More The online hate speech is proliferating with several organization and countries implementing laws to ban such harmful speech. While these restrictions might reduce the amount of such hateful content, it does so by restricting freedom of speech. Thus, an promising alternative supported by several organizations is to counter such hate speech with more speech. In this paper, We analyze hate speech and the corresponding counters (aka counterspeech) on Twitter. We perform several lexical, linguistic and psycholinguistic analysis on these user accounts and obverse that counter speakers employ several strategies depending on the target community. The hateful accounts express more negative sentiments and are more profane. We also find that the hate tweets by verified accounts have much more virality as compared to a tweet by a non-verified account. While the hate users seem to use words more about envy, hate, negative emotion, swearing terms, ugliness, the counter users use more words related to government, law, leader. We also build a supervised model for classifying the hateful and counterspeech accounts on Twitter and obtain an F-score of 0.77. We also make our dataset public to help advance the research on hate speech. △ Less

Submitted 6 December, 2018; originally announced December 2018.

Comments: 11 pages, 8 figures, and 5 tables

Spread of hate speech in online social media

Authors: Binny Mathew, Ritam Dutt, Pawan Goyal, Animesh Mukherjee

Abstract: The present online social media platform is afflicted with several issues, with hate speech being on the predominant forefront. The prevalence of online hate speech has fueled horrific real-world hate-crime such as the mass-genocide of Rohingya Muslims, communal violence in Colombo and the recent massacre in the Pittsburgh synagogue. Consequently, It is imperative to understand the diffusion of su… ▽ More The present online social media platform is afflicted with several issues, with hate speech being on the predominant forefront. The prevalence of online hate speech has fueled horrific real-world hate-crime such as the mass-genocide of Rohingya Muslims, communal violence in Colombo and the recent massacre in the Pittsburgh synagogue. Consequently, It is imperative to understand the diffusion of such hateful content in an online setting. We conduct the first study that analyses the flow and dynamics of posts generated by hateful and non-hateful users on Gab (gab.com) over a massive dataset of 341K users and 21M posts. Our observations confirms that hateful content diffuse farther, wider and faster and have a greater outreach than those of non-hateful users. A deeper inspection into the profiles and network of hateful and non-hateful users reveals that the former are more influential, popular and cohesive. Thus, our research explores the interesting facets of diffusion dynamics of hateful users and broadens our understanding of hate speech in the online world. △ Less

Submitted 4 December, 2018; originally announced December 2018.

Comments: 8 pages, 5 figures, and 4 table

Deep Dive into Anonymity: A Large Scale Analysis of Quora Questions

Authors: Binny Mathew, Ritam Dutt, Suman Kalyan Maity, Pawan Goyal, Animesh Mukherjee

Abstract: Anonymity forms an integral and important part of our digital life. It enables us to express our true selves without the fear of judgment. In this paper, we investigate the different aspects of anonymity in the social Q&A site Quora. The choice of Quora is motivated by the fact that this is one of the rare social Q&A sites that allow users to explicitly post anonymous questions and such activity i… ▽ More Anonymity forms an integral and important part of our digital life. It enables us to express our true selves without the fear of judgment. In this paper, we investigate the different aspects of anonymity in the social Q&A site Quora. The choice of Quora is motivated by the fact that this is one of the rare social Q&A sites that allow users to explicitly post anonymous questions and such activity in this forum has become normative rather than a taboo. Through an analysis of 5.1 million questions, we observe that at a global scale almost no difference manifests between the linguistic structure of the anonymous and the non-anonymous questions. We find that topical mixing at the global scale to be the primary reason for the absence. However, the differences start to feature once we "deep dive" and (topically) cluster the questions and compare the clusters that have high volumes of anonymous questions with those that have low volumes of anonymous questions. In particular, we observe that the choice to post the question as anonymous is dependent on the user's perception of anonymity and they often choose to speak about depression, anxiety, social ties and personal issues under the guise of anonymity. We further perform personality trait analysis and observe that the anonymous group of users has positive correlation with extraversion, agreeableness, and negative correlation with openness. Subsequently, to gain further insights, we build an anonymity grid to identify the differences in the perception on anonymity of the user posting the question and the community of users answering it. We also look into the first response time of the questions and observe that it is lowest for topics which talk about personal and sensitive issues, which hints toward a higher degree of community support and user engagement. △ Less

Submitted 17 November, 2018; originally announced November 2018.

Comments: 12 pages, 6 figures, and 12 tables

Thou shalt not hate: Countering Online Hate Speech

Authors: Binny Mathew, Punyajoy Saha, Hardik Tharad, Subham Rajgaria, Prajwal Singhania, Suman Kalyan Maity, Pawan Goyal, Animesh Mukherje

Abstract: Hate content in social media is ever-increasing. While Facebook, Twitter, Google have attempted to take several steps to tackle the hateful content, they have mostly been unsuccessful. Counterspeech is seen as an effective way of tackling the online hate without any harm to the freedom of speech. Thus, an alternative strategy for these platforms could be to promote counterspeech as a defense again… ▽ More Hate content in social media is ever-increasing. While Facebook, Twitter, Google have attempted to take several steps to tackle the hateful content, they have mostly been unsuccessful. Counterspeech is seen as an effective way of tackling the online hate without any harm to the freedom of speech. Thus, an alternative strategy for these platforms could be to promote counterspeech as a defense against hate content. However, in order to have a successful promotion of such counterspeech, one has to have a deep understanding of its dynamics in the online world. Lack of carefully curated data largely inhibits such understanding. In this paper, we create and release the first ever dataset for counterspeech using comments from YouTube. The data contains 13,924 manually annotated comments where the labels indicate whether a comment is a counterspeech or not. This data allows us to perform a rigorous measurement study characterizing the linguistic structure of counterspeech for the first time. This analysis results in various interesting insights such as: the counterspeech comments receive much more likes as compared to the non-counterspeech comments, for certain communities majority of the non-counterspeech comments tend to be hate speech, the different types of counterspeech are not all equally effective and the language choice of users posting counterspeech is largely different from those posting non-counterspeech as revealed by a detailed psycholinguistic analysis. Finally, we build a set of machine learning models that are able to automatically detect counterspeech in YouTube videos with an F1-score of 0.71. We also build multilabel models that can detect different types of counterspeech in a comment with an F1-score of 0.60. △ Less

Submitted 4 April, 2019; v1 submitted 13 August, 2018; originally announced August 2018.

Comments: Accepted at ICWSM 2019. 12 Pages, 5 Figures, and 7 Tables. The dataset and models are available here: https://github.com/binny-mathew/Countering_Hate_Speech_ICWSM2019

Mining Twitter Conversations around E-commerce Promotional Events

Authors: Binny Mathew, Unnikrishnan T A, Tanmoy Chakraborty, Niloy Ganguly, Samik Datta

Abstract: With Social Media platforms establishing themselves as the de facto destinations for their customers views and opinions, brands around the World are investing heavily on invigorating their customer connects by utilizing such platforms to their fullest. In this paper, we develop a novel technique for mining conversations in Twitter by weaving together all conversations around an event into one unif… ▽ More With Social Media platforms establishing themselves as the de facto destinations for their customers views and opinions, brands around the World are investing heavily on invigorating their customer connects by utilizing such platforms to their fullest. In this paper, we develop a novel technique for mining conversations in Twitter by weaving together all conversations around an event into one unified graph (Conversation Graph, henceforth). The structure of the Conversation Graph emerges as a variant of the BOWTIE structure (dubbed ASKEWBOWTIE henceforth) as a result of the complex communication patterns amongst these players. Finally, we investigate the structural properties of the ASKEWBOWTIE structure to understand the configuration of the components and their temporal evolution. △ Less

Submitted 4 February, 2018; originally announced February 2018.

Comments: 4 pages, 5 tables, 3 figures

Adapting predominant and novel sense discovery algorithms for identifying corpus-specific sense differences

Authors: Binny Mathew, Suman Kalyan Maity, Pratip Sarkar, Animesh Mukherjee, Pawan Goyal

Abstract: Word senses are not static and may have temporal, spatial or corpus-specific scopes. Identifying such scopes might benefit the existing WSD systems largely. In this paper, while studying corpus specific word senses, we adapt three existing predominant and novel-sense discovery algorithms to identify these corpus-specific senses. We make use of text data available in the form of millions of digitiz… ▽ More Word senses are not static and may have temporal, spatial or corpus-specific scopes. Identifying such scopes might benefit the existing WSD systems largely. In this paper, while studying corpus specific word senses, we adapt three existing predominant and novel-sense discovery algorithms to identify these corpus-specific senses. We make use of text data available in the form of millions of digitized books and newspaper archives as two different sources of corpora and propose automated methods to identify corpus-specific word senses at various time points. We conduct an extensive and thorough human judgment experiment to rigorously evaluate and compare the performance of these approaches. Post adaptation, the output of the three algorithms are in the same format and the accuracy results are also comparable, with roughly 45-60% of the reported corpus-specific senses being judged as genuine. △ Less

Submitted 1 February, 2018; originally announced February 2018.

Comments: 10 pages,2 figures, Accepted in TextGraphs-11

A Retraction Theorem for Distributed Synthesis

Authors: Dietmar Berwanger, Anup Basil Mathew, R. Ramanujam

Abstract: We present a general theorem for distributed synthesis problems in coordination games with $ω$-regular objectives of the form: If there exists a winning strategy for the coalition, then there exists an "essential" winning strategy, that is obtained by a retraction of the given one. In general, this does not lead to finite-state winning strategies, but when the knowledge of agents remains bounded,… ▽ More We present a general theorem for distributed synthesis problems in coordination games with $ω$-regular objectives of the form: If there exists a winning strategy for the coalition, then there exists an "essential" winning strategy, that is obtained by a retraction of the given one. In general, this does not lead to finite-state winning strategies, but when the knowledge of agents remains bounded, we can solve the synthesis problem. Our study is carried out in a setting where objectives are expressed in terms of events that may \emph{not} be observable. This is natural in games of imperfect information, rather than the common assumption that objectives are expressed in terms of events that are observable to all agents. We characterise decidable distributed synthesis problems in terms of finiteness of knowledge states and finite congruence classes induced by them. △ Less

Submitted 26 April, 2016; originally announced April 2016.

MSC Class: 05C57; 68M14; 91A06; 91A28; 93B50 ACM Class: C.2.4; F.1.2

Hierarchical Information and the Synthesis of Distributed Strategies

Authors: Dietmar Berwanger, Anup Basil Mathew, Marie van den Bogaard

Abstract: Infinite games with imperfect information are known to be undecidable unless the information flow is severely restricted. One fundamental decidable case occurs when there is a total ordering among players, such that each player has access to all the information that the following ones receive. In this paper we consider variations of this hierarchy principle for synchronous games with perfect rec… ▽ More Infinite games with imperfect information are known to be undecidable unless the information flow is severely restricted. One fundamental decidable case occurs when there is a total ordering among players, such that each player has access to all the information that the following ones receive. In this paper we consider variations of this hierarchy principle for synchronous games with perfect recall, and identify new decidable classes for which the distributed synthesis problem is solvable with finite-state strategies. In particular, we show that decidability is maintained when the information hierarchy may change along the play, or when transient phases without hierarchical information are allowed. Finally, we interpret our result in terms of distributed system architectures. △ Less

Submitted 16 July, 2016; v1 submitted 11 June, 2015; originally announced June 2015.

Comments: 35 pages, 6 figures; extended version of a paper presented at ATVA 2015

MSC Class: 91A06; 68M14; 93B50 ACM Class: C.1.4

Infinite games with finite knowledge gaps

Authors: Dietmar Berwanger, Anup Basil Mathew

Abstract: Infinite games where several players seek to coordinate under imperfect information are deemed to be undecidable, unless the information is hierarchically ordered among the players. We identify a class of games for which joint winning strategies can be constructed effectively without restricting the direction of information flow. Instead, our condition requires that the players attain common kno… ▽ More Infinite games where several players seek to coordinate under imperfect information are deemed to be undecidable, unless the information is hierarchically ordered among the players. We identify a class of games for which joint winning strategies can be constructed effectively without restricting the direction of information flow. Instead, our condition requires that the players attain common knowledge about the actual state of the game over and over again along every play. We show that it is decidable whether a given game satisfies the condition, and prove tight complexity bounds for the strategy synthesis problem under $ω$-regular winning conditions given by parity automata. △ Less

Submitted 28 July, 2015; v1 submitted 21 November, 2014; originally announced November 2014.

Comments: 39 pages; 2nd revision; submitted to Information and Computation

MSC Class: 05C57; 68M14; 91A06; 91A28; 93B50

Games with recurring certainty

Authors: Dietmar Berwanger, Anup Basil Mathew

Abstract: Infinite games where several players seek to coordinate under imperfect information are known to be intractable, unless the information flow is severely restricted. Examples of undecidable cases typically feature a situation where players become uncertain about the current state of the game, and this uncertainty lasts forever. Here we consider games where the players attain certainty about the cur… ▽ More Infinite games where several players seek to coordinate under imperfect information are known to be intractable, unless the information flow is severely restricted. Examples of undecidable cases typically feature a situation where players become uncertain about the current state of the game, and this uncertainty lasts forever. Here we consider games where the players attain certainty about the current state over and over again along any play. For finite-state games, we note that this kind of recurring certainty implies a stronger condition of periodic certainty, that is, the events of state certainty ultimately occur at uniform, regular intervals. We show that it is decidable whether a given game presents recurring certainty, and that, if so, the problem of synthesising coordination strategies under w-regular winning conditions is solvable. △ Less

Submitted 3 April, 2014; originally announced April 2014.

Comments: In Proceedings SR 2014, arXiv:1404.0414

Journal ref: EPTCS 146, 2014, pp. 91-96