-
PlagBench: Exploring the Duality of Large Language Models in Plagiarism Generation and Detection
Authors:
Jooyoung Lee,
Toshini Agrawal,
Adaku Uchendu,
Thai Le,
Jinghui Chen,
Dongwon Lee
Abstract:
Recent literature has highlighted potential risks to academic integrity associated with large language models (LLMs), as they can memorize parts of training instances and reproduce them in the generated texts without proper attribution. In addition, given their capabilities in generating high-quality texts, plagiarists can exploit LLMs to generate realistic paraphrases or summaries indistinguishab…
▽ More
Recent literature has highlighted potential risks to academic integrity associated with large language models (LLMs), as they can memorize parts of training instances and reproduce them in the generated texts without proper attribution. In addition, given their capabilities in generating high-quality texts, plagiarists can exploit LLMs to generate realistic paraphrases or summaries indistinguishable from original work. In response to possible malicious use of LLMs in plagiarism, we introduce PlagBench, a comprehensive dataset consisting of 46.5K synthetic plagiarism cases generated using three instruction-tuned LLMs across three writing domains. The quality of PlagBench is ensured through fine-grained automatic evaluation for each type of plagiarism, complemented by human annotation. We then leverage our proposed dataset to evaluate the plagiarism detection performance of five modern LLMs and three specialized plagiarism checkers. Our findings reveal that GPT-3.5 tends to generates paraphrases and summaries of higher quality compared to Llama2 and GPT-4. Despite LLMs' weak performance in summary plagiarism identification, they can surpass current commercial plagiarism detectors. Overall, our results highlight the potential of LLMs to serve as robust plagiarism detection tools.
△ Less
Submitted 23 June, 2024;
originally announced June 2024.
-
CountCLIP -- [Re] Teaching CLIP to Count to Ten
Authors:
Harshvardhan Mestha,
Tejas Agrawal,
Karan Bania,
Shreyas V,
Yash Bhisikar
Abstract:
Large vision-language models (VLMs) are shown to learn rich joint image-text representations enabling high performances in relevant downstream tasks. However, they fail to showcase their quantitative understanding of objects, and they lack good counting-aware representation. This paper conducts a reproducibility study of 'Teaching CLIP to Count to Ten' (Paiss et al., 2023), which presents a method…
▽ More
Large vision-language models (VLMs) are shown to learn rich joint image-text representations enabling high performances in relevant downstream tasks. However, they fail to showcase their quantitative understanding of objects, and they lack good counting-aware representation. This paper conducts a reproducibility study of 'Teaching CLIP to Count to Ten' (Paiss et al., 2023), which presents a method to finetune a CLIP model (Radford et al., 2021) to improve zero-shot counting accuracy in an image while maintaining the performance for zero-shot classification by introducing a counting-contrastive loss term. We improve the model's performance on a smaller subset of their training data with lower computational resources. We verify these claims by reproducing their study with our own code. The implementation can be found at https://github.com/SforAiDl/CountCLIP.
△ Less
Submitted 10 June, 2024; v1 submitted 5 June, 2024;
originally announced June 2024.
-
Multimodal Vision Transformers with Forced Attention for Behavior Analysis
Authors:
Tanay Agrawal,
Michal Balazia,
Philipp Müller,
François Brémond
Abstract:
Human behavior understanding requires looking at minute details in the large context of a scene containing multiple input modalities. It is necessary as it allows the design of more human-like machines. While transformer approaches have shown great improvements, they face multiple challenges such as lack of data or background noise. To tackle these, we introduce the Forced Attention (FAt) Transfor…
▽ More
Human behavior understanding requires looking at minute details in the large context of a scene containing multiple input modalities. It is necessary as it allows the design of more human-like machines. While transformer approaches have shown great improvements, they face multiple challenges such as lack of data or background noise. To tackle these, we introduce the Forced Attention (FAt) Transformer which utilize forced attention with a modified backbone for input encoding and a use of additional inputs. In addition to improving the performance on different tasks and inputs, the modification requires less time and memory resources. We provide a model for a generalised feature extraction for tasks concerning social signals and behavior analysis. Our focus is on understanding behavior in videos where people are interacting with each other or talking into the camera which simulates the first person point of view in social interaction. FAt Transformers are applied to two downstream tasks: personality recognition and body language recognition. We achieve state-of-the-art results for Udiva v0.5, First Impressions v2 and MPII Group Interaction datasets. We further provide an extensive ablation study of the proposed architecture.
△ Less
Submitted 7 December, 2022;
originally announced December 2022.
-
Multimodal Personality Recognition using Cross-Attention Transformer and Behaviour Encoding
Authors:
Tanay Agrawal,
Dhruv Agarwal,
Michal Balazia,
Neelabh Sinha,
Francois Bremond
Abstract:
Personality computing and affective computing have gained recent interest in many research areas. The datasets for the task generally have multiple modalities like video, audio, language and bio-signals. In this paper, we propose a flexible model for the task which exploits all available data. The task involves complex relations and to avoid using a large model for video processing specifically, w…
▽ More
Personality computing and affective computing have gained recent interest in many research areas. The datasets for the task generally have multiple modalities like video, audio, language and bio-signals. In this paper, we propose a flexible model for the task which exploits all available data. The task involves complex relations and to avoid using a large model for video processing specifically, we propose the use of behaviour encoding which boosts performance with minimal change to the model. Cross-attention using transformers has become popular in recent times and is utilised for fusion of different modalities. Since long term relations may exist, breaking the input into chunks is not desirable, thus the proposed model processes the entire input together. Our experiments show the importance of each of the above contributions
△ Less
Submitted 12 January, 2023; v1 submitted 22 December, 2021;
originally announced December 2021.
-
From Multimodal to Unimodal Attention in Transformers using Knowledge Distillation
Authors:
Dhruv Agarwal,
Tanay Agrawal,
Laura M. Ferrari,
François Bremond
Abstract:
Multimodal Deep Learning has garnered much interest, and transformers have triggered novel approaches, thanks to the cross-attention mechanism. Here we propose an approach to deal with two key existing challenges: the high computational resource demanded and the issue of missing modalities. We introduce for the first time the concept of knowledge distillation in transformers to use only one modali…
▽ More
Multimodal Deep Learning has garnered much interest, and transformers have triggered novel approaches, thanks to the cross-attention mechanism. Here we propose an approach to deal with two key existing challenges: the high computational resource demanded and the issue of missing modalities. We introduce for the first time the concept of knowledge distillation in transformers to use only one modality at inference time. We report a full study analyzing multiple student-teacher configurations, levels at which distillation is applied, and different methodologies. With the best configuration, we improved the state-of-the-art accuracy by 3%, we reduced the number of parameters by 2.5 times and the inference time by 22%. Such performance-computation tradeoff can be exploited in many applications and we aim at opening a new research area where the deployment of complex models with limited resources is demanded.
△ Less
Submitted 19 October, 2021; v1 submitted 15 October, 2021;
originally announced October 2021.
-
On evaluating CNN representations for low resource medical image classification
Authors:
Taruna Agrawal,
Rahul Gupta,
Shrikanth Narayanan
Abstract:
Convolutional Neural Networks (CNNs) have revolutionized performances in several machine learning tasks such as image classification, object tracking, and keyword spotting. However, given that they contain a large number of parameters, their direct applicability into low resource tasks is not straightforward. In this work, we experiment with an application of CNN models to gastrointestinal landmar…
▽ More
Convolutional Neural Networks (CNNs) have revolutionized performances in several machine learning tasks such as image classification, object tracking, and keyword spotting. However, given that they contain a large number of parameters, their direct applicability into low resource tasks is not straightforward. In this work, we experiment with an application of CNN models to gastrointestinal landmark classification with only a few thousands of training samples through transfer learning. As in a standard transfer learning approach, we train CNNs on a large external corpus, followed by representation extraction for the medical images. Finally, a classifier is trained on these CNN representations. However, given that several variants of CNNs exist, the choice of CNN is not obvious. To address this, we develop a novel metric that can be used to predict test performances, given CNN representations on the training set. Not only we demonstrate the superiority of the CNN based transfer learning approach against an assembly of knowledge driven features, but the proposed metric also carries an 87% correlation with the test set performances as obtained using various CNN representations.
△ Less
Submitted 26 March, 2019;
originally announced March 2019.
-
Bollywood Movie Corpus for Text, Images and Videos
Authors:
Nishtha Madaan,
Sameep Mehta,
Mayank Saxena,
Aditi Aggarwal,
Taneea S Agrawaal,
Vrinda Malhotra
Abstract:
In past few years, several data-sets have been released for text and images. We present an approach to create the data-set for use in detecting and removing gender bias from text. We also include a set of challenges we have faced while creating this corpora. In this work, we have worked with movie data from Wikipedia plots and movie trailers from YouTube. Our Bollywood Movie corpus contains 4000 m…
▽ More
In past few years, several data-sets have been released for text and images. We present an approach to create the data-set for use in detecting and removing gender bias from text. We also include a set of challenges we have faced while creating this corpora. In this work, we have worked with movie data from Wikipedia plots and movie trailers from YouTube. Our Bollywood Movie corpus contains 4000 movies extracted from Wikipedia and 880 trailers extracted from YouTube which were released from 1970-2017. The corpus contains csv files with the following data about each movie - Wikipedia title of movie, cast, plot text, co-referenced plot text, soundtrack information, link to movie poster, caption of movie poster, number of males in poster, number of females in poster. In addition to that, corresponding to each cast member the following data is available - cast name, cast gender, cast verbs, cast adjectives, cast relations, cast centrality, cast mentions. We present some preliminary results on the task of bias removal which suggest that the data-set is quite useful for performing such tasks.
△ Less
Submitted 11 October, 2017;
originally announced October 2017.
-
Analyzing Gender Stereotyping in Bollywood Movies
Authors:
Nishtha Madaan,
Sameep Mehta,
Taneea S Agrawaal,
Vrinda Malhotra,
Aditi Aggarwal,
Mayank Saxena
Abstract:
The presence of gender stereotypes in many aspects of society is a well-known phenomenon. In this paper, we focus on studying such stereotypes and bias in Hindi movie industry (Bollywood). We analyze movie plots and posters for all movies released since 1970. The gender bias is detected by semantic modeling of plots at inter-sentence and intra-sentence level. Different features like occupation, in…
▽ More
The presence of gender stereotypes in many aspects of society is a well-known phenomenon. In this paper, we focus on studying such stereotypes and bias in Hindi movie industry (Bollywood). We analyze movie plots and posters for all movies released since 1970. The gender bias is detected by semantic modeling of plots at inter-sentence and intra-sentence level. Different features like occupation, introduction of cast in text, associated actions and descriptions are captured to show the pervasiveness of gender bias and stereo- type in movies. We derive a semantic graph and compute centrality of each character and observe similar bias there. We also show that such bias is not applicable for movie posters where females get equal importance even though their character has little or no impact on the movie plot. Furthermore, we explore the movie trailers to estimate on-screen time for males and females and also study the portrayal of emotions by gender in them. The silver lining is that our system was able to identify 30 movies over last 3 years where such stereotypes were broken.
△ Less
Submitted 11 October, 2017;
originally announced October 2017.
-
Secure Analog Network Coding in Layered Networks
Authors:
Tulika Agrawal,
Samar Agnihotri
Abstract:
We consider a class of Gaussian layered networks where a source communicates with a destination through $L$ intermediate relay layers with $N$ nodes in each layer in the presence of a single eavesdropper which can overhear the transmissions of the nodes in any one layer. The problem of maximum secrecy rate achievable with analog network coding for a unicast communication over such layered wireless…
▽ More
We consider a class of Gaussian layered networks where a source communicates with a destination through $L$ intermediate relay layers with $N$ nodes in each layer in the presence of a single eavesdropper which can overhear the transmissions of the nodes in any one layer. The problem of maximum secrecy rate achievable with analog network coding for a unicast communication over such layered wireless relay networks with directed links is considered. A relay node performing analog network coding scales and forwards the signals received at its input. The key contribution of this work is a lemma that provides the globally optimal set of scaling factors for the nodes that maximizes the end-to-end secrecy rate for a class of layered networks. We also show that in the high-SNR regime, ANC achieves secrecy rates within a constant gap of the cutset upper bound on the secrecy capacity. To the best of our knowledge, this work offers the first characterization of the performance of secure ANC in multi-layered networks in the presence of an eavesdropper.
△ Less
Submitted 15 July, 2016;
originally announced July 2016.
-
Network Simplification for Secure AF Relaying
Authors:
Tulika Agrawal,
Samar Agnihotri
Abstract:
We consider a class of Gaussian layered networks where a source communicates with a destination through L intermediate relay layers with N nodes in each layer in the presence of a single eavesdropper which can overhear the transmissions of the nodes in the last layer. For such networks we address the question: what fraction of maximum secure achievable rate can be maintained if only a fraction of…
▽ More
We consider a class of Gaussian layered networks where a source communicates with a destination through L intermediate relay layers with N nodes in each layer in the presence of a single eavesdropper which can overhear the transmissions of the nodes in the last layer. For such networks we address the question: what fraction of maximum secure achievable rate can be maintained if only a fraction of available relay nodes are used in each layer? In particular, we provide upper bounds on additive and multiplicative gaps between the optimal secure AF when all N relays in each layer are used and when only k, 1 <= k < N, relays are used in each layer. We show that asymptotically (in source power), the additive gap increases at most logarithmically with ratio N/k and L, and the corresponding multiplicative gap increases at most quadratically with ratio N/k and L. To the best of our knowledge, this work offers the first characterization of the performance of network simplification in layered amplify-and-forward relay networks in the presence of an eavesdropper.
△ Less
Submitted 1 July, 2016;
originally announced July 2016.
-
Scheduling Chained Multiprocessor Tasks onto Large Multiprocessor System
Authors:
T. K. Agrawal,
R. Sharma,
M. Ghose,
A. Sahu
Abstract:
In this paper, we proposed an effective approach for scheduling of multiprocessor unit time tasks with chain precedence on to large multiprocessor system. The proposed longest chain maximum processor scheduling algorithm is proved to be optimal for uniform chains and monotone (non-increasing/non-decreasing) chains for both splitable and non-splitable multiprocessor unit time tasks chain. Schedulin…
▽ More
In this paper, we proposed an effective approach for scheduling of multiprocessor unit time tasks with chain precedence on to large multiprocessor system. The proposed longest chain maximum processor scheduling algorithm is proved to be optimal for uniform chains and monotone (non-increasing/non-decreasing) chains for both splitable and non-splitable multiprocessor unit time tasks chain. Scheduling arbitrary chains of non-splitable multiprocessor unit time tasks is proved to be NP-complete problem. But scheduling arbitrary chains of splitable multiprocessor unit time tasks is still an open problem to be proved whether it is NP-complete or can be solved in polynomial time. We have used three heuristics (a) maximum criticality first, (b) longest chain maximum criticality first and (c) longest chain maximum processor first for scheduling of arbitrary chains. Also compared performance of all three scheduling heuristics and found out that the proposed longest chain maximum processor first performs better in most of the cases.
△ Less
Submitted 12 August, 2015;
originally announced August 2015.
-
Securing the Data in Clouds with Hyperelliptic Curve Cryptography
Authors:
Debajyoti Mukhopadhyay,
Ashay Shirwadkar,
Pratik Gaikar,
Tanmay Agrawal
Abstract:
In todays world, Cloud computing has attracted research communities as it provides services in reduced cost due to virtualizing all the necessary resources. Even modern business architecture depends upon Cloud computing .As it is a internet based utility, which provides various services over a network, it is prone to network based attacks. Hence security in clouds is the most important in case of…
▽ More
In todays world, Cloud computing has attracted research communities as it provides services in reduced cost due to virtualizing all the necessary resources. Even modern business architecture depends upon Cloud computing .As it is a internet based utility, which provides various services over a network, it is prone to network based attacks. Hence security in clouds is the most important in case of cloud computing. Cloud Security concerns the customer to fully rely on storing data on clouds. That is why Cloud security has attracted attention of the research community. This paper will discuss securing the data in clouds by implementing key agreement, encryption and signature verification/generation with hyperelliptic curve cryptography.
△ Less
Submitted 25 November, 2014;
originally announced November 2014.