InfFeed: Influence Functions as a Feedback to Improve the Performance of Subjective Tasks

In most of the classification problems, the real-world data (training and test instances) are not evenly distributed into classes Bengio et al. (2020). As a result, the performance of the model suffers significantly, providing motivation to use pre-trained large-scale models. Despite these large models’ excellent performance, most deep neural architectures are implemented as a black box and lack algorithmic transparency Lipton (2016). Transparency in the method improves the explainability of the model and makes it more trustworthy.

Some previous works attempt to explain the predictions of a model (i.e., why the model takes a particular decision) by perturbing the train instances or locally fitting the model on train data Ribeiro et al. (2016). In addition, to explain the model, the authors in Koh and Liang (2017) formulate influence functions to understand how the model predictions are affected by up-weighting a small amount of training instance loss. The idea is to estimate how much each training sample affects the model’s predictions over the test set. Any training sample that causes the test loss to go up is considered less useful and is down-weighted afterward. Given the efficacy of influence-based data resampling in this work, we set a twofold objective. First we show that influence functions can be passed as a feedback to the model to improve its overall performance. Second, for the purposes of extension of annotated datasets, we show that influence functions can automatically identify those data points whose labels need to be cross-checked (and corrected) by annotators out of the full extension set that have been initially ‘silver’ annotated by some existing model.

Our main contributions to this paper are as follows.

• •

  We propose a framework called InfFeed where we employ the influence function as feedback to adjust the label of a candidate data point based on the labels of its influencers in order to increase the performance of the model.

• •

  We evaluate the proposed framework on six datasets which are on subjective tasks such as hate speech detection, stance classification, irony, and sarcasm detection.

• •

  We observe that our framework results in an improvement of 4%, 3.5%, 3% and 2% F1 score in the model performance over state-of-the-art baselines for hate speech, stance and irony, and sarcasm classification, respectively.

• •

  For the dataset extension exercise, we show that just manually correcting the labels of the data points that impart a negative influence can result in a performance very close to the case where the whole extension set is gold annotated. The reduction is huge since the negatively influencing set is $\sim\frac{1}{1000}^{\textrm{th}}$, of the size of the full extension set.

This, we believe, is a first-of-its-kind approach to use influence functions play the role of a pseudo-annotator deciding whether to update the label of target instances in a text classification model in order to improve its performance over state-of-the-art baselines.

One of the most critical issues with deep learning models is their interpretability Guidotti et al. (2018); Lipton and Steinhardt (2018), and the proneness to learn ambiguous correlations instead of understanding the true nature of the task Sagawa et al. (2020). These two reasons result in poor outcomes on datasets and cannot meet the expectations Gururangan et al. (2018); Jia and Liang (2017); Glockner et al. (2018) resulting in severe biases in model decisions Blodgett et al. (2020); Sun et al. (2019). This further brings down the overall confidence in the technology Ribeiro et al. (2016); Ehsan et al. (2019). Despite great success, the question of “why does the model predict what it predicts?” needs a succinct answer. A satisfactory answer to this question can result in the improvement of the model Amershi et al. (2015), lead to the development of newer perspectives Shrikumar et al. (2017), and benefit users by providing explanations of the model actions Goodman and Flaxman (2017).

Understanding black-box models by approaches like locally fitting a simpler model around the test point Ribeiro et al. (2016) or by perturbing the train point to see how the prediction changes Simonyan et al. (2013), Li et al. (2016), Datta et al. (2016) do not satisfactorily indicate where the model came from Koh and Liang (2017). To answer this question, the influence function Hampel (1974) was introduced; it was a classic technique based on robust statistics through which the learning algorithm can be inspected, and can be traced back to the most influential training data points which impacts the model to predict what it predicts. A simple and efficient methodology was introduced to align and fit the influence function to the machine learning paradigm, which required access to gradients and Hessian-vector products Koh and Liang (2017). It was further demonstrated by Basu et al. (2020) that non-convex and non-differentiable models, which seem to have limited usefulness, successfully provide significant information while approximated by influence function analysis. On linear models, it can be observed that the influence function is useful in – explaining model predictions, tracking and reducing errors in datasets, debugging models, and even fabricating indistinguishable training set impact¹¹1https://christophm.github.io/interpretable-ml-book/. The influence function indicates ‘influential’ training data points during model prediction and has a plethora of applications. The authors in Han et al. (2020) employed them to explain model predictions and uncover data artifacts. They were used by Yang et al. (2020) in order to determine the quality of synthetic training samples within the framework of data augmentation. The authors in Kobayashi et al. (2020) investigated what would happen if they used gradient-based approaches in conjunction with influence functions to investigate training history and test stimuli simultaneously. One of the drawbacks of influence functions is that it is highly compute intensive. To circumvent this problem FastIf Guo et al. (2021), a collection of simple modifications were proposed to significantly improve the runtime for computing influence functions.

Of late, there have been a rising interest in debugging models using explainability techniques Teso and Kersting (2019); Lertvittayakumjorn et al. (2020); Guo et al. (2021); Xu and Du (2020); Nuamah and Bundy (2020); Banerjee et al. (2021). In Rajani et al. (2020), the authors suggest utilizing kNN representations to identify training instances responsible for a model’s predictions and acquire a corpus-level knowledge of the model’s behavior. A recent research Zylberajch et al. (2021) (HILDIF) has sought to use explainability feedback as input to fine-tune the model for the MNLI dataset. Recently, some comparable tests were carried out using image data, randomly flipping two labels using the influence function Hao et al. (2020); Teso et al. (2021); Wang et al. (2018). UIDS by Wang et al. (2020) and RDIA by Kong et al. (2022), can both relabel data points based on influence capability using just numeric attributes. To the best of our knowledge, RDIA is the most recent study that addresses the problem of data relabeling followed by a classification task.  Mozes et al. (2023) tried to incorporate LLM and utilized influence functions to relabel the predictions. There is a major gap between these works and what we can accomplish with the available textual data. Our work differs from these in that it employs influence functions as a pseudo-annotator and leverages the influential instances as feedback to adjust the gold annotation for a target instance, thereby, improving the overall model performance.

In this section, we detail the methodology that we adopt to incorporate the influence function as feedback into the classification model. We also discuss the baselines used in this paper. Our proposals: Our proposals include two systems – System 1 and System 2. While System 1 is the standard classification model, System 2 is our proposal for incorporating the influence function into the System 1. System 1 is the vanilla approach where one usually uses a transformer-based classification model having three divisions of a dataset marked as train $(T_{R})$, valid $V$ and test $T_{S}$. We first train a model with $T_{R}$ and save the model snapshot $\theta$ where the validation loss is minimum and then evaluate the performance using the test data $T_{S}$. As shown in Figure 2 (System 1) the input text (post/tweet etc.) is split into tokens $\{w_{1},w_{2},w_{3}\cdots w_{m}\}$ and is passed through a transformer encoder followed by a softmax layer to make the final prediction.

The method proposed by us is generic in nature. However, to demonstrate the real effectiveness of the approach, we choose datasets that involve subjective tasks. Our datasets are chosen in a way to cover a wide spectrum of problems and comprise both binary and multiclass scenarios. In specific, we focus on four types of subjective tasks – hate speech detection, stance classification, sarcasm, and irony detection. We evaluate our method on state-of-the-art datasets including – (a) HateXplain Mathew et al. (2021) and (b) Davidson Davidson et al. (2017) for hate speech (c) WTWT Conforti et al. (2020) and (d) Mohammad et al. (2016) for stance classification, (e) isarcasm Oprea and Magdy (2020) for sarcasm detection, (f) Van Hee et al. (2018) for irony detection. The basic statistics for each of these datasets are given in Table 1.

We use three different setups in our experiment to observe the importance of increasing data. The setups are as follows – (i) $S_{1}$: Here, we randomly sample 2500 instances from the dataset. Then, we split these into four parts : $T_{PR}$ (1000 instances), $T_{CR}$ (800 instances), $V$ (200 instances) and $T_{S}$ (500 instances). (ii) $S_{2}$: Here we have 6000 randomly sampled instances and the number of instances in $T_{PR}$, $T_{CR}$, $V$ and $T_{S}$ are 4200, 800, 500 and 500 respectively. (iii) $S_{3}$: In this case, the number of randomly sampled instances is 10000. The number of instances in $T_{PR}$, $T_{CR}$, $V$ and $T_{S}$ are 7500, 1500, 500 and 500 respectively. For each setup, we sample the union of $T_{PR}$, $T_{CR}$, $V$ three times and compute the performance. We keep the test set $T_{S}$ fixed across all the setups. We take the average of the three macro F1 scores as the final performance. This result is representative, and the trends remain similar for setups with more than 10000 randomly sampled instances. In the case of the datasets which have less number of instances (less than the total instances in $S_{2}$ but more than $S_{1}$), we oversample the instances in training data ($T_{PR}$) using random selection with repetition.

For the baselines Hao et al. (2020); Rajani et al. (2020); Wang et al. (2020); Kong et al. (2022) also, we have three such setups; however, during training, we merge $T_{PR}$ and $T_{CR}$ to form a single training set. We let the validation ($V$) and test ($T_{S}$) sets remain the same. For the LLM baselines we query the models with each entry from the test set $T_{S}$ and record the classification labels in each case.
Model setup: For System 1 and System 2 (i.e., InfFeed), we have used two models – BERT-base and DistilBERT. During the fine-tuning, we freeze the first nine layers based on the findings in Lee et al. (2019) to limit the amount of computation. This leaves us with approximately 14.7M trainable parameters. In the case of DistilBERT, we freeze the first 4 layers to bring down the overall computation cost. For both models, we consider a maximum of 350 tokens. After parameter tuning, the learning rate is set at $2e-5$, the number of epochs at 12, and the batch size at 64. Further, for InfFeed, the weight decay is set to 0.005, the $k$ in kNN to 100, and the Hessian approximation value to 800.

For Hao et al. (2020), everything else remaining same as System 1, the learning rate has been set to $5e-5$. In the case of this baseline, we treat the hate speech datasets as a two-class classification scenario whereby we merge the ‘hateful’ and the ‘offensive’ classes into a single ‘abusive’ class. Now, during classification, we randomly select 10% of the instances from the entire dataset along with their original labels; we then flip the label for each instance to ‘abusive’ if the original label is ‘normal’ and vice versa. We did the same for the sarcasm dataset. For Rajani et al. (2020), the learning rate and the $k$ in kNN have been set to $5e-5$ and 16, respectively, while everything else remains the same as System 1.

For the baselines UIDS Wang et al. (2020) and RDIA Kong et al. (2022) we use the Newton-CG algorithm Martens (2010) to calculate Influence Functions as mentioned in the paper. For the logistic regression model mentioned in RDIA, we select the regularization term $C=0.1$.
System setup: We run all of the models described in this study on a Windows-based system equipped with 64 gigabytes of RAM, two 24 gigabytes RTX 3090 GPU connected through SLI, and a Ryzen 9 with a fifth generation, twelve-core CPU.

In Table 2, we summarize our main results. As our dataset does not have numerical features, we represent the data points using BERT based pretrained embeddings that are fed to UIDS and RDIA as inputs. The BBU and DB columns show the results using BERT-base-uncased and DistilBERT as the transformer architectures, respectively. All the results are averaged over the three setups $S_{1}$, $S_{2}$ and $S_{3}$. We observe that InfFeed (majority/weighted voting) always outperforms the most competing baselines except for the Mohammad et al. (2016) dataset, where it is the same as the baseline. In all cases where our models win, the results are statistically significant. In general, InfFeed weighted voting is slightly better than majority voting. Further, for both InfFeed models, the DistilBERT architecture performs better than BERT-base-uncased in most cases. For the baselines Hao et al. (2020) and Rajani et al. (2020), the trends are reversed; BERT-base-uncased generally works better than DistilBERT here.
Our models also outperform the LLM based baselines. The largest performance margin is for the iSarcasm dataset with GPT-4 reporting a macro F1 score of 0.541 compared to InfFeed (WV) at 0.635.
Effect of varying data size: Here we report the performance of the best performing model, InfFeed (majority voting) separately for the three setups – $S_{1}$, $S_{2}$ and $S_{3}$. Figure 3, shows how the performance of the model improves as we increase the dataset size. For some datasets, e.g., Davidson et al. (2017) and Conforti et al. (2020), one observes a gain close to 20% as one sweeps from setup $S_{1}$ to $S_{3}$.
Remark: According to the study by Koh and Liang (2017), with $N$ training data points and $P$ parameters, the Hessian matrix computation requires $O(NP^{2}+P^{3})$ operations, which is unacceptably expensive for massive datasets/models. This is the primary reason for the popularity of the FastIf Guo et al. (2021) algorithm which is also what we have used here.
Ablation studies: In order to understand the effectiveness of the influence function as a ‘pseudo-expert’ annotator, we perform two ablation experiments. These are – (a) random flipping and (b) vanilla fine-tuning.
Random flipping: This system uses the same parameters as mentioned in System 1. However, here we randomly flip the labels of some of the training instances (around 5%, which is similar in tune to the number of instances updated on average by InfFeed).
Vanilla fine-tuning: As in System 2, here also we obtain a model $\theta_{A}$ by training it on $T_{CR}$. Now rather than computing influence functions, we fine-tune $\theta_{A}$ using $T_{CR}$.
Subsequently, we use the held-out validation set $V$ and save the new model $\theta_{B}$ where the validation loss is minimum and evaluate the performance with the held-out test set $T_{S}$.
The results from the two ablations are reported in Table 3. For random flipping, in case of the hate speech datasets, there is an average performance drop of almost 20%. For the stance detection datasets, we can see an average 16% drop, while for the irony and sarcasm datasets, the average drops are nearly 13% and 18%, respectively. In vanilla fine-tuning for all the datasets we see an average drop in the range of 2% – 2.5%. Clearly, both the approaches perform worse than InfFeed showing the effectiveness of the influence functions.
Example instances: In Table 4 we show some examples where the incorrect original label gets updated to the correct label based on the votes from the influential instances. This is one of the basic reasons for the better performance of our models.

In this section, we analyze the incorrect predictions from InfFeed and group them into the following types. Here we identify the frequent mistakes made by the model and place them into different categories – (i) Presence of slur words may not always correspond to the text being hateful/offensive. Thus, since the model observes such examples in both classes during training it is sometimes confused during prediction. For instance, in the HateXplain dataset, the text “kevin gates make music for dykes and chubby ni**a” is wrongly marked as hate speech by the model owing to the presence of the word “ni**a”, (ii) When the major text feature appears only once in a sentence, that feature is given higher importance during annotation. For example, in the Van Hee et al. (2018) dataset, the statement “Most of us didn’t focus in the #ADHD lecture” is labelled as situational irony while the correct class should be other verbal irony. The influential instances also support this same label even though they do not have any contextual overlap, (iii) If there are several targets in the dataset, one individual target instance may be taking influence from another target cluster and producing the incorrect collection of influential instances. For example, in the Mohammad et al. (2016) dataset, “Jeb Bush is the only sane candidate in this republican lineup” has ‘Donald Trump’ as the target but is influenced by “Letś hope the VOTERS remember" which has ‘Hilary Clinton’ as the target. This may result in possible incorrect predictions, (iv) If a post’s contextual information and some label-related input are limited, influential instances are often incapable of capturing the latent semantics of the post. They rather pick up cues from the surface structure. As an example, “Thank user for being so entertaining at the Edinburgh signings! You did not disappoint! I made my flight so will have plenty time to read user” is influenced by instances where the words ‘entertainment’ or ‘flight’ appear in an instance, and (v) If the gold annotations are incorrect, influence functions may relate to the wrong instance, resulting in incorrect voting. For example, in the HateXplain dataset, “bare naked islam brought to you by bare naked ladies” is annotated as ‘hate speech’ while it should be actually annotated as ‘offensive’. If this instance is an influencer for a particular target instance then the silver label of the target instance might get incorrectly updated resulting in possible incorrect prediction.

We present InfFeed, which, by leveraging influence as feedback, attempts to simulate a pseudo-expert annotator by updating the label of a target instance. This simple approach results in significantly better performance as compared to the state-of-the-art baselines for a series of classification tasks that are subjective in nature. In the dataset extension setting, we observe that even by manually annotating $\sim\frac{1}{1000}^{\textrm{th}}$ of the full dataset that need to be extended we obtain comparable performance with the scenario where all the dataset to be extended is gold-annotated. In the future, we would like to investigate if this scheme can be effectively used to replace the need for an expert annotator in a real-world deployment scenario through faster computation.

In our research, we responsibly use social subjective data, originally published in another study and used with appropriate permissions. Acknowledging the sensitive nature of this data, we have undertaken diligent steps to maintain ethical standards. Specifically, we employed expert annotators to revisit and correct any potential misannotations, enhancing the reliability of our data. This process reinforces our commitment to upholding stringent ethical guidelines in our research.