SubRegWeigh: Effective and Efficient Annotation Weighing
with Subword Regularization

Our work is in the line of CrossWeigh Wang et al. (2019). CrossWeigh detects annotation errors by performing $K$-fold cross-validation on the training dataset $T$ times and weighs the loss value according to the number of wrong predictions, which degrades the effect of training samples with annotation errors in the NER task. This method requires large computational costs because it requires training $K\times T$ models for detecting annotation errors. Our work is different from CrossWeigh in that we use multiple tokenization candidates to obtain various outputs from one model instead of $K$-fold cross-validation. Compared to CrossWeigh, the proposed method can remarkably reduce the time for detecting annotation errors because we train the model only once and use the output from pseudo-models by inputting various tokenization candidates. Furthermore, our method can use the entire training dataset to make the model because we do not employ $K$-fold cross-validation, which contributes to the more accurate detection of annotation errors.

In addition to methods that pre-assign annotation correctness weights to the training data set, methods have been studied to correct labels in partially annotated training data using a small set of data known to have correct annotations Yang et al. (2018); Mayhew et al. (2019); Ding et al. (2023). In other lines, some methods are developed to infer the correctness of the labels during training. For instance, in the method introduced in Zhou and Chen (2021), multiple models are trained simultaneously with different initial values, and the KL divergence of outputs for the same sentence is added to the loss function. This method can be applied in conjunction with the weighting of training datasets used by CrossWeigh and our method.

For detecting annotation errors in datasets beyond NER, annotation correction methods with distant supervision Mintz et al. (2009) have been studied in the relation extraction tasks Huang and Du (2019); Qin et al. (2018). However, these methods require an additional clean dataset or are not always applicable. Furthermore, methods for correcting POS tagging tasks have also been studied Nakagawa and Matsumoto (2002); Helgadóttir et al. (2014), but these do not apply to tasks other than POS tagging. Maini et al. (2023) introduced a method to analyze the memorization of the model, which can be used to detect training samples with ambiguous labels.

The proposed method exploits the technique of tokenization for the efficient detection of annotation errors. Tokenization Song et al. (2021); Sennrich et al. (2016); Kudo (2018) is widely used in recent natural language processing. This technique reduces the number of unknown words and low-frequency words and limits the number of dictionaries by treating words as a sequence of finer subwords. However, because tokenization is deterministic processed on training data, the subword sequences that appear during inference may not always match the sequences learned during training.

Subword regularization Kudo (2018); Provilkov et al. (2020); Hiraoka (2022) alleviates this problem of deterministic tokenization. This technique generates multiple tokenization candidates for a single text. While subword regularization is typically used during the training step to learn from various tokenizations, some studies use it during inference. Takase et al. (2022) uses subword regularization during inference to ensemble the various outputs of a single model by inputting some different tokenization candidates to the model. Wang et al. (2021a) proposed a method to reduce the effect of tokenization difference. However, there has been no research on using subword regularization during inference to reduce noise in the training data.

Subword regularization using a unigram language model Kudo (2018) is a method to make the model more robust by sampling tokenization candidates using the unigram probability of tokens. For the cases where the model does not employ a tokenizer with the unigram language model like Byte Pair Encoding (BPE) Sennrich et al. (2016) used in RoBERTa Liu et al. (2019) or WordPiece Song et al. (2021) used in BERT Devlin et al. (2019), we can use BPE-Dropout Provilkov et al. (2020) and MaxMatch-Dropout Hiraoka (2022) for the subword regularization, respectively. BPE-Dropout randomly avoids merges during the combination process to obtain multiple subword sequences. MaxMatch-Dropout randomly rejects the longest match candidates for each word to obtain multiple subword sequences.

Inspired by CossWeigh Wang et al. (2019), we first create the scouting model $M$ (see the left top of Figure 1). This model is used to scout the correctness of the annotated label in the training dataset in the next step. Let $(X,Y)$ be a pair of the text and the label corresponding to a single training sample. When the text is composed of $I$ words, $X=x_{1},...,x_{I}$. $Y$ is a single value in the text classification (e.g., positive and negative labels for sentiment classification) or a sequence of labels $Y=y_{1},...,y_{I}$ in sequential labeling tasks such as NER.

In this step, we train the scouting model $M$ in the usual manner of training or fine-tuning for general NLP datasets. The single input text $X$ is tokenized into a sequence of subwords $X^{\prime}=s_{1},...,s_{J}$ that is composed of $J$ subwords¹¹1The numbers of words $I$ and subwords $J$ are not necessarily matched depending on the tokenization methods. In the case of sequential labeling tasks with $I\neq J$, we aligned the subwords and labels following the existing literature Liu et al. (2019); Yamada et al. (2020).. After tokenizing all texts in the training dataset, we train the scouting model $M$ with the tokenized texts and labels. Note that we use the deterministic tokenization here (e.g., the default encoding of the BPE tokenizer or the $1$-best tokenization with the Viterbi algorithm for the unigram language model-based tokenizer). In other words, we do not use subword regularization (i.e., stochastic tokenization) for the training of the scouting model $M$.

In the second step, we use the trained scouting model $M$ to scout the correctness of the annotated label in the training dataset (see the right part of Figure 1). We assume that the trained scouting model $M$ can correctly predict the labels even for the differently tokenized input if the label is not wrongly annotated. Specifically, the input text $X$ is tokenized into $K$ different tokenization candidates $\{X^{\prime}_{1},...,X^{\prime}_{k},...,X^{\prime}_{K}\}$. Then each tokenization candidate $X^{\prime}_{k}$ is fed into the trained model $M$ and we obtain the corresponding prediction $\hat{Y}_{k}$²²2As well as $Y$, $\hat{Y}_{k}$ can be single or multiple labels depending on target tasks. After the inference, we evaluate the output $\hat{Y}_{k}$ with the gold annotation $Y$. This process can be regarded as the “mistake-reweighing” process of CrossWeigh with $K$ pseudo-models, which is realized with the $K$ different tokenization candidates for the single input text and the single model.

We can obtain various tokenization candidates using a technique of subword regularization. When we use BPE or WordPiece, BPE-Dropout and MaxMatch-Dropout can be used, respectively. When one uses a unigram language model-based tokenizer, we can also use $N$-best tokenization instead of subword regularization.

We consider it important to make a variety of the $K$ tokenization candidates to weigh the correctness of the annotation from diverse aspects. To avoid using similar subword sequences as the inputs for the model $M$, we select $K$ much different candidates from a large number $N$ of tokenization candidates. We propose three options of the way to select such $K$ tokenization candidates as follows.

To reduce the effect of training samples with annotation errors, we calculate weights for each sample depending on the results of the inference step (see the left bottom of Figure 1). Concretely, we calculate the weight $w$ corresponding to the training sample $(X,Y)$ as the follows:

where $\hat{Y}_{k}$ is the output of the scouting model $M$ when inputting the tokenization candidate $X^{\prime}_{k}$. If all predictions were different from the original label $Y$, the weights would be 0 and the data would not be used for training. To avoid this, we use a pre-defined minimum weight $w_{\mathrm{min}}$. We use this minimum weight because Wang et al. (2019) reported that it is better to give a small weight to samples with annotation errors than not to use them.

Using the training dataset with weights calculated in the above section, we train the final model $M^{\prime}$ that is independent of $M$. Following CrossWeigh Wang et al. (2019), the loss value $L_{\mathrm{weighted}}$ for the training sample $(X,Y)$ with the weight $w$ is defined as the follows:

where $L_{\mathrm{original}}(\cdot)$ is the loss function used in the default training.

We briefly overview the dataset used in the experiments here. The detailed information can be found in Appendix B.

We compared the following five methods including three options of SubRegWeigh explained in §3.2.

For the baselines, we selected Vanilla and CrossWeigh. Vanilla is the final model that is trained on the original training split without any annotation weighing. CrossWeigh is the final model that is trained with the dataset weighed by the official code⁴⁴4https://github.com/ZihanWangKi/CrossWeigh of the existing work Wang et al. (2019). Although CrossWeigh is specialized to NER, we utilize it for the text classification setting without Entity Disjoint, which excludes sentences containing named entities in the validation data from the training data in each fold. proposed in Wang et al. (2019). This is because Entity Disjoint is unavailable in the text classification.

For the proposed method, we evaluated the three options introduced in §3.2, which are represented as SubRegWeigh (Random), SubRegWeigh (Cos-Sim), and SubRegWeigh ($K$-means), respectively. We used these proposed methods to weight the training dataset and evaluate the final model trained with this weighted dataset as well as other baselines.

As the pre-trained language model, we employed $\mathrm{RoBERTa_{LARGE}}$ for both NER and text classification. Besides $\mathrm{LUKE_{LARGE}}$ Yamada et al. (2020) is used for the NER setting because LUKE is an architecture for entity-related tasks. In this experimental setting, we used the same backbone model for both the scouting model and the final model⁵⁵5§5.3 discusses the case where using different pre-trained models between the scouting and final models.. For example, when RoBERTa is used in the scouting model, the final model will also use RoBERTa. The detailed model settings such as hyperparameters are provided in the Appendix C.

CrossWeigh has three hyperparameters: the number of folds in mistake estimation $K$, the number of iterations of mistake estimation $T$, and the weight scaling factor $\epsilon$. We use $K=10$, $T=3$, and $\epsilon=0.7$, respectively. BPE-Dropout Provilkov et al. (2020) was used to obtain the multiple tokenization candidates for SubRegWeigh because both RoBERTa and LUKE employ BPE for their tokenizers. BPE-Dropout has a hyperparameter $p$, where a higher $p$ results in finer segmentation (i.e., more different from the original segmentation). In the experiments, $p$ was set to $0.1$ unless otherwise noted. Additionally, by default, the hyperparameters for SubRegWeigh were set to $N=500$, $K=10$, and $w_{\mathrm{min}}=1/3$.

We evaluate the methods from the viewpoints of speed and performance. On the speed side, we measured the time for detecting the annotation errors and generating the weighted data for the training dataset in NER⁶⁶6We measured the processing time with Xeon Platinum 8167M + NVIDIA V100 for RoBERTa and Xeon Gold 6230R + NVIDIA A100 for LUKE.. On the performance side, we used the weighted data to train the final models and measured the $F_{1}$ score in NER and the accuracy score in text classification for the outputs of the models on each test dataset.

The results of our experiments are shown in Table 1. We reported the averaged $F_{1}$ scores and accuracy scores over the five independent runs and the standard deviations.

We verified whether the proposed method can accurately detect annotation errors in the dataset. We evaluate this capability using a modified dataset with some labels flipped to incorrect labels artificially.

Assuming that the original label should be correctly annotated, we replaced 10% of the labels in the CoNLL2003 training dataset with different labels that do not conflict with the IOB2 format (the detail of the replacement is provided in the Appendix D). As a result, 3,329 sentences including pseudo-incorrect labels, referred to as replaced, and 5,356 sentences without replacements of annotations, referred to as unreplaced are generated by this method from 8,685 sentences.

We assign weights to this modified dataset using CrossWeigh and SubRegWeigh⁷⁷7The hyperparameter settings are the same as those described in Section §4.3 and Appendix C.. Then we compared the average weights between the replaced and the original samples to confirm whether annotation error detection is correct. Here, we calculate the weights as (number of the same inferences as labels) / (number of inferences) instead of $w$ in Eq. (1). This is because we consider that the calculation using the minimum weight $w_{min}$ and the weight correction in CrossWeigh could have a negative effect on the fair analysis.

In Table 2, $\overline{w}_{incor}$ is the averaged weights assigned to the samples with pseudo-incorrect labels, where we expect them should be lower weights as incorrect labels. Similarly, $\overline{w}_{cor}$ is the averaged weights assigned to the original labels, where higher weights should be assigned as the correct labels.

From the results in this table, SubRegWeigh ($K$-means) assigns the most distinct weights between the replaced and the unreplaced data. Additionally, all methods assign weights approximately 100 times lower to replaced data than unreplaced data. This indicates that all methods can detect errors effectively. The lowest average weight for unreplaced data is CrossWeigh. This is because CrossWeigh infers only 3 times while SubRegWeigh infers 10 times. Even one incorrect prediction reduced the weight to 2/3. Among SubRegWeigh methods, Cos-Sim assigns lower weights to sentences without pseudo-incorrect labels, but these do not mean lower performance improvements compared to other methods, especially SubRegWeigh (Random). According to this, if the weights of sentences containing errors are low, it does not matter if the average weight is slightly lower.

In addition to the investigation of weights, we also analyze the obtained performance by training final models with the modified dataset. For the training of the final models, we used the weights calculated in the manner of Eq. (1). The results are shown in the right two columns of Table 2. Because the training data is noisy, the entire scores are worse than the ones in Table 1. However, the total tendency of the performance is similar to the results in §4. From the entire results, we cannot find a clear relationship between the averaged weights and the final performance.

In the proposed method, $K$ is an important hyperparameter affecting both speed and performance. We investigated the difference in the time for the annotation weighing and the $F_{1}$ score of the final model with three options of the number of tokenization candidates $K$ in the NER dataset. We examined the difference in the range of $K={10,50,500}$ with RoBERTa. The dropout rate for BPE-Dropout was $p=0.1$. Since $K$-means and Cos-Sim are techniques to reduce the variations in subword segmentation, a large value like $K=500$ makes them almost indistinguishable from the Random method. Therefore, only the Random method was investigated for $K=500$.

The results are shown in Table 3. Comparing the results between the ones with $K=10$ and $K=50$, one can see that $K$ does not have a large effect on the $F_{1}$ scores. However, the larger $K$ remarkably increases the time for the annotation weighing. For $K=500$, the $F_{1}$ score improved compared to Random with $K=10$ and $K=50$ on the CoNLL and CoNLL-CW datasets. This suggests that using a large number of subword sequences can effectively weigh annotation errors. However, such much large $K$ damages the speed of annotation weighing, making it a trade-off between the speed vs. the performance. While increasing $K$ to 500 in the Random method showed performance improvement, $K$-means exhibited the highest performance even with smaller $K$. This indicates that $K$-means can select sufficiently diverse subword sequences even with a small $K$.

In the experiment shown in §4, we used the same backbone for the scouting model $M$ for annotation weighing and the final model $M^{\prime}$. Herein, we investigated the performance difference when using different backbones in the NER dataset. Specifically, we examined the effect in $F_{1}$ scores when $M$ was RoBERTa and $M^{\prime}$ was LUKE. Similarly, we examined the case where $M$ was LUKE and $M^{\prime}$ was RoBERTa. Both models were trained with the hyperparameters explained in Table 7. We used $K$-means for selecting the tokenization candidates because it showed the best performance in the main results (§4).

The results in Table 4 indicate that, in most cases, the data weighted by the same model tends to achieve higher $f_{1}$ scores. Although we can save time by preparing the weighed dataset with a single model and reusing it to train models with other backbones, this observation suggests that we should prepare the weighed dataset depending on the backbone used for the final models to obtain performance improvement. This suggestion also supports the importance of the fast method of annotation weighing.

Several examples of the CoNLL-2003 training dataset weighed by each method are shown in Table 5. All methods successfully assigned low weights for clearly incorrect sentences like the top example in the table. However, for ambiguous labels like the example at the bottom of the table, CrossWeigh tended to assign high weights, whereas SubRegWeigh more frequently assigned low weights. Additionally, we discovered a specific weakness of SubRegWeigh: it tends to assign lower weights to sentences composed entirely of uppercase letters (see the example in the middle of the table). This is because only a few uppercase-only subwords are included in the tokenizer’s vocabulary, and even with a low $p$, subword regularization significantly changes the tokenization candidates for uppercase-only sentences, leading to incorrect inferences. We believe this issue is not unique to our method but rather a general problem with subword regularization, which we plan to investigate further in future research.