Seeing Like an AI: How LLMs Apply (and Misapply) Wikipedia Neutrality Norms

To test whether LLMs could apply Wikipedia’s NPOV policy, we conducted detection and generation experiments. In both experiments, LLM decisions and edits were compared against an existing corpus of NPOV edits by Wikipedia editors, also known as Wikipedians. In the detection experiment, we used ChatGPT 3.5, Mistral-Medium, and GPT-4 to classify whether edits violated NPOV, varying the amount of policy detail and example decisions provided. In the first generation experiment, we used GPT-4 (using both Zero-Shot and Constitutional AI approaches) to neutralize NPOV-violating edits. We used computational metrics to compare the LLM-neutralized edits to the Wikipedian-neutralized edits. In the second generation experiment, humans gave masked evaluations of LLM and Wikipedian rewrites. We also conducted a qualitative analysis of LLM vs. Wikipedian rewrites to better understand differences in how each neutralizes text. We summarize our main findings and implications below:

1. (1)

   Large language models largely failed at neutrality detection. Across models, prompts, and optimization strategies, performance was low (64% accuracy for the best prompt; the dataset was balanced so random chance accuracy is 50%). Analysis of LLM rationales and errors suggested LLMs relied (sometimes to a fault) on simple heuristics like the presence of a highly subjective adjective. Perhaps modern models still struggle with applying subtle rules to real-world cases without task-specific training.

2. (2)

   Models exhibited biases that persisted across prompts. ChatGPT 3.5 over-predicted edits as biased, and Mistral-Medium did the opposite. GPT-4 was balanced. Pretrained LLMs may internalize distinct priors about what constitutes neutrality that persist across prompts. Understanding model idiosyncrasies is crucial. It is not clear that, when a new model appears on the scene, we will know what its priors are.

3. (3)

   LLMs applied rules in different ways than expert editors. Computational experiments showed LLMs generally did remove words that Wikipedia editors removed in the generation phase. But LLMs also made many other changes. Crowdworkers also rated Constitutional AI (CAI)-generated rewrites as adding unnecessary information more frequently than Wikipedian rewrites. In other words, LLM editors are high-recall but low-precision. And while Wikipedians make more removals than additions, LLM editors do the opposite. One risk for large-scale deployment of models within communities is that they may follow rules in different ways than community members would.

4. (4)

   Crowd-workers preferred LLM neutralizations to Wikipedia-editor neutralizations. Given the finding that LLMs generally made more changes than Wikipedia editors, we conducted human experiments to understand if these generations are in fact preferred to Wikipedian neutralizations. We find that crowd-workers prefer AI edits over human edits on both fluency (61% of choices) and bias reduction (70% of choices). Instruction-tuned models may be highly capable of applying rules in ways that resonate with a broader public, even if these applications differ from community experts.

5. (5)

   Qualitative analysis showed LLMs are ‘NPOV+’. To reconcile LLMs’ low detection performance with high generation evaluations, we conducted a qualitative analysis. Despite being instructed to make only minimal (NPOV) changes, we find LLMs made many additional grammatical and stylistic edits that may have influenced participants’ judgments. This maximalist editing approach could (1) increase labor costs as moderators may need to check for AI hallucinations and (2) reduce editor agency due to extensive rewrites. But in other cases, AI models arguably applied NPOV more faithfully than human editors (as judged by the authors), possibly due to varying community norms on what would constitute neutrality. These findings highlight tradeoffs in general-purpose LLM generations.

Our study has practical implications for Wikipedia, model builders, and perhaps for policy. For Wikipedia, LLMs have low NPOV-violation detection accuracy but we see potential in generating initial drafts of neutral content, with the caveats discussed above. We also found that LLMs applied rules differently from Wikipedians yet were preferred by laypeople, indicating a potential tension between reader and contributor satisfaction. For model builders, we suggest several actionable steps to improve detection and generation abilities: Retrieval-augmented generation (e.g., incorporating Wikipedia Talk page conversations to simulate editors’ community knowledge) or multi-agent systems (irving_ai_2018, ; khan_debating_2024, ) (simulating Wikipedia-style debates) may enhance detection capabilities. Fine-tuning models on expert feedback can increase alignment with domain specialists. Refining Constitutional AI to emphasize preserving original text may improve revision precision. For policy, our findings highlight the limitations of a “set and forget” approach for rules and AI systems. Human oversight and stakeholder input are required to prevent LLMs from overstepping instructions, as LLMs may generate content appealing to the general public but not to the intended community.

Wikipedia is one of the most visited websites in the world (semrush_top_nodate, ). It is a widely-regarded success in peer production. It is well studied in CSCW and beyond (nagar_what_2012, ; matei_wikipedias_2011, ; bryant_becoming_2005, ; reagle_is_2007, ; kittur_harnessing_2008, ; swarts_collaborative_2009, ; viegas_studying_2004, ). To support a crowd-sourced encyclopedia editable by anyone, Wikipedia developed many policies, processes, and norms (butler_dont_2008, ). “Neutral Point of View” (NPOV) is one of the most central. For example, NPOV is listed first among the three core content policies (wikipedia_core_2023, ). According to Wikipedia.org (wikipedia_wikipedianpov_2024, ):

“All encyclopedic content on Wikipedia must be written from a neutral point of view (NPOV), which means representing fairly, proportionately, and, as far as possible, without editorial bias, all the significant views that have been published by reliable sources on a topic.”

Jimmy Wales (a co-founder of Wikipedia) has said (wales_jimmy_2006, ) (nagar_what_2012, , p. 5):

“So how do we do this? …How does it work? …So the biggest and the most important thing is our neutral point of view policy. This is something that I set down from the very beginning, as a core principle of the community that’s completely not debatable.”

NPOV is crucial for Wikipedia’s content quality. Wikipedia’s iterative revision process improves neutrality over time (greenstein_experts_2018, ). This is due in part to the role of NPOV in defending against disinformation, low-quality information (steinsson_rule_2024, ; mcdowell_it_2020, ), and delineating fringe points of view (steinsson_rule_2024, ). The result is (1) a public perception that Wikipedia editors are committed to unbiased information and (2) increased reader trust (elmimouni_why_2022, ).

Because of NPOV’s importance, Wikipedia goes to great lengths to codify exactly what a “neutral point of view” means. Specifically, NPOV is described by a set of principles²²2Avoid stating opinions as facts; Avoid stating seriously contested assertions as facts; Avoid stating facts as opinions; Prefer nonjudgmental language; Indicate the relative prominence of opposing views. (wikipedia_wikipedianeutral_2024-1, ). These principles are expounded and clarified in numerous FAQs, tutorials, and examples. Editors are trained in NPOV before they can edit. And if an article violates NPOV, then it is flagged as “NPOV Disputed” so that an editor can bring it into compliance. As a community norm, NPOV is clearly defined and often invoked in day-to-day community activity. The ‘rules’ are clear.

However, despite its clear articulation, NPOV has always been complicated in practice. Reagle claims that “in the Wikipedia culture, the notion of ‘neutrality’ is not understood so much as an end result, but as a process”  (reagle_is_2007, ). Wikipedia articles often go through many rewrites, with debates occurring as to what constitutes neutrality. For example, over three years the Wikipedia entry for ‘clean coal technology’ had 39 distinct facts and 142 different rewrites of these facts  (swarts_collaborative_2009, ). Norm-consistent Wikipedia content is often the result of conflict, coordination, and deliberation—more complex than lone editors applying rules in isolation (kittur_he_2007, ; swarts_collaborative_2009, ; nagar_what_2012, ; kittur_harnessing_2008, ; viegas_studying_2004, ). Past work (kittur_harnessing_2008, ) estimated that 40% of Wikipedia edits are dedicated to coordination between Wikipedians (e.g., achieving consensus). And despite NPOV being 5 principles, there are many pages and debates dedicated to this topic (matei_wikipedias_2011, ).

Wikipedia’s NPOV policy is an interesting test case of general-purpose LLM abilities because of a tension: It is both clear and nuanced. On one hand, NPOV is clearly defined and documented. In theory, augmenting LLMs with the text of NPOV guidelines should enable LLMs to apply these guidelines effectively. But even though NPOV is clear in theory, it is complex in practice. To what extent, then, can general-purpose LLMs—absent any task-specific training—be steered to follow nuanced community guidelines such as NPOV?

Researchers have developed algorithms to understand and ameliorate problems with Wikipedia article content (e.g., (nicholson_measuring_2021, ; berube_wiki-gendersort_2020, ; wang_assessing_2020, ; sathe_automated_2020, ; halfaker_ores_2020, ; schmahl_is_2020, )). Most relevant to the present work, several papers specifically tested whether models can neutralize NPOV-violating edits. Recasens et al. (recasens_linguistic_2013, ) experimented on a subset of NPOV-violating edits where the bias was due to one word. Both humans and their system struggled to guess the biased word (37% and 34% accuracy, respectively). Building on this result, Pryzant et al. (pryzant_automatically_2020, ) collected a similar dataset of NPOV-violating edits and their corrections called the Wikipedia Neutrality Corpus (WNC). We use this dataset in our work. Both projects (pryzant_automatically_2020, ; recasens_linguistic_2013, ) conducted experiments on a subset of edits in which the bias was attributable to only one word. With the WNC corpus, a sequence-to-sequence system was trained to detect the NPOV-violating word and then predict its replacement, achieving 46% generation accuracy (measured as the proportion of generations that matched Wikipedia-editor rewrites).

Our study departs from these two studies in two ways. First, we conduct experiments on a more representative set of NPOV-violating edits (i.e., not the one-word subset), which is arguably a harder task. Second, we test whether general-purpose models, absent any task-specific training, can perform this task. Both Recasens et al. and Pryzant et al. show that applying NPOV in a way that matches how expert editors behave is difficult for both laypeople and task-specific NLP systems. In our work, we can answer the question: How well can new generations of general-purpose LLMs neutralize text?

Wikipedia already uses automated moderation tools (wikipedia_wikipedia_2024, ), and our results can inform the benefits and risks of using general-purpose LLMs. Past and current services include WikiTrust (wikipedia_wikipedia_2024-1, ), an extension that helped editors detect untrustworthy edits. The ORES project used machine learning to rank the usefulness of edits (halfaker_ores_2020, ). The Automod project was an abuse detection model (rawat_automatic_2019, ). More recently, Wikipedia editors have debated the utility of ChatGPT for generating Wikipedia edits (harrison_should_2023, ). By systematically evaluating NPOV detection and generation from general-purpose LLMs, our results can add evidence to this debate, possibly informing community practice.

Given how much content there is to moderate, platforms often employ automatic content moderation tools such as Reddit’s AutoMod (jhaver_human-machine_2019, ). Because general-purpose LLMs show remarkable zero-shot (gilardi_chatgpt_2023, ) and few-shot (brown_language_2020, ) capabilities even without fine-tuning (i.e., training the model on task-specific data), it is reasonable to think general-purpose LLMs would be effective content moderators. The zero-shot and few-shot capabilities of LLMs are useful, since fine-tuning requires resources (compute, instances) that some communities may not have. Hence, online communities may rightfully wonder how well these off-the-shelf LLMs can apply their rules for online moderation. Our work engages with this question.

Several studies have explored whether general-purpose LLMs can apply highly specific sets of online community rules without fine-tuning. One study (kolla_llm-mod_2024, ) tested whether LLM moderators can detect rule-violating posts across nine subreddits when the rules of these subreddits are provided to the LLM. Their system achieved a high true-negative rate (92%) but a low true-positive rate (43%). A similar study found that ChatGPT 3.5 was 64% accurate in predicting subreddit moderation decisions (kumar_using_2023, ). Cao et al. tested how well LLMs could detect rule violations of r/AskHistorians, with precision and recall showing variance depending on the specific rule (cao_toxicity_2024, ). They also asked moderators what would be an acceptable precision and recall of models for the models to be useful. In general, moderators indicated models should have recall and precision above 70%. Although not subreddit-specific, Barbarestani et al. found ChatGPT largely agrees with both crowds and experts when detecting inappropriate words in Reddit comments (barbarestani_content_2024, ). However, when considering errors, they found that differences in subjective interpretations of appropriateness accounted for 41% of the disagreement amongst experts but 69% of the disagreement between experts and ChatGPT. This may suggest that some familiarity with the topic is required to apply nuanced guidelines reliably.

Our study complements and extends existing research on using general-purpose LLMs for online community content moderation. It complements prior work by testing if general-purpose LLMs can effectively apply Wikipedia NPOV. NPOV is both intrinsically important (being a central norm to one of the most visited websites) and theoretically interesting (since it is highly documented while also requiring nuanced reasoning to apply). Our study extends these works, too. In addition to detection, we focus on how well LLMs can generate content consistent with a community’s rules. As LLM usage grows, understanding the implications for generation within communities is crucial. Because LLMs are explicitly trained for natural language generation, they may be more effective at generation than detection. We conduct both computational and human subject experiments to measure generative quality.

The data for our detection and generation experiments come from the Wikipedia Neutrality Corpus (WNC) (pryzant_automatically_2020, ). The WNC is a collection of edit pairs: A biased edit (an edit flagged by an editor for violating the NPOV) and a neutral edit (the rewrite an editor made to ensure NPOV compliance). While NPOV is a set of multiple rules, the WNC consists of edits that violated a particular subset of NPOV—biased language (framing, epistemological, and demographic bias). This stipulation is useful for testing the abilities of LLMs. Unlike other NPOV rules (e.g., those concerning citations), a violation of neutral language does not require access to external information. For detection experiments, we use the subset of these edits where the data also has tags for which topic an edit belongs to (Appendix for Table 6 topic counts).

This WNC was collected by crawling Wikipedia revisions between 2004 and 2019. In an initial pass, Pryzant et al. (pryzant_automatically_2020, ) filtered for revisions where editors provided a justification indicating an NPOV violation. In subsequent filtering passes, Pryzant et al. (pryzant_automatically_2020, ) excluded revisions below a minimal and above a maximal number of characters and applied additional rules to maximize the probability that the change was related to bias in particular (e.g., excluding revisions that involved references).

We conducted a multi-model prompt experiment ($N$ = 5,348 annotations) where the task was to classify if a given Wikipedia edit was biased or neutral. Our two experimental factors were (Factor 1) the definition given to an LLM on what constitutes neutrality, and (Factor 2) whether or not we provided examples (i.e., few-shot or zero-shot). The rationale for varying definitions is that we are interested in whether augmenting LLMs with community-specific definitions of neutrality increases adherence to community-specific norms. The rationale for our second factor is to test whether providing example decisions from a community helps models adhere to these community norms. Appendix B has all the prompts we used in this paper.

• Factor 1: Definitions

  • –

    Minimal definition (Minimal)

  • –

    Wikipedia’s Neutral Point of View definition (NPOV)

  • –

    Wikipedia’s definition focused on neutral language (NPOV-Scoped)

• Factor 2: Examples

  • –

    No examples: Zero-shot (ZeroShot)

  • –

    With examples: 10-shot examples from the same topic (10-Shot)

We employed three models: gpt-4-0125-preview, gpt-3.5-turbo, and mistral-medium-latest. We refer to these as GPT-4, ChatGPT 3.5, and Mistral-Medium from here on. We chose these on the basis that—at the time of this study—the first is a state-of-the-art LLM, the second is commonly used (largely due to speed and cost), and the third is from a popular open-source developer. For each of the 18 (model $\times$ definition $\times$ example) combinations, we classify a balanced (between biased edits and neutralized rewrites) sample of 300 edits stratified by topic, yielding 5,400 attempted classifications. For each classification, we instruct the model to return the correct label (‘neutral’ or ‘biased’), a rationale, and the policy violated (if the prompt contains policies). Of 5,400 attempted annotations, 5,358 yielded compliant annotations (i.e., one of the two classes). We classified this number of edits due to API costs; we wanted to use more expensive state-of-the-art models such as GPT-4. Confidence intervals for quantities of interest are narrow.

All models performed poorly and prompts made little difference (Table 1; 1(a)). Across all conditions, the accuracy was 0.58. Averaging across models, ChatGPT 3.5 performed the worst with accuracy at 0.55 and GPT-4 performed the best at 0.61. The top combination was GPT-4 with a zero-shot NPOV prompt (0.63). For each of the models, there were no statistically significant differences in accuracy between prompt conditions, based on p-values from two-tailed permutation tests. Models were more accurate for biased edits (0.63, 95% CI = [0.61, 0.65]) than neutral edits (0.53, 95% CI =[0.51, 0.55]), $p<0.001$ from two-tailed permutation tests.

We ran a logistic regression (Appendix Table 7) modeling accuracy as a function of definitions, few-shot or zero-shot, model, topic, edit word count, and normalized edit distance tertile. Normalized edit distance is the edit distance between NPOV-violating and compliant edits, divided by the longer string’s length. Higher values indicate that it took more characters to bring the NPOV-violating edit into compliance, so we may expect AI systems to find these edits easier to detect. Compared to ChatGPT 3.5, GPT-4 (OR = 1.25, 95% CI = [1.09, 1.43]) and Mistral-Medium (OR = 1.15, 95% CI = [1.01, 1.31]) were more accurate. Definitions and few-shot examples were not significant. Edit word count was weakly associated with higher accuracy (OR = 1.10, 95% CI = [1.02, 1.18]). Medium (OR = 1.86, 95% CI = [1.60, 2.16]) and high (OR = 1.43, 95% CI = [1.21, 1.70]) edit distance had higher accuracy than low edit distance, suggesting models were better at detecting violations that required more characters to correct. We also found that providing examples tempered ChatGPT 3.5’s predictions. Across zero-shot conditions, ChatGPT 3.5 predicted that 83% of classifications were biased but across few-shot conditions, ChatGPT 3.5 predicted 59% were biased, two-tailed permutation test $p<0.001$.

We experimented with more advanced reasoning techniques and LLM self-optimizations (see Appendix B for these LLM-assisted prompts). First, we selected our top model (GPT-4) and the NPOV and NPOV-Scoped prompts to use with chain-of-thought (CoT) reasoning and DSPy’s ‘BootstrapFewShot’ module. In CoT, a model is instructed to reason step by step to get to its answer, which can improve performance (wei_chain–thought_2023, ). DSPy (khattab_dspy_2023, ) is a state-of-the-art framework for self-improving prompt pipelines. In DSPy’s BootstrapFewShot module, the LLM will learn rationales for labeled instances in a training phase. Then on the test set, instead of simply seeing an example and a label (standard few-shot), the LLM will also see these ‘bootstrapped’ rationales when making classifications. For both prompts, we tested 300 edits with a 70-30 train-test split and 20-shot bootstrapped-rationales. (We doubled the number of examples from our 10-shot experiment to amplify the effect of LLM-generated rationales.) The 20-shot augmented NPOV-Scoped prompt achieved 64% accuracy on the held-out set (+1% higher but statistically indistinguishable from the best unaugmented prompt); the 20-shot augmented NPOV prompt achieved 62% accuracy. We also experimented with an LLM optimizing its prompt on its own. To do this, we used the COPRO module from DSPy (khattab_dspy_2023, ). Briefly, a model is initialized with an initial minimal prompt, and then in a training phase the model repeatedly generates and refines the prompt based on performance on a subset of examples. We similarly used 300 examples with a 70-30 train-test split. The top AI-generated prompt from the training phase achieved a test-set accuracy of 61%. For the rest of the detection analysis, we analyze the results from our original experiments.

We based our evaluation on a balanced dataset of neutral and biased edits. In real-world applications, models would encounter unbalanced data. We examined the error distribution of different models. We find that not all models failed the same way (2(a) and 2(b)). ChatGPT 3.5 was far more likely than other models to predict that edits were biased when they were neutral. Mistral-Medium, on the other hand, erred in the opposite direction. It over-predicted neutral edits. GPT-4 was balanced. We assessed the statistical significance of these patterns with two-tailed binomial tests (Appendix C.1; 2(b)), rejecting a null hypothesis of balanced errors for ChatGPT 3.5 and Mistral-Medium but not GPT-4. Note that prompts and specific edits were held constant, so these differences in model predictions arise from the model itself. Our results suggest that prompting may not be enough. If models have idiosyncrasies, these may persist through different prompts.

We calculated the edit-level probability of correct classification (POCC) to identify which edits are easier to detect (Figure 3). POCC is the probability of a model correctly classifying an edit across all models and prompts. POCCs were bimodal (Figure 3), indicating that edits tend to be either easy or hard to classify.

We conducted a qualitative analysis of ‘easy’ vs. ‘hard’ biased edits (i.e., top or bottom POCC quartile). Top-POCC biased edits tend to have some highly subjective word that alerts models these edits are biased. See below (emphasis added).

• •

  “one of the central characters of the novel, akili kuwale, provides a brilliant demonstration of this change and its implications, together with excellent characterization.”

• •

  “environmentalists complain that before shipbreaking began there in june 1983 the beach at alang was pristine and unspoiled.”

• •

  “colchester has a proud tradition of its citizen volunteers serving in the territorial army.”

By contrast, low-POCC biased edits generally do not contain an overtly subjective adjective.

• •

  “the bill protects americans against discrimination based on their genetic information when it comes to health insurance and employment.”

• •

  “confucianism (; pinyin: rxu ; literally means ”the school of the scholars”, see also names for confucianism for details) is an east asian ethical and philosophical system derived from the teachings of the early chinese sage confucius.”

• •

  “they can see far over the great plains of illinois and across lake michigan on a clear day.”

We examined if interpretable features of LLM explanations correlate with accuracy (Appendix C.2 for variables and standardized logistic regression coefficients). We focused on three features:

• •

  Complexity: (e.g., word count, LIWC  (pennebaker_development_2007, ) cognitive processing words, conjunctions (li_detection_2019, ))

• •

  Prompt and rule reference: (e.g., SBERT (reimers_sentence-bert_2019, ) semantic similarity to prompt, NPOV-related words)

• •

  Subjective language (number of LIWC emotion words)

Increased word count (OR = 0.80, 95% CI = [0.74, 0.86]) and more conjunctions (OR = 0.91, 95% CI = [0.85, 0.96]) correlates with lower accuracy. Higher semantic similarity to the prompt (OR = 1.15, 95% CI = [1.07, 1.23]), NPOV-related words (OR = 1.12, 95% CI = [1.05, 1.20]), and emotion words (OR = 1.09, 95% CI = [1.03, 1.15]) correlate with higher accuracy. These results suggest that the depth of reasoning alone does not predict accuracy. Instead, LLMs performed better when their rationales incorporated specific instructions and referenced relevant content. However, due to a low pseudo-$R^{2}$, we consider these findings preliminary.

We next conducted a TF-IDF logistic regression (Figure 4) to understand what specific words in an explanation correlate with accuracy. We used an 80-20 train-test split over {Explanation, Is Accurate} tuples and 5-fold cross-validated grid search to tune hyperparameters. The best model achieved 0.8 accuracy, 0.8 precision, and 0.79 recall. Referencing highly subjective words such as ‘sadly’, were associated with accuracy. Occasionally, the model also flagged edits as biased when they contained a subjective-sounding word even when the edit was itself not biased. As an example, models misclassified the following edit as biased (emphasis added):

the advertisement called obama a hypocrite for not supporting armed guards in schools while noting that the children of the us president receive special protection by armed agents of the us secret service.

In this edit, the writer is referencing the advertisement—but LLMs flagged the edit as biased even though it was factually recalling what an ad was saying. Here is GPT-4’s explanation:

The edit uses loaded language by calling Obama a ‘hypocrite’ which is an opinion stated as a fact, and also uses judgmental language which violates the neutral point of view policy.

These dynamics suggest LLMs relied (sometimes to a fault) on simple heuristics like the presence of a highly subjective adjective. It is worth noting that LLMs did effectively identify the subjective-sounding part of the text in many of these false positive cases. This suggests that general-purpose LLMs may be able to neutralize biased text—but possibly at the cost of precision. That is what we find in the next section.

We used our top-performing model (GPT-4) and the NPOV-Scoped³³3We used this prompt since GPT-4 performance did not significantly differ between any prompt condition, but the dataset contains violations of neutral language, in particular. prompt to neutralize biased Wikipedia edits. Specifically, we instructed the model to revise an NPOV-violating edit, varying two factors. We varied (Factor 1) whether we conducted this generation using zero-shot reasoning or Constitutional AI (CAI). In the latter, the model first critiqued why an edit violated NPOV and then used that critique to revise the edit. Since initial results suggested LLMs changed more words than human editors, we experimented with (Factor 2) instructions to only edit what was necessary. We conducted this generation on 200 edits, varying both prompt type (CAI or Zero-Shot) and whether to add constraining instructions (Yes or No), yielding 800 generations. For all conditions, we set the temperature to zero to further reduce extraneous changes.

We evaluated both the intensity and accuracy of AI changes. To measure intensity, we computed (1) ‘AI Edit Distance’ as the normalized edit distance between the AI-neutralized edit and the original NPOV-violating edit, (2) the word count of AI edits, and (3) the number of changed words (excluding stopwords) in each edit by summing additions and deletions (this process discussed more below). We compared these intensity metrics to analogous metrics for human-modified Wikipedia edits to determine if AI makes more or fewer changes than human editors.

We evaluated the accuracy of AI edits by comparing them to Wikipedia editors’ changes, which we treated as the gold standard. Our approach is similar to that of Pryzant et al. (pryzant_automatically_2020, ). We first computed differences (diffs) between the original biased edits ($w_{i}^{\text{Bias}}$), human-modified edits ($w_{i}^{\text{ModH}}$), and AI-modified edits ($w_{i}^{\text{ModAI}}$) using Python’s difflib library. ‘Diffs’ index substrings that were removed or added. We focus on removals as a measure of neutrality adjustment. After preprocessing diffs (lowercasing, stripping punctuation, removing stopwords) we defined sets $A$ (words removed by AI) and $B$ (words removed by humans). From these, we define true positives (TP) as words removed by both AI and humans ($|A\cap B|$); false positives (FP) as words removed by the AI that were not removed by humans ($|A-B|$); and false negatives (FN) as words removed by humans that were not removed by AI ($|B-A|$). We calculated various metrics based on these values (see Table 2). We also computed the BLEU score (a standard machine translation metric) of AI edits, using human edits as the reference text. We treated undefined precision metrics as 0, undefined recall metrics as missing, and used BLEU-score smoothing from (lin_orange_2004, ). For robustness, we re-analyzed the data under 32 analysis decisions (Appendix Figure 10), yielding results similar to those in Table 4.

Finding 1: AI changes more than humans. LLMs changed more than humans (Figure 5). AI edit distance ($M$ = 0.34, $SD$ = 0.21) was larger than human edit distance ($M$ = 0.15, $SD$ = 0.13), $t$(799) = 24.06, $p$¡0.001, Cohen’s $d$ = 0.85. Considering both additions and removals, AI made more changes ($M$ = 13.88, $SD$ = 12.91) than humans ($M$ = 3.79, $SD$ = 4.01), $t$(799) = 22.10, $p$¡0.001, Cohen’s $d$ = 0.78. Driven by CAI, AI edits were longer ($M$ = 33.53, $SD$ = 21.19) than human edits ($M$ = 27.53, $SD$ = 14.98), $t$(799) = 10.87, $p$¡0.001, Cohen’s $d$ = 0.38. See Appendix Table 9 for regressions and Table 3 for descriptive statistics. CAI makes more changes than zero-shot and constraining instructions are more effective in reducing changes for CAI than for zero-shot.

Finding 2: Humans are more likely to remove while AI is more likely to add. We find different editing patterns for AI vs human editors (Figure 6). AI additions ($M$ = 8.04, $SD$ = 9.33) were higher than removals ($M$ = 5.83, $SD$ = 4.61), $t$(799) = 8.86, $p$¡0.001, Cohen’s $d$ = 0.31. Meanwhile, human removals ($M$ = 2.40, $SD$ = 2.31) were higher than additions ($M$ = 1.39, $SD$ = 2.14), $t$(199) = 7.34, $p$¡0.001, Cohen’s $d$ = 0.52. Condition-level dynamics of additions and removals (Appendix Figure 9; Appendix Table 11) are similar to Finding 1.

Finding 3: AI changes are high recall but low precision. Across AI conditions, recall (0.79; 95% CI = [0.76-0.81]) was twice as high as precision (0.37; 95% CI = [0.35-0.40]). While AI editors remove content that human editors remove AI editors also make many other changes that human editors do not. See Table 4 for condition-level breakdowns and see Appendix Table 10 for regressions.

Label Quality. Participants completed the task in a median of 19 minutes $(M=21,SD=9)$. The median correctness for the three training questions was 2 $(M=2,SD=1)$. Agreement was generally high. The proportion of respondents agreeing with the majority answer for each question was as follows: neutrality $(Mdn=0.75,M=0.74,SD=0.13)$, fluency $(Mdn=0.68,M=0.69,SD=0.13)$, additions $(Mdn=0.72,M=0.72,SD=0.12)$, and removals $(Mdn=0.64,M=0.66,SD=0.11)$. We conducted robustness checks (Appendix Table 15) where we re-analyzed data after excluding participants with duration below the first quartile and including only those participants who got every training module question correct. Main results are highly similar to using the full dataset. See Appendix E.1 for participant demographics.

Neutrality. See Table 5 and 7(b) for raw data and 7(b) for logistic regression odds ratios. Participants rated AI edits as more neutral than human edits in 70% of zero-shot choices (95% CI = [67, 73]) and 69% of CAI choices (95% CI = [66, 72]), $p<0.0001$ for both by two-tailed binomial tests and conditional logistic regressions. After our initial (surprising) results, we conducted a pilot study to rule out the possibility that the preference for AI edits was simply an artifact of forcing participants to choose between edits rather than allowing a “both equal” option. We find the proportional gap (probability of choosing the AI-generated edit over the human one) replicated, but in 21% of cases, the options were rated as comparable (Appendix Table 14). At the end of our experiment, participants guessed how often other participants selected the AI edit as more neutral. The actual rate of AI neutrality preference (70%) was generally higher than participants’ forecasts $(M=58\%;SD=16)$, but 75% of participants predicted that AI edits would be chosen over 50% of the time (Appendix Figure 11).

Fluency. Participants rated AI edits as more fluent (Table 5; 7(b)) than human edits in 65% of zero-shot choices (95% CI = [61, 68]) and 57% of CAI choices (95% CI = [53, 60]), $p<0.001$ for both by two-tailed binomial tests and conditional logistic regression.

Additions and removals. Participants evaluated whether each rewrite added or removed information from the original edit, excluding changes that increased NPOV compliance (Table 5). Participants were shown human and AI edits simultaneously but answered questions for each separately. We analyzed judgments of human edits in two ways. First, aggregating across both AI matchups, we found CAI edits added information more frequently (34%, 95% CI = [31, 38]) than human edits (28%; 95% CI = [26, 31]), $\chi^{2}$ $p<0.001$. However, when evaluating human edits alongside CAI edits, participants reported a higher information addition rate (33%, 95% CI = [30, 37]) than when evaluating human edits alongside zero-shot edits (23%; 95% CI = [20, 26]). This led to a directionally similar but not statistically significant difference between human and CAI additions at the matchup level (Appendix Table 13). High-addition CAI edits may have heightened participants’ awareness or changed perceptions of additions in human edits.

Our results provide insight into the capabilities and limitations of LLMs in applying community-specific neutrality norms. Across models, prompts, reasoning strategies, and self-optimizations, performance was low—even with targeted prompts that directly incorporated Wikipedia’s guidelines. LLMs are trained on Wikipedia data, so this poor performance existed despite a potential data leakage. Low detection performance suggests that applying subtle guidelines to real-world cases is difficult for today’s general-purpose models. Notably, the largest model did the best. Perhaps even larger models would do better. Alternatively, low performance may be due to a more fundamental aspect of LLMs. For example, it may be that LLMs effectively ‘over-learn’ a notion (e.g., neutrality) from broad web corpora, and specializing this notion to a community’s norms requires changing a model’s parameters. In any case, our study suggests two future directions. First, fine-tuning may improve performance. However, fine-tuning may also risk overfitting (ma_adapting_2024, ). Another approach involves more advanced use of retrieval-augmented-generation (RAG). Can incorporating conversations from Wikipedia Talk pages or other Wikipedia data simulate the community knowledge that editors have?

However, we must view LLM performance in the context of what is, essentially, a hard task. NPOV edits are typically made by senior editors (pryzant_automatically_2020, ). And crowdworkers could only guess the biased word in a Wikipedia edit 37% of the time (recasens_linguistic_2013, ). General-purpose LLMs might be better than random individuals at NPOV detection but worse than expert editors.

Different models exhibited distinct biases. ChatGPT 3.5 was far more likely to over-predict edits as biased, while Mistral-Medium had the opposite tendency. Contrasting failure modes suggest that pre-trained LLMs may internalize distinct priors about what constitutes neutrality. These distinctions highlight one other shortcoming of relying on LLMs for such tasks. Models get introduced and updated rapidly, and our understanding of how LLMs work can get outdated just as rapidly. As language models become more ubiquitous, understanding these idiosyncratic biases will be crucial. Beyond understanding these biases, they may also be utilized. Considering Wikipedia NPOV specifically, editors engaging with opposing views play a role in maintaining neutrality (greenstein_ideology_2021, ). Perhaps a multi-agent system simulating ‘debates’ (irving_ai_2018, ; khan_debating_2024, ) can better apply nuanced rules than an individual agent in isolation.

AI editors neutralize text differently from human editors. Computational metrics indicate that LLMs typically remove the NPOV-violating words that human editors remove (79% average recall), but they also make many other changes (37% average precision). Furthermore, while Wikipedians were more likely to neutralize text through deletions than additions, AI editors showed the opposite pattern. This divergence suggests LLMs may internalize and operationalize rules like NPOV in fundamentally different ways compared to a community’s human experts. Though different is not necessarily better or worse. Of course, one clear risk is that AI moderators can reduce the cohesiveness of community content. Then again, LLMs may also inspire new practices. The fact that LLMs did not apply rules like human experts represents both a risk (discord) and an opportunity (innovation). Future work can explore innovative uses of LLMs for content moderation that go beyond attempting to ‘mimic’ human content moderators.

We find that crowdworkers prefer AI edits over human edits on both fluency (61% of choices) and neutrality (70% of choices). We emphasize that participants were not Wikipedia editors. Their judgment may not match those who would actually be using these systems for content moderation. We view crowdworkers as more representative of Wikipedia readership than editors, and our results should be interpreted in this context.

We hypothesize that our human evaluation findings are driven by both model size and reinforcement learning from human feedback (RLHF). First, LLMs have been trained on large corpora to predict the highest-likelihood next token (zheng_secrets_2023, ), and so their output is often highly grammatically correct or readable—which is not always the case for human Wikipedia edits. This tendency to produce “expected” text may have caused participants to evaluate LLM generations more favorably. Relatedly, through RLHF, LLMs have been trained to produce text that people—in general—will like (zheng_secrets_2023, ). And when evaluated by crowdworkers, their generations are preferred to Wikipedia-editor generations. Instruction-tuned models may be highly capable of applying rules in ways that resonate with a broader public, even if these applications differ from community experts. A promising area of future research is instruction-tuning models on expert feedback.

To reconcile LLMs’ low detection performance with high generation evaluations, we conducted a qualitative analysis of LLM generations. Qualitative analysis confirmed what computational experiments found: LLMs are high-recall neutralizers. In most cases, they did change the words that human editors changed. And we also found cases in which LLMs changed words human editors arguably should have changed. If a community prioritizes recall over precision, then LLM generations are highly valuable.

But LLMs also make many changes, and these changes are not always related to NPOV (e.g., grammar or readability). Participants rated CAI-generated rewrites as adding unnecessary information more frequently (34%) than human rewrites (28%). However, this difference was relatively small compared to the large AI-human gap in changes measured by edit distance and diffs, suggesting that LLMs typically make many small and unnecessary tweaks rather than add new information. If deployed, we do not know how volunteer editors would react to AI systems making unnecessary edits to their content. This risks engendering a loss of agency, decreasing stylistic variation, and increasing moderator burden if moderators need to constantly check that the LLM changes are not hallucinations.

Our findings suggest promise for LLMs in mixed-initiative community moderation systems. LLMs performed better at generation than detection, so they can create ‘first drafts’ of texts flagged by human moderators as norm-violating. However, LLMs’ poor detection performance limits their usability. Generating norm-compliant text once a violation is identified is valuable—but it is impractical to have LLMs rewrite every text in a community. Therefore, effective detection would have amplified the usability of effective generation. Future work could explore hybrid systems that leverage LLM generation abilities while reducing reliance on costly human judgment for detection. For instance, models could monitor Wikipedia Talk pages for comments suggesting NPOV violations. Based on automated triggers, these models could then offer neutralizations. This approach would use indirect human input to address LLMs’ detection shortfalls.

Here, we summarize some of the practical implications of our study.