The Sociolinguistic Foundations
of Language Modeling

NLP systems generally suffer from social bias: their real-world application leads to outcomes that unfairly disadvantage or harm specific social groups [58, 59, 60, 61]. Social bias can be introduced at various points during the development and deployment of NLP systems [62], but given the unsupervised nature of language modeling, training corpora are a key source of social bias in LLMs [10, 63]. While bias in NLP systems can harm people in various ways [59], in this section, we primarily focus on two common harmful outcomes of social bias. These two types of harms are most commonly discussed in terms of quality-of-service harms and stereotyping harms (e.g., [64, 65, 60, 66]), although many different systems have been proposed for classifying biases and harms in NLP, which define these terms in somewhat different ways, along with many additional and often overlapping categories [59]. Both of these types of harms are especially relevant to LLMs, and crucially, we believe both can be better understood and addressed in language modeling by adopting a sociolinguistic perspective (see Figure 3).

First, social bias can be characterized by poor system performance for certain social groups that are interacting with LLMs and applications based on language models: token prediction will be more or less accurate depending on the social origins of the language inputted into the system. For example, ChatGPT might have difficulty correctly understanding prompts written by people from certain social groups due to their use of non-standard or socially restricted language patterns. This type of bias leads to what is known as quality-of-service harms, where the performance of these systems varies depending on the social background of the user [64, 60]. These types of quality-of-service harms can often be the product of selection bias, as they result from how training data is selected from across the society whose language is being modeled [58]: in general, if language data from certain social groups is under-represented in the training data for a language model, we should expect that NLP applications based on that model will process language structures produced by these groups less accurately and consequently exhibit poorer performance for these groups [59, 67]. Notably, quality-of-service harms, especially those resulting from selection bias, have been one of the central concerns in computational sociolinguistics [68, 69, 51]. Researchers in this emerging field have stressed for the past decade that the performance of NLP systems generally varies for people from different social groups and have called for engagement with description and theory from sociolinguistics to help address this basic form of social bias (e.g. [70, 71, 72, 73]).

Second, social bias can be characterized by systems that produce outputs that directly harm or discriminate against certain social groups even when they are not directly engaging with these systems themselves. For example, when prompted, ChatGPT might be more likely to produce negative portrayals about ethnicities and genders, no matter who is doing the prompting [8, 67]. Most notably, this type of bias can lead to what is known as stereotyping harms [64], as well as related harms like disparagement and dehumanization [60], where negative viewpoints about specific social groups are propagated, as has been widely discussed in regards to LLMs [10]. Once again this issue can be traced back to the data the language model was trained on. If the training corpus contains relatively frequent expression of harmful or inaccurate ideas about certain social groups – as we can safely assume any large, unconstrained sample of internet writings will – language models will inevitably reproduce those biases [10, 63]. As Bender et al. (2021, 613) state, “large, uncurated, Internet-based datasets encode the dominant/hegemonic view, which further harms people at the margins” [10]. These types of harms are generally the product of semantic bias, as they result from the meaning relationships between words inferred by the language model based on patterns of co-occurrence observed in the training corpus [58].

From a sociolinguistic perspective, we believe that social bias in language modeling can generally be addressed by training on corpora that more accurately represent the target variety of language. It is especially important that the training corpus represents the internal structure of the target variety, in the sense that the sub-varieties of that variety of language, including most importantly the major dialects of that variety of language, are adequately represented in the training corpus (see Figure 3). For example, a corpus intended to represent American English, but which is primarily composed of texts collected from a specific dialect of American English (e.g., texts written by highly educated, middle-class, white Americans from major coastal cities), cannot adequately represent the full diversity of American English. Any language model trained on such a corpus should therefore be expected to be biased against social groups that are underrepresented in the training data, compared to a language model trained on a corpus that more accurately represents variation in American English.

The link between corpus design and quality-of-service harms in LLMs is especially clear: because language varies in systematic ways, to ensure a language model can accurately process language from a wide range of social groups, it must be trained on corpora that represent the language used by a wide range of social groups, i.e., their dialects (see Figure 3). For example, consider lexical variation in British and American English: if a model were only trained on American English, it would be much more likely to misinterpret the meaning of words that tend to have different meanings in British English, like boot (for trunk) or underground (for subway). Consequently, the quality of service provided by applications based on that model for speakers of British English would be degraded.

Stereotyping and related forms of discrimination generated by LLMs have also often been assumed to result from careless data collection and a lack of data curation [10]. A sociolinguistic perspective provides a principled solution to this problem: in general, stereotyping harms can be addressed by using training data that better represents the language produced by a wider range of social groups. One reason that certain social groups are negatively portrayed by LLMs is because they are not allowed to portray themselves in the data used for training. By training on corpora that equitably and deliberately represent the internal varietal structure of the target variety of language, especially the range of dialects of which it is composed, stereotyping and other forms of semantic bias can be mitigated (see Figure 3). In other words, modeling data from a wider range of dialects helps ensure that a wider range of viewpoints will be represented by a language model. Stratified corpora that accurately represent the sociolinguistic structure of the target variety can also be used to evaluate and probe a model, allowing for social bias to be identified and interpreted directly.

The sociolinguistic approach to language modeling advocated for in this paper therefore provides a simple yet theoretically grounded basis for understanding the general source of social bias in language modeling, including for addressing both quality-of-service and stereotyping harms, as well as other related types of harms. In addition, a sociolinguistic approach offers a clear pathway for both interpreting and addressing these different forms of social bias during pre-training through careful corpus compilation informed by theories and descriptions of sociolinguistic variation. Crucially, however, such sociolinguistic interventions need not necessarily occur during the initial pre-training of the base model, but can be pursued through the further pre-training of base models, as we discuss in the next section.

Despite their remarkable fluency and general applicability, LLMs generally benefit from some form of domain adaptation before deployment [5, 30]. In NLP, domain adaptation is the task of improving the performance of a system that was developed using language data collected in one domain for a different and often more specific domain where the system is to be applied [74]. Although there are many approaches for adapting language models, including for different downstream tasks (e.g., through forms of supervised and reinforcement learning), in this case, we focus on the process of fine-tuning a base model by extending unsupervised language modeling on a corpus of texts sampled from a specific target domain – the real-world context where the system is used, such as texts about a particular topic or from a particular genre [30, 75, 76].

This approach is often referred to as further pre-training because it involves extending the basic form of unsupervised language modeling used to train the base model to new data from the more specific target domain [30]. The goal is simply to improve the accuracy of token prediction in the target domain, while preserving the underlying fluency of the base model. For example, a base model trained on huge amounts of unrestricted online language data could be adapted to the specific domain of customer service: based on a corpus of customer service transcripts, the parameters of the base model would be adjusted to improve the ability of the model to predict word tokens in texts from that domain given the topics of discussion and the specific types of interactions that characterize that domain.

In the context of language modeling, the process of domain adaptation can be straightforwardly reframed directly in sociolinguistic terms (see Figure 4). If the goal of the base model is seen as accurately predicting word tokens in a broadly defined variety of language, like the English language, then the goal of domain adaptation can be seen as the process of fine-tuning the base model to allow it to predict word tokens more accurately in a more narrowly defined variety of that language – the sub-variety associated with the target domain. Crucially, the adapted model should be expected to be more accurate because more narrowly defined varieties of language must be characterized by less variation than any larger variety that encompasses it. This process can also potentially be carried out in an iterative manner, where a base model is repeatedly adapted on corpora representing more narrowly defined varieties of language.

A sociolinguistic perspective on domain adaptation therefore sees the target domain as a variety of language, which means that the process of domain adaptation can be meaningfully informed by linguistic analysis that rigorously identifies maximally distinctive varieties of language. This can include both existing research in sociolinguistics, dialectology, and related fields, as well as new research conducted directly to support model training for specified domains. For example, if a base model is adapted for a specific region of the US, research in American dialect geography (e.g., [53]) should be consulted to precisely define the sub-region that is being targeted for adaptation (see Figure 3). Similarly, if a base model is adapted for a specific type of blog writing, research on register variation in blogs (e.g., [55]) should be consulted to precisely define the sub-type of blog writing that is being targeted for adaptation.

Crucially, however, sociolinguistics not only provides a basis for identifying valid targets for domain adaptation but for mapping and modeling the internal structure of these target varieties (see Figure 4). This is especially important because target varieties for domain adaptation are often well-defined by default. For example, if a fine-tuning corpus is collected by sampling data from a particular social media platform, a relatively homogeneous variety of language will have naturally been targeted; however, a random sample of texts from that variety, drawn without taking into account its internal structure, might severely under-represent sub-varieties of interest (see Figure 2). For example, a social media corpus may be dominated by certain sub-registers (e.g., abusive or promotional posts) that are not the target of adaptation, while the sub-registers that are the target of adaptation (e.g., interactive or informational posts) may be limited. Similarly, people from certain social groups may be underrepresented in specific domains, resulting in social bias being inadvertently exacerbated by naive domain adaptation. In many cases, the target variety cannot even be accurately defined until the overall structure of the larger variety in which it is subsumed is understood through careful sociolinguistic analysis.

A sociolinguistic perspective also highlights a more general problem with domain adaptation: the success of this process depends on the relationship between the larger variety represented by the base model and the smaller target variety towards which the base model is being adapted. Ideally the variety of language represented by the base model would completely subsume the target variety: the target variety would be a sub-variety of the base variety, regardless of whether it was represented directly in the base training data. However, the target variety may not be adequately represented in the data sampled for training the base model. For example, the target variety could be associated with a social group or a social context that is severely underrepresented in the base training corpus. In such situations, fine-tuning regimes informed by sociolinguistic theory and description would likely be beneficial.

Finally, understanding the sociolinguistic structure of the larger variety of language could also allow models to be adapted to represent target varieties with missing data. For example, if empirical research in linguistics has found that a target dialect or register for which data is lacking falls between multiple dialects or registers for which data is available, a model could be adapted for the target variety by training on a combination of the available corpora. Overlap between varieties could also be exploited in a similar way: for example, if data is lacking for a target variety defined in terms of a specific register and a specific dialect, a model could be adapted for the target variety by fine-tuning on a combination of corpora that represent that specific dialect and that specific register. These types of techniques could even be used to create a model of a variety of language that does not yet exist – engineered by training on corpora representing different registers and dialects.

The related challenges of social bias and domain adaptation can be seen as forms of the more general alignment problem – how to ensure that the behavior of AI systems aligns with the values and expectations of society [77, 78, 79, 80, 81]. Misalignment arises not simply when AI systems fail to achieve their intended goals, but when they pursue these goals, even successfully, in ways that have negative or unforeseen consequences or that are not in accordance with societal values, for example, in ways society finds to be inappropriate, unethical, immoral, or dishonest. Alignment is therefore the general process of guiding AI systems to behave in ways that are consistent with the broader expectations of society, while discouraging them from behaving in ways that are inconsistent with these expectations, especially to avoid unintended risks and harms. Crucially, the challenge is not only how to guide AI systems but where to guide them [77].

Although alignment is a long-standing concern in AI [82], attention has grown in recent years due to the growing complexity and ubiquity of real-world AI systems, especially systems based on language modeling [83, 84, 85, 86, 87], which potentially allow for misalignment to emerge on many different levels [77, 81]. For example, consider a generative language model that automatically produces reviews of scientific literature on a specified topic. An obviously misaligned system might produce reviews that are clearly wrong – incoherent or incorrect – while a less obviously misaligned system might produce fluent reviews, completing the task successfully in a superficial way, but getting facts wrong, for example, referencing publications that do not exist. This type of a hallucination – the presentation of false information as if it is true – is a common form of misalignment in LLMs [88, 89]. A more insidiously misaligned system, however, might produce perfectly accurate and fluent syntheses that cite relevant literature, but exhibit other problematic behaviors, such as limiting references to certain ideas or researchers in certain fields, thereby effectively suppressing certain viewpoints [10].

One solution for aligning language models with the values of society is by training these models using corpora that are in some way deemed to be more aligned with these values (e.g., [90]. As we have argued throughout this paper, we believe sociolinguistic theory provides a meaningful, interpretable, and productive way to guide this process – allowing us to better understand how corpora can be compiled so as to allow for societal expectations to be captured, crucially without pre-specifying what these exact expectations are.

Our view is that alignment is possible if training corpora accurately represent the range of dialects and registers of the target variety. In terms of dialects, as we discussed when we considered the challenge of social bias, we believe that, by balancing training data originating from different social groups, language models can be trained to better align with the general values of society, as opposed to the values of some particular social group. Similarly, in terms of registers, as we discussed when we considered the challenge of domain adaptation, we believe that, by balancing training data originating from different communicative contexts, language models can be trained to better align with the expectation that they will perform adequately across the range of contexts found in that society. In other words, the values and expectations of a society are instantiated in their patterns of language use. In general, we therefore believe that a major source of LLM misalignment results from what we call varietal misalignment and that LLM misalignment can therefore be addressed, at least in part, by aligning training corpora to the varietal structure of the target variety.

In addition to addressing alignment issues related to social bias and domain adaptation, we believe this sociolinguistic approach can potentially help us train models that are less susceptible to unethical and dishonest behavior in general, because respecting sociolinguistic diversity entails training models on data that represents a greater diversity of viewpoints, experiences, and contexts. As LLMs are models of varieties of language, they will be better models, more aligned with the needs, expectations, and values of society, when they account for the full range of sub-varieties, and hence the full range of perspectives, found within that society.

Finally, it is important to acknowledge that while a sociolinguistic perspective provides a basis for aligning a language model to the general viewpoints of the society that it is intended to serve, this approach does not ensure that the resultant language model will be aligned with the ethical and moral aspirations of that society. For example, a generative language model trained on a socially balanced corpus of the English language will still potentially produce texts that express racist viewpoints because a portion of English texts expresses racist viewpoints. There might be greater equity in the types of stereotypes it spreads, but such behavior can still be seen as a form of misalignment. A sociolinguistic perspective, however, also provides a possible solution to this problem – by deliberately weighting the varieties of language represented in the training corpus. For example, if a particular social group has been broadly disadvantaged or has a worldview that society wishes to encourage, the portion of the corpus representing the relevant varieties of language can be more heavily weighted during training. In this way, a sociolinguistic perspective can provide a theoretical basis not only for balancing but for controlling the alignment of language models.

Thus far, our discussion has focused on how a series of challenges in language modeling related to bias, adaptation, and alignment can be addressed, in principle, by building training corpora that better represent the dialects and registers of the target variety. Another form of this basic problem involves ensuring that language models and applications based on language models are responsive to language change and cultural change more generally [10, 8]. All varieties of language change over time, often in ways that are difficult, if not impossible, to predict [91]. If language models are to maintain their fluency and not become obsolete, they must therefore be continuously updated using training corpora that consist of examples of contemporary language use. In principle, this problem can be resolved by compiling new corpora over time that consistently represent the target variety and its evolving internal varietal structure. The challenge is therefore to understand how the sociolinguistic landscape of registers and dialects of that variety of language has changed over time, which can only be accomplished accurately through detailed and ongoing sociolinguistic analysis.

A related issue that has caused growing concern in language modeling is that over time more and more real-world language will presumably be produced with the assistance of LLMs, which will make it increasingly difficult to compile contemporary corpora of real human language for training new models or updating existing ones [92]. Proposed solutions to these problems of data contamination [93] and task contamination [94] generally involve finding ways to exclude machine-generated language from future training data, including through watermarking systems [95]. These types of solutions, however, seem easy to confound, if only because they do not generally allow texts written collaboratively by human and machine to be identified, which is likely to become increasingly common and diversified in everyday life.

Despite real concerns about LLM detection in certain contexts, the rising use of LLMs to generate language is not difficult to reconcile with sociolinguistic theory and practice. Over time, AI systems based on language models will undoubtedly start to change how we use language. Texts generated with the help of language models will increasingly enter into the real world. At this point, from an externalist perspective [47], these texts will be part of language – produced, transmitted, and understood by humans as language, often indistinguishable from human-generated language in the regular flow of real-world language use. Ultimately, the distinction between human- and machine-generated language can therefore be seen as simply another aspect of register that defines variation within varieties of language, just like all communicative technologies that have come before, including the invention of writing and digital communication.

Taking a sociolinguistic perspective, it is also important to acknowledge that the rise of language models is creating new varieties of language, including those characterized by the linguistic interaction between humans and machines, such as dialogues with ChatGPT. These new varieties, which will only continue to diversify over time, will also need to be accounted for, like all varieties of language, both by theories of sociolinguistic variation and by the evolving language models designed to represent contemporary language use. If language models are to be kept up-to-date, machine-generated language cannot be excluded, as its production will become a significant driver of language change.

In addition to more specific insights into the development and deployment of language models, we believe a sociolinguistic perspective can also help to explain the remarkable success of LLMs more generally, which has been attributed both to the development of new deep learning architectures and the use of extremely large corpora of natural language for training [96, 10, 8]. Although there is a clear relationship between the scale of the training data and the success of these systems, it is not altogether clear why increasing the amount of training data results in such great increases in performance. Is there a limit to how much performance can be gained simply by increasing the scale of the training data? How can more powerful models be developed with less data? These are fundamental questions for LLM development [8], especially because of the significant costs and environmental impacts associated with increases in scale [10]. We believe these are questions that can be uniquely informed by a sociolinguistic perspective.

The obvious reason why increasing the amount of training data provided to the model improves the performance of a language model is that it provides access to a wider range of language patterns. Working with extremely large corpora is clearly necessary – the complexity of language demands it – but it is also clear that scale is not sufficient on its own. For example, a language model trained repeatedly on the same dataset will not improve. What therefore matters is not simply the scale of the training data but the diversity of the training data.

Although the importance of the diversity of training data has often been stressed in critiques of LLMs [27, 10], the sociolinguistic perspective advocated in this paper provides a theoretical basis for understanding this relationship with greater precision: diversity in the training corpus, in terms of both its linguistic structure and its semantic content, can be seen as directly reflecting the diversity of the varieties of language represented by that corpus. To maximize the performance of language models and the efficiency with which these improvements can be obtained, in our view, it is therefore far more important to focus on increasing the varietal diversity of the training data than purely its scale. This can be achieved by carefully representing a wider range of contemporary language varieties in the training corpora, including both dialects and registers, as we have discussed throughout this paper.

This sociolinguistic perspective also provides an answer to questions about the limits of increasing the scale of training data [8]. At what point should increasing the size of the training corpus no longer lead to substantial improvements in model performance? Our hypothesis is that increasing the scale of training data will continue to increase the performance of language models so long as it also results in an increase in the sociolinguistic diversity in the training corpus. Crucially, this implies that attempts to empirically assess the limits of scale simply by comparing model performance as the amount of training data increases will not be accurate, unless the sociolinguistic diversity of the corpus is also controlled for and measured alongside corpus size.

Finally, a sociolinguistic perspective also offers clear direction for training models using limited amounts of data, for example, for under-resourced languages [10, 97]: models can be developed on a smaller scale by taking care to maximize the amount of sociolinguistic diversity in the training data, given the target variety. Moving forward, we therefore believe that optimizing the development and performance of LLMs will necessarily involve incorporating insights from sociolinguistics to enhance the diversity and representativeness of language data used for training.