Actuation without production bias

A common assumption made in many models is that while on some level sound change involves language learners/users selecting from what Ohala memorably called the ‘pool of synchronic variation’ (Ohala 1989), the variants in the pool are determined by universal aspects of speech production and perception, or phonetic bias factors (Ohala 1993, Blevins 2004, 2006, Garrett & Johnson 2013, and many others). Phonetic bias factors are asymmetries in patterns of phonetic realization which make certain sound changes more likely than others. Phonetic biases were traditionally understood to be production biases – physiological constraints on speech production – but acoustic-perceptual biases may also play a role (for reviews see Garrett & Johnson 2013, Ohala 1997, Blevins 2015). In this chapter, we use the term production bias to refer specifically to incremental and asymmetric phonetic biases with a articulatory basis. For example, vowel undershoot is the result of a production bias affecting the speed with which the articulators involved can achieve a particular spatial target. This is to be distinguished from vowel reduction, the outcome of phonologizing undershoot. What is important for present purposes is that production bias refers to the constraints giving rise to asymmetric, but non-phonologized variability.

Importantly, the mere existence of phonetic biases does not entail that they will lead to change at the population level. For as long as the existence of external forces impinging on speech patterns has been noted, so too has the observation that they don’t make change inevitable: indeed, the equilibrium state of language is arguably stability, not change (Milroy 1992, Janda 2003, Kiparsky 2015). At a minimum, a bias needs to result in a change in the speech behavior of at least a single individual, the classic Ohalaian ‘mini-sound change’. The idea that sound changes can take hold in the speech of individuals has led to work investigating why certain individuals (the innovators of Milroy & Milroy 1985) may be more poised to adopt bias variants than others. Proposals include individual differences in perception and/or cognitive processing style (Yu et al. 2013, Yu & Zellou 2019), production (Dediu & Moisik 2019), and/or learning experience (Bermúdez-Otero 2020). For more discussion of the role of individual differences in sound change, see the papers in Hall-Lew et al. 2021b). The critical point for the present discussion is simply that while biases exist, they do not necessarily entail change, even at the level of the individual.

Much as not all biases result in innovations in the speech of individual speakers, not all innovations propagate in a population. Why not? Here a wide range of proposals have been advanced. Lindblom et al. (1995) suggest that both articulatory ease and perceptual confusability may contribute to making some changes more or less likely to take hold. Other researchers propose that bias factors may be kept in check by a countervailing structural force, such as categoricity bias (Kirby & Sonderegger 2013, 2015) or contrast maintenance (Sóskuthy 2015). Kirby & Sonderegger (2015), building on the tradition of modeling population-level language change of Niyogi et al. (Niyogi & Berwick 1996, Niyogi 2006, Sonderegger & Niyogi 2010, 2013), show how population dynamics can be a source of stability: a change present in some individuals rooted in the same bias factor can spread or not spread through the population, depending on the learning algorithm assumed at the level of individuals and on population structure (e.g., how connected agents are), with population-level dynamics differing crucially depending on whether each agent learns from one or multiple teachers (Niyogi 2006, Niyogi & Berwick 2009, Smith 2009, Kirby & Sonderegger 2013, 2015).

There is, however, something of a disconnect between computational models of sound change propagation and theoretical and empirical work on change in the population. As noted above, much work on sound change from a phonetic perspective has emphasized the role of phonetic bias factors, and these have been implemented in models such as Pierrehumbert (2001) or Sóskuthy (2015). Yet, the broader literature on language variation and change often refers to other forces driving spread of a variant through a population: contact between groups with and without the variant, and social meaning attached to the variant or those who use it. As the above quote from Ohala suggests, the forces which give rise to the initial conditions for change at the level of the individual may be distinct from those which drive transmission and propagation at the population level. To understand the extent to which phonetic biases actually shape sound change typology, then, it is necessary to study them within the context of a population as well as within individuals, and to consider the full spectrum of forces which cause or inhibit the spread of a variant.

Perhaps most prosaically, frequency of interaction between and/or accommodation to individuals or groups of individuals impacts both transmission and propagation of variants (Trudgill 1986). Indeed, the many decades of research in the Labovian tradition argues that propagation of change in a speech community involves orderly shifts in the frequency of competing variants along demographic dimensions (Labov 2001). All else being equal, speakers are more likely to imitate the speech of groups (however defined, e.g., speakers of a different dialect, a certain gender, or a locally-defined category such as ‘burnout’ in Eckert 2000) they are more often in contact with, with group contact potentially magnifying existing phonetic bias-driven asymmetries (Harrington et al. 2018). Yet frequency of contact cannot be considered in a social vacuum. Work in the variationist sociolinguistic tradition (including Eckert 2000, Labov 2001, 2007, Eckert & Labov 2017, etc. and associated formal/computational work such as Burnett 2019 and Kauhanen 2020) emphasizes the centrality of social evaluation in determining who imitates who and under what circumstances. An important idea in this line of work is that not only innovative groups, but also innovative speakers are more likely to be imitated (Labov 1990, 2001, Harrington & Schiel 2017, Harrington et al. 2018). Approaches stemming from accommodation theory, including ‘identity-projection’ models and the notion of ‘style design’ (for a review, see Auer & Hinskens 2005), further emphasize that imitation is not automatic, but conditional on the social relationship between speaker and hearer. From the sociolinguistic perspective, then, understanding the dynamics of sound change requires understanding not just the source of a novel variant (which may indeed have come about due to some phonetic bias), but population-level variability in how variants are evaluated.

Other work raises the possibility that the likelihood of propagation could involve evaluation of specific variants themselves. For example, Baker et al. (2011) argue that adoption proceeds by imitation of variation, but that awareness of variation is limited to extreme targets. If learners only adopt variants they are aware of, and since extreme variants will by definition be rare, this helps explain why sound change itself is rare. Garrett & Johnson (2013) consider the possibility that whether or not a given token (exemplar) is retained (stored) may be socially motivated. This could potentially mean one of two things: that certain learners are more prone to retain innovative variants, potentially for reasons outlined above, or that a given variant has some socioindexical value affecting the probability with which it will be stored or retained, independent of the speaker who utters it (or the hearer’s relationship with/evaluation of that speaker). The idea that particular variants may be more salient to listeners is consistent with the sociolinguistic proposal that a variant must somehow be ‘marked’ before it can take on valence (social meaning) (Hall-Lew et al. 2021a, Silverstein 2003).

The foregoing review, while far from comprehensive, highlights the range of external forces that may be involved in the actuation of sound change at the population level above and beyond phonetic biases. That is to say, while change might take place at the population level due to the enhancement of an asymmetric phonetic bias, we must also consider other situations characterized by asymmetries like frequency of interaction, social evaluation, or individual differences. This leads naturally to asking: what do the dynamics of actuation look like when invoking forces besides phonetic bias? There may well be substantive differences in the evolution of a variant which are dependent on our assumptions about how that variant is propagated: due to positive evaluation of a group, of particular individuals, or of the variant itself. Because of the non-trivial mapping between individual learning/usage and population-level dynamics, computational modeling provides an attractive approach to understanding the potential contributions of different factors.

Empirically evaluating the time course of sound changes usually requires access to phonetic data gathered over long timespans (whether from panel studies or using apparent-time data), such as the Philadelphia Neighborhood Corpus (Labov et al. 2013), the Atlas of North American English (Labov et al. 2006), the Queen’s Christmas broadcasts (Harrington et al. 2000), or the Sounds of the City corpus (Stuart-Smith et al. 2017, Sonderegger et al. 2020, Sóskuthy & Stuart-Smith 2020; see also Cox, Palethorpe, and Penney, this volume). Such datasets enable rich inferences about the dynamics of how changes unfold over time in a population setting. Computational modeling is a complementary approach, providing not only a way to study changes for which the necessary empirical data are lacking, but also a (relatively) quick way to test hypotheses and make empirical predictions. Moreover, a well-specified computational model provides a baseline against which to judge alternative explanations, and the implementation of a model – the process of translating theoretical concepts into concrete parameters – can prove invaluable in terms of evaluating and adjudicating between possible explanations, and generally helps to avoid the “hazards of unaided reasoning”.²²2“With purely verbal arguments about evolutionary processes, it is too often the case that our conclusions do not follow from our assumptions. Unaided reasoning about the mass effects of many small forces operating over many generations has proven to be hazardous. Formalizing our arguments helps us understand which stories are possible explanations.” (McElreath & Boyd 2007, 6) This is the case not only because intuitions based on purely verbal descriptions can often be misleading, but also because in complex settings like language change, it is far from obvious how different factors will interact: iterating weak forces over time generally leads to surprises.

A good example of this is Baker (2008), a computational exploration of the classical Neogrammarian hypothesis that sound change is fundamentally the accumulation of error. In his simulations, agents are connected in a social network based on relative prestige and interact only with other agents in their subnetwork. Tokens produced by each agent are subject to a small amount of random noise. Baker shows that such a model is unable to produce the typical s-shaped curve familiar from empirical studies of sound change, in which transition between periods of stability is initially slow but then proceeds quite rapidly. Baker’s model also underscores an important point: because most variation does not result in change (what Kiparsky 2015 calls the ‘non-phonologization problem’), any adequate model of sound (indeed language) change must be able to model stability, as well as change. As Baker aptly demonstrates, simple-minded implementations of phonetic bias are insufficient in this regard, since the introduction of the bias in such models inevitably leads to its adoption (cf. Pierrehumbert 2001 and Wedel 2004, who show something similar in terms of how unconstrained memorization of new exemplars leads to increasingly diffuse phonetic category distributions).

Much like actuation may be initiated by forces other than production bias, stability may come about due to different mechanisms in different models. Several lines of modeling work have found that to meet the stability goal, it is necessary to include some kind of force promoting contrast maintenance, to keep separate phonetic categories stable, alongside an external force, such as a production bias, which induces change (Pierrehumbert 2001, Wedel 2006, Kirby 2013, Kirby & Sonderegger 2015, Sóskuthy 2015). Other work shows how individual- or group-level differences in evaluation can promote stability. For example, Garrett & Johnson (2013) demonstrate how both stability and change can come about in a model where a listener or listener group disregards biased variants. These simulations are instructive in that they show how different results may arise from the same starting conditions under different assumptions about the nature of bias.

In previous work, we have described a modeling framework (outlined in §2 below) in which categoricity bias can enforce stability even in the face of production bias, and in which production bias can induce change even in the presence of categoricity bias. Such a model is adequate in the sense that it allows for both stability and change. However, as discussed in §1, production biases are not the only forces which can perturb a population from equilibrium. This leads us to pose the following research questions:

1. 1.

   Does the use of production bias as a perturbing force have a unique dynamics, or can a nonlinear transition from one stable state to another emerge when a different force is employed?

2. 2.

   If it can, will any kind of external force produce the same dynamics at the population level, or do different forces have different dynamics?

Note that in this setting, ‘dynamics’ means the deterministic dynamics of well-mixed and infinitely large populations which define a landscape of possible trajectories of change, rather than the properties of specific temporal trajectories in simulations corresponding to a real-world instance of a particular sound change (as is typically the focus in agent-based simulations such as the IP-ABM).

We consider models containing two external forces beyond phonetic production bias: (1) contact between subpopulations with different stable speech patterns, and (2) the weight agents give to tokens depending on their source (based either on the speaker’s group membership or individual identity) or the extent to which a token is phonetically innovative. We assess each model based on whether both stability and change are possible as model parameters (= the strength of the external force) are varied. In this paper, we focus on the particular example of the propagation of a phonologized coarticulatory variant, but the general framework is applicable to many types of change involving a wide range of non-phonetic production biases.

The broader question addressed by this exercise is: can we safely assume that any proposed force driving change could lead to change, iterated over time in a population? This fundamental assumption is implicit in a great deal of theorizing about sound change, but remains untested. The answer is not obvious, because (as we will see) even in simple models where individual agents are anything but sophisticated, unintuitive outcomes may still result at the population level (see also Jochim and Kleber, this volume). If the answer turns out to be ‘no’, this would mean that a thorough understanding of population dynamics, in addition to their source, will be necessary to get to grips with how particular changes actuate.

Agents in our model are represented by a simple lexicon consisting of three vowel categories {V₁, V₂, V₁₂}, where V₁₂ represents V₁ in the coarticulation-inducing context of V₂ (e.g., /a   i/, as opposed to plain /a/ or /i/)⁴⁴4We employ this notation, rather than using e.g. /e/, to underscore the fact that phonemically we are modeling a state where there are two categories: that is, /a   i/ is a (stable) variant of /a/, rather than a separate phoneme.. Vowels are represented as first formant (F1) values. For simplicity, the F1 distributions of V₁ and V₂ are assumed to be normal ($V_{1}\sim N(\mu_{a},\sigma_{a}^{2})$, $V_{2}\sim N(\mu_{i},\sigma_{i}^{2})$), known to all learners, the same for all learners, and stable over time.

The F1 distribution of V₁₂ is normal, with fixed variance as for V₁ and a mean we denote by $c$:

We will sometimes refer to V₁₂ (or equivalently, $c$, which determines the distribution of V₁₂) as the contextual variant. $c$ lies between the means of V₁ and V₂, and $\mu_{a}-c$ is the extent to which /a/ (V₁) is coarticulated in the context of /i/ (V₂).

In addition, we assume that productions of $V_{12}$ may be subject to phonetic production bias, represented by a quantity $\lambda$ which describes the tendency of a speaker to over- or undershoot articulatory targets.⁵⁵5In Kirby & Sonderegger (2015), the production bias is allowed to vary normally across tokens. For simplicity we assume a single fixed production bias here. Thus, the actual productions of an agent with contextual variant $c$ follow the distribution:

(Note that the bias adjusts the mean of V₁₂ by negative $\lambda$, because V₂ has lower F1 than V₁.) This scenario is illustrated in Fig. 1.

We assume that agents are divided into discrete generations, each containing a very large number of agents.⁶⁶6In the limit of infinitely-large populations, the evolution of the distribution of $c$ becomes deterministic, and can be analyzed as a dynamical system (see e.g., Niyogi 2006, chapter 5). Learners in generation $t+1$ receive $n$ examples of $V_{12}$ drawn from generation $t$. Each learner’s task is to infer $c$. We assume that learners apply a learning algorithm which is ‘rational’, in the sense that they assume each token in their learning data to be generated according to Equation 1, and estimate the most probable value of $c$. That is, each learner’s task is simply to infer how much /a/ is produced like /i/ in the context of /i/ (/a   i/).

In Kirby & Sonderegger (2015), we allowed this basic model to vary in two ways. The first was changing the learning algorithm agents applied, by adding a categoricity bias: the degree to which agents preferred /a   i/ tokens to conform to the fixed distributions of /a/ and /i/. This was implemented as a prior over the distribution of $c$. We explored two cases, a simple Gaussian prior where $c$ was biased to be near $\mu_{a}$ (corresponding to a preference/expectation for /a   i/ to be realized as /a/) and a more complex polynomial prior representing a bias for $c$ to be a value around either $\mu_{a}$ oder $\mu_{i}$ (corresponding to either no coarticulation, or full coarticulation). The strength of this ‘complex prior’ is controlled by a parameter $a$, as shown in Fig. 2. All simulations in the current paper use this prior to capture categoricity bias.

The second issue we addressed was that of population structure: whether agents were assumed to learn from single teachers, a frequent assumption in computational modeling of language change (often called ‘iterated learning’: Smith 2009, Griffiths & Kalish 2007, Kirby et al. 2007, S.), or multiple teachers, as illustrated in Fig. 3. Because we observed significant differences between the dynamics in single- vs. multiple-teacher settings (c.f. Niyogi & Berwick 2009), and because only the results from the multiple-teacher settings were consistent with the extant sociolinguistic evidence, we restrict our attention to the multiple-teacher setting in the present paper.⁷⁷7Kirby & Sonderegger (2015) also considers the case where each agent learns from two (randomly chosen) teachers, which we do not consider here because the dynamics were similar to the multiple-teacher case.

Considering the ensemble of all agents in generation $t$, the state of the population at $t$ can be characterized by a probability distribution describing how likely different values of $c$ are. Formally, this is the distribution of a random variable, which we write as $C^{t}$. Similarly, the values of $c$ learned by agents in generation $t+1$ can be characterized by $C^{t+1}$, whose probability distribution describes how likely different values of $c$ now are. For simplicity, we assume that the number of agents per generation is very large. The evolution of the distribution of $c$ is then deterministic, making its behavior more easily analyzed as a dynamical system. This and several other aspects of our modeling framework, such as the assumption that generations are discrete, are shared with the broader literature on dynamical systems models of language change (e.g., Niyogi & Berwick 1996, Niyogi 2006, Yang 2000; see §1.3.1).

In this framework, given a choice of learning algorithm, population structure, and degree of phonetic bias, we may characterize the evolution of the distribution of $c$ and determine the extent to which it satisfies our modeling goals. When learners are assumed to have a categoricity bias like that in Fig. 2, analytic solutions become intractable, and we must proceed by simulation to determine how the distribution of $c$ evolves. For practical reasons, in this chapter we present simulations over a limited number of generations (between 50 and 500), but it is important to keep in mind that the stable distribution of $c$ remains a fundamentally deterministic, rather than stochastic, function of the parameter settings. As a result, the number of generations our simulations require to converge to a stable state is not directly comparable to simulation time in stochastic agent-based simulations such as Harrington & Schiel (2017), Stevens et al. (2019), or Jochim and Kleber (this volume). Our goal is less to characterize the speed, slope, or trajectory of change than to determine the parameter regimes in which change occurs at all, as well as which change will in a occur in a particular regime.

Fig. 4 shows an example for the setting where all agents have a minimally coarticulated variant in generation 1 ($c$ is normally distributed with mean $\mu_{a}-10$), and there some production bias ($\lambda=$ 2) along with weak categoricity bias ($a=$ 0.02). The results can be visualized in two ways. The left panel shows the evolution of the entire distribution of $c$, while the right panel summarizes this distribution by showing just the evolution of its mean (representing the ‘average pronunciation’ of V₁₂). From either representation, it is clear that $P(C^{t})$ quickly converges to having most mass around 530 Hz (= $\mu_{i}$, the F1 of V₂), meaning that the population converges on a state of full coarticulation.⁸⁸8Settings of other parameters for this simulation and those shown in Fig. 5: $\mu_{a}=530$, $\mu_{i}=730$, $\sigma_{a}=\sigma_{i}=50$, $M=500$, $n=100$.

In Kirby & Sonderegger (2013, 2015), it was shown that only a model with both production and categoricity biases could achieve all three modeling goals (stable limited coarticulation, stable full coarticulation, and change from one to the other). When the degree of categoricity bias is held constant, increasing $\lambda$ past a critical value causes a rapid transition from stable limited coarticulation (low $\lambda$) to stable full coarticulation (high $\lambda$). This dynamics of this case are characterized by a trade-off between the strength of categoricity bias and production bias, with a sudden change from limited to full coarticulation. This ‘adaptive landscape’ is illustrated in Fig. 5, which shows the degree of coarticulation the population will end up with, as $a$ and $\lambda$ are varied, for the same starting state (limited coarticulation), in a concrete example. To generate this figure, we ran a simulation like the one shown in Fig. 4 using a range of values of $a$ and $\lambda$, and recorded where the average degree of coarticulation stabilizes after a large number of generations ($t=2500$). In Fig. 4, this stable state is near $\mu_{i}$ (= 530 Hz).

Before moving on to our extensions, we emphasize two further aspects of this earlier work. First, in models with both categoricity bias and production bias, phonologization is not inevitable (cf. Baker 2008): the mere presence of a production bias ($\lambda>0$) does not entail that a novel variant will inevitably become the dominant variant in the speech community. Second, the dynamics of phonologization in this model are non-linear: a very small change in the degree of bias can result in phonologization (sudden population-level change) or not (population-level stability), simply depending on the state of the system (e.g., the amount of production vs. categoricity bias). In particular, there is a clear bifurcation: once $\lambda$ is large enough relative to $a$, the stable state will be one of full coarticulation. It is thus not necessary to postulate any additional mechanism to explain both phonologization and ‘non-phonologization’.

In this setting we consider a population consisting of two groups. Group A is characterized by a contextual variant exhibiting little or no coarticulation, while for Group B, this same variant is extremely coarticulated (Fig.  6). Let $aProb$ be the probability that a Group B agent learns from (equivalently, imitates/stores an exemplar from) a Group A agent, and let $bProb$ be the probability a Group A agent learns from a Group B agent. In this setting, there is no production bias ($\lambda=0$), and the $a$ parameter controlling the strength of the categoricity bias was set at 0.01, encoding a strong dispreference for intermediate variants (see Fig. 2). In other respects, the simulation procedure was the same as outlined in §2.

A selection of the results of these simulations after 50 simulation epochs are given in Fig. 7, which shows the evolution of $c$ for each group. The main result is that the starting mean F1 value $c$ characterizing each group’s stable state (Fig. 6) appears to be stable even when there is some degree of interaction between them (upper left panels). However, obtaining (and storing/imitating) just 5% of training examples from a different group can be enough to induce the entire population to converge to one or the other group’s mean. There may also exist regimes in which both groups converge on an intermediate distribution, i.e., a more extreme version of the $aProb=bProb=0.06$ panel shown in Fig. 7.

In regimes in which there is a sizable asymmetry between $aProb$ and $bProb$, the direction of convergence appears to be predictable. The panels in the lower right quadrant of Fig. 7 in which $aProb=bProb$, particularly the case where $aProb=bProb=0.1$, merit additional mention. That both groups are seen converging to the group B mean in the lower right panel of Fig. 7 is unlikely to consistently replicate; repeating this simulation many times, with the same parameter settings, should yield approximately as many outcomes which converge to the group A mean. This is because simulation requires that we work with a finite population sample, whereas the ‘true’ (deterministic) stable state is contingent on the assumption of infinite populations.

This simplistic scenario could be further complicated in many ways, e.g., by introducing differences in the orientations of the distributions between groups. However, it already shows us that, in the context of our modeling framework, the subpopulations in contact scenario fulfills our modeling goals, with a dynamics very similar to that seen by the introduction of a production bias: there exist parameter regimes in which both stability and change are possible.

The next three models explore whether both stability and change are possible in the presence of social value associated with more coarticulated variants, with valued speakers who coarticulate more, and with groups characterized by coarticulation. We are intentionally remaining agnostic as to exactly what motivates the association of ‘social value’ with a variant, a speaker, or a group, but see Section 1, as well as Garrett & Johnson (2013) and Salmons (2021, chapter 7), for some discussion.

In our social weighting settings, each token $y_{i}$ is transmitted as a vector of an F1 value together with a social weight value $w_{i}\in[1,w_{max}]$. How this value is determined varies across the three models. In Model 2, higher social weight is associated with more coarticulated tokens, i.e., tokens of /a   i/ are more highly weighted the more similar they are to /i/. In Model 3, tokens of /a   i/ from a high-coarticulation group are given greater weight, so exposure to highly coarticulated tokens of /a   i/ is modulated by probability of group interaction. And in Model 4, the weight is associated with tokens of /a   i/ from individual teachers who coarticulate more. These weights are distinct from the probabilities (aProb and bProb) with which an agent of one group learns a token from another.

As before, the learner’s task is to estimate $c$ using the weighted average of $y_{i}$. In these scenarios, however, tokens with values of $w_{1}>1$ (i.e., those that are either more coarticulated, or from teachers who coarticulate more), will have greater influence on the evolution of $c$ in the population.