In Silico Sociology: Forecasting COVID-19 Polarization with Large Language Models

For decades, the term “artificial intelligence” was used to describe computational capabilities that remained out of reach—in a quote often attributed to Larry Tesler, “artificial intelligence is whatever hasn’t been done yet.”¹¹1Tesler himself maintains that this is a misquote, and that his actual statement was “intelligence is whatever machines haven’t done yet,” critiquing the tendency to continuously shift the criteria for intelligence to keep it one step ahead of ever-encroaching machines \parenciteTesler2010-ty. But in recent years, algorithms have gained fluency in such complex tasks as composing novel texts, generating photo-realistic images, and advanced coding, and the term “artificial intelligence” is now part of daily parlance.

This rapid progress is largely the product of two overarching technological developments. First, continuous improvements to hardware have driven decades of exponential growth in computational power. New capabilities emerge as models scale up, and modern chips now make it possible to train models with over a trillion parameters \parenciteKaplan2020-lo. Second, the proliferation of online digital content has supplied abundant training data. Current models are commonly trained on a near-complete record of all text on the internet, and “multi-modal” models are trained on vast collections of online images and videos as well \parenciteBrown2020-nh. These two complementary developments have resulted in algorithms capable of generating vast and varied forms of content; deep neural architectures make it possible to learn subtle and complex patterns, and large training data provide abundant examples of patterns to learn.

Particularly striking advances have been made in the training of large language models (LLMs), algorithms capable of generating text by predicting the next word in a sequence. LLMs form the foundation of AI conversational agents such as OpenAI’s ChatGPT, Anthropic’s Claude, and Deep Mind’s Gemini. These “chatbots” have swiftly gained widespread public exposure; ChatGPT alone reached over 100 million users within two months of its public release, and a large share of workers in fields ranging from education to computer programming report regularly using LLMs to improve their productivity \parenciteDellAcqua2023-fo, Mollick2023-wy. Seemingly overnight, the ability of algorithms to successfully impersonate human interaction – commonly known as “the Turing Test” \parenciteTuring1950-in – shifted from aspiration to expectation.

Because LLMs are typically trained on the wide variety of texts published on the internet, they learn to reproduce many distinct discursive styles. They achieve this not by memorizing specific sentences (although they occasionally do this), but by learning the latent probability distributions of word sequences constitutive of discourses. Publicly released LLMs are commonly fine-tuned to speak in the style of a helpful professional assistant and avoid making statements that are offensive, biased, or politically charged \parenciteOuyang2022-xw. But if prompted to do so, even these fine-tuned models can generate texts that mirror the diverse cultural and linguistic styles represented in their training data, ranging from sarcastic wisecracking to postmodern literary criticism to extreme political rhetoric \parenciteArgyle2023-ii, Kim2023-ji, Park2022-py.

We argue that this capacity to mimic and reproduce human responses opens fruitful new forms of socio-cultural analysis. First, LLMs are able to reproduce the discourses of populations not available for interviews or surveys, including populations from the past. Second, because LLMs can quickly and cheaply generate responses, they facilitate testing wide varieties of wordings for each question, improving robustness and identifying how specific words and associated framings can steer responses \parenciteGarcia-Pardina2022-cx. Third, they allow open-ended responses to be both generated and effectively machine-coded at scale, enabling a high-level view into the system of considerations that inform a given response. Establishing the validity of these LLM methods enables the generation of useful “social science fictions” or simulations that open up a wide range of possibilities that lie beyond the scope of this empirical investigation.

We apply this novel approach to investigate a longstanding question in the study of culture – to what extent are new cultural developments constrained by the existing ideological landscape \parenciteConverse1964-kv, DellaPosta2020-ta, Hunzaker2019-ds? More specifically, when a new issue arises, are public responses to that issue predictable given the systematization of attitudes across other topics? Answering this question requires observing how individuals respond to an emerging issue before it is framed by commentators, public figures, or personal acquaintances. Yet this analytic approach presents an empirical challenge, as new issues are rapidly subject to public discussion and interpretation, leaving social scientists little opportunity to measure responses prior to top-down framing by opinion leaders.

LLMs can shed new light on this question by serving as a “cultural time capsule,” capable of reproducing the most plausible responses from the time of the model’s training texts. An analyst can present an LLM with issues that had not yet emerged when the model was trained and compare these LLM-generated responses with survey responses gathered in the following years. To the extent that the LLM generates the pattern of attitudes that empirically manifest in later years, it suggests that public response to the new issue was prefigured in prior discourse, and that the public reception of the issue was a “predictable” development.

We use this method to examine a pivotal issue of our time, the public response to the COVID-19 pandemic. The spread of COVID-19 foregrounded several topics with little political precedent, such as vaccine mandates, face masks, and lockdowns. The public response to these issues rapidly politicized, with liberals endorsing cautious approaches to the virus and conservatives opposing more drastic measures \parenciteGadarian2021-su. What remains unclear is whether this politicization reflects deep characteristics of American liberalism and conservatism as idea systems, or whether the public simply responded to the cues of partisan elites who quickly espoused opposing responses. Fortunately for our analytic aims, GPT-3, the first LLM to accurately reproduce patterns of public opinion, was trained on texts published through October 2019, and therefore has no knowledge of the COVID-19 pandemic \parenciteOpenAI2023-wo. We can therefore use GPT-3 to reproduce how liberals and conservatives likely would have responded to questions pertaining to a pandemic before any top-down framing emerged around COVID-19 \parenciteKaplan2008-nm.

We find that liberal and conservative responses simulated with GPT-3 largely anticipate future politicization on a wide variety of issues pertaining to COVID-19. When prompted to speak in the style of a liberal Democrat, the model exhibits a greater likelihood of choosing to be vaccinated, choosing to wear a mask, endorsing vaccine and mask mandates, and supporting lockdowns. By contrast, when prompted to speak as a conservative Republican, responses are more likely to agree that wearing a mask or getting vaccinated should be personal decisions rather than government mandates and that both mask and vaccine mandates should be ended. To shed light on why the model associates liberals with more cautious responses than conservatives, we machine code more than 4,500 open-ended justifications for those responses and identify common themes corresponding to partisan gaps. Specifically, we find that levels of trust in the government and scientific community as well as the prioritization of safety versus freedom emerge as common considerations across questions, grounding novel pandemic-related issues in longstanding ideological principles.

These findings suggest that certain features of contemporary American liberalism and conservatism structured the way COVID-19 politicization unfolded. While these findings do not imply that “discourse is destiny,” they provide compelling evidence that existing ideological systems channeled the reception to this novel issue in a way that ultimately undermined a unified public response. Our results suggest that COVID-19 did not fundamentally alter American political ideology, but our research design could similarly be applied to identify instances where history does deviate from the projections based on prior discourse, suggesting an unanticipated development of discourse. When social movements or influential public figures cultivate surprising new discourses that do not conform to existing patterns, this would result in a disjuncture between prior models and subsequent empirical observations. The failure of models to anticipate future developments may therefore suggest points when “history matters,” and key events reshape public discourse in unpredictable ways.

Interviewing simulated respondents with an AI model presents obvious limitations. Most critically, an LLM may inaccurately reproduce a discourse, leading to incorrect or misleading inferences \parenciteJi2023-ut. Nevertheless, in situations where the target population cannot be interviewed or a question requires measuring responses at a scale that is practically infeasible, simulated respondents may provide the best available evidence into such questions that otherwise elude empirical analysis \parenciteKim2023-ji, Kozlowski2019-vh. Investigations with simulated respondents thus occupy the edge of empirical sociology; the information they provide may not definitively resolve a debate, but can place meaningful evidentiary weight on important questions that lie outside the reach of conventional methods.

Advocates for the use of artificial intelligence in science often emphasize algorithms’ capacity to surpass human performance. Indeed, such “superhuman” abilities are already facilitating important scientific contributions. In social science, machine learning algorithms assist researchers by identifying objects in videos and photographs, transcribing audio into text, and classifying texts into typologies, all at speed and scale far surpassing human capabilities \parenciteBonikowski2022-th, Grimmer2022-fu, Hannan2022-bs, Le_Mens2023-wa, Vicinanza_undated-cj. In the natural sciences, algorithms are beginning to make new discoveries by combining computational power and speed with sophisticated knowledge bases. Arguably the most important advance has been AlphaFold’s success at solving protein folding \parenciteSenior2020-ee, but advances in drug \parenciteJimenez-Luna2020-tg and materials discovery \parenciteWilkins2023-mx, Zhou2018-cw, the control of complex nuclear fusion reactors \parenciteDegrave2022-zi, and even the identification of novel auction and market policies are also promising \parenciteJiao2021-mb, Mosavi2020-hy.

Social scientists, however, may gain more from artificial intelligence by capitalizing not on its capacity to surpass human performance, but its ability to mimic it \parenciteBrynjolfsson2023-yu, Sourati2023-aa. Simulation studies in the social sciences have historically favored elegant models with simplistic agents over empirically realistic ones. Such formal models provide important insight into how complex social patterns emerge from simple interactions, but they tell us comparatively little about the dynamics of specific empirical cultural systems, organizations, or institutions. Progress on this front has long been hindered by the difficulty of specifying empirically realistic agents to populate complex social simulations. Fortunately, due to their training on massive archives of rich cultural data, modern AI models can now generate “digital doubles” of human respondents, capable of faithfully reproducing the knowledge, preferences, and behaviors characteristic of a specific social group.

Social simulations with empirically realistic agents open productive avenues for research that would be impossible with human subjects. First, AI models can use textual records to reproduce the discourse of social groups that no longer exist or that otherwise cannot be interviewed. Because these models are generative, they enable analysts to go beyond the exact statements made in the textual archive and extrapolate likely out-of-sample utterances consistent with the semantic associations in observed texts. Second, digital doubles can produce simulated data at scale. Millions of interactions between digital actors can be quickly and affordably simulated over a wide array of initial conditions and differently parameterized agents, detailing a richer and denser high-dimensional interaction space than would be possible with human actors \parenciteLai2024-qg. Lastly, the internal representations of AI agents are directly observable in a way that human representations are not. Although the activation patterns of deep neural networks are commonly described as black boxes, these representations can be directly analyzed and explored for deeper understanding.

For the following analyses, we analyze completions generated by the GPT-3 language model. This approach presents a notable departure from conventional methods of text analysis. LLMs are generative models, and the most straightforward way to learn from these models is not to examine their internal representations but to study the outputs they produce. As a result, the data we analyze are not actual statements made by members of our target populations nor are they representations of such statements like topic models or word embeddings. Our data are novel word sequences learned to occur with high probability given the discursive patterns learned in a vast training corpus.

Such an approach deviates from the simulation studies typical of formal sociology. Formal models commonly attempt to parsimoniously capture social dynamics by precisely specifying minimal conditions under which empirically-observed patterns can be reproduced. Using LLM outputs as data, by contrast, constitutes a hybrid of empirical and formal sociology; we analyze simulated data, but the simulation is trained to produce outputs that closely approximate empirical distributions.⁷⁷7Previous social simulation studies have used parameters estimated by empirical models to improve correspondence to observed contexts, but these models still typically strove to parsimoniously reproduce the system of interacting factors, prioritizing interpretability over predictive accuracy. To the extent that the model successfully reproduces a population’s response distributions, the model’s outputs can stand in for human responses from that population and be analyzed at scale. We adopt this approach to simulate the opinions of American liberals and conservatives in October of 2019, the historic point immediately prior to the emergence of COVID-19.

We begin by presenting results from prompts on vaccine-related topics. Each of the figures below is a coefficient plot showing the effect of “partisan priming” on the classification scores for a given prompt’s outputs. Partisan priming is a binary variable coded 1 = “liberal”, so positive coefficients indicate that liberal prompts were more likely to endorse the first anchoring term in the response dichotomy (e.g. “good idea”). This equivalently means that conservative prompts are more likely than the liberal prompt to endorse the second anchoring term (e.g. “bad idea”). Coefficients with significant positive effects (p<0.05) are colored blue. Negative coefficients conversely signify that conservatives are more likely to endorse the first anchoring term than liberals. Coefficients with significant negative effects are colored red.

Figure 6 displays coefficient plots of “partisan priming” predicting output classification scores for vaccine-related issues. The plots in the top left of Figure 5 examine the effect of partisanship on the intention to receive the vaccine. The positive coefficients in the first plots show that liberal prompts were more likely to report an intention to get the vaccine than conservative prompts. The effect is significant for both wording variations (“Personally, I will” and “I have decided that I will”), and for two question variations (“…to solve the spread of the virus” and “…to protect yourself against the virus”).

The next plots show results to questions regarding vaccine mandates. In the first question, liberal prompts are more likely to claim that a vaccine mandate is a “good idea” than conservative prompts, with the effect significant for three of four wordings. The second question replaces the word “mandate” with “requirement.” The effect remains positive for three of four wordings, although some effects are attenuated. Conversely, when the question is changed to be about ending vaccine mandates, conservative prompts are more likely to state that this is a “good idea.”

In the second row of Figure 5, we test the effect of partisanship on responses to vaccine mandates for government workers and students. Although many of the coefficients fall short of statistical significance, the significant effects are consistent with later observed polarization: the liberal prompt is associated with stating that “requiring government workers to get the vaccine” is a good idea, while conservative prompts is associated with saying “allowing government workers to return to work without getting the vaccine” is a good idea. Similarly, liberal prompts endorse “requiring students to get the vaccine” while conservative prompts endorse “allowing students to return to school without getting the vaccine.” However, when the question is worded to probe a response to “opening schools without a vaccine requirement,” no significant effect for partisanship is identified.

The third row of Figure 5 shows the effects of partisanship on responses to whether getting the vaccine should be a choice. On the questions of allowing government workers or students to “opt-out” of the vaccine, conservative prompts were more likely in both cases to say this is a good idea than liberal prompts. Similarly, the conservative prompts were more likely than liberal prompts to say that “letting individuals choose” and “letting people decide” to get the vaccine is a good idea.

We see the first instances of model inferences being inconsistent with historical observed patterns of polarization in the bottom row of Figure 5, where we ask about requiring proof of vaccination for various activities. Conservative prompts are more likely to say it is a good idea to require proof of vaccination to “travel by plane” or “enter bars or restaurants.” The same pattern emerges when the question is reworded, such that liberal prompts are more likely to state that “allowing unvaccinated people” to travel by plane or enter bars and restaurants is a good idea.

We next run a similar set of tests for opinions regarding face mask requirements. Results are presented in Figure 6. First, we see that liberal prompts are more likely to express intent to wear face masks both to slow the spread of the virus and for personal protection. Liberal prompts were also more likely than conservative prompts to say that mask “mandates” and mask “requirements” are good ideas. Consistent with this, conservative prompts were more likely to respond that ending mask mandates is a good idea.

The second row shows responses to prompts about requiring/mandating masks in stores, workplaces, or schools. For both “requiring” and “mandating” wordings, and for both stores/workplaces and schools, liberal prompts exhibit a greater likelihood of framing these measures as a good idea.

In the third and final rows, we see results from prompts on whether masking should be a personal choice and whether masks are effective. The first plots show that conservative prompts were more likely to state that “letting individuals choose” or “letting people decide” whether to wear a mask is a good idea, with no apparent difference between these wordings. The next plot shows that, when asked whether wearing masks in public should be mandatory or optional, liberal prompts were more likely to state that they should be “mandatory” than conservative prompts. Lastly, partisanship had no statistically significant effect on responses to whether face masks are an effective way to slow the spread of the virus.

Next, in Figure 7 we examine responses to questions about lockdowns. The first two plots in the first row ask about closing business and closing bars and restaurants. For both of these issues, the liberal prompts are more likely to endorse the lockdowns. The next two issues concern prohibiting large gatherings and avoiding small gatherings, and again the liberal prompts are more likely to say each of these measures is a “good idea.” By contrast, the final question asks if keeping businesses open is a good idea. The results for this question are mixed; for completions beginning with “I think,” conservative prompts were more likely to say keeping businesses open is a good idea. Yet for completions beginning with “I believe that,” liberal prompts were more likely to support the action. Although these results are mixed, they still represent a movement in the correct direction relative to previous questions.

In the second row of Figure 7 are plots representing the effect of partisanship on prompts regarding school lockdowns and halting international travel. Results on the topic of school lockdowns are mixed. Liberal prompts are more likely to say that “keeping schools open” is a good idea than conservative prompts, but they are also more likely to say “closing schools and conducting classes online” or “switching to remote schooling” are good ideas as well. Thus, we do not see a clear partisan leaning in either direction in the question of school lockdowns. Halting travel similarly exhibits some inconsistency. Conservative prompts are more likely to endorse a ban on visitors from countries with COVID-19 outbreaks, but are not more likely to endorse “stopping international travel.” Partisanship effects for “stopping international travel” are non-significant, except liberal prompts were more likely to endorse “stopping international travel” for completions that began with “I believe that.” Results for halting travel are therefore also inconsistent, but lean slightly in the correct direction.¹³¹³13Gadarian et al. (2021) find that Democrats were less likely to support air travel restrictions or banning visitors from countries with COVID-19 outbreaks

Lastly, Figure 8 shows results from general questions on the COVID-19 pandemic. The first two questions assess confidence in the CDC. Liberal prompts were more likely to state that the CDC is “doing a good job” handling the situation, while conservative prompts were more likely to state that the CDC is “exaggerating the danger posed by the virus.” The next prompt poses whether the virus is something to be afraid of. Intriguingly, conservative prompts were more likely to state that the virus is dangerous than the liberal prompt, an apparent inconsistency with the prior tendency of conservative prompts to state that the CDC is exaggerating the threat. Finally, we ask whether the virus originated in a lab or a wild animal. In these initial prompts, we do not observe a partisan difference. However, when we mention in the prompt that COVID-19 originated “in China,” conservative prompts are associated with higher likelihood than liberal prompts of speculating that the virus originated in a lab.

A collective overview of the results suggests that the partisan associations generated by GPT-3 mirror historical associations at a rate far outperforming chance. To assure that this assessment is correct, we conduct multiple tests of overall statistical significance in the Appendix. All tests, ranging from simple binomial tests to multi-level cross-classified models all find that GPT-3’s predictive capabilities outperform chance with p-values consistently below 0.001. Models are described and results are presented in Appendix.

LLMs can serve as powerful tools for the analysis of language and culture. These models are unique in that they do not merely provide a map of associations, they generate new texts consistent with the linguistic patterns and discursive styles on which they were trained. It is therefore possible to use LLMs to create “digital doubles” of actors fluent in discourses included in the model’s training texts. In this study, we reconstructed the political opinion landscape of a pivotal period, the year preceding the spread of COVID-19, to determine whether the following politicization of the pandemic was predictable given the existing regime of discourse and politics. We find that the model predicts the correct direction of politicization far better than chance across a wide array of pandemic-related issues. To gain insight into how the model made these predictions, we prompt the LLM to produce justifications for its responses. We find on key issues that the distrust of institutions and the prioritization of personal freedom characteristic of American conservatism corresponded with their greater rates of rejecting sweeping policies to restrict the spread of the virus, whereas liberal prompts tend to justify strong collective responses to the virus with appeals to government responsibility and public safety. These results suggest that the way the pandemic politicized was largely consistent with existing repertoires of American liberalism and conservatism, and that the pandemic was not an occasion of substantial political innovation or surprise.

These findings speak to a fundamental question in the study of culture – to what extent are historical developments constrained by culture? When a new issue emerges, is it interpreted within an existing system of understanding, or is it “up for grabs,” with political and cultural entrepreneurs offering competing frames for interpretation? Our findings suggest that, in the case of COVID-19’s polarization, existing schemas of political sense-making steered the issue’s reception. While President Trump’s early statements downplaying the need for drastic responses may have helped crystallize the observed pattern of polarization, our evidence suggests that this stance was consistent with a general ideological tendency that predated the virus’s emergence and channeled a zeitgeist already present among Trump’s constituency. We do not deny that the political response to the COVID-19 pandemic could have unfolded differently, but our evidence suggests that in this case, history manifested along the predictable path consistent with the features of liberalism and conservatism in 2019.

It is noteworthy that the considerations the LLM uses to justify its stances often deviate from those emphasized within political psychology. Many of the pillars of conservatism offered by political psychologists — uncertainty avoidance, fear of threat, need for order and structure, and respect for authority \parenciteJost2006-wm — would have likely motivated opposite responses to many COVID-19 policies. Nevertheless, these considerations were not well represented among produced justifications. Instead, LLM-generated justifications more often made appeals to personal freedom and skepticism toward governmental and scientific authorities, which were consistent with conservatives’ observed resistance to large-scale restrictions.

Our study presents one potential application of LLMs to the analysis of cultural systems, but a variety of alternative approaches are likely to prove fruitful for future inquiry. First, our analytic approach benefits from historical happenstance — the first human-level LLM was trained on texts published immediately prior to an unprecedented socio-political event. In order for LLMs to be a general tool for the analysis of socio-cultural change, researchers should intentionally train models sequentially on ordered time periods, releasing model “checkpoints” with training restricted to different periods of training texts. This pattern of sequential training mirrors “curriculum learning” in which training examples are presented to a model in sequence from simple to complex \parenciteBengio2009-ig, but we propose a method that instead proceeds chronologically, encouraging the model to learn the present in terms of the past, as in the historic unfolding of social life. This sequential training approach enables two key analytic opportunities. First, by comparing models trained on subsequent periods, an analyst can identify the rate and direction of historical cultural shifts. Second, by comparing periods immediately prior and following moments of important social framing, the influence of an event can be assessed.

Figure 12 schematically represents how time-stamped, sequentially trained LLMs could enable a “causal cultural analysis.” The top row displays representations of cultural systems drawn from time-stamped LLMs, here rendered as two dimensional point clouds. “Historical events” are rendered below as networked associations between three focal concepts (e.g., red, green, and blue). Linking these two levels of observation could facilitate an empirical operationalization of Giddens’ structuration theory \parenciteGiddens1984-vx or Coleman’s “boat” models of macro-micro-macro relations \parenciteColeman1994-ay. In our example, cultural world models are checkpointed at two times: t₁ and t₂. Irregular events e₁ through e₄ each render the focal three concepts, which could represent ideas, facts, stereotypes, or narrative elements with different orientations to one another and the rest of the cultural world. In this scenario, event e₂ is wholly unsurprising, as it perfectly reproduces the standing system of cultural relations. By contrast, event e₃ not only deviates from the prior cultural system, but changes the cultural world, becoming typical in the subsequent time period. Most surprising associations (e.g., e₁ and e₄) do not change the cultural world. These represent the failed innovations, unfunny jokes, and unfortunate accidents of history that do not cause anything but their own forgetting.

Using LLMs as cultural world models, a wide range of causal identification strategies \parencitePearl2009-on could be deployed in order to do causal cultural history—to identify when cultural events like speeches, concerts, new products, or viral memes change the space of associations; and when the space of associations changes the distribution of cultural events that follow. Our ability to generate speech events from cultural world models, and evolve cultural world models from speech events through fine-tuning enables the production of richly situated counterfactuals for probabilistic identification. This potential for cultural measurement and identification could allow us to relax the assumed conservation of influence from the macro-cultural ether to individual cultural behaviors and back again, suggesting when change is conditioned and disproportionately driven from the “bottom-up” or the “top-down”.

We note that there are several practical reasons why LLMs are currently not developed through chronological training, including: (1) the lower average quality of earlier text, leading of path-dependent low-quality models; (2) the inaccurately dated character of text on the web, LLM’s dominant corpus; (3) the phenomena of catastrophic forgetting whereby deep neural networks learn new tasks without retaining the ability to perform past ones \parenciteMcCloskey1989-rg, Ratcliff1990-bk; and (4) the associated problem of mixing styles that allows LLMs to efficiently link text across a large asynchronous corpus. However, many of these limitations can be engineered around, albeit with non-trivial expense. Early text can be corrected, or translated into contemporary formats. Web-text can be dated using time-stamped informal text and news. Earlier texts can be incorporated into new training samples in proportion to metrics reflecting their circulation and presence in current collective memory, such as their recent citation rate.

Beyond this, LLMs present tremendous new opportunities to simulate complex interactions between “encultured” agents. Our study uses LLMs to generate simulated data that resemble responses to open-ended survey questions, in which each response is treated as an independent sample from a population of possible responses. But social life emerges through interaction, and LLMs enable the simulation of semantically-rich interaction at scale. Many forms of consequential social interactions could be analyzed in silico that would be challenging to reproduce in a laboratory or online, such as parliamentary discourse, scientific conferences, or cross-cultural collaborations. Moreover, simulated interactions can be easily run millions of times over with initial conditions experimentally manipulated to encourage wide coverage of the space of possible interaction outcomes. Findings inductively discovered from an extensive search over the interaction space could then be validated with human subjects, or for certain hard-to-observe social phenomena for which validation is impossible, simulation may ultimately provide the best obtainable evidence.

Lastly, the internal representations of AI agents are directly observable in a way that human internal representations are not, and may therefore yield key insights into how complex meaning systems can be efficiently modeled and compressed. Although the activation patterns of deep neural networks are commonly described as black boxes, the growing field of mechanistic interpretability is making progress in mapping relations between neuronal activation and model behavior \parenciteBills2023-db, Elhage2022-bh. More and more findings demonstrate the lower-level “neurons” with the transformer’s self-attention architecture facilitate novel interpretive and simulational capacities \parenciteHendel2023-ij. As this field matures, it may enable an “AI neuroscience,” whereby differences in behavior or discourse can be translated into differences in latent representation. Although it is likely that lessons from artificial neural networks cannot be directly applied to human brains, their visibility and manipulability make them a valuable site for exploratory studies linking distributed representations to phenomena of culture and cognition.

As AI models continue to achieve more accurate approximations of human behavior, they will become increasingly powerful tools for the analysis of complex social processes. The advent of LLMs has now made simulation studies of discourse possible for the first time, opening many new avenues for investigation into language, culture, and meaning. Moreover, just as LLMs trained on massive collections of text learn the patterns of language, multi-modal models trained on images, videos, and other records of social life can similarly distill underlying patterns across these diverse domains \parenciteGuilbeault2024-wz, Ludwig2024-nl. Social scientists are fundamentally interested in what people do, and direct observation of human action remains the essential cornerstone of our science. But as empirically realistic agents can be achieved in silico, social simulations will offer opportunities to explore social phenomena beyond the observable, shedding light on the underlying patterns and branching pathways that structure social life and give rise to its complex and varied forms.