Showing 1–22 of 22 results for author: Evans, J A

In Silico Sociology: Forecasting COVID-19 Polarization with Large Language Models

Authors: Austin C. Kozlowski, Hyunku Kwon, James A. Evans

Abstract: By training deep neural networks on massive archives of digitized text, large language models (LLMs) learn the complex linguistic patterns that constitute historic and contemporary discourses. We argue that LLMs can serve as a valuable tool for sociological inquiry by enabling accurate simulation of respondents from specific social and cultural contexts. Applying LLMs in this capacity, we reconstr… ▽ More By training deep neural networks on massive archives of digitized text, large language models (LLMs) learn the complex linguistic patterns that constitute historic and contemporary discourses. We argue that LLMs can serve as a valuable tool for sociological inquiry by enabling accurate simulation of respondents from specific social and cultural contexts. Applying LLMs in this capacity, we reconstruct the public opinion landscape of 2019 to examine the extent to which the future polarization over COVID-19 was prefigured in existing political discourse. Using an LLM trained on texts published through 2019, we simulate the responses of American liberals and conservatives to a battery of pandemic-related questions. We find that the simulated respondents reproduce observed partisan differences in COVID-19 attitudes in 84% of cases, significantly greater than chance. Prompting the simulated respondents to justify their responses, we find that much of the observed partisan gap corresponds to differing appeals to freedom, safety, and institutional trust. Our findings suggest that the politicization of COVID-19 was largely consistent with the prior ideological landscape, and this unprecedented event served to advance history along its track rather than change the rails. △ Less

Submitted 23 May, 2024; originally announced July 2024.

In Situ Framework for Coupling Simulation and Machine Learning with Application to CFD

Authors: Riccardo Balin, Filippo Simini, Cooper Simpson, Andrew Shao, Alessandro Rigazzi, Matthew Ellis, Stephen Becker, Alireza Doostan, John A. Evans, Kenneth E. Jansen

Abstract: Recent years have seen many successful applications of machine learning (ML) to facilitate fluid dynamic computations. As simulations grow, generating new training datasets for traditional offline learning creates I/O and storage bottlenecks. Additionally, performing inference at runtime requires non-trivial coupling of ML framework libraries with simulation codes. This work offers a solution to b… ▽ More Recent years have seen many successful applications of machine learning (ML) to facilitate fluid dynamic computations. As simulations grow, generating new training datasets for traditional offline learning creates I/O and storage bottlenecks. Additionally, performing inference at runtime requires non-trivial coupling of ML framework libraries with simulation codes. This work offers a solution to both limitations by simplifying this coupling and enabling in situ training and inference workflows on heterogeneous clusters. Leveraging SmartSim, the presented framework deploys a database to store data and ML models in memory, thus circumventing the file system. On the Polaris supercomputer, we demonstrate perfect scaling efficiency to the full machine size of the data transfer and inference costs thanks to a novel co-located deployment of the database. Moreover, we train an autoencoder in situ from a turbulent flow simulation, showing that the framework overhead is negligible relative to a solver time step and training epoch. △ Less

Submitted 22 June, 2023; originally announced June 2023.

China and the U.S. produce more impactful AI research when collaborating together

Authors: Bedoor AlShebli, Shahan Ali Memon, James A. Evans, Talal Rahwan

Abstract: Artificial Intelligence (AI) has become a disruptive technology, promising to grant a significant economic and strategic advantage to the nations that harness its power. China, with its recent push towards AI adoption, is challenging the U.S.'s position as the global leader in this field. Given AI's massive potential, as well as the fierce geopolitical tensions between the two nations, a number of… ▽ More Artificial Intelligence (AI) has become a disruptive technology, promising to grant a significant economic and strategic advantage to the nations that harness its power. China, with its recent push towards AI adoption, is challenging the U.S.'s position as the global leader in this field. Given AI's massive potential, as well as the fierce geopolitical tensions between the two nations, a number of policies have been put in place that discourage AI scientists from migrating to, or collaborating with, the other country. However, the extents of such brain drain and cross-border collaboration are not fully understood. Here, we analyze a dataset of over 350,000 AI scientists and 5,000,000 AI papers. We find that, since the year 2000, China and the U.S. have been leading the field in terms of impact, novelty, productivity, and workforce. Most AI scientists who migrate to China come from the U.S., and most who migrate to the U.S. come from China, highlighting a notable brain drain in both directions. Upon migrating from one country to the other, scientists continue to collaborate frequently with the origin country. Although the number of collaborations between the two countries has been increasing since the dawn of the millennium, such collaborations continue to be relatively rare. A matching experiment reveals that the two countries have always been more impactful when collaborating than when each of them works without the other. These findings suggest that instead of suppressing cross-border migration and collaboration between the two nations, the field could benefit from promoting such activities. △ Less

Submitted 21 April, 2023; originally announced April 2023.

Comments: 23 pages, 5 figures, 4 tables

Limited Diffusion of Scientific Knowledge Forecasts Collapse

Authors: Donghyun Kang, Robert S. Danziger, Jalees Rehman, James A. Evans

Abstract: Market bubbles emerge when asset prices are driven unsustainably higher than asset values and shifts in belief burst them. We demonstrate the same phenomenon for biomedical knowledge when promising research receives inflated attention. We predict deflationary events by developing a diffusion index that captures whether research areas have been amplified within social and scientific bubbles or have… ▽ More Market bubbles emerge when asset prices are driven unsustainably higher than asset values and shifts in belief burst them. We demonstrate the same phenomenon for biomedical knowledge when promising research receives inflated attention. We predict deflationary events by developing a diffusion index that captures whether research areas have been amplified within social and scientific bubbles or have diffused and become evaluated more broadly. We illustrate our diffusion approach contrasting the trajectories of cardiac stem cell research and cancer immunotherapy. We then trace the diffusion of unique 28,504 subfields in biomedicine comprising nearly 1.9M papers and more than 80M citations and demonstrate that limited diffusion of biomedical knowledge anticipates abrupt decreases in popularity. Our analysis emphasizes that restricted diffusion, implying a socio-epistemic bubble, leads to dramatic collapses in relevance and attention accorded to scientific knowledge. △ Less

Submitted 18 April, 2024; v1 submitted 15 April, 2023; originally announced April 2023.

Disrupted Routines Anticipate Musical Exploration

Authors: Khwan Kim, Noah Askin, James A. Evans

Abstract: Prior research suggests that taste preferences relate to personality traits, values, shifts in mood, and immigration destination, but understanding everyday patterns of listening and the function music plays in life have remained elusive, despite speculations that musical nostalgia may compensate for local disruption. Using more than a hundred million streams of 4 million songs by tens of thousand… ▽ More Prior research suggests that taste preferences relate to personality traits, values, shifts in mood, and immigration destination, but understanding everyday patterns of listening and the function music plays in life have remained elusive, despite speculations that musical nostalgia may compensate for local disruption. Using more than a hundred million streams of 4 million songs by tens of thousands of international listeners from a global music service catering to local tastes, here we show that breaches in personal routine are systematically associated with personal musical exploration. As people visited new cities and countries, their preferences diversified, converging towards their destinations. As people experienced COVID-19 lock-downs, and then again when they experienced reopenings, their preferences diversified further. △ Less

Submitted 9 January, 2023; originally announced January 2023.

Quantifying hierarchy in scientific teams

Authors: Fengli Xu, Lingfei Wu, James A. Evans

Abstract: This paper provides a detailed description of the data collection and machine learning model used in our recent PNAS paper "Flat Teams Drive Scientific Innovation" Xu et al. [2022a]. Here, we discuss how the features of scientific publication can be used to estimate the implicit hierarchy in the corresponding author teams. Besides, we also describe the method of evaluating the impact of team hiera… ▽ More This paper provides a detailed description of the data collection and machine learning model used in our recent PNAS paper "Flat Teams Drive Scientific Innovation" Xu et al. [2022a]. Here, we discuss how the features of scientific publication can be used to estimate the implicit hierarchy in the corresponding author teams. Besides, we also describe the method of evaluating the impact of team hierarchy on scientific outputs. More details will be updated in this article continuously. Raw data and Readme document can be accessed in this GitHub repository Xu et al. [2022b]. △ Less

Submitted 11 October, 2022; originally announced October 2022.

Aging Scientists and Slowed Advance

Authors: Haochuan Cui, Lingfei Wu, James A. Evans

Abstract: What is the relationship between aging and the character of scientific advance? Prior research focuses on star scientists, their changing dates, and rates of breakthrough success through history. Analyzing more than 244 million scholars across 241 million articles over the last two centuries, we show that for all fields, periods, and impact levels, scientists research ideas and references age over… ▽ More What is the relationship between aging and the character of scientific advance? Prior research focuses on star scientists, their changing dates, and rates of breakthrough success through history. Analyzing more than 244 million scholars across 241 million articles over the last two centuries, we show that for all fields, periods, and impact levels, scientists research ideas and references age over time, their research is less likely to disrupt the state of science and more likely to criticize emerging work. Early success accelerates scientist aging; while changing institutions and fields and collaborating with young scientists slows it. These patterns aggregate within fields such that those with a higher proportion of older scientists experience a lower churn of ideas and more rapid individual aging, suggesting a universal link between aging, activity, and advance. △ Less

Submitted 8 February, 2022; originally announced February 2022.

Comments: 37 pages, 18 figures

Learning from One and Only One Shot

Authors: Haizi Yu, Igor Mineyev, Lav R. Varshney, James A. Evans

Abstract: Humans can generalize from only a few examples and from little pretraining on similar tasks. Yet, machine learning (ML) typically requires large data to learn or pre-learn to transfer. Motivated by nativism and artificial general intelligence, we directly model human-innate priors in abstract visual tasks such as character and doodle recognition. This yields a white-box model that learns general-a… ▽ More Humans can generalize from only a few examples and from little pretraining on similar tasks. Yet, machine learning (ML) typically requires large data to learn or pre-learn to transfer. Motivated by nativism and artificial general intelligence, we directly model human-innate priors in abstract visual tasks such as character and doodle recognition. This yields a white-box model that learns general-appearance similarity by mimicking how humans naturally ``distort'' an object at first sight. Using just nearest-neighbor classification on this cognitively-inspired similarity space, we achieve human-level recognition with only $1$--$10$ examples per class and no pretraining. This differs from few-shot learning that uses massive pretraining. In the tiny-data regime of MNIST, EMNIST, Omniglot, and QuickDraw benchmarks, we outperform both modern neural networks and classical ML. For unsupervised learning, by learning the non-Euclidean, general-appearance similarity space in a $k$-means style, we achieve multifarious visual realizations of abstract concepts by generating human-intuitive archetypes as cluster centroids. △ Less

Submitted 21 May, 2024; v1 submitted 14 January, 2022; originally announced January 2022.

Human Languages with Greater Information Density Increase Communication Speed, but Decrease Conversation Breadth

Authors: Pedro Aceves, James A. Evans

Abstract: Human languages vary widely in how they encode information within circumscribed semantic domains (e.g., time, space, color, human body parts and activities), but little is known about the global structure of semantic information and nothing about its relation to human communication. We first show that across a sample of ~1,000 languages, there is broad variation in how densely languages encode inf… ▽ More Human languages vary widely in how they encode information within circumscribed semantic domains (e.g., time, space, color, human body parts and activities), but little is known about the global structure of semantic information and nothing about its relation to human communication. We first show that across a sample of ~1,000 languages, there is broad variation in how densely languages encode information into their words. Second, we show that this language information density is associated with a denser configuration of semantic information. Finally, we trace the relationship between language information density and patterns of communication, showing that informationally denser languages tend toward (1) faster communication, but (2) conceptually narrower conversations within which topics of conversation are discussed at greater depth. These results highlight an important source of variation across the human communicative channel, revealing that the structure of language shapes the nature and texture of human engagement, with consequences for human behavior across levels of society. △ Less

Submitted 29 September, 2023; v1 submitted 15 December, 2021; originally announced December 2021.

Comments: Nat Hum Behav (2024)

New Directions in Science Emerge from Disconnection and Discord

Authors: Yiling Lin, James Allen Evans, Lingfei Wu

Abstract: Science is built on the scholarly consensus that shifts with time. This raises the question of how new and revolutionary ideas are evaluated and become accepted into the canon of science. Using two recently proposed metrics, we identify papers with high atypicality, which models how research draws upon novel combinations of prior research, and evaluate disruption, which captures the degree to whic… ▽ More Science is built on the scholarly consensus that shifts with time. This raises the question of how new and revolutionary ideas are evaluated and become accepted into the canon of science. Using two recently proposed metrics, we identify papers with high atypicality, which models how research draws upon novel combinations of prior research, and evaluate disruption, which captures the degree to which a study creates a new direction by eclipsing its intellectual forebears. Atypical papers are nearly two times more likely to disrupt science than conventional papers, but this is a slow process taking ten years or longer for disruption scores to converge. We provide the first computational model reformulating atypicality as the distance across latent knowledge spaces learned by neural networks. The evolution of this knowledge space characterizes how yesterday's novelty forms today's scientific conventions, which condition the novelty--and surprise--of tomorrow's breakthroughs. △ Less

Submitted 19 November, 2021; v1 submitted 4 March, 2021; originally announced March 2021.

Policy-Aware Mobility Model Explains the Growth of COVID-19 in Cities

Authors: Zhenyu Han, Fengli Xu, Yong Li, Tao Jiang, Depeng Jin, Jianhua Lu, James A. Evans

Abstract: With the continued spread of coronavirus, the task of forecasting distinctive COVID-19 growth curves in different cities, which remain inadequately explained by standard epidemiological models, is critical for medical supply and treatment. Predictions must take into account non-pharmaceutical interventions to slow the spread of coronavirus, including stay-at-home orders, social distancing, quarant… ▽ More With the continued spread of coronavirus, the task of forecasting distinctive COVID-19 growth curves in different cities, which remain inadequately explained by standard epidemiological models, is critical for medical supply and treatment. Predictions must take into account non-pharmaceutical interventions to slow the spread of coronavirus, including stay-at-home orders, social distancing, quarantine and compulsory mask-wearing, leading to reductions in intra-city mobility and viral transmission. Moreover, recent work associating coronavirus with human mobility and detailed movement data suggest the need to consider urban mobility in disease forecasts. Here we show that by incorporating intra-city mobility and policy adoption into a novel metapopulation SEIR model, we can accurately predict complex COVID-19 growth patterns in U.S. cities ($R^2$ = 0.990). Estimated mobility change due to policy interventions is consistent with empirical observation from Apple Mobility Trends Reports (Pearson's R = 0.872), suggesting the utility of model-based predictions where data are limited. Our model also reproduces urban "superspreading", where a few neighborhoods account for most secondary infections across urban space, arising from uneven neighborhood populations and heightened intra-city churn in popular neighborhoods. Therefore, our model can facilitate location-aware mobility reduction policy that more effectively mitigates disease transmission at similar social cost. Finally, we demonstrate our model can serve as a fine-grained analytic and simulation framework that informs the design of rational non-pharmaceutical interventions policies. △ Less

Submitted 21 February, 2021; originally announced February 2021.

Low-skilled Occupations Face the Highest Upskilling Pressure

Authors: Di Tong, Lingfei Wu, James Allen Evans

Abstract: Substantial scholarship has estimated the susceptibility of jobs to automation, but little has examined how job contents evolve in the information age as new technologies substitute for tasks, shifting required skills rather than eliminating entire jobs. Here we explore patterns and consequences of changes in occupational skill and characterize occupations and workers subject to the greatest re-sk… ▽ More Substantial scholarship has estimated the susceptibility of jobs to automation, but little has examined how job contents evolve in the information age as new technologies substitute for tasks, shifting required skills rather than eliminating entire jobs. Here we explore patterns and consequences of changes in occupational skill and characterize occupations and workers subject to the greatest re-skilling pressure. Recent work found that changing skill requirements are greatest for STEM occupations. Nevertheless, analyzing 167 million online job posts covering 727 occupations over the last decade, we find that re-skilling pressure is greatest for low-skilled occupations when accounting for distance between skills. We further investigate the differences in skill change across employer and market size, as well as social demographic groups, and find that these differences tend to widen the economic divide. Jobs from large employers and markets experienced less change relative to small employers and markets, and non-white workers in low-skilled jobs are most demographically vulnerable. We conclude by showcasing our model's potential to precisely chart job evolution towards machine-interface integration using skill embedding spaces. △ Less

Submitted 7 December, 2023; v1 submitted 27 January, 2021; originally announced January 2021.

Comments: 44 pages; 10 figures

Spectral mesh-free quadrature for planar regions bounded by rational parametric curves

Authors: David Gunderman, Kenneth Weiss, John A. Evans

Abstract: This work presents spectral, mesh-free, Green's theorem-based numerical quadrature schemes for integrating functions over planar regions bounded by rational parametric curves. Our algorithm proceeds in two steps: (1) We first find intermediate quadrature rules for line integrals along the region's boundary curves corresponding to Green's theorem. (2) We then use a high-order quadrature rule to com… ▽ More This work presents spectral, mesh-free, Green's theorem-based numerical quadrature schemes for integrating functions over planar regions bounded by rational parametric curves. Our algorithm proceeds in two steps: (1) We first find intermediate quadrature rules for line integrals along the region's boundary curves corresponding to Green's theorem. (2) We then use a high-order quadrature rule to compute the numerical antiderivative of the integrand along a coordinate axis, which is used to evaluate the Green's theorem line integral. We present two methods to compute the intermediate quadrature rule. The first is spectrally accurate (it converges faster than any algebraic order with respect to number of quadrature points) and is relatively easy to implement, but has no guarantee of polynomial exactness. The second guarantees exactness for polynomial integrands up to a pre-specified degree k with an a priori-known number of quadrature points and retains the convergence properties of the first, but is slightly more complicated. The quadrature schemes have applications to computation of geometric moments, immersogeometric analysis, conservative field transfer between high-order meshes, and initialization of simulations with rational geometry. We compare the quadrature schemes produced using our method to other methods in the literature and show that they are much more efficient both in terms of number of quadrature points and computational time. △ Less

Submitted 14 September, 2020; v1 submitted 15 May, 2020; originally announced May 2020.

Interactive Geometry Modification of High Performance Finite Element Simulations

Authors: Corey Wetterer-Nelson, Kenneth E. Jansen, John A. Evans

Abstract: In the context of high performance finite element analysis, the cost of iteratively modifying a computational domain via re-meshing and restarting the analysis becomes time prohibitive as the size of simulations increases. In this paper, we demonstrate a new interactive simulation pipeline targeting high performance finite element simulations where the computational domain is modifiable in situ, t… ▽ More In the context of high performance finite element analysis, the cost of iteratively modifying a computational domain via re-meshing and restarting the analysis becomes time prohibitive as the size of simulations increases. In this paper, we demonstrate a new interactive simulation pipeline targeting high performance finite element simulations where the computational domain is modifiable in situ, that is, while the simulation is ongoing. This pipeline is designed to be modular so that it may interface with any existing finite element simulation framework. A server-client architecture is employed to manage simulation mesh data existing on a high performance computing resource while user-prescribed freeform geometric modifications take place on a separate workstation. We employ existing in situ visualization techniques to rapidly inform the user of simulation progression, enabling computational steering. By expressing the simulation domain in a reduced fashion on the client application, this pipeline manages highly refined finite element simulation domains on the server while maintaining good performance on the client application. △ Less

Submitted 19 May, 2021; v1 submitted 30 April, 2020; originally announced May 2020.

Human Evaluation of Interpretability: The Case of AI-Generated Music Knowledge

Authors: Haizi Yu, Heinrich Taube, James A. Evans, Lav R. Varshney

Abstract: Interpretability of machine learning models has gained more and more attention among researchers in the artificial intelligence (AI) and human-computer interaction (HCI) communities. Most existing work focuses on decision making, whereas we consider knowledge discovery. In particular, we focus on evaluating AI-discovered knowledge/rules in the arts and humanities. From a specific scenario, we pres… ▽ More Interpretability of machine learning models has gained more and more attention among researchers in the artificial intelligence (AI) and human-computer interaction (HCI) communities. Most existing work focuses on decision making, whereas we consider knowledge discovery. In particular, we focus on evaluating AI-discovered knowledge/rules in the arts and humanities. From a specific scenario, we present an experimental procedure to collect and assess human-generated verbal interpretations of AI-generated music theory/rules rendered as sophisticated symbolic/numeric objects. Our goal is to reveal both the possibilities and the challenges in such a process of decoding expressive messages from AI sources. We treat this as a first step towards 1) better design of AI representations that are human interpretable and 2) a general methodology to evaluate interpretability of AI-discovered knowledge representations. △ Less

Submitted 15 April, 2020; originally announced April 2020.

Too many cooks: Bayesian inference for coordinating multi-agent collaboration

Authors: Rose E. Wang, Sarah A. Wu, James A. Evans, Joshua B. Tenenbaum, David C. Parkes, Max Kleiman-Weiner

Abstract: Collaboration requires agents to coordinate their behavior on the fly, sometimes cooperating to solve a single task together and other times dividing it up into sub-tasks to work on in parallel. Underlying the human ability to collaborate is theory-of-mind, the ability to infer the hidden mental states that drive others to act. Here, we develop Bayesian Delegation, a decentralized multi-agent lear… ▽ More Collaboration requires agents to coordinate their behavior on the fly, sometimes cooperating to solve a single task together and other times dividing it up into sub-tasks to work on in parallel. Underlying the human ability to collaborate is theory-of-mind, the ability to infer the hidden mental states that drive others to act. Here, we develop Bayesian Delegation, a decentralized multi-agent learning mechanism with these abilities. Bayesian Delegation enables agents to rapidly infer the hidden intentions of others by inverse planning. We test Bayesian Delegation in a suite of multi-agent Markov decision processes inspired by cooking problems. On these tasks, agents with Bayesian Delegation coordinate both their high-level plans (e.g. what sub-task they should work on) and their low-level actions (e.g. avoiding getting in each other's way). In a self-play evaluation, Bayesian Delegation outperforms alternative algorithms. Bayesian Delegation is also a capable ad-hoc collaborator and successfully coordinates with other agent types even in the absence of prior experience. Finally, in a behavioral experiment, we show that Bayesian Delegation makes inferences similar to human observers about the intent of others. Together, these results demonstrate the power of Bayesian Delegation for decentralized multi-agent collaboration. △ Less

Submitted 5 July, 2020; v1 submitted 26 March, 2020; originally announced March 2020.

Comments: Rose E. Wang and Sarah A. Wu contributed equally

Social Centralization and Semantic Collapse: Hyperbolic Embeddings of Networks and Text

Authors: Linzhuo Li, Lingfei Wu, James Allen Evans

Abstract: Modern advances in transportation and communication technology from airplanes to the internet alongside global expansions of media, migration, and trade have made the modern world more connected than ever before. But what does this bode for the convergence of global culture? Here we explore the relationship between centralization in social networks and contraction or collapse in the diversity of s… ▽ More Modern advances in transportation and communication technology from airplanes to the internet alongside global expansions of media, migration, and trade have made the modern world more connected than ever before. But what does this bode for the convergence of global culture? Here we explore the relationship between centralization in social networks and contraction or collapse in the diversity of semantic expressions such as ideas, opinions, and tastes. We advance formal examination of this relationship by introducing new methods of manifold learning that allow us to map social networks and semantic combinations into comparable hyperbolic spaces. Hyperbolic representations natively represent both hierarchy and diversity within a system. We illustrate this method by examining the relationship between social centralization and semantic diversity within 21st Century physics, empirically demonstrating how dense, centralized collaboration is associated with a reduction in the space of ideas and how these patterns generalize to all modern scholarship and science. We discuss the complex of causes underlying this association, and theorize the dynamic interplay between structural centralization and semantic contraction, arguing that it introduces an essential tension between the supply and demand of difference. △ Less

Submitted 26 January, 2020; originally announced January 2020.

Comments: 55 pages. arXiv admin note: substantial text overlap with arXiv:1807.10216

MSC Class: 91-08; 91C20; 91D30 ACM Class: J.4

Quantifying dynamics of failure across science, startups, and security

Authors: Yian Yin, Yang Wang, James A. Evans, Dashun Wang

Abstract: Human achievements are often preceded by repeated attempts that initially fail, yet little is known about the mechanisms governing the dynamics of failure. Here, building on the rich literature on innovation, human dynamics and learning, we develop a simple one-parameter model that mimics how successful future attempts build on those past. Analytically solving this model reveals a phase transition… ▽ More Human achievements are often preceded by repeated attempts that initially fail, yet little is known about the mechanisms governing the dynamics of failure. Here, building on the rich literature on innovation, human dynamics and learning, we develop a simple one-parameter model that mimics how successful future attempts build on those past. Analytically solving this model reveals a phase transition that separates dynamics of failure into regions of stagnation or progression, predicting that near the critical threshold, agents who share similar characteristics and learning strategies may experience fundamentally different outcomes following failures. Below the critical point, we see those who explore disjoint opportunities without a pattern of improvement, and above it, those who exploit incremental refinements to systematically advance toward success. The model makes several empirically testable predictions, demonstrating that those who eventually succeed and those who do not may be initially similar, yet are characterized by fundamentally distinct failure dynamics in terms of the efficiency and quality of each subsequent attempt. We collected large-scale data from three disparate domains, tracing repeated attempts by (i) NIH investigators to fund their research, (ii) innovators to successfully exit their startup ventures, and (iii) terrorist organizations to post casualties in violent attacks, finding broadly consistent empirical support across all three domains. Together, our findings unveil identifiable yet previously unknown early signals that allow us to identify failure dynamics that will lead to ultimate victory or defeat. Given the ubiquitous nature of failures and the paucity of quantitative approaches to understand them, these results represent a crucial step toward deeper understanding of the complex dynamics beneath failures, the essential prerequisites for success. △ Less

Submitted 18 March, 2019; originally announced March 2019.

The Geometry of Culture: Analyzing Meaning through Word Embeddings

Authors: Austin C. Kozlowski, Matt Taddy, James A. Evans

Abstract: We demonstrate the utility of a new methodological tool, neural-network word embedding models, for large-scale text analysis, revealing how these models produce richer insights into cultural associations and categories than possible with prior methods. Word embeddings represent semantic relations between words as geometric relationships between vectors in a high-dimensional space, operationalizing… ▽ More We demonstrate the utility of a new methodological tool, neural-network word embedding models, for large-scale text analysis, revealing how these models produce richer insights into cultural associations and categories than possible with prior methods. Word embeddings represent semantic relations between words as geometric relationships between vectors in a high-dimensional space, operationalizing a relational model of meaning consistent with contemporary theories of identity and culture. We show that dimensions induced by word differences (e.g. man - woman, rich - poor, black - white, liberal - conservative) in these vector spaces closely correspond to dimensions of cultural meaning, and the projection of words onto these dimensions reflects widely shared cultural connotations when compared to surveyed responses and labeled historical data. We pilot a method for testing the stability of these associations, then demonstrate applications of word embeddings for macro-cultural investigation with a longitudinal analysis of the coevolution of gender and class associations in the United States over the 20th century and a comparative analysis of historic distinctions between markers of gender and class in the U.S. and Britain. We argue that the success of these high-dimensional models motivates a move towards "high-dimensional theorizing" of meanings, identities and cultural processes. △ Less

Submitted 25 March, 2018; originally announced March 2018.

Journal ref: American Sociological Review 2019, Vol. 84(5) 905-949

Centralized "big science" communities more likely generate non-replicable results

Authors: Valentin Danchev, Andrey Rzhetsky, James A. Evans

Abstract: Growing concern that most published results, including those widely agreed upon, may be false are rarely examined against rapidly expanding research production. Replications have only occurred on small scales due to prohibitive expense and limited professional incentive. We introduce a novel, high-throughput replication strategy aligning 51,292 published claims about drug-gene interactions with hi… ▽ More Growing concern that most published results, including those widely agreed upon, may be false are rarely examined against rapidly expanding research production. Replications have only occurred on small scales due to prohibitive expense and limited professional incentive. We introduce a novel, high-throughput replication strategy aligning 51,292 published claims about drug-gene interactions with high-throughput experiments performed through the NIH LINCS L1000 program. We show (1) that unique claims replicate 19% more frequently than at random, while those widely agreed upon replicate 45% more frequently, manifesting collective correction mechanisms in science; but (2) centralized scientific communities perpetuate claims that are less likely to replicate even if widely agreed upon, demonstrating how centralized, overlapping collaborations weaken collective understanding. Decentralized research communities involve more independent teams and use more diverse methodologies, generating the most robust, replicable results. Our findings highlight the importance of science policies that foster decentralized collaboration to promote robust scientific advance. △ Less

Submitted 15 January, 2018; originally announced January 2018.

Large Teams Have Developed Science and Technology; Small Teams Have Disrupted It

Authors: Lingfei Wu, Dashun Wang, James A. Evans

Abstract: Teams dominate the production of high-impact science and technology. Analyzing teamwork from more than 50 million papers, patents, and software products, 1954-2014, we demonstrate across this period that larger teams developed recent, popular ideas, while small teams disrupted the system by drawing on older and less prevalent ideas. Attention to work from large teams came immediately, while advanc… ▽ More Teams dominate the production of high-impact science and technology. Analyzing teamwork from more than 50 million papers, patents, and software products, 1954-2014, we demonstrate across this period that larger teams developed recent, popular ideas, while small teams disrupted the system by drawing on older and less prevalent ideas. Attention to work from large teams came immediately, while advances by small teams succeeded further into the future. Differences between small and large teams magnify with impact - small teams have become known for disruptive work and large teams for developing work. Differences in topic and re- search design account for part of the relationship between team size and disruption, but most of the effect occurs within people, controlling for detailed subject and article type. Our findings suggest the importance of supporting both small and large teams for the sustainable vitality of science and technology. △ Less

Submitted 11 September, 2017; v1 submitted 7 September, 2017; originally announced September 2017.

Comments: 3 figures

Tradition and Innovation in Scientists' Research Strategies

Authors: Jacob G. Foster, Andrey Rzhetsky, James A. Evans

Abstract: What factors affect a scientist's choice of research problem? Qualitative research in the history, philosophy, and sociology of science suggests that this choice is shaped by an "essential tension" between the professional demand for productivity and a conflicting drive toward risky innovation. We examine this tension empirically in the context of biomedical chemistry. We use complex networks to r… ▽ More What factors affect a scientist's choice of research problem? Qualitative research in the history, philosophy, and sociology of science suggests that this choice is shaped by an "essential tension" between the professional demand for productivity and a conflicting drive toward risky innovation. We examine this tension empirically in the context of biomedical chemistry. We use complex networks to represent the evolving state of scientific knowledge, as expressed in publications. We then define research strategies relative to these networks. Scientists can introduce novel chemicals or chemical relationships--or delve deeper into known ones. They can consolidate existing knowledge clusters, or bridge distant ones. Analyzing such choices in aggregate, we find that the distribution of strategies remains remarkably stable, even as chemical knowledge grows dramatically. High-risk strategies, which explore new chemical relationships, are less prevalent in the literature, reflecting a growing focus on established knowledge at the expense of new opportunities. Research following a risky strategy is more likely to be ignored but also more likely to achieve high impact and recognition. While the outcome of a risky strategy has a higher expected reward than the outcome of a conservative strategy, the additional reward is insufficient to compensate for the additional risk. By studying the winners of 137 different prizes in biomedicine and chemistry, we show that the occasional "gamble" for extraordinary impact is the most plausible explanation for observed levels of risk-taking. Our empirical demonstration and unpacking of the "essential tension" suggests policy interventions that may foster more innovative research. △ Less

Submitted 27 February, 2013; originally announced February 2013.

Comments: 18 pages, 4 figures