The Peer-Review Dilemma

Updated at 11:06 a.m. ET on October 3, 2023

In recent months, two loud public discussions have taken up the question of what scientists really think of their research. “I left out the full truth to get my climate change paper published,” the climatologist Patrick Brown wrote in an essay posted earlier this month, just days after his research had appeared in the journal Nature. The paper’s main finding, that global warming makes extreme wildfires more common, was based on a willful oversimplification of reality, he confessed—and it did not represent his private view that other factors are as or more important.

Another story came out this summer, during the congressional inquiry into COVID’s origins. In this case, House Republicans accused the virologist Kristian Andersen and his colleagues of downplaying the possibility that the coronavirus had leaked out of a lab. A House subcommittee found that the language of a crucial, early paper written by this group that ruled out the “lab-leak theory” had been altered during peer review to make its conclusions “stronger.” Andersen acknowledged that the paper’s blanket dismissal of “any type of laboratory-based scenario” was added in response to comments by the journal’s editor and peer reviewers. But he said this was just part of the normal, iterative scientific process, and that the updated text reflected his beliefs at the time. “The process of peer review means that you incorporate changes, you shorten it down, you make some of the language punchier because you don’t have three sentences to write the same thing, you only have one,” Andersen told investigators. “That’s just part of peer review, scientific publishing.” House Republicans described the publication process differently: as a cover-up. (Of course, some Republicans have seized on the COVID-origins debate for their own political gain.)

Each of these incidents brought demands that the affected papers be retracted—not because they contained fraudulent data or false facts but rather on the grounds that their authors had supposedly been hiding doubts about their own conclusions. Neither paper has been taken down. The editor in chief of Nature did call out Brown’s “poor research practices” and say that the journal is “carefully considering the implications of his stated actions.” (Brown has said that he stands by his research and that “there is nothing explicitly wrong with the paper,” but that “molding the presentation of the research for a high-impact journal made it less useful than it could have been.”) The editor of Nature Medicine, the journal that published Andersen’s paper, said that retraction was unwarranted.

Researchers tend to get into trouble when their published numbers have been faked, or when their math is incorrect. Other matters of dubious judgment—whether pertaining to a study’s design or its interpretation—would fall under the more permissive auspices of scholarly debate. The accusations against Brown and Andersen, however, propose a novel form of misbehavior: the crime of insincerity.

This newly prominent offense aligns with the nation’s mood. In today’s skeptical environment, any outside influence on the work of scientists may be cast as covert manipulation, if not censorship. Brown publicly confessed that he held back his true feelings in order to get the work published in a top journal—that sort of publication is a near-requirement for academic scientists. Prestige periodicals, he claimed, demand obsequious devotion to the most alarming possible narrative about climate change. If he’s right, then peer review—once a means of making scientific work balanced and consensus-driven—now serves to stifle disagreements, and deferring to it would be a form of surrender to establishment elites. The most important aspect of an article would be whether it is heartfelt.

But that is far too simple—and cynical—a way to look at science, which is a muddy, human endeavor. Every study is strategic. Each requires choices about how to design the analysis and explain the results. Yes, Brown made his choices with a particular conclusion in mind, one he thought would be acceptable to scientific gatekeepers. And while Andersen’s COVID-origins paper was in progress, the authors were clearly attuned (like all of us were) to the political environment of early 2020. But their stories are not exceptional. As an academic physician, I’ve contributed to papers for medical journals and fielded the demands of peer reviewers, however parochial they may be. I can’t say that I’ve always held the line on my every belief or conviction. I’ve toned down criticism of professional colleagues. I’ve hewed to the preferred phrasing of my editors. (I’ve also played the part of narrow-minded reviewer myself.)

The academic half of me feels that this represents at most a minor breach of principles: Getting useful data or an interesting idea into the literature is worth a few compromises around the edges. But the physician half becomes indignant at the prospect of insincerity. The framing of a paper helps determine how research is received and understood. Subtle choices in its assumptions, figures, and conclusion may, for instance, encourage readers to believe that the most apocalyptic predictions about climate change are inevitable, or that the lab-leak hypothesis has never been more than a conspiracy theory. Anti-vaccine ideas also gain traction in this way. By tempering their rhetoric and zooming in on discrete claims, vaccine doubters can transform a questionable ideology into a facsimile of healthy skepticism, and publish watered-down versions of their core theories in peer-reviewed medical journals.

Joseph Ladapo, the vaccine-skeptical surgeon general of Florida, has been a prominent user of this motte-and-bailey strategy. He has consistently been a vocal detractor of the COVID shots. He’s called them an “unsafe medication” and said he’s not sure that anyone should be getting inoculated this far into the pandemic—yet when his department released a scientific analysis last year suggesting that vaccination may increase the risk of cardiac death, its conclusion was presented with the mealymouthed restraint of formal scientific inquiry: “The risk associated with mRNA vaccination should be weighed against the risk associated with COVID-19 infection,” it said, then cautioned that the results were preliminary. But shortly after putting out these data, Ladapo recommended that all men under 40 avoid the vaccine. If the language in Andersen’s paper was punched up for publication, the language in the surgeon general’s must have been tamped down. (Ladapo’s office did not respond to inquiries.)

Ladapo has faced more scrutiny than other vaccine skeptics because of his influential public post and affiliation with Governor Ron DeSantis. Numerous media outlets have run stories undermining the validity of Florida’s vaccine study, and the Tampa Bay Times obtained draft versions of the paper revealing that, prior to release, results showing that COVID infection posed a greater risk of cardiac death than the vaccines did were removed. Ladapo also cut an analysis showing that the risk of cardiac death to young men from vaccination was not significant. Critics called the changes a “lie by omission.” A review by a faculty committee at the University of Florida, where Ladapo is a tenured professor, found that he may have violated the school’s research-integrity standards. The committee report did not allege that he had committed “research misconduct” in the classic sense. Instead, it raised the squishier concern that he had engaged in “careless, irregular, and contentious” research practices. (The university declined to investigate, saying that Ladapo’s work as surgeon general was outside their purview.)

Ladapo brushed off the criticism, saying that he had revised the paper based on his scientific expertise. He was simply making choices about how to present his research, and those choices happened to support the conclusion that would be most amenable to a specific audience. Truthfully, his behavior may be dangerous, but it is not all that unusual. A large swath of academic literature could reasonably be described as “careless” or “contentious.” In 2019, I published an article describing how poor methodology in my own field of pathology commonly leads to some of the very same questionable scientific behaviors that the faculty report accused Ladapo of committing. The blowback to Ladapo’s paper—and to Andersen’s and Brown’s—has more to do with ongoing political conflicts than any specific, egregious details of its presentation. No researcher is immune to bias or the impulse to spin their findings in order to advance their private goals. Some are just better at keeping themselves out of the spotlight.

Is scientific insincerity really a problem? Facts, as the saying goes, don’t care about our feelings; science is supposed to be the land of facts. Data are presented, discussed, confirmed, or discredited—all on their own terms. Belief has nothing to do with it, and forensically dissecting an author’s motivations has little practical value. But the public’s skepticism of science remains significant. People want to know what the research community might be keeping from them. Brown’s essay, which accused scientific journals of bias, was published by The Free Press, an outlet devoted to “stories that are ignored or misconstrued in the service of an ideological narrative.” The Free Press’s science section is awash in references to censorship, deception, and lies. Only bad news is newsworthy in some corners of the media; shady science has become a dominant narrative in its own right.

The Andersen, Brown, and Ladapo controversies suggest that scientists’ personal views—and the way they get run through the publication meat grinder—are likely to remain a source of scandal. When an unpalatable result cannot be dismissed out of hand, we turn to a simpler explanation: human nature. The science is wrong because the scientists are being insincere. It’s too easy to assume that if they’d only tell us what they really think, the facts would be on our side.