Why has disruptive, paradigm-shifting science declined?

Apollo crew in the mission operation control room

Is all impactful science paradigm-shifting? Here: 1969s view of activity in the Mission Control Center during the Apollo 12 lunar landing mission. Image credit: NASA, CC0.

In the last 70 years, over 50 million new scientific papers have been published, alongside over five million patents. The number of papers published each year has grown exponentially over this period: one estimate puts it at 1.8 million per year as of 2014.

So why then have we seen a steady decrease in the disruptiveness of research? Academics see disruptive papers in science as research that shifts perspectives on existing knowledge around the topic, often rendering it obsolete. One such paradigm shift was the discovery of the double helix of DNA put forward in Watson and Crick’s 1953 paper, thereby nullifying Linus Pauling’s triple-helix model. Papers like this are seen as golden tickets in science. Catalysing new discovery and research, they push science in a direction not previously explored. But evidence suggests a stagnation in the number of such disruptive papers since 1945, despite considerable annual growth in the rate of papers published.

Figure showing the decline of paradigm-shifting science
Graphs show a steady decline in disruption of scientific work over the 55 years. The declines for each research area examined range from 91.9-100% for papers, and 93.5-96.4% for patents. Image credit: Park et al., 2023 (CC BY 4.0, Wikimedia Commons).

What could explain this paradox? Research led by Michael Park from the University of Minnesota, and published recently in the journal Nature, tried to address this. Using a previously developed metric called the CD index (consolidation-or-destabilisation index), which measures the disruptiveness of work, they examined the disruption caused by 45 million papers and 3.9 million patents found online to determine the cause of this decline.

The results are quite telling. The group could rule out certain common theories, including a lack of “low-hanging fruits” left in science for us to discover, or the idea that the quality of current science being published has diminished over time. In fact, while the authors reach no definitive conclusion, they do attribute the loss partially to narrowing research areas.

They argue the “narrow slices” of science currently being explored mean scientists are less likely to approach a topic holistically. This diminishes researchers’ capacity to bring in new ideas and thought processes that may lead to that Eureka! moment so familiar to disruptive work. Their evidence centres on  a reduction in vocabulary used in publications, including a loss of unique word pairings and a shift in academic language away from words associated with discovery to those of consolidation. You could argue that this loss in diversity is an attempt to preserve the accessibility of scientific writing, as standardised language across a field aims to clarify research for both expert and novice readers. 

Such a focus on semantic consistency is motivated by a desire to improve the reproducibility of research. Encouraging authors to explain concepts using pre-defined terms ensures audiences are not alienated by the introduction of confusing new jargon. In fact, several recent publications have examined this, and call for authors to adopt a more standardised model of academic writing.

Modern-day researchers tend to focus on narrow areas of science, working on advancing incremental increases in knowledge. Some attribute this to a decline in disruption.

At first, the evidence for a decrease in disruptive science may seem quite convincing. It is true that modern-day researchers tend to focus on narrow areas of science, working on advancing incremental increases in knowledge that aim to deepen our understanding of the world around us. Unsurprisingly, the diversity of work cited would decrease, as relevant papers become more concentrated in one specific area. This would correspond with an increase in authors’ self-citation of past work as topic areas become more esoteric with fewer researchers in the field. 

Park and colleagues attribute this to a decline in disruption, and state that scientists are disadvantaged by this tunnel-vision approach. However, how much of this evidence just reflects the changes in modern science writing practices, rather than a lack of ability for specialised scientists to innovate? This emphasis on the necessity of disruption in science for innovation seems in itself a slightly archaic approach to measuring the relevance of academic work. With the authors examining innovation under the dichotomy of disruption vs. consolidation, they seem to miss the importance of innovation being an exercise in problem solving; as Shina Kamerlin, a professor of computational biochemistry, puts it, ‘there are as many ways to innovate as there are problems to solve‘. 

When publishing scientific work today, the burden of evidence required for acceptance into an academic journal is far greater than it was in 1945. In my own field of chemistry, for example, it is now standard for a journal to require significant supplementary information packets, often trailing into triple-figure documents of data. This need for detailed proof may be time-consuming, but is incredibly important for the credibility of scientists’ work. It does, however, drive the scientific community to draw new ideas from well established evidence, perhaps rendering the threshold for publishing highly disruptive work too high to surpass peer-review.

Instead, authors may want to be more cautious in their claims of innovation, or instead publish several incremental papers rather than one. Alternatively, there may be examples where conclusions have been drawn before the scientific technique could provide enough evidence for the phenomena. It still brings into question why such prestige is given to the idea that disruptive science is a highly important factor in scientific progression.

When publishing scientific work today, the burden of evidence required for acceptance into an academic journal is far greater than it was in 1945.

This culture can also mean academics push to publish disruptive work that later must be retracted once the results are found to be false. Such examples include work from 2013 suggesting that the Mediterranean diet prevented cardiovascular disease, the 2005 paper claiming the discovery of a new insulin-acting protein, or the papers of the early 2000s announcing they had found materials that displayed negative thermal expansion

Instead, I would argue that valuable science should be the result of in-depth and well-examined work published incrementally over time, gradually consolidating prior knowledge whilst also introducing new paradigms, perhaps over several papers. In fact, Park and colleagues mention some notably disruptive papers from recent years, their key example being that by Abbott and colleagues on the first detection of gravitational waves.

But to suggest that this work alone was disruptive belies the years of incremental and meticulous work to reach these findings by countless scientists since Einstein first hypothesised the existence of gravitational waves in 1916. Additionally, if you examine the citation practices of the subsequent publications from this paper, you will find that they all continue to cite the preceding works, indicating that the paper would not in fact score very highly in the authors’ CD index. 

There are many other modern examples of important innovative work that would not score highly, such as AlphaFold, CRISPR, and mRNA technologies, or Google’s original patent for the core algorithm that underpins their search engine.

Furthermore, I can’t help but disagree with the premise that focusing on a particularly narrow area of science restricts creativity and innovation. My work on my “narrow slice” requires not only a deep-dive into a particular topic, but the ability to approach problems from new angles, an appreciation and understanding for a range of techniques in my field and a creative mindset to furthering the forefront of chemistry through practical research. Kamerlin writes on her own expertise in computational biology and biophysics that innovation is alive and, well, moving at “warp-speed“. In chemistry, more and more groups are pursuing the formation of spinouts and startups through universities’ innovation programmes. Though our work is proportionally less disruptive than it might have been 50 years ago, perhaps the focus has just shifted towards more carefully planned innovative steps. 

How we highlight the impact of current science could profoundly shape the future research landscape—after all, if Newton saw further by standing on the shoulders of giants, why can’t we?  

Lastly, citation practices, used as their main metric for measurement of disruption, have changed enormously since 1945. Journals now place more emphasis on citing both corroborating literature and opposing viewpoints. Where the diversity of citations might have decreased, the number has enormously grown, with papers published in 2022 having an average of 51-54 citations. Park and co-workers do state in their paper that the CD index does have its limitations in terms of accounting for citation practices, as well as being a new metric that perhaps is not as fully developed and tested as it could be. 

Across social media and publishing platforms, the recent publication has captured the attention of academics and the public alike. While the authors have produced strong evidence to show that the level of disruption in science has diminished, I do not believe that this is a bad thing. Disruptive science certainly has its place in research, but I feel that the loss in proportional “disruption” is a result of better-quality research, publication, and citation practices among academics across scientific fields. The idea of disruptive science embodies leaping scientific progress.

So perhaps when we discuss the significance of its decline, we should place more emphasis on how academic work is still innovative, albeit in changing ways. In practice, maybe this is a call for scientists to re-examine how we communicate science, and define disruption. How we highlight the impact of current science could profoundly shape the future research landscape—after all, if Newton saw further by standing on the shoulders of giants, why can’t we?