Big Science & Technology Isn’t Always Better: Small Teams Produce More Innovation

Analysis of 65 million projects finds that large science & technology teams develop while small teams disrupt

In today’s science and business worlds, it’s increasingly common to hear that solving big problems requires a big team. But a new analysis of over 65 million papers, patents, and software projects found that smaller teams produce much more disruptive and innovative research. 

In a new paper published by Nature, researchers from the University of Chicago’s Knowledge Lab examined 60 years of publications and found that smaller teams were far more likely to introduce new ideas to science and technology, while larger teams more often developed and consolidated existing knowledge.

While both large and small teams are essential for scientific progress, write authors Lingfei Wu, Dashun Wang, and James Evans, findings suggest that recent trends in research policy and funding towards big teams should be reassessed.

“Big teams are almost always more conservative. The work they produce is like blockbuster sequels; very reactive and low-risk.” said Evans, Professor of Sociology and Director of Knowledge Lab at UChicago, External Professor at the Santa Fe Institute, and a leading scholar in the quantitative study of how ideas and technologies emerge. “Bigger teams are always searching the immediate past, always building on yesterday's hits. Whereas the small teams, they do weird stuff—they're reaching further into the past, and it takes longer for others to understand and appreciate the potential of what they are doing.”

Knowledge Lab is a unique research center that combines “science of science” approaches from sociology with the explosion of digital information now available on the history of research and discovery. By using advanced computational techniques and developing new tools, Knowledge Lab researchers reconstruct and examine how knowledge over time grows and influences our world, generating insights that can fuel future innovation.

The Nature study collected 44 million articles and over 600 million citations from the  Web of Science database, 5 million patents from the U.S. Patent and Trademark Office, and 16 million software projects from the Github platform. Each individual work in this massive dataset was then computationally assessed for how much it disrupted versus developed its field of science or technology. 

“Intuitively, a disruptive paper is like the moon during the lunar eclipse; it overshadows the sun — the idea it builds upon — and redirects all future attention to itself,” said Wu, a postdoctoral researcher with the University of Chicago and Knowledge Lab. “The fact that most of the future works only cite the focal paper and not its references is evidence for the ‘novelty’ of the focal paper. Therefore, we can use this measure, originally proposed by Funk and Owen-Smith, as a proxy for the creation of new directions in the history of science and technology.” 

Across papers, patents and software products, disruption dramatically declined with the addition of each additional team member. The same relationship appeared when the authors controlled for publication year, topic, or author, or tested subsets of data, such as Nobel Prize-winning articles. Even review articles, which simply aggregate the findings of previous publications, are more disruptive when authored by fewer individuals, the study found.

The main driver of the difference in disruption between large and small teams appeared to be how each treat the history of their field. Larger teams were more likely to cite more recent, highly-cited research in their work, building upon past successes and acknowledging problems already in their field’s zeitgeist. By contrast, smaller teams more often cited older, less popular ideas, a deeper and wider information search that creates new directions in science and technology.

“Small teams and large teams are different in nature,” Wu said. “Small teams remember forgotten ideas, ask questions, and create new directions, whereas large teams chase hotspots and forget less popular ideas, answer questions, and stabilize established paradigms.”

The analysis shows that both small and large teams play important roles in the research ecosystem, with the former generating new, promising insights that are rapidly developed and refined by larger teams. Some experiments are so expensive, like the Large Hadron Collider or the search for dark energy, that they can only be answered by a single, massive collaboration. But other complex scientific questions may be more effectively pursued by an ensemble of independent, risk-taking small teams rather than a large consortium, the authors argue.

“In the context of science, funders around the world are funding bigger and bigger teams,” Evans said. “What our research proposes is that you really want to fund a greater diversity of approaches. It suggests that if you really want to build science and technology, you need to act like a venture capitalist rather than a big bank — you want to fund a bunch of smaller and largely disconnected efforts to improve the likelihood of major, pathbreaking success.” 

“Most things are going to fail, or are not going to push the needle within a field. As a result it's really about optimizing failure,” Evans added. “If you want to do discovery, you have to gamble.” 

The study, “Large Teams Develop and Small Teams Disrupt Science and Technology” was published on February 13th, 2019 in Nature. The third co-author, Dashun Wang, is a associate professor of Management and Organizations at the Kellogg School of Management, the McCormick School of Engineering, and the Northwestern Institute on Complex Systems at Northwestern University. The research was funded by a DARPA Big Mechanism grant, the John Templeton Foundation, the National Science Foundation, and the Air Force Office of Scientific Research.