How does a country get the biggest bang for its buck when it comes to science? Comparing science systems between countries can be a good way to see what’s working well and what’s not. But it’s difficult to do.
Want to speed up scientific progress? First understand how science policy works
Different countries analyse different things; relevant data can be hard to find or access; and the same concepts can be defined and measured in various ways.
Scientific outputs such as numbers of publications, citations and patents are easy to quantify. But they say little about what factors influenced them. Standardized statistics published by the Organisation for Economic Co-operation and Development (OECD), for example on investments, allow for easy comparisons between countries. But these measurements are not enough to analyse the intertwined effects that different approaches to funding, running institutions and supporting researchers can have over time. Some metrics — such as how funding shapes researchers’ careers — are rarely measured systematically.
And even when countries report on similar measures, differences in data collection can make comparisons difficult. For example, one country reporting information about its fundamental research might publish detailed spending statistics by sector and funding source; another might report only aggregated spending that includes applied research. This makes it hard to form a coherent global picture.
To overcome these problems, a group that I lead has developed an approach to analysing national research systems — gathering data from a broad range of sources and developing measures for often-overlooked aspects of such systems. I hope that other researchers, funders and international organizations will use our framework, and find ways to improve it.
China’s shift towards ‘organized research’: how can coordination and innovation co-exist?
Our project, the China Fundamental Research Report, examines research systems in 25 countries and regions, including the European Union (www.cfrr.net). The chosen regions have systems of various sizes, structures and stages of development. In particular, China offers a useful case study because its scientific landscape has transformed over the past 40 years, through sustained investment in funding, institutions, talent and research infrastructure.
Our work was possible thanks to a rapid expansion in the amount of research information that is now openly available — from publication and funding databases to data repositories and more. This has made it much easier to compare many facets of research between countries. We used large-scale computational methods to analyse data sets that were previously difficult to study together.
One key part of our work was developing fresh indicators to capture areas of research that are not usually measured. For example, we used large language models to identify signals of novelty in research questions and methods — how far a study ventures beyond established knowledge. We also linked publications to patent citations to trace connections between fundamental research and technological development. And we combined funding and workforce data to trace how, in China, junior researchers make a substantial contribution to scientific output. This group, we found, was supported by a long-term expansion of funding for early-career researchers from the National Natural Science Foundation of China.