The systems used to measure how well research investments are working keep changing

Terry Gerton All right, this month we’re talking about science and research data. Denise, we’ve seen a lot of changes in how the Trump administration approaches research and science funding, everything from grants and grant administration to the relationship between the government and research institutions. If someone wanted to understand the actual effects of those policy changes, where would they look for evidence, not just stories?

Denice Ross That is a great question. And one of my favorite types of federal data is data that help hold government accountable and make government activities and priorities transparent to the American people. And there is a fantastic collection of data about science and technology and research and development that the federal government has been producing. And so when I think about the changes happening in the federal R&D ecosystem right now, what keeps me up at night a little bit is that as all of these changes to the federal science ecosystem are happening, we’re also seeing a degradation of federal data capacity. And so if we’re going to be able to make informed policy decisions about where our nation’s priorities are, we need to keep those data about the federal research and development ecosystem flowing.

Terry Gerton Well, Chris, you’ve done kind of an inventory of the R&D data landscape. Which data sets are doing the most important work to help us understand, as Denice describes, where all of this money is going and what it’s doing?

Christopher Marcum Yeah, so I want to start by saying that I think when we think about research development data in the United States, we have to look at sort of the crown jewel of R&D data providers, and that’s the National Center for Science and Engineering Statistics, which is a statistical agency that’s located within the National Science Foundation. And as a statistical agency, its primary responsibility is collecting and providing data on the United States R&D and science technology enterprise and it covers topics from national funding expenditures to business expenditures in RD to what the federal research labs are doing and so that’s where I would start and they have a huge collection of various research products and statistical products that really provide data that leads insights into how the nation’s doing on a year-to-year basis.

Terry Gerton The National Science Foundation though, has been a little bit in the crosshairs. How’s that data set itself doing?

Christopher Marcum Yeah, so NCSES has, of course, there’s been a reduction in capacity across the National Science Foundation. NCSES itself has fared, I think, a little bit better. You might be aware that the administration had restructured the National Science Foundation to eliminate the social, behavioral, and economic directorate. NCSES actually sits within that. And rather than being eliminated, it was actually pulled out and given its own place within the organization.

Terry Gerton That’s good to hear and good to know. Denice, you mentioned earlier that you were a little concerned about our ability to see the impacts of the current policy changes, given the degradation of data. As we’re changing policy and changing data, what are we going to be able to look at? Are we going be able see forward? Are we gonna be able the see back?

Denice Ross Seeing forward and seeing back and seeing details are all really important for understanding what’s happening. The larger patterns happening with data are impacting our ability to understand what’s happened with science. One big one has to do with the STEM workforce and that day one executive order that mandated the removal of gender identity variables from major workforce surveys and also targeted DEI. So, how that’s manifesting across federal data is that there’s a slowing of implementation of Statistical Policy Directive 15, which was a huge improvement on how we collect data about race and ethnicity, making it simpler and including Middle Eastern, North African as a category for the first time. And we’ve seen not only a slowing implementation, but we’re also seeing race and ethnicity getting pulled from surveys, including the current proposed changes to the decennial census to remove race and ethnicity from our count every decade of all the people living in the United States. And so with key demographic data disappearing, it’s going to be so much harder to understand in the sciences, the climate in the workplace, how scientists are progressing through their careers, and how well different organizations are able to retain their workforce. And that means it will be harder for the federal government and society to be more intentional about making sure that STEM careers are accessible to all Americans. And speaking of science, it’s well-documented that diversity breeds innovation. So that’s one concern that I have. Another one that we’re seeing is just in general, data are becoming harder to access and used to make decisions and take actions. And so we’ve seen a couple of tools that have been diminished in particular. The U.S. Patent and Trademark Office had a very friendly patents view program that allowed people to more easily navigate the data about patents and that has been diminished. And so now technically the data are available but it’s just requires more skills, expertise and time to access them. So I would look for patterns like that to potentially continue, including with many of their reports that made it easier to take stock of where we are as a nation on science and technology and research and development.

Terry Gerton Denice Ross is the Director of the Data Policy Institute at the Federation of American Scientists, and Chris Markham is Senior Advisor for Federal Data Policy at the Federation of American Scientists. Chris, let me come back to you. Denise mentioned the loss of demographic data in the STEM workforce, but when we’re talking about R&D data, we’re taking about funding of projects, specific research things, and the people who need to fund them. How are we seeing the two parts of that come together?

Christopher Marcum Yeah, so I’d like to add, just to put a finer point on something that Denice mentioned just at the end of what she just said, which was the loss of reports. And really, when we think about how we’re tracking the development of the STEM workforce, this is really the human factor in American R&D. And we often rely on not just underlying data, but how that data gets interpreted. And so one of the — lots of R&D data end up in national reports on the state of the science. And among those are a biennial report from the National Science Board, which works very closely with NCSES for its data needs. And in that report that was released just this last year, one of the things that we learned that as of 2024, which was the final year of the Biden-Harris administration, that China has overtaken the United States in terms of global R&D spending, that data that’s used in that report, more than two years old at this point. And so we should really be keeping an eye on upcoming releases of the underlying data from NCSES to see how the R&D reform efforts that are currently happening from this administration are faring. When it comes to STEM and to workforce development, specifically, we can look to specific NCSES surveys, such as National Training Education Workforce Survey. That covers all of the technical skills in the workforce. And this is not just within the academy and training that happens at universities. It’s training that happened through technical programs at community colleges and keeps laboratories, clean rooms, advanced manufacturing lines, sort of… operational with high quality work. And that’s the data that we rely on for that.

Terry Gerton Denice, as you think about the implications of the availability and accuracy of this science and technology R&D data, what should policymakers be doing now to make sure that we both understand the impacts of our policy, but also have this data available to make future policies?

Denice Ross That is the big question. As with all of the data that the federal government produces, it needs to be trustworthy. And part of that is making sure that political interests aren’t influencing the data in a way that makes people not trust the data. And Chris just talked about some upcoming data on our R&D expenditures compared to other nations, including China. And when those data come out, it will show how the current policies are faring and we need to trust the data that are being released. As I’m making sure that there’s no political interference and data helps us trust the the data and make collective decisions as a society about what we want to prioritize. So that’s number one. Number two is detailed demographic and other data like on geography and other details are really what help us understand what’s happening in America. And they also are necessary for AI applications. The more detail you have, the better AI models can operate. And so improving and restoring the demographic data collection about our S&T workforce is going to be a critical path for making sure that we’ve got the right talent at the table. And then the third thing is maintaining the user-friendly data like in the Patents View Explorer, or currently the Indicators Explorer that National Science Foundation has on national and international trends in science, including details about the STEM workforce across America and the rest of the world. So, really delivering on the Open Government Data Act and its requirement that agencies make data that are going to be usable for public stakeholders. And then lastly, we’ve always had the challenge of sort of starving at our banquet of data because the data haven’t been connected in a way that are meaningful. And so there’s such a great opportunity to connect data from the appropriations in Congress, to apportionments from OMB, to grants and awards and contracts. And the R&D programs spending at the agencies and including the careers and outputs of the STEM workforce under these programs. And that type of data integration will give us such better insights on what’s happening with R& D and how we can really regain our competitiveness as a nation. And of course, this needs to be done responsibly and with participation from the scientific community.

Terry Gerton And Chris, your point earlier was about international competitiveness and the investment in our own economy that this represents. What do you want to add to Denice’s wish list?

Christopher Marcum Oh my goodness. I think this is, I think where I find sort of the most inspiring aspect of federal R&D statistics is really on the immediate return of federal investments. And when we, when we think about that, we have to look towards how federal agencies provide navigation data to help the public locate and access the peer reviewed scholarly manuscripts that are generated by federally funded research grants and the data associated with those grants. Those repositories can include like PubMed Central from the NIH, NSF has their public access repository, the Department of Energy has a repository called Pages, and NASA has one called PubSpace. Of course, they have a space pun in the name of their repository. In 2022, the Office of Science and Technology policy from the White House issued guidance to agencies to make that research and underlying data immediately and freely available to the public. You can think of how important this could be for not just researchers but the general public to really be able to see the immediate returns from federal taxpayer-supported investments in research and development.