AI could turn scientific papers into research assistants

Scientific papers may be getting a 21st century update.

A system called Paper2Agent lets scientists do more than read a new scientific study. When a paper comes with code, methods and data, the system packages these into a tool set that an AI assistant can use to work with new data, researchers report September 16 in Nature.

The system is essentially “a virtual author of the paper,” available around the clock, says James Zou, a computer scientist at Stanford University.

In tests, Zou and colleagues tried to convert 100 computational biology papers into working tool sets and succeeded with 74. The others had obstacles such as missing code or data, or software that wouldn’t run. With the papers that were converted, the Paper2Agent tool set could run analyses of the published work and answer questions about the results. It did so more accurately and also more than twice as fast as the same agent working from the original papers and accompanying code.

Making another scientist’s code usable can take substantial work, says Peter Clark, a senior research director at the Allen Institute for Artificial Intelligence in Seattle who was not involved in developing Paper2Agent. Researchers already share code on platforms such as GitHub. But Clark says he often has to “grub around” for half a day to figure out how to get it running.

A general-purpose AI assistant such as Claude can help, he says, but it typically generates new code on the fly. Paper2Agent instead packages code associated with a paper into reusable tools, tests them against reference outputs and writes step-by-step instructions guiding an AI assistant on which tools to use and in what order. This streamlines and standardizes the process of using AI to interact with scientific research, Clark says. It’s “an exciting step” toward papers that actively help scientists.

To Zou, transforming scientific papers into agents seem like a natural next step for transmitting knowledge. We’ve already moved from stone tablets and papyrus to paper and then to digital formats, he says, and now it’s time to move from passive to active formats for human knowledge. A paper-agent can help one human scientist quickly use another’s code and methods. The tool can also search for paper-agents about similar topics with the aim of alerting their human authors about potential collaborations.

Zou says his team has already created “over 100 different paper-agents” from their own work and that these have already helped him find other scientists to collaborate with.

Clark notes that Paper2Agent specializes in helping scientists reuse methods and code, and not in answering general questions, like a true virtual author should be able to do. Also, he worries about what could happen if millions of low-quality papers get turned into agents capable of pestering scientists. That could make things “worse rather than better,” he says, recalling the days of Clippy, the much-maligned Microsoft Office assistant. But he’d love for a “non-annoying agent” to pop up and tell him about work that he should be paying attention to.

For more open-ended questions about a paper, Clark says he would still want to talk to the people who did the research. “The reason I would write to a corresponding author is for precisely the things they didn’t put in the paper.”