Private, offline and yours: how to search the literature with local AI systems

When he wants to scratch an intellectual itch, Ian Gaudet does what people are increasingly doing on the Internet: he asks a chatbot. But when it comes to his research data, Gaudet, a neuroscientist at Florida Atlantic University in Boca Raton, prefers to stay offline. “I do not want Claude to start telling people my research results,” he explains.

AI co-scientists are revolutionizing how research is done

Cloud-based large language models (LLMs), such as those underlying the chatbots Claude (Anthropic), Gemini (Google) and ChatGPT (OpenAI), represent the cutting edge of artificial-intelligence performance. However, these systems might use what is typed into them for training purposes. For researchers such as Gaudet, state-of-the-art performance isn’t worth the risk to data protection, reproducibility or budgets. Instead, they turn to a growing collection of tools, such as Ollama and LM Studio, that enable researchers to download LLMs to their computer and use them offline.

These programs can help researchers to sort and query their own literature libraries to gather insights, for instance. But two fresh barriers have replaced the programming and set-up issues that once kept researchers from using them: the hardware needed to run them has become much more expensive, and the tools’ output still requires scrutiny to assess its veracity.

Going local

Ask researchers why they want to run an AI model on their desktop rather than in the cloud, and the answer almost always starts with data. Eric Böhnisch-Volkmann, president of DEVONtechnologies in Coeur d’Alene, Idaho, hears it constantly from users of the firm’s document-management software DEVONthink, which integrates LLMs.

“Medical records is one thing, lawsuits is one other thing, research is another,” he says. “They just want to keep their hands on [their data] and not give it to someone else they don’t trust.”

Privacy concerns are well founded. A 2025 analysis found that the six AI developers studied — Amazon, Anthropic, Google, Meta, Microsoft and OpenAI — trained their models on users’ input data by default (J. King et al. Preprint at arXiv https://doi.org/g967b4; 2025).

AI for research: the ultimate guide to choosing the right tool

Ahmed Farooq, a computer scientist at Tampere University in Finland, built a local system called TAUCHI-GPT, so that papers, reports and study data protected under the European Union’s General Data Protection Regulation could be searched without leaving the university’s network.

Reproducibility is another benefit: cloud-based AI providers update their models without notice, so a query that works today might return different results next month. Casey Greene, a biomedical informatician at the University of Colorado Anschutz School of Medicine in Aurora, notes that if a local model converts his documents into a searchable index using the same settings each time, the results will always be identical. “If you want to use that for a scientific workflow, being able to know that is important,” he says.

Another benefit of local LLMs is economic. Jayashree Kalpathy-Cramer, who leads ophthalmology AI research at Anschutz, used Qwen (developed by Alibaba Cloud) and NVIDIA Nemotron through Ollama to index a decade’s worth of open-access publications on her machines. The corpus is public, so she could send it to a remote, commercial AI system. But local inference is free, and that changes how she works. Her system sends a daily briefing to her department’s chair highlighting new papers worth reading. The same query is run through three or four AI models, before another model reconciles the outputs, and every summarized finding is checked against the passage it came from. Because all of the field’s records sit in one searchable index, she can cluster them, map them and ask the LLM where any research gaps are.

The search for meaning

Preliminary findings of a survey by Wiley, a scientific publisher in Hoboken, New Jersey, indicate that 68% of some 2,400 researchers worldwide had used AI for literature searches or reviews (see go.nature.com/4rpuvwj). The tasks that were embraced fastest include reviewing large collections of studies, monitoring the latest publications and assisting with writing. Those are jobs that local models already do well, says Kalpathy-Cramer.

Greene built a local database of embeddings — numerical representations of each document’s overall meaning — to process the public abstracts of grants awarded to his university. When a federal-funding call for proposals in HIV research came through, he searched his database for potential collaborators. A keyword search for HIV highlighted a group researching the virus. But a semantic search of the same database, which looks at meaning rather than exact words, also came up with a researcher whose grants did not mention HIV in the title but who studied antiretroviral therapies, he says. He then asked that researcher about collaborating.

Dmitrijs Lvovs, a bioinformatician at the Institute for Genome Sciences at the University of Maryland School of Medicine in Baltimore, uses a computational pipeline to analyse spatial data from tumour tissue, producing hundreds of tables containing gene activity data, cell-type interaction information and records of treatment responses. To make sense of those data, he and his colleagues paste them into Microsoft Copilot, the cloud-based AI service that their university provides. The tool then searches the published literature and suggests what any patterns might mean. But Lvovs wants to bring that step in-house. So, he started using a tool called llmize to feed the tables into Google Gemma 4 through Ollama, in a fully offline environment.

Because the model is not connected to the Internet, Gemma cannot search the literature in the way that Copilot can. But the tables represent people’s data, and Lvovs would rather accept a ‘weaker’ model than send such data sets to the cloud. Even if study participants have given their consent for data collection, “it feels wrong to be concentrating all that data in the hands of a few companies”, he says.

Getting started

In the world of LLMs, ‘weaker’ is a moving target, and the gap between local and cloud-based models is shrinking fast, Lvovs says. “The competition is now more about the harness, the actual program that uses the model as a back end, not the model itself.”