I got my hands on Lenovo's Pro 9n with an NVIDIA RTX Spark chip last week, and I've been using it as a daily driver ever since. Equipped with a powerful onboard GPU with unified memory — meaning it can 'borrow' from the system RAM when VRAM is needed, like in AI tasks — I figured it was time to finally set up a local agent on a laptop.
Why go with local instead of cloud? I lay out a few solid reasons after the short installation and setup tutorial below, but it mostly has to do with external token usage, privacy, and convenience.
Of course, you don't necessarily need a powerful RTX Spark chip inside your PC to do the same. If you're often using AI to chat, create images, study, summarize, and handle other light tasks, there aren't many reasons why you can’t make use of local AI on your PC.
As a relatively novice AI user, I found it shockingly easy to get everything up and running in about 5 minutes.
What you need before you start
A powerful GPU matters a lot more than a powerful CPU when it comes to local AI. Fitting an entire AI model into your PC’s VRAM will ensure the best performance.
For smaller local models, say within the 3 billion (3B) to 8 billion (8B) parameter range, you should aim for having a GPU with at least 8GB of VRAM. For medium-size models, say 8B to 14B parameters, aim for 12GB of VRAM.
For large models at 30B or more parameters, you’re getting into workstation territory where VRAM is either available from very expensive GPUs or from unified memory pools (as is the case with RTX Spark laptops).
I don’t recommend running local AI on a CPU only, but it is possible. Just be prepared for far slower performance, even with a powerful, modern chip with lots of cores.
PC gamers are likely the best equipped for this task, as they’re far more likely to have a strong GPU in their system.
How to install local, free AI on your Windows PC
For this walkthrough, I’ll show you just how easy it is to get AI running locally on your PC. It only took me about five minutes between launching a browser to begin downloading the necessary software and my first prompt sent to the agent.
The TL;DR? Download LM Studio. Pick an AI model to download. Mount it within LM Studio. Start chatting. Those are the basics, and I explain in more detail below.
1. Install LM Studio — It's free and easy to use
I have some experience with Ollama (another free piece of software for managing local LLMs) on my desktop PC, but in this case, I decided to test out LM Studio.
It’s a free app designed to make downloading, managing, and working with local AI models as easy as possible. The interface is modern and clean, and I’ve had no performance issues so far.
You can download it directly from the official LM Studio website. It runs on Windows (x86 and ARM), Linux, and macOS.
2. Download an AI model — there’s likely something to fit your hardware
Your next step is to choose and download an AI model (all within the LM Studio app) for local use. This is done with the Model Search feature in the app, which will recommend certain models to you right off the bat.
The open-source Qwen3.8 model is super popular right now, so that’s what I went with. Here’s where things get a tad tricky, but just for a moment.
AI rookies should check out LM Studio paired with a 7B or 8B AI model. You get the privacy and convenience perks of offline use, all without stressing your PC too much.
The Qwen model I chose sits at 27B parameters, which is rather heavy and not built for the average PC. I’m using it to test the powerful RTX Spark laptop with unified VRAM.
Most casual users will have a lot more luck with AI models using fewer parameters. The beauty of LM Studio is that the search is all built right in, with flags for compatibility and performance. Websites like Codersera also have tools to help you find a suitable model for your specific PC.
If you find a popular model you think suits your needs and it comes with the “Full GPU Offload Possible” tag, chances are good that you’re in the right place.
Click the Download button on your preferred model and let it finish before proceeding.
3. Load the model and start chatting
Once your preferred AI model has finished downloading, the next step is to load the LLM into LM Studio. This can be done in a number of ways.
Probably the easiest way is to just click the Chat button in the top-left corner of the app, then click the “Select a model to load” button that appears in the top-center of the window.
Click the model you want to use and let it load. It will take a few moments to finish.
Once the LLM is loaded locally, you’re free to start chatting locally with your AI agent.
Local AI isn’t just for advanced PC users
Setting up a local LLM is about as easy as downloading and installing any other piece of software. If you’re interested in AI but want to increase privacy, use it offline, and not worry about expensive token usage, it’s basically the easiest method forward.
If you have a modern PC and a bit of time to spare, you can probably get a local AI agent running in the same time it just took you to read this walkthrough.
Local AI models — FAQ
What is "local AI" and why should I use it?
Local AI means that the LLMs are hosted on your PC rather than in a cloud server. Your PC's hardware powers the AI's thinking, meaning you don't need an internet connection and don't need to worry about your information being sent off to the cloud.
Why use local AI instead of cloud AI?
Cloud AI services like Claude, ChatGPT, and Copilot are convenient, but not everyone is comfortable with their communications with a bot being sent out into cyberspace.
Not everyone wants to pay for tokens or subscriptions, either, which often arise during your AI journey.
Are there any limitations to local AI compared to cloud AI?
Yes, there are some limitations.
Local models are usually weaker than the top cloud models. For the average user, that's OK, but more advanced users will likely still find a need for paid subscriptions.
You also need a fairly powerful PC in order to run local AI, which not everyone has. Setup is super easy, but performance will vary greatly depending on your hardware.
Do I need an internet connection to use LM Studio?
You only need an internet connection to download the LM Studio app and your preferred AI model. After that, everything runs offline.
Can local LLMs run on a laptop's NPU?
Neural Processing Units (NPU) came into focus with the launch of Copilot+ PCs, and they're designed specifically to run AI.
Although weaker first-gen laptop NPUs don't have a ton of power, newer hardware from the Snapdragon X2 lineup with 80 TOPS can run lighter local LLMs.
Is Copilot considered local AI?
The Copilot app that comes installed on Windows 11 PCs relies on the cloud to provide answers. It will not work without an internet connection.
Copilot+ tools built into Windows 11 — like Recall and Studio Effects — use the PC's NPU for local acceleration, meaning they don't need to call out to the cloud.
Join us on Reddit at r/WindowsCentral to share your insights and discuss our latest news, reviews, and more.