My local models are worse than Claude at almost everything, but they still run most of my workflow for $0

Abhinav pivoted from a career in banking to pursue his first love in writing. Even while working full-time, he continued contributing as an editor-at-large, a role he has held for more than 7 years. A lifelong tech enthusiast who has built three gaming and productivity powerhouse PCs since 2018, his passion for technology keeps him closely following the semiconductor industry, from NVIDIA and AMD to ARM. His MSc dissertation explored how artificial intelligence will reshape the future of work, reflecting his curiosity about the wider social impact of emerging technologies.

It's not lost on me that any one of Anthropic's models can outperform the models I run on my 16GB Nvidia GPU. I'd wager even Sonnet 5.5 can write much better code, keep track of conversations that just go on and on, and catch mistakes that both my Qwen3.6 and Gemma 4 would walk right past. If you put them side by side on any benchmark, Claude will take the win in literally any workflow, and it won't be close.

And yet, when I look back on a typical week, my local AI stack running on my RTX 4070 Ti Super gets more prompts from me than Claude ever does. It's not because I'm an advocate for self-hosting or have any clever tweaks that make my local models work better, but because some of what I ask simply involves data I'd rather not upload, some of it is code that doesn't need a frontier model on the first attempt, and the remainder are tasks too trivial to spend my cloud tokens on.

Here's why I'm relying more on the models that run on my PC than on Anthropic's servers, and why I think it's the better approach, both fiscally and logically.

My local models offer me complete privacy

I'm not losing any sleep giving my financial data to Gemma 4 E4B

Once every month, my bank's app generates a spending report that can be exported to a PDF. It gives me an insight into the many ways I've splurged, which is descriptive, but not quite informative in the way I'd like it to be.

Often I'd find myself thinking just how convenient it would be if there was some way to query these documents and ask a model that has all of this context to see if any of this data could be turned into actionable information that I can use to hit the savings target I set at the start of the year, which category is eating into the biggest share and which months I overshoot without noticing. All of these questions warrant a conversation, and a bar graph and a flow chart can't give me that.

I'm also cautious enough to not just drag and drop all of that data onto a Claude chat, so instead, I drag and drop it into a session with Gemma 4 E4B. Since this variant has vision capabilities and supports up to 128K context, it's perfectly suited to the job of parsing a dozen PDFs in a single setting and holding the thread across a conversation that can take up an entire afternoon. Since I'm only looking for answers to simple one-liner questions that don't involve deep reasoning, it doesn't really matter if it can't draw up interactive visuals of my month-on-month expenditure, and does the job just fine.

Not all of my code needs a frontier model to fix

I save all my cloud tokens for the biggest blunders, and everything else goes to Qwen3:30B

I've previously talked about how Claude on any tier can feel strangely restrictive when it comes to iterative coding sessions, unless you're a part of the highest paid Max or Enterprise tiers. For a customer like myself on a Claude Pro plan, there are many kinds of usage limits in place that can potentially break a workflow, like the dreaded 5-hour rolling session limit, the weekly limit, or the shared pool cap that applies at busy hours.

These feel like setbacks when I'm in the middle of a long session. Of course, when the limits kick in, anyone would argue that it's time to switch to a local model and keep going, but that comes with its own task-switching cost. All the context needs to be moved from A to B, and that is a problem since that very movement can break the trail of reasoning that got the session this far.

That's why I found it prudent to flip the order instead. I find that when my Qwen3:30B MoE model gets the first pass at the coding, it greatly reduces my token usage on Claude, which can be reserved for the final finishing touches of whatever Python app that I'm working on.

There's also a cost benefit to it, since in an experiment I ran a few weeks ago on a Pygame project, I found that handing Claude Code a working build from Qwen3:30B and asking it to refine it from there only cost $0.77, while asking it to build the same Pygame from scratch cost $1.35, amounting to a cost reduction of 43%.

Claude's limits are too prohibitive for trivia

Gemma 4 does a fine job recommending games to play or shows to watch anyway

Not everything I use an AI model for relates to work, and anything that isn't strictly for work can't justify the expenditure of the expensive cloud usage credits or tokens on. Obviously, that's because it is eating into my weekly usage limits, and I don't want them to act as a stopper in the middle of a project that disrupts the entire chain of ideation and refinement.

But that doesn't mean I can't get my fix of recommendation lists from an AI model altogether. For the kind of queries that fall in the trivia bucket, like what game I should play next, which show would fill the void that got created by the last one I finished, which book would suit the mood I am in, Gemma 4 handles just as well. I've previously tested this capability in action, wherein I had the model recommend games against Steam's Interactive Recommender, and Gemma 4 gave me better recommendations than the store that sold me all my games in the first place.

Claude may be the best, but my local AI stack does more for me

There's nothing defeating Claude and its various models in the AI service industry today when it comes to deep reasoning, zero-shot coding, debugging, or perhaps even research. It's clearly the superior service. But for many day-to-day tasks, you simply don't require frontier reasoning capabilities, and that's precisely where I reach for my local AI stack that costs me $0 to run.