The better I get at AI, the less I trust it

The better I get at using AI, the less I trust it.

I know, I know — that sounds backwards. We’re being sold a future where increasingly capable AI assistants take more and more work off our plates. But after spending a frankly embarrassing amount of time building an elaborate AI-powered NFL betting project over the last few months — and especially after suffering through an infuriating $1,500 catastrophe this week — I’ve come to the opposite conclusion.

AI works best when you treat it like an idiot.

Some context: I co-host a small NFL player-prop show as a side project. Because I am apparently incapable of enjoying hobbies like a normal person, I started building an AI research system to augment my own analysis. I want to know ball on camera, obviously.

It started simply: Analyze the games. Find interesting player props. Tell me what you think. Let’s make some money!

That worked surprisingly well — until it didn’t.

AI desperately wants to be helpful. Ask it a question and it wants to give you an answer, even when the correct answer is “I don’t know.” Give it a theory and it’ll happily find evidence supporting it. Find a promising new approach and, if you aren’t careful, suddenly that approach has somehow become part of the methodology that supposedly produced last month’s results. I could go on.

So BWR, as I call my increasingly deranged side project, evolved. And as I saw how stupid and forgetful AI can be, I started putting the AI in a cage.

Research ideas couldn’t automatically become part of the production model. New approaches had to compete against the existing one. Historical testing could only use information available at the time. Results were preserved so failures couldn’t conveniently disappear. I created separate AI challengers whose job was essentially to tell the main system that it might be full of shit, because AI is so often deeply full of shit.

I even had to teach the damned thing that “I don’t know” is an acceptable answer. Seriously!

After endless days of trials, errors, and hammering rules into place: It worked! The less freedom I gave the AI, the more useful it became.

That’s been my biggest lesson from this rabbit hole. AI can do things that still blow my mind, but capability and reliability aren’t the same thing. The more I care about the result, the less I trust AI to simply figure things out. I want boundaries, checks, receipts, and explicit permission to fail rather than make something up.

I thought I’d built enough of those safeguards.

Then BWR friggin’ blew up this week.

AI stumbles over its own feet

Without drowning you in the deeply uninteresting details of my fake football laboratory, I discovered that critical pieces of the project’s underlying state weren’t preserved and recoverable the way they should have been, despite my safeguards.

And fixing the mess created an even nastier problem. I’d built safeguards to stop AI from hallucinating about football. I hadn’t adequately safeguarded against AI hallucinating about itself.

Now I was asking AI to reconstruct its own missing history — and a plausible reconstruction isn’t necessarily the real one. I couldn’t just ask, “Is everything fixed now?” and trust the cheerful answer. Every recovered piece had to be treated like evidence at a crime scene.

I spent roughly 25 hours this week, during a frantic period here at my normal job, doing AI forensics on a system I’d specifically designed to protect itself from AI failure. Between my time and the obscene number of tokens I burned fixing it, I figure the fiasco would’ve cost north of $1,500 at standard rates.

For an NFL betting side project. (Again: Normal hobbies aren’t really my thing.)

BWR survived. My sanity mostly did. But the experience changed how I think about local AI.

Seizing my own AI destiny

Running a model locally won’t magically make AI trustworthy. A local model can confidently make stuff up just as well as a cloud model. But I can control everything around it.

I want the logs. I want the checkpoints. I want the artifacts. If I’m going to spend months building something increasingly complex, I want its state sitting somewhere I control rather than trusting a black box to remember what matters, and to be fully recoverable if the brittle house of cards ever falls down again.

Cloud AI isn’t going anywhere for me; having some frontier hooks is crucial in my quest to break Las Vegas. But local AI suddenly looks less like a fun enthusiast experiment and more like infrastructure if you’re doing serious work.

Fortunately, I have an RTX 4090 system sitting a few feet away.

It’s funny how quickly “running AI locally” becomes appealing after catastrophic AI hallucinations eat 20 hours of your life.

So yeah, the better I get at using AI, the less I trust it. Maybe it’s time I owned more of it, too.

That’s it from me. AI guru Ben Patterson will be back in the saddle (and your inbox) next week! Don’t forget to sign up to start receiving this newsletter directly.