Try asking an AI assistant what a mid-range hotel in Lisbon costs for next weekend. You’ll get a number, usually stated with complete confidence. And if you then ask where it came from, the explanation gets vague fast.
Sometimes the agent really did check a booking site a few seconds earlier. And sometimes it checked nothing and handed you a figure from its training data, possibly from a summer two years back. It does so in the same even voice it uses when it’s right.
People tend to call this a hallucination. I’d push back on the word a little, since in many of these cases the model recalled a price that was real at some point. Nothing in the system asked it to confirm the figure still held.
Checking the web costs time, and agents know it
Before an agent answers anything, it chooses between replying from memory and going out to find and read a page. In the developer and AI communities, it’s called the fetch decision.
The user never sees it. Almost none of the teams we work with at Decodo record it anywhere, even though their dashboards will show the cost of every query to a fraction of a cent.
In most agent frameworks, an engineer doesn’t even make that call. Web search is exposed to the model as a tool it may use. The model judges in the moment whether the question needs it.
That judgment is weaker than it looks. A model has no sense of how old any single fact in its memory is. So a hotel rate from 2024 sits in its weights with the same undated certainty as the boiling point of water. Nothing prompts it to treat one differently from the other.
The lookup also involves more steps than most people assume. The agent needs a search call to find the right page, and then a fetch. The fetch is where it tends to break: the site blocks automated traffic, or the price only appears once JavaScript runs, and the agent receives a cookie consent banner instead.
Whatever does come back has to be stripped down before the model reads it. Every paragraph is paid for in tokens and seconds. Recall costs almost nothing by comparison.
How skipping becomes the default
Product teams then judge agents on speed and smoothness. In a review meeting, an agent that replies “I couldn’t load that page” looks broken while the one that quotes a stale price passes.
Over a few release cycles, every tweaked prompt and every default picked under deadline pressure makes the lookup a little easier to skip, until answering from memory becomes the agent’s normal behaviour.
How to tell whether an agent looked up the web
You can’t see the fetch decision when you use an AI assistant, but you can usually spot what came out of it. An answer built on a live page tends to show where it came from, with a link to the page and often the date it was checked. An answer from memory reads just as smoothly, only without the link.
For anything that changes often, like a price or a flight time, I’d treat an answer with no source as a guess. Ask the assistant to look it up and show you the source it took information from. That one extra request often changes the answer.
Companies buying AI tools can ask the same question before they sign. Ask the vendor how often its agent checks the web before answering a question about price or availability. Also ask what the user sees when that check fails.
Web browsing is becoming a standard feature, and most AI agents can already do it. However, not all can. A few still can’t reach the web on their own.
For those, check whether you can connect a third-party web data service through a plugin or an API. That way, the agent searches and reads live pages instead.
Making the lookup the cheaper option
Switching to a newer model is usually the first fix developers try, and it does less than they hope. The newest models and AI tools can browse the web, yet they purposely decide to skip it.
If the model judges a search unnecessary, it answers from what it learned in training. For something like a price, that can be well out of date.
Wording often decides whether a search happens. A question with “today” or “latest” in it will usually send the agent to the web.
“What does our main competitor charge for its entry-level plan?” can still get an answer from memory, even though that price may have changed twice since the model was trained.
Four things to ask for
The fix is in how the product is built around the model. If I were reviewing an agent product this week, I’d ask for four things:
- Track how often the agent checks: measure the share of answers based on a page the agent actually read, broken down by topic. If it looks up prices only half the time, start there.
- Make checking mandatory for some questions: prices, stock levels, and anything legal or financial should always trigger a lookup. It should be written into the product’s code so the model can’t skip it.
- Show when a check fails: if the agent couldn’t load a page, it should tell the user it couldn’t confirm the answer. The team should be able to see how often that happens.
- Keep the lookup simple: teams often stitch together separate services to search the web and to read pages. Every handoff between them adds delay and another point where the request can fail.
Where Decodo fits
That last point is the one we work on most at Decodo, and it’s why we built the Web Data API.
With one API key, an agent can search the web, see how a site is organised, collect pages across it, or pull the content of a single page.
It runs on the ethically-sourced IP pool, which is what lets it read sites that block automated visitors. The page comes back as clean text the agent can use straight away, rather than raw website code.
Go back to the same competitor question. With live web access in place, the agent opens the competitor’s live pricing page and reads today’s price there. The answer comes from the page, not from what the model remembers.
Most AI tools can already search the web, and many show their sources (when they want to). What they still lack is consistency.
An agent that checks the latest price on one question and answers from memory on the next, with no way to tell which is which, is hard to trust with anything that costs money.