What are AI hallucinations?
Language models generate text by calculating likely continuations. The result usually reads fluently and convincingly, regardless of whether it is true. A hallucination is a statement of this kind that is factually wrong or has no basis in the available sources. Because it is written in the same confident tone as correct information, users often do not notice it.
This matters for brands because more and more people ask AI systems such as ChatGPT, Gemini, Claude or Google’s AI Overviews about providers, products and contact details.
Why language models hallucinate
In a paper from September 2025, researchers from OpenAI and Georgia Tech argue that language models hallucinate because training and evaluation reward guessing over acknowledging uncertainty. Models, they say, are optimised like exam candidates who would rather guess on hard questions than leave a gap. Hallucinations are therefore not mysterious but a statistical consequence of training.
There are practical causes too: training data has a cut-off date and goes out of date. And when a system retrieves web pages for an answer, it also picks up their errors, gaps and contradictions.
Typical hallucinations about businesses
- Wrong core details: in one client project, Gemini gave a winery a wrong address and wrong opening hours.
- Wrong terms: for the same client, Claude quoted shipping costs that did not match those in the online shop.
- Wrong people: Gemini and Claude named owners and contacts for the winery who were not correct.
- Invented URLs: in a log file analysis for a large organisation, around 15 per cent of Claude-User requests ended on error pages (404) because the model had invented the addresses of subpages. For ChatGPT-User, the figure was just 0.1 per cent.
A fact check for the same large organisation shows how common this is. Of 450 AI answers (150 prompts in ChatGPT, Claude and Gemini), 20 per cent were fully correct, 37 per cent correct but imprecise, 38 per cent partly wrong and 2 per cent wrong. The remaining 3 per cent or so did not deal with the organisation and were not assessed. Around a third of the errors could be traced back to the organisation’s own website: information available only in PDFs or in the online portal, or scattered across many pages, gaps in content and contradictory figures on different pages.
Grounding: how brands reduce hallucinations
Grounding means that an AI system ties its answer to verifiable sources, such as search results or documents, rather than relying only on what it learned in training. A well-known method for this is retrieval-augmented generation (Lewis et al., 2020). Google Cloud, for example, offers grounding for Gemini models, including grounding with Google Search (Google Cloud).
Businesses cannot correct the model directly. They can, however, influence what a system finds when it relies on sources:
- Core facts in HTML: publish address, opening hours, prices, services and key figures on regular pages, not only in PDFs.
- One fact, one place: resolve contradictory figures across pages and update or redirect outdated pages.
- Structured data: mark up organisation details with Organization markup that matches the visible content. See schema and structured data.
- Maintain external profiles: according to Google’s guide, Business Profile and Merchant Center can help your products and services appear in AI responses. Keep the details there up to date, and do the same for industry directories and review sites.
- Allow crawlers: fetchers such as ChatGPT-User or Claude-User should be able to reach the relevant pages.
- Catch invented URLs: identify frequently requested but non-existent addresses in your log files and, where it makes sense, redirect them to the right page.
Detecting and monitoring hallucinations
Hallucinations usually only come to light when someone looks for them. A proven approach is a fixed set of questions about core details, offering, prices and differentiators, asked regularly in several AI systems. The answers are checked against your own facts and classified by severity. Note that monitoring tools do not recognise every name variant, so manual spot checks are part of the process. This check is part of our AI monitoring. How overall presence can be measured is described in the article on AI visibility.
What businesses can realistically achieve
Hallucinations cannot be prevented entirely, because they arise from the way the models work. The error rate about your own brand can be reduced, however, and the biggest lever is often your own website. If you publish facts clearly, keep them current and make them accessible, you give AI systems the basis for correct answers. A GEO audit offers a structured starting point.
Would you like to find and reduce wrong AI statements about your company and need support? Then get in touch or read more about our AI monitoring! Would you like to find out more or have your team trained on this topic? Then take a look at our GEO seminar.
