How Can I Test Whether an AI Tool Is Hallucinating About My Business?

Why do AI tools get facts about a business wrong?

Most errors about businesses have a small number of causes, and knowing them tells you what to test:

• Too little information. When there is little about you online, a model fills gaps with what is typical for your category, such as a standard price, a common feature or a likely location.
• Old sources. A directory listing, press release or archived page from years ago can be retrieved and presented as current.
• Name collisions. Another company, product or person with a similar name gets blended with you.
• Leading questions. A question that assumes something false ("Why did X stop offering Y?") often gets an answer that goes along with it.
• Competitor blending. Features or prices from rivals in the same list are attributed to you.
• Summaries of summaries. Engines copy third-party descriptions that were already slightly wrong.

A good test is built to expose each of these, not just to ask "what is X?".

How do you set up a hallucination test?

The whole test has five steps. For a small business, a basic first run of about 20 questions, three runs on four engines (roughly 240 answers), takes most of a working day to run and score; later runs of only the fragile facts take a few hours.

1. Write a fact sheet. List 20–40 facts that matter to buyers, each with the correct value and the page where it is stated. Include what you sell, who it is for, prices or price ranges, locations or service areas, size, founding date, key people, certifications, and anything that changed recently.
2. Write a fixed question set. Map each important fact to at least one question, and add the question types below. Keep the exact wording, because small wording changes produce different answers.
3. Run the questions. Ask each question three to five times per engine in clean sessions, and record the full answer, the date, the engine and mode, and any cited sources.
4. Score each answer against the fact sheet. Use the scale further down, per fact, not per answer.
5. Trace errors. For each wrong or invented fact, look at the cited sources, search for the wrong value, and note where it appears.

Keep everything in one sheet: one row per answer, one column per fact checked. That becomes the baseline you re-run after fixes. For the general method of running and logging questions, see how to manually track AI visibility.

What questions should you ask an AI about your product?

Use several question types. Each exposes a different kind of error:

The false-premise and recent-change questions are the ones people skip, and they often produce the most damaging errors. An engine that invents a closed office or a lawsuit to fit a question can do the same when a real buyer asks a loaded question.

How do you check whether an AI model knows your brand at all?

Separate two things that are easy to confuse: what a model learned in training, and what it finds by searching when you ask.

• Search-based answers. Perplexity and Google's AI features build answers from search results. ChatGPT, Claude and Gemini also search the web for many questions. When an answer cites sources, it is usually based on retrieved pages.
• Memory-based answers. When a model answers without searching, it relies on what it learned in training. Asking a model through its developer API without any web search tool attached is a direct way to test this layer.

Test both. A brand the model has no memory of, but that search finds, usually gets described correctly when the engine searches and invented or omitted when it does not. A brand the model "remembers" wrongly can keep getting old facts even when the web has the new ones.

Also watch for the difference between "I don't have information about that company" and a confident invented answer. The first is a visibility problem; the second is an accuracy problem and more urgent.

How can you check whether AI recognises a business as an entity?

Recognition means the engine treats you as one specific business, separate from others with similar names. To check it:

• Ask for your company by name alone, then with industry and city added. If the answers describe different businesses, the engine does not have a stable picture of you.
• Ask for facts only you would have: founding year, head office, founder, a flagship product. Mixed answers suggest blending with another entity.
• Check whether Google shows a knowledge panel for your name, and whether your business exists in Wikidata. Google's Knowledge Graph Search API can also be queried by name to see what entity, if any, it returns.
• Check that your name, address, website and description are consistent across your site, Google Business Profile, directories and social profiles.

For agencies checking a client, run this before any visibility work. Do not push for more mentions of a client that an engine confuses with another business until the confusion is fixed. More in what an entity is in GEO and how to become a clear entity in Google's Knowledge Graph.

How do you test whether ChatGPT understands what your company sells?

Understanding goes beyond naming you. Test it three ways:

1. Ask for a one-sentence description and compare it with how you describe yourself. Watch for the wrong category ("a marketing agency" when you are a software company) or the wrong customer ("for enterprises" when you sell to small firms).
2. Ask who your competitors are. If the list is made of companies in a different market, the engine misunderstands what you do, even if the description sounds close.
3. Ask buyer questions without your name for the job you do. If you appear for the wrong jobs, or not at all for the main one, the understanding is off.

Do the same in Gemini, Claude and Perplexity. They often differ, and the differences point at which sources each one relies on.

How do you check whether AI describes your product accurately?

Score each fact in each answer on one scale, so results can be compared across engines and over time:

Report the result per fact: "starting price: correct in 11 of 15 answers, outdated in 4". That shows which facts are fragile and which engines get them wrong, which is more useful than a single accuracy score.

How do you test specialised facts?

Generic questions miss the facts that matter most in specialist businesses. Build questions around the details a buyer would check before shortlisting you.

How can a specialist firm check whether AI understands its specialties?

A construction engineering firm, for example, might specialise in seismic retrofits of concrete buildings, not general structural work. Test with questions a client would ask: "Which firms in [region] do seismic retrofits for concrete buildings?", "What does [firm] specialise in?", "Has [firm] worked on hospitals?". Score whether the specialties come back specific and correct, generic ("structural engineering services"), or wrong. Generic answers mean your pages and listings describe you too broadly; engines can only repeat the level of detail they find.

How can you check whether AI gets project locations and capacities right?

Businesses with named projects, such as renewable energy developers, often find engines mixing up sites, sizes and status. Put every project in the fact sheet with location, capacity, stage and date, then ask: "Where are [company]'s solar projects?", "How large is the [project name] project?", "Is [project] operating yet?". Watch for units (MW and MWh confused), planned capacity reported as built, and projects attributed to the wrong developer. A clear project page per site, with capacity, status and location in plain text, is usually the fix.

How can you check whether AI understands a loyalty programme's tiers, rewards and launch date?

Programme details change often, which makes them prone to outdated and invented answers. List each tier, its threshold, the rewards, and the launch or change dates. Ask: "What are the tiers in [programme]?", "How many points do I need for [tier]?", "When did [programme] launch?", and a false premise such as "Why did [company] end its [programme]?". Pay special attention to answers that describe a rival's scheme or the previous version of yours.

How do you check whether AI misrepresents your company size?

Size is often guessed. Ask: "How big is [company]?", "How many employees does [company] have?", "Is [company] a good fit for large enterprises?". Engines often take headcount from third-party profiles that are years out of date, or infer size from category. If the answers are wrong, check professional network pages, business databases and "about" pages for old numbers, and state current size plainly on your own site.

How can you monitor how different engines interpret your business?

Engines differ in where they get facts, so run the same question set on each and compare:

Re-run the full set monthly, or after any change to prices, locations, products or name. Stable errors that appear in most runs matter far more than one odd answer. For many brands, a subset of the questions, the fragile facts, can be checked weekly.

How do you fix what AI gets wrong about your business?

The fix depends on the cause you traced:

• Wrong fact on a source engines cite: correct it at the source, whether that is your page, a directory, a profile or a third-party article.
• Missing information: publish a clear page that states the fact plainly in text, such as prices, specialties, locations or programme terms.
• Outdated pages: update or redirect old pages and ask third parties to update their copies.
• Name collision: make your identifying details consistent everywhere and state them clearly on your site.
• Model memory: takes longer; correct and strengthen web sources so search-based answers override it.

In-app feedback, such as a thumbs-down, rarely fixes a wrong fact by itself; see whether thumbs-down feedback fixes a wrong answer. For the full correction process, see how to correct false or outdated information that AI repeats from its sources.

Is monitoring AI hallucinations about your business worth it?

For most businesses, a periodic test is worth it; constant monitoring may not be. A basic check of about 20 questions on four engines takes most of a working day; a full 40-question set with five runs on five engines is closer to a week of work. That is why the full set is worth running on a fixed schedule and after major changes, with only the fragile facts checked more often.

More frequent monitoring pays off when mistakes are expensive: regulated industries, prices that change often, a recent rebrand or merger, or a business whose name collides with another. The damage from a wrong answer, such as a buyer ruling you out or a partner doubting you, is covered in how AI hallucinations can damage a business's reputation.

What do people ask about testing AI accuracy?

How many times should I ask each question?

Three to five times per engine is enough to separate stable errors from one-off variation. An error that appears in most runs is worth fixing first.

Can I just ask the AI where it got a fact?

You can, but the explanation may itself be invented. Cited sources in search-based answers are more reliable than a model's own account of where it learned something.

Why does the same engine give different answers on different days?

Answers are generated fresh each time, search results change, and models are updated. That is why a fixed question set and repeated runs matter.

What if an AI tool says it doesn't know my business?

That is safer than a wrong answer but still a visibility gap. Check whether your site can be crawled and whether other sites describe you.