How to Diagnose a Change or Drop in AI Visibility

Step 1: Is the drop real?

AI answers vary between runs, so small samples swing. A rough check for comparing two periods is to work out the margin of error:

• With a 35% average mention rate and 60 runs in each period, the margin is about ±17 percentage points. A fall from 40% to 30% is within normal variation.
• With 300 runs in each period, the margin shrinks to about ±8 points, and the same fall starts to look real. These margins assume independent answers; repeated runs of the same prompt are correlated, so count distinct prompts (aim for 20–30) and check that the fall shows up across most of them.

If the change is within the margin, collect more runs before acting. Look for drops that persist across several weeks and several engines.

What changes in temperature or model version can invalidate an AI visibility comparison?

The second step, before blaming the market, is to check your own set-up. Comparisons break when any of these change between periods:

• Model version: the engine moved to a new default model, or your tool switched models.
• Mode: fast answers compared with reasoning answers, or web search on in one period and off in the other.
• Temperature and sampling, for tools that call APIs. Higher temperature gives more varied answers. Some reasoning models do not let you set it at all, so a switch of model can change variability too.
• Collection method: consumer app versus API, logged-in versus logged-out, or a change of tracking vendor.
• Location and language settings.
• The question set: questions added, removed or reworded.

Record these settings with every run. If any changed, mark a break in the trend and compare only like with like.

How can you tell whether an AI visibility drop is market-wide or brand-specific?

Compare your drop with three reference points:

1. Competitors. If the main competitors' mention rates fell at the same time, the cause is probably the engine, not you.
2. Control questions. Track a few questions unrelated to your campaigns. If they shifted too, something changed in the engine.
3. Answer structure. Count brands named per answer and sources cited per answer. If both fell across the board, the engine is now giving shorter lists or citing less.

A brand-specific drop shows up as your rate falling while competitors hold steady or rise, on the same questions.

Why can AI model updates make a brand disappear from recommendations?

New models and product changes alter answers in several ways:

• Different training data, which changes what the model knows about brands without searching.
• Different search behaviour, such as searching less often, running different queries or citing fewer sources.
• Shorter lists, which squeeze out brands that used to appear fourth or fifth.
• Different source preferences, favouring some types of site over others.

These shifts can be large. seoClarity tracked ChatGPT across five markets in 2026 and found platform-level drops on 8 March and 19 April. Citation volumes fell by 86–94% between February and April before rebounding in May. Google also made Gemini 3 the default model for AI Overviews in January 2026. Changes like that can reshuffle which sources are cited.

What evidence shows that a model update, not competitors, changed our AI visibility?

Build the case from several signals:

• Timing matches a provider announcement or release note, or a widely reported shift.
• Breadth: the change appears across unrelated questions and markets, not just your topics.
• Competitors moved in the same direction, or no competitor gained what you lost.
• Structure: answers became shorter, named fewer brands or cited fewer sources.
• Sources: the pages you were cited from are still live and unchanged.

If a competitor gained exactly where you lost, on the same questions, the more likely cause is a competitor's new page, coverage or reviews.

Why did my brand disappear from ChatGPT recommendations?

When the drop is real, brand-specific and in ChatGPT, the usual causes are:

• a cited page of yours broke, moved or changed its content;
• your site started blocking OpenAI's crawlers, often through a firewall or CDN setting;
• a third-party source that named you was updated, removed or outranked;
• a competitor published stronger comparison content or earned new coverage;
• review themes turned negative;
• your rankings fell in the search sources ChatGPT relies on.

Check the sources answers cited before the drop and after it. The difference usually points to the cause. Redesign-related losses are covered in why ChatGPT citations disappear after a website redesign or content update.

Why does a client stop appearing in ChatGPT answers without website changes?

Visibility depends on much more than the client's own site. Without any site change, a client can drop because:

• A source changed: a list they were on was rewritten, or a review platform changed its category pages.
• A competitor improved their content, reviews or coverage.
• The platform changed its model, search behaviour or citation habits.
• Infrastructure changed without anyone editing the site. Hosting or CDN providers sometimes change default bot settings, which can block AI crawlers.
• Reviews shifted, with recent negative reviews changing the summary.

Agencies should check these before telling a client that "nothing changed". The website did not change, but the evidence around it did.

What should you record for every run?

A drop is much easier to diagnose if each run records the date, engine, model where known and mode. It should also record location, language and account state, the question ID, the full answer, the brands named and the cited URLs. How to show uncertainty and model releases on dashboards is covered in how an AI visibility dashboard should show uncertainty and model releases.

What might an illustrative diagnosis look like?

Consider an illustrative agency whose client, a skincare brand, falls from 24 to 6 mentions in 30 ChatGPT runs within a week. The fall is well outside normal variation. The client made no site changes. Competitors' mentions are stable, which rules out a platform shift. Comparing sources, the agency finds the client used to be cited from a beauty publication's "best serums" list, and the list has since been rewritten without the client. The agency contacts the publication with the client's new clinical results, and pursues two other lists that ChatGPT cites. Mentions recover to about half their earlier level within six weeks.

What do people ask about AI visibility drops?

How long should we wait before reacting to a drop?

Collect at least one more week of runs, unless the drop is dramatic and brand-specific. Many apparent drops reverse.

Should we change our content immediately after a drop?

Not until you know the cause. Changing content in the middle of a platform-wide shift makes the diagnosis harder.

Do model updates affect all engines at once?

No. Each provider updates on its own schedule, so a drop in one engine alone often points to that provider.

Can drops be prevented?

Not entirely. Diverse sources, several strong pages and consistent monitoring make them smaller and easier to explain.