Are 20 Monthly Queries Enough to Detect AI Visibility Trends?

Why does run-to-run variation matter so much?

Ask an assistant the same buying question twice and you will often get a different list. SparkToro's study of runs in late 2025 found less than a one-in-a-hundred chance of ChatGPT or Google's AI returning the same list of brands twice for the same prompt. Your visibility on any question is therefore a rate, how often you appear, not a fixed fact. Measuring a rate needs several observations, and measuring a change in a rate needs more.

How big a change can each design detect?

The table below uses simple binomial arithmetic: a brand that appears in about 30 per cent of answers, answers treated as independent, and a 95 per cent level of confidence. Real answers are not fully independent, because runs of the same question resemble each other, so treat these figures as the best case. In practice the smallest detectable change is larger.

The realistic column reflects that the distinct question-and-engine pairs, here up to 100, set the floor: extra runs of the same pairs narrow the margin far less than the binomial figures suggest.

The point is not the exact numbers but the shape: more distinct observations narrow the margin, and repeated runs of the same question count for less than new questions or engines because they resemble each other. Twenty questions on five engines can support a trend for large moves; twenty answers cannot.

What about trends on a single question?

Per-question trends need even more care. A question asked three times on five engines gives 15 answers per reading, enough to see that you are named in most answers or in almost none, but not enough to trust a move from, say, six appearances to eight. For single questions, look for consistent direction across several readings and across engines rather than one jump.

That is also why the overall figure and the per-question view do different jobs. The overall figure tells you whether things are moving; the per-question view tells you where, and the cited pages tell you why.

What else can break a trend?

• Changing the questions. Edited questions are new questions. Their history should start again, and the rest should stay untouched.
• Engine and model updates. Providers change models and search behaviour without notice. A sudden shift on one engine only is often their change, not yours.
• Settings. Location, language, signed-in state and search mode all change answers. Keep them fixed.
• Seasonality. Some categories change answers with the season, prices or news.
• Mixing methods. A month of manual checks followed by a month of a tool's readings is not a trend.

How should you read a trend honestly?

1. Check the size against the table above before reporting a change.
2. Look for agreement across engines and across weekly readings. A change that appears everywhere is more believable than one that appears once.
3. Read the answers behind the change. If a new comparison page started being cited, the trend has a cause you can name.
4. Line it up with your work. Note the dates of page changes, outreach and listings fixes, and allow time for engines to recrawl.
5. Say what you do not know. A flat month with wide margins is "no clear change", not "no effect".

Our guides on measuring the noise floor for ChatGPT citation tracking and calculating appearance rate across repeated runs go deeper on the method.

What might reading a month look like?

Take an illustrative business that appears in 27 per cent of baseline answers in September and 31 per cent in October, on the same 20 questions, five engines and three runs. The four-point rise is inside the margin in the table, so on its own it is "no clear change". Now look underneath: three of the twenty questions moved from near-absent to named in most answers on two of the five engines, all three relate to a service page rewritten in early September, and the new page is cited in those answers. The overall figure barely moved, but the per-question view and the cited pages show a real effect on the questions the work targeted. That is the report worth writing; the headline number is the least informative part of it.

What should a monthly trend report include?

• The overall figure with its rough margin, and whether the change is outside it.
• The questions that moved, in which direction, and on how many engines.
• The pages newly cited or no longer cited.
• The work done since the last reading, with dates.
• Anything that changed in the method, such as replaced questions.

Keep the format identical from month to month, so readers learn where to look and can compare readings quickly.

How does Kovmere schedule its readings?

Kovmere asks up to 20 questions for one market, up to 18 buyer questions that do not name you and up to 2 brand checks reported separately. On Pro, each billing period starts with a baseline in which every question is asked three times on each of OpenAI, Perplexity, Google Gemini, Anthropic Claude and Google AI Overviews, up to 300 answers, followed by single-run re-tests in weeks two, three and four, up to 100 answers each. The questions stay frozen; if you replace some, up to 5 per round, those start a fresh history from the next baseline and the rest keep theirs. The answers and the pages they cite can be read, with the 60 most-cited pages labelled by type and whether they mention you.

That schedule gives Kovmere up to 300 answers in a baseline and up to 600 across a billing period, but repeated runs of the same questions resemble each other, so treat month-to-month moves under about 10 to 15 points as unconfirmed and look for clear per-question patterns; it is not built to detect small shifts. A one-time Check gives a single baseline, which is a starting point rather than a trend.

What do people ask about detecting AI visibility trends?

How long before I can see a trend?

Usually two or three full readings on the same questions, often a few months, depending on how large the change is and how quickly engines pick up your work.

Is weekly tracking better than monthly?

Weekly readings add answers and show timing; they do not remove variation. Combine them with a fuller baseline and read them as a series, not one at a time.

Should I use more questions or more runs?

For trend detection on a fixed set, more engines and more distinct questions help most; extra runs of the same question mainly show whether that one question's result is typical. For coverage of a broad topic, more questions help.

Can a tool tell me a change is statistically significant?

Some report confidence intervals; many do not. Either way, check whether the change is consistent across engines and readings before acting on it.