What Should an Agency Check Before Sending a Client an AI Visibility Report?
Before sending an AI visibility report, an agency should check five things. First, the method: same questions, engines, settings and run counts as last time, all stated in the report. Second, detection: brand and competitor mentions were identified correctly, without counting similarly named companies or missing name variants. Third, citations: the counted URLs really belong to the client. Fourth, the numbers: totals and percentages reconcile, and comparisons use the same baseline. Fifth, the story: screenshots are representative, claims about accuracy are verified against the client's facts, and every recommendation follows from the evidence. Read every answer behind a headline claim, and a random tenth of the rest to catch common detection problems. Reports that fail these checks damage trust quickly, because clients can rerun a few questions themselves.
What is on the pre-send checklist?
Method - The question set is the one agreed with the client, unchanged since the last report, or changes are listed. - Brand-name questions are reported separately from category questions. - Engines, modes, locations, languages, run counts and dates are stated.
Detection - Brand name variants, abbreviations and product names were all counted. - Similarly named companies were not counted as the client. - The competitor list matches what was agreed, and new competitors are flagged rather than silently added.
Citations - Counted URLs belong to the client's domains, including subdomains and regional sites, and not to namesakes. - Third-party pages that mention the client are reported separately from the client's own pages.
Numbers - Percentages show their base, such as "14 of 60 answers". - Totals add up across engines and question groups. - Changes are calculated against the correct baseline period. - Changes are larger than the noise before they are called a gain or a loss. For a rate measured on 60 answers, a month-to-month difference under about 15 points is within normal variation; say "no clear change" instead.
Content and claims - Statements about what AI "says" about the client are quoted from saved answers. - Inaccuracies flagged in answers have been checked against the client's current facts. - Screenshots are representative of the pattern, not the best single run.
Presentation - Client name, logo and details are correct, especially in white-label templates. - Links in the report work, and saved answers can be opened. - Caveats about run-to-run variation are included.
What errors are most common?
| Error | How it happens | How to catch it |
|---|---|---|
| Namesake counted as the client | Automatic detection matches a similar company name | Spot-check answers where the client is "mentioned" |
| Missed mentions | Product names or abbreviations not in the detection list | Search saved answers for variants |
| Inflated visibility | Brand-name questions mixed into category results | Report them separately |
| Wrong baseline | Question set or engines changed between periods | Compare method notes month to month |
| Stale "inaccuracy" | The client changed prices or hours after the run | Confirm facts with the client before reporting errors |
| Cherry-picked screenshot | Best run chosen to illustrate a win | Show counts next to screenshots |
| Noise reported as a trend | A few answers moved between months | Compare the change with the margin for the sample size before writing the headline |
| Template leftovers | Previous client's name or figures in a white-label report | Read the report end to end before sending |
How do you spot-check the data?
- Sample about 10% of answers at random, plus every answer behind a headline claim.
- Read each answer and confirm the detected mentions, their roles and the cited URLs.
- Rerun three to five questions yourself in the named engines to confirm the pattern still holds.
- Record any corrections and fix the underlying detection rules, not just the report.
If the spot check finds more than an occasional error, hold the report and fix the pipeline first.
What should the report say about limits?
State plainly that AI answers vary between runs, users and locations, and that the figures come from a defined sample of questions and runs. Explain any engine or model changes during the period that might affect results. Clients who understand the method trust the numbers more, and are less alarmed when their own checks differ slightly. Why clients sometimes cannot reproduce agency results is covered in why an agency's GEO report shows citations the client cannot reproduce.
Do recommendations follow from the evidence?
Before sending, check that each recommendation links to a finding. For example: "The comparison article cited in 9 of 12 comparison answers does not list you, so pitch its author with updated information." Recommendations that could apply to any client, such as "add more FAQs", signal that the analysis is thin. How agencies track many clients consistently is covered in how agencies track AI visibility across multiple clients and locations.
What do people ask about checking AI visibility reports?
How long should the pre-send check take?
For a typical monthly report, an hour or two, most of it on the spot check.
Should the client see the raw answers?
Offer access to them. Transparency makes disagreements quick to resolve.
What if the numbers look worse than last month?
Report them honestly, with the likely reasons and what you will do. Hiding a decline damages trust more than the decline itself.
Can automation replace the manual check?
Automation handles collection and counting; a person should still read a sample of answers each month.
Published by The Kovmere Team · updated Sep 29, 2026 · More resources