AI Visibility Report Teardown: Seven Panels, Two That Lead to a Fix

The typical AI visibility report looks a lot like the free Google Ads grader I used to send clients: a big score, a few gauges, and no clear instruction to fix anything. I am tearing down this report because four buyers asked me the same question in different words: which tool helps us get featured in Google AI Overviews instead of merely watching us fail to show up?

The artefact is familiar across vendors. It runs a fixed set of buyer prompts across ChatGPT, Perplexity, AI Overviews and sometimes Copilot or Claude, then records mentions, citation rate, sentiment and share of voice against named competitors. Monitoring has a real job: 63% of B2B buyers used an AI tool during a purchase in the past year, up from 46%, and 72% say they verify AI answers against reviews and comparisons. A rank tracker cannot show you that verification step.

But after reading a dozen exports side by side, I keep coming back to the grader. Most panels describe the problem. Only two of the usual seven can tell you what to change on which page: cited sources and content gaps, and the second earns its place only when it gets specific. The other panels can help you trust or interpret the report. That is not the same as triggering a fix. I used to mistake the distinction myself.

Panel 1: The visibility score is only as good as its prompt list

The first panel is the biggest number on the page. Your brand appears in 22 of 100 answers, so the report prints 22% visibility. The standard calculation is answers mentioning your brand divided by total answers run. It looks like market share. It reads like a grade. I treated Quality Score with similar reverence in 2016, until I watched it move two points after a landing page swap nobody logged.

The denominator here is not the market. It is the prompt list selected for the report. Add five branded prompts and the score can jump 10 points while nothing about buyer behavior changes, because prompts containing your name cite you near every time. Start with a small set, such as 15 prompts on Otterly Lite, 25 on Semrush or 50 on Profound Starter, and a few different answers can make the trend look dramatic. At 20 prompts, a 20% score carries a 95% interval from roughly 2.5% to 37.5%; even at 100 prompts, the margin is still plus or minus 7.8 points.

That does not make the score worthless. If the prompts stay fixed, a change may be worth investigating. If the prompts change without a note, you cannot tell whether visibility moved or someone rewrote the test. Verdict: use the score as a change detector only when the prompt set is frozen and versioned. Otherwise, it is decoration with a percent sign.

Panel 2: Mention rate must be split by engine

The next panel breaks the headline number into ChatGPT, Gemini and AI Overviews. I would keep this one as a diagnostic, because a blended rate can hide the engine where you are absent. Across 680M citations, ChatGPT’s top-10 sources are dominated by Wikipedia at 47.9%, Perplexity leans on Reddit at 46.7%, and AI Overviews split between Reddit at 21% and YouTube at 18.8%. In a separate 3.7M citation study, 91% of cited URLs appeared in only one model. A good ChatGPT rate tells you little about Gemini. Paid and organic blends taught me the same lesson: the average can conceal the channel that needs work.

Before you trust a zero, check what the plan actually measures. Entry plans differ: Otterly lists $29 a month for 15 prompts, Peec $95 a month for 50 prompts across three of six models, Profound Starter $99 a month for ChatGPT-only tracking, and Profound Growth $399 a month for three engines. A ChatGPT-only report cannot tell you whether you appear in Gemini, Claude or AI Mode. A blank cell is not evidence of absence.

Verdict: keep the per-engine rates, with sample size beside each one. They tell you where to investigate, not which page to edit.

Panel 3: The share-of-voice donut moves when its denominator moves

Third comes the donut chart: you against three competitors. It is the panel most likely to change when nobody did any work. The usual calculation is your mentions divided by all tracked brand mentions, so 30 of 100 counted mentions prints as 30%. That denominator contains only the brands being tracked. Add a competitor mid-period and everyone’s share can fall on otherwise unchanged data.

I have seen the same confusion in paid search auction insights, where a new entrant changes how people read the whole chart. The repair is mechanical: freeze the competitor set, annotate any change, or recompute history against the new set. Without that, a falling slice may mean your brand lost ground. It may also mean somebody added a name to a spreadsheet. Verdict: directional context when the competitor set is stable; decoration when it drifts. Do not assign a page edit from a moving donut.

Panel 4: Sentiment needs a split, not a reassuring gauge

Then comes a single sentiment score, often with a pleasant green tint. Say your brand appears in ten answers: eight positive, two highly critical. The gauge can look healthy even if those two critical answers appear on commercial prompts, where the wording matters most.

A useful view separates positive, neutral and critical mentions, then lets you inspect the answers behind the labels. It also needs a monthly hand audit of around 50 labels, because LLM classifiers can drift when the labelling model changes. Otherwise, a cleaner-looking month might reflect a new classifier rather than a different account of your brand. That matters when an engine uses a bad review or a negative comparison as its source.

Verdict: keep the stacked split and the audit sample; ignore the net score. The critical answer and its source deserve attention. The green gauge does not tell you where to start.

Panel 5: The prompt list reveals whether the test was fair

Scroll to the appendix and you find the lever behind every panel above it. The tracked prompt list helps determine the score before a single answer runs. Prompts containing your company name cite you near every time, so a handful of branded prompts in a 30-prompt set can lift the headline by 10 points or more. Broad terms and narrow questions behave differently, too: one study found less than a 1 in 100 chance that two prompts return the same brand list in the same order.

I used to muddy PPC reports without meaning to by leaving a brand campaign inside a blended CPA. The overall number looked better, but it answered a different question from the one the client had asked. Here, the discipline is to freeze and version the prompt set, split branded from unbranded and commercial from informational, and build the list from questions buyers actually type. If the report shows none of that, read the headline score accordingly.

Verdict: the most honest diagnostic panel, because it exposes the test. It tells you whether to trust the numbers. It still does not name the page to fix.

Panel 6: Cited sources finally name the work

This is the panel I print. The sources view lists the URLs and domains behind each answer; better implementations connect those citations to backlink data for outreach. Read it beside the mention-versus-citation split. A mention means the engine knows your name but sources the claim elsewhere; a citation means it treated your page as the source.

That distinction gives an operator something to do. If mentioned-but-not-cited runs high on commercial prompts, buyers hear your name but may get sent to a review site, a forum thread or a competitor page to decide. Now you can inspect the page that won, the question it answered and the page of yours that should have answered it. That gap changed my work more than any movement in a visibility score.

Google’s eligibility rule sets a useful boundary. To appear as a supporting link in AI Overviews or AI Mode, a page must be indexed and eligible for Search with a snippet; there is no extra technical requirement, and eligibility does not guarantee serving. Check that first. Then compare the answers: if your page buries a direct answer that another source states plainly, give the answer a clear block with a date and a quotable fact. That makes it easier to use; it does not guarantee a citation. If the same directories keep winning your prompts, earn coverage there. If a competitor page keeps winning, tighten your page around the question.

This is where monitoring stops and execution starts. A list of winning URLs changes nothing until someone acts on what it shows. That is the distinction I want from groas: not another export, but search work tied to the outcome the export is supposed to measure. Verdict: keep cited sources. This is the first panel that names the work.

Panel 7: A content gap must name the page and the edit

Most reports end with a content gap panel that reads like a brainstorm: write about pricing, add an FAQ, cover topic X. That is the point where the report quietly becomes a to-do list. I have paid for that list from three vendors and written it myself for clients. I know the failure mode. “Competitors cited for return policy questions” sounds actionable until somebody has to open the site and decide what to change.

Compare it with an instruction tied to the evidence: “Your /shipping-returns page lacks a 40-word answer block that Reddit threads provide on prompt 14; add it under the H2 and retest.” One sentence names the page, the missing block, the competing source and the test. The other starts a meeting. A comparison of 15-plus tools separates metric depth from actionable output for this reason. A matrix of 12 AI Overview tools makes the contrast sharper: only one vendor restructures pages while the rest report only.

Verdict: keep the gap panel only when each gap names an existing URL, the change to make and the source it aims to displace. Otherwise, it is a brainstorm attached to a subscription.

Seven panels in, the answer to “how do we measure visibility across ChatGPT, Gemini and AI Overviews?” is not “ignore the report.” Per-engine rates show where to look. A frozen prompt list tells you whether the test was fair. The sentiment split can warn you that a mention is not necessarily a good one. But cited sources and specific content gaps are the two sections that can turn a finding into a page edit.

The cheapest way to look busy is to automate the reporting while leaving the fixing to somebody else. That is the trap in asking for software that automates visibility reporting without hiring an agency: the charts may arrive on schedule, but the pages do not rewrite themselves because a donut changed colour. I stopped asking how polished the dashboard looked and started asking who executes.

The vendor-demo checklist: make the report produce a work order

Bring this list to a demo for software that promises AI search visibility reporting or content gaps that block citations. Ask for each answer on screen, not in slides.

1. Name the exact page to change for each gap. A gap without a URL creates no work order.
2. Split branded from unbranded prompts. A blended score can reward adding your own name.
3. Print sample size beside every per-engine rate. A ChatGPT-only plan cannot speak for Gemini.
4. Freeze and version the prompt and competitor sets. An unmarked change can make a trend impossible to interpret.
5. Show cited URLs beside your mentions, including the source you aim to displace. This connects the prompt to the page work.
6. Say who executes the edit and when. A chart waiting two weeks for a ticket is still just monitoring.
7. Log the edit against the metric it was meant to move. This is the item most often skipped; without it, you keep paying for reports and never learn which fix earned a citation.

Verdict: keep the evidence that can change a page

I would keep the per-engine mention rate with its sample size, the frozen prompt list that proves the test was fair, and the cited URLs that show which sources won. I would keep a content gap only when it points to a page and an edit. I would demote the headline score, the share-of-voice donut that shifts with its competitor list and the single sentiment gauge. I used to sell that cut pile as insight. It mostly told clients what they already suspected, in better colours.

My overall verdict is simple: the report earns its fee only if someone uses its evidence to make and check a change. If you have builders waiting for precise targets, buy monitoring that gives them URLs, sources and a fair retest. If you do not, buy execution rather than another export. That is the part of the grader I would keep: the evidence. Not the grade.