Why Search Console AI data is misleading marketers

Yes, the new generative AI data in Search Console is useful. It is also a trap if you read it like classic SEO reporting.

The core problem is simple. Google now gives site owners a clearer view of where their pages appear in AI Overviews and AI Mode, but the dedicated report emphasizes impressions, not economic outcomes. That means teams can watch AI visibility rise while traffic, leads, and revenue stay flat or even fall.

It gets worse. In AI Overviews, every linked URL inside the block can inherit the same position when the overview sits at the top of the page. Then average position can blend those AI appearances with normal organic rankings into a single number that looks healthier than reality. If your reporting celebrates AI impressions without cross-checking first-party analytics, conversions, and brand citations, you may be telling a success story that the business cannot cash.

What did Google actually ship in Search Console?

In June 2026, Google rolled out a dedicated Generative AI performance report in Search Console. It is meant to show how your site appears in AI Overviews and AI Mode. For many SEO teams, that looked like the missing visibility layer they had been waiting for.

But the report is narrower than the excitement around it suggests. Google’s own help documentation shows impressions broken down by dimensions such as page, country, date, and device. What you do not get in that dedicated view is the full decision context marketers usually want: no clean query-level breakdown for those AI appearances and no dedicated click view that lets you isolate traffic generated specifically by those generative surfaces.

That difference matters. A normal search report helps you connect query, rank, click-through rate, landing page, and outcome. The generative AI report mostly tells you one thing: your URL was shown somewhere inside an AI feature.

Take a simple example. Imagine a software comparison page that receives 25,000 AI impressions in a month. That sounds promising. But if you cannot see which buying-intent questions triggered those appearances or how many visits came from them specifically, the number is closer to a signal of presence than a measurement of performance.

• What the report is good at: spotting which pages are surfacing in Google’s AI features.
• What it is weak at: proving whether those appearances changed user behavior.
• What teams often assume anyway: that more impressions automatically means more demand.

That last assumption is where the trouble starts.

Why are impressions a dangerous comfort metric?

An impression is not a visit. In generative search, that gap becomes brutal.

AI Overviews and AI Mode often answer a large share of the user’s question before any click happens. The user sees the synthesis, extracts the key fact, and moves on. Your brand may have been present in the experience, but presence alone does not pay for content, headcount, or pipeline.

This is why the new data can become a comfort metric. Comfort metrics look impressive in dashboards, trend lines, and board slides, but they do not tell you whether the business got stronger. A rising impression chart can hide a declining traffic chart. It can also hide a weaker conversion funnel if users are getting enough information on Google to skip your site entirely.

To be fair, Google has said that clicks from pages with AI Overviews can be higher quality, with users spending more time on site. That may be true in some cases. The issue is that the dedicated generative AI report does not give marketers the reporting depth needed to validate that claim at the page, query, and conversion level inside the report itself.

Here is a practical example. A glossary page answering “what is retrieval-augmented generation” might earn a surge of AI impressions because Google can summarize the definition directly. That page could look like a winner in Search Console while producing almost no incremental business value. By contrast, a lower-volume product page that earns fewer impressions but generates demo requests is the page you actually want to protect.

The trap, then, is not bad data. It is incomplete data being used as if it were complete.

How can AI Overviews turn position 1 into a vanity number?

Google’s position methodology for AI Overviews is the most underappreciated problem in the whole reporting stack. Google says an AI Overview occupies a single position in search results, and all links inside that overview are assigned that same position.

In plain English, if the AI Overview sits at the top of the search results page, every linked URL inside it can be treated as position 1 in Search Console.

That sounds better than it is. Not all links inside an AI Overview receive the same level of attention from the user. One citation may be prominent and immediately visible. Another may sit deeper in an expandable list or secondary reference area. Yet both can inherit the same top position label in reporting.

For marketers, that creates a false equivalence between very different levels of exposure. Position 1 starts to mean at least three different things:

• Primary visibility: your link is one of the most visible citations in the overview.
• Secondary visibility: your link is present but not central to the answer.
• Technical inclusion: your link is counted in the block even if most users never meaningfully notice it.

Example: your pricing guide appears as a secondary citation inside an AI Overview for “best CRM for startups.” Search Console may effectively label that appearance as position 1 if the overview is at the top. But your page did not rank first organically, and it may not even have been one of the first links users saw. Reporting that as “we now rank number one” is not analysis. It is wishful translation.

This is exactly why old ranking language breaks in AI search. Inclusion matters. Placement matters. Visibility within the generated answer matters. But the familiar label of “position 1” no longer carries its old meaning.

Why does average position get worse, not better, in hybrid search results?

Average position was already a slippery metric before generative search. AI Overviews make it worse because they mix fundamentally different kinds of visibility into one neat-looking number.

Suppose a page appears inside an AI Overview at position 1 and also ranks organically at position 4 on the same results page. Search Console can report an average position of 2.5. On paper, that looks like a strong page-one performance. In reality, the traditional organic listing at rank 4 may be doing the real traffic work, while the AI Overview appearance contributes little or nothing in clicks.

Now imagine the page used to rank organically at position 2 but slipped to position 4 over time. If it also gained AI Overview inclusion during that period, the blended average might still look stable or even improved. A leadership team reading only the average position trend could conclude that performance is holding up. In practice, your organic click potential may have weakened.

This is why blended metrics can be dangerous in hybrid search environments. They flatten unlike things into a clean statistic. Organic rankings, AI citations, and generated-answer placements are not interchangeable units of value.

If your team insists on tracking position, treat average position as a rough signal, not a KPI. At minimum, pair it with manual result reviews, landing-page traffic, and conversion data. Better yet, stop letting average position lead the conversation.

What should teams measure instead?

They should measure outcomes first, then visibility quality, then exposure. In that order.

AI visibility is the measurable presence of your brand inside generated answers. In practice, that means not just whether your URL appeared, but whether your brand was mentioned, how it was described, which source pages were cited, and whether those appearances correlate with business results. If you want a fuller framework, BotRank already mapped out the GEO metrics that matter in AI search.

A stronger reporting stack usually includes five layers:

• Organic revenue and pipeline contribution. If AI visibility grows but revenue from organic landing pages falls, the growth story is incomplete.
• Qualified leads and assisted conversions. For B2B teams, demo requests, trials, and influenced opportunities are more valuable than impression spikes.
• Brand mentions and citations inside AI answers. In AI search, a mention can shape consideration before a click ever happens.
• First-party behavioral data. Time on page, engaged sessions, return visits, and CRM connection tell you whether visits are useful.
• Source-level analysis. Which pages and external sources are actually feeding AI systems, and are they representing the brand accurately?

Notice what dropped down the list: raw impression growth and average position. They are not useless. They are just weak lead indicators unless they connect to the layers above.

This is also where operational discipline matters. Teams need a repeatable way to turn noisy visibility signals into action items. A workflow built around prioritized GEO recommendations is more valuable than another dashboard tab that everyone misreads the same way.

BotRank's Take

The biggest mistake teams will make with Google’s new AI reporting is treating it as a more modern ranking report. It is not. It is an exposure report for a search environment where exposure, recommendation, and click are no longer the same event.

That is why BotRank would not use Search Console as the center of an AI search strategy. It is one input, not the operating system. The more useful layer is AI Visibility: a workflow that lets teams create reusable prompts, run them across multiple models, compare how their brand and competitors are described, and track which sources keep showing up behind those answers. That matters because the real question is no longer “Did Google show my URL?” It is “Was my brand included, represented correctly, and cited in commercially relevant answers?”

If that distinction feels new, it is worth reading why AI visibility starts before the prompt and ends with citations. That is the measurement shift many SEO teams still have not fully absorbed.

How should SEO teams rebuild reporting for AI search?

Start by separating four reporting layers that used to blur together in classic SEO.

• Exposure: where and how often your pages appeared in AI features.
• Traffic: what first-party analytics show after those appearances.
• Outcome: leads, revenue, assisted conversions, and retention impact.
• Influence: whether your brand is cited, recommended, or framed positively in AI answers.

Once you split the stack that way, reporting gets clearer fast.

For example, a content team might learn that a category page earns strong AI exposure but weak traffic. Instead of calling that a failure, they can ask a better question: did the page still increase branded searches, direct visits, or conversion rate on later sessions? If yes, the page may be influencing demand rather than harvesting it. If no, the impression growth may be mostly cosmetic.

This is also where Source Analysis becomes valuable. In AI search, the page you want cited and the page the model actually trusts are often different. Source-level analysis helps teams see whether Google and other models are relying on the right assets, outdated pages, or third-party summaries that barely mention the brand.

From there, build a weekly review process:

• Track AI-feature impressions by page and device in Search Console.
• Compare those pages against first-party session and conversion data.
• Review the exact prompts and answer types that matter commercially.
• Inspect which pages and external sources get cited most often.
• Prioritize fixes around clarity, evidence, page structure, and source trust.

That last step is where many teams stall. They can see the problem but cannot turn it into work. A practical starting point is to study what increases citation likelihood in generative search, especially for Google’s own interfaces. This is why pieces like How to get cited in Google AI Overviews matter more right now than another debate about average position.

The short version: AI search reporting should be built like revenue reporting, not vanity reporting. If a metric cannot help you decide what to do next, it should not lead the dashboard.

FAQ: what should marketers do with Search Console AI data?

Is the generative AI report in Search Console useless?

No. It is useful for identifying where your pages appear in AI Overviews and AI Mode. It becomes misleading only when teams treat impression growth as proof of business impact.

Should we stop tracking AI impressions?

Also no. Track them as an exposure signal, but never as the headline KPI. They need to be paired with first-party traffic, conversion, and citation data.

Can an AI Overview mention still matter without a click?

Yes. In AI search, a brand mention can shape recall, comparison, and later demand even when the user does not visit immediately. That is why citation and brand-perception tracking now matter alongside traffic analytics.

What is the best way to explain this to leadership?

Show them two views side by side: impression growth and business outcomes. If impressions rise while leads, revenue, or assisted conversions do not, the reporting story needs to change.

The practical takeaway is blunt: stop asking whether AI search made your charts look better. Ask whether it made your business stronger. Search Console can help you spot presence, but it cannot tell the whole story of demand, influence, and commercial value on its own. If your team needs a more operational way to turn AI visibility into action, build that workflow with a structured GEO roadmap rather than another vanity dashboard.