Discussion starter: an SEO measurement question, posted with an answer from Niraj Raut to open the thread. If you have dealt with this on a site, reply with what you saw, especially where it differs.
- AI Search
- SEO Analytics
Impressions up, clicks down, average position stable: how do you separate AI Overviews, SERP layout changes and reporting artefacts?
Disclosure: Niraj Raut, who posted this answer, runs the SEO consultancy linked at the end of it.
1 reply
-
adminAdmin
When impressions rise, clicks fall and average position holds steady, the cause is usually a change in composition before it is a change in rankings. Average position is an impression-weighted average of the topmost position you held for each query. It can stay flat while the queries, surfaces and SERP features underneath it shift a lot.
You separate AI Overviews, layout changes and reporting artefacts by working at query-cohort level, not at property totals. Work in this order: rule out counting changes (cheapest), then split new impressions from lost CTR on queries you already had, then check which of those queries changed on the results page. The property-level chart cannot answer this for you. It is where the question starts.
Why a flat average position hides the answer
Search Console’s metric definitions explain most of the confusion. Position is the topmost position a link to your site occupied, averaged across impressions. An AI Overview occupies a single position, and every link inside it gets that same position. An impression counts when a result appears on the current results page, whether or not the user scrolled to it. In AI Mode, a follow-up question is treated as a new query, with its own impressions, positions and clicks.
Three consequences follow. Being cited in an AI Overview can improve your reported position on a query even when your organic listing sits lower and gets fewer clicks. A position 3 on a page stacked with ads, shopping units and an AI Overview is recorded the same as a position 3 on a clean page, even though it may sit far below the fold. And conversational follow-ups can add impressions on queries that barely existed a year ago.
Hypothetical example:
A site has 100,000 impressions at an average position of 6.0 and 3,000 clicks. Next period, the same queries still produce 100,000 impressions at position 6.0, but their CTR falls from 3% to 2.2%. The site also gains 25,000 impressions from AI Overview citations at around position 1.5 (very few clicks) and 15,000 from new long-tail queries at around position 14 (almost no clicks).
Total impressions rise 40% to 140,000. Average position moves from 6.0 to about 6.1. Clicks fall to roughly 2,370. Every headline number looks like “rankings stable, demand up, CTR collapsed”, but there are three separate stories underneath, and only one of them (the CTR drop on existing queries) is a real loss.
Step 1: rule out reporting artefacts
Artefacts are the cheapest explanation to eliminate, and they are more common than most reports admit.
- Known anomalies. Check Google’s data anomalies page for both comparison periods. Recent entries include a logging error that reduced generative AI impressions between 13 and 17 August 2026 (since restored) and a drop in FAQ impressions from 7 May 2026, when FAQ rich results stopped appearing.
- The new multimodal split. From 24 September 2026, Search Console splits Web into text-based and multimodal traffic (Lens, Circle to Search, image uploads and Chrome’s “Search this image”). John Mueller said that this data was not previously in the counts, and Search Engine Journal noted it is unclear whether the split affects existing Web totals. If your window touches late September 2026, compare text-only Web data as well.
- Counting regimes across years. When Google stopped supporting the
num=100parameter in September 2025, most sites in one 319-property analysis saw impressions fall, with fewer queries on page 3 and beyond, because rank trackers and scrapers had been generating impressions. That change pushed impressions down, so it does not create your pattern directly. It does show that automated requests have been counted before, and that a year-on-year comparison straddling mid-September 2025 compares two different counting regimes. - Totals that will never reconcile. Anonymised queries are included in chart totals unless a query filter is applied, the table is truncated, and the graph is always aggregated by property. Summing the query table will not match the chart. Use the bulk export or API for anything you plan to decompose.
- Property and filter hygiene. Confirm you are comparing the same property type, the same country and device filters, and windows with the same weekday mix. A newly verified subdomain or a switch from URL-prefix to domain property produces “growth” that is not growth.
Step 2: split the change into query cohorts
Export query-level data for both periods and classify every query into one of three cohorts. Then compute how much of the impression gain and the click loss each cohort contributes.
Cohort Definition What it tells you Continuing queries Impressions in both periods; sub-split by position change (stable within about one position, improved, worsened) If stable-position queries lost CTR, something on the results page changed. This is the only cohort where a CTR comparison is fair. New queries Impressions only in the recent period If most impression growth sits here with near-zero clicks, you have a mix shift: AI citations, AI Mode follow-ups or long-tail expansion. Lost queries Impressions only in the earlier period If these were click-heavy, part of the click loss is lost coverage, which average position hides because the query no longer contributes to it. Run the split separately for branded and non-branded queries using the branded queries filter, and separately by device. Brand CTR behaves differently from non-brand, and desktop and mobile layouts differ enough to hide opposite trends in one average.
Query shape is a useful hint, not proof. A rise in long, conversational or question-style queries with impressions but almost no clicks fits the AI Mode behaviour described above, but long-tail content expansion produces a similar shape. Treat it as a lead for Step 3.
Step 3: separate AI Overviews from other layout changes
For continuing queries with stable positions and falling CTR, you need to know what the results page looked like in each period. Search Console will not tell you. Use a rank tracker that stores SERP feature history, or manually sample your top non-branded queries by device and country. Record whether an AI Overview appears, whether you are cited in it, the number of ads, and whether shopping, video, forum or People Also Ask blocks appear. Also note whether you lost a rich result or had your title rewritten.
Then compare two groups: continuing queries that gained an AI Overview between periods, and continuing queries whose SERP stayed free of one. This is a simple difference-in-differences comparison.
- If CTR fell mainly in the group that gained an AI Overview, AI Overviews are the leading explanation.
- If CTR fell by a similar amount in both groups, the cause is broader: ads, other features, seasonality, intent drift, or a lost rich result such as the FAQ results Google removed in May 2026.
- If CTR fell only on one device or in one country, look for a layout test or feature rollout specific to that market before blaming AI features in general.
External data gives you a sense of scale, not your answer. Pew Research’s March 2025 browsing panel found users clicked a traditional result in 8% of visits when an AI summary appeared, versus 15% without one. Ahrefs’ February 2026 analysis estimated position-one CTR was about 58% lower than forecast for keywords with an AI Overview. The same analysis found position-one CTR on informational keywords without AI Overviews also roughly halved between December 2023 and December 2025. That second finding is the one practitioners skip: not every CTR decline at a stable position is an AI Overview effect.
Google’s generative AI performance report helps here, within limits. It shows impressions only (no clicks, CTR or queries) for AI Overviews and AI Mode, by page, country, device and date. Google’s help page says it draws on the Web search type data, and that not every property has access yet. Search Engine Journal reported a worldwide rollout on 31 August 2026. Use it to check whether the pages driving your impression growth are the same pages gaining AI feature impressions. I would not subtract it from Web totals to estimate “classic” impressions, because the documentation does not explain how a URL appearing in both an AI feature and the standard results is handled.
Watch for this:
Analytics cannot settle this one. AI Mode is reported inside the Web search type in Search Console, and practitioners report that clicks from AI Mode arrive in GA4 as google / organic with no distinguishing referrer. Filtering GA4 for “AI traffic” will not isolate Google’s own AI features.
A diagnostic matrix for the common patterns
Pattern in the data Most likely explanation How to confirm What would weaken it Impression growth concentrated on desktop and in deep positions, with no click change on those queries Counting or automated-request artefact Device and position-bucket split; anomalies page; check the dates against known changes Growth spread across devices and producing some clicks Growth concentrated in new, long or question-style queries with near-zero clicks Mix shift from AI Mode, AI citations on queries you do not rank for organically, or long-tail expansion Cohort split; matching growth in the generative AI report for the same pages New queries are short head terms where you rank in the top five Continuing queries at stable positions lose CTR, and an AI Overview newly appears AI Overviews absorbing clicks Difference-in-differences against queries without a new AI Overview CTR fell just as much where no AI Overview appeared Continuing queries at stable positions lose CTR, with no AI Overview change Other layout changes: ads, shopping, video, forums, lost rich result, rewritten title SERP feature history; search appearance filter; snippet review SERP unchanged; then check seasonality and intent drift Position improves on queries where you are cited, while clicks fall Topmost-position counting for AI Overview links Compare cited versus non-cited queries for the same pages Organic listing genuinely moved up on those queries One caveat on the search appearance filter: Google’s help page notes that when you filter by appearance, it is not guaranteed that every click for a URL came from the link with that feature. Treat appearance-level CTR as indicative.
Making the conclusion hold up
Compare 28-day windows aligned by weekday, and add a year-on-year view for seasonal businesses. Keep a log of confounders: the March and May 2026 core updates, the multimodal split, site releases and title changes. Decide in advance what result would change your mind. If the AI Overview cohort and the control cohort lost CTR equally, stop attributing the decline to AI Overviews, however plausible it sounds.
Then check whether the click loss matters commercially. Google states that clicks from results pages with AI Overviews are “higher quality”, meaning people spend more time on the site. That is Google’s claim, not an independent finding, so test it: compare conversions and engaged sessions per organic landing page for the affected cohort before and after. Fewer clicks with stable leads is a reporting problem for the SEO team, not a revenue problem for the business.
The decision rule
Do not diagnose this from the property chart. Rule out artefacts, split queries into continuing, new and lost cohorts, and then compare CTR only on continuing queries at stable positions, split by whether an AI Overview appeared. If the loss sits in the new-query cohort, it is mostly mix and may cost you nothing. If it sits in continuing queries that gained AI Overviews, the fix is about being the cited source and earning the remaining clicks. If it sits in continuing queries whose SERP changed in other ways, start with the features that took the space, because that is where the clicks went.
Need help with this on your own site?
Niraj Raut works with businesses in Nepal, Australia, the UK, Europe and the US on technical SEO, ecommerce SEO, local SEO and AI search.
Add a reply Cancel reply
Replies are open to approved members. Log in or apply to join.