Discussion starter: an SEO measurement question, posted with an answer from Niraj Raut to open the thread. If you have dealt with this on a site, reply with what you saw, especially where it differs.
- SEO Analytics
How do you call a traffic change real when last year’s comparison period included a core update?
Disclosure: Niraj Raut, who posted this answer, runs the SEO consultancy linked at the end of it.
1 reply
-
adminAdmin
Don’t let the year-on-year figure make the call. When last year’s comparison period contains a core update, the YoY change mixes three things: what your site is doing now, what the update did to last year’s numbers, and ordinary seasonality. One ratio cannot separate them.
I would call a change real only when it holds up in comparisons that avoid the update. That means like-for-like windows that do not cross the update’s anniversary, a before-and-after comparison around the date the change started, and a control segment that had the same seasonality but not the suspected cause. If those agree in direction, and the gap stays outside normal week-to-week noise for several weeks in a row, it is real enough to report. If only the YoY view shows it, treat it as an artefact of the base period until the other comparisons say otherwise.
How an update in the base period invents changes this year
If a core update moved your site, the effect shows up as a step in the series: traffic shifts to a new level over the rollout weeks and stays near it until something else moves it. On the rollout’s anniversary, your YoY line shows the same step in reverse, whether or not anything happened this year. This is a base effect: the ratio moves because the denominator moved.
Think of judging this year’s harvest against a year when a storm flattened half the crop in July. From the storm’s anniversary onwards, this year looks like a bumper season, even if the crop is ordinary. Nothing happened on that date this year. Only the comparison changed. In traffic terms:
- Last year’s update hurt you: YoY growth jumps at the anniversary, and whatever you launched around then gets the credit.
- Last year’s update helped you: YoY growth drops at the anniversary and looks like a decline that needs explaining, even if this year’s traffic is flat.
- The window straddles the rollout: the YoY figure averages a pre-update state and a post-update state, so it describes neither.
The current year can carry breaks of its own. Google completed core updates in March 2026 and May 2026. Google also says it keeps making smaller core updates that it does not announce, because they are not widely noticeable. No base period is guaranteed clean, which is why the control segment and the noise band below matter more than hunting for a perfect year.
Where the 2025 updates show up in 2026 comparisons
Start dates and durations come from the Google Search Status Dashboard. End dates are approximate.
Update Rollout began Ran for about Where a YoY line can step in 2026 March 2025 core update 13 March 2025 14 days Mid to late March. The March 2026 core update began on 27 March 2026, so comparisons spanning late March have an update on both sides. June 2025 core update 30 June 2025 17 days Early to mid July August 2025 spam update 26 August 2025 27 days Late August to late September, for sites the spam systems affected December 2025 core update 11 December 2025 18 days Mid to late December 2026, still ahead Three tests before calling a change real
- It is not a measurement change. Several reporting changes sit between 2025 and 2026. When Google stopped supporting the
&num=100results parameter in September 2025, many sites saw desktop impressions fall and average position improve in Search Console, while clicks were affected far less, according to Brodie Clark’s analysis. Any impressions or position comparison that crosses that point is partly measuring a reporting change. AI Mode data has been counted in Performance totals since June 2025, with no separate click breakdown. In GA4, the AI Assistant default channel group added in May 2026 gives recognised AI assistants their own channel, so channel-level comparisons across that date are not like for like. Add your own tagging, consent and template changes to the list. When a reporting change sits inside either window, lean on clicks and conversions, not impressions. - It is bigger than normal noise. Take a stretch of weeks with no known update or site change and measure how far weekly clicks for the segment swing around their trend. If normal weeks swing by plus or minus 6% around their trend, a week or two running 4% ahead of last year is well inside the noise. My working rule is four consecutive weekly readings outside that band, because a single week can be one news cycle, one bot or one outage. Low-volume segments have wide bands, so aggregate them or lengthen the window.
- It persists, and it sits where the cause would act. A template change should show on that template first, and a content refresh on the refreshed pages. An anniversary effect shows across everything that moved last year, so a change spread evenly across unrelated sections points to a base effect or a market shift, not your work.
Baselines that avoid the update
No single baseline is clean. Use at least two that fail in different ways, and trust the change when they agree.
Comparison How to build it What it controls for Where it fails Like-for-like YoY Drop the rollout weeks from both windows, and split any window that crosses the anniversary, so pre-update weeks meet pre-update weeks and post-update weeks meet post-update weeks Last year’s step Only covers the part of the year on the same side of the step, and ignores other changes Two years back The same weeks two years earlier, if that year had no update in the window The update, when the older year was clean Two years of site, SERP and AI feature changes pile up. Search Console keeps 16 months of Performance data, so you need exports (API, BigQuery) you saved earlier Before and after the change date A few weeks before against the same number of weeks after your change, adjusted with the seasonal shape of a year that had no update in those weeks Last year entirely Short windows pick up in-season swings; weak for changes that build slowly Target versus control segment The change in the segment your work touched, minus the change in a similar segment it did not touch (difference in differences) Seasonality, demand and shared algorithm movement A core update can hit segments differently, so pick a control that moved like the target in last year’s update Counterfactual model Fit a model on a clean pre-period using control series, then predict the post-period All of the above, with an uncertainty range Only as good as its controls and pre-period The before-and-after row is the same logic Google recommends for judging a core update itself: wait at least a full week after the update completes, then compare that week with a week before the rollout started. It works because both weeks come from the same year, so last year’s base never enters the calculation.
For the model route, Google’s CausalImpact package for R builds a Bayesian structural time-series model from the series you care about and a set of control series, then predicts what would have happened without the intervention. Its documentation spells out two assumptions: the controls were not themselves affected by the intervention, and the relationship between controls and target, as established in the pre-period, stays stable in the post-period. A core update inside either period can break both, so train on a pre-period without one, and treat a wide uncertainty range as a “not proven” result rather than a small positive one.
Split the change before you attribute it
Clicks move because demand changed, visibility changed or click-through changed. Each leaves a different fingerprint:
- Demand: check an outside signal for the same terms, such as Google Trends or ad platform volume. If demand fell 10% and clicks fell 10%, rankings may not have moved at all.
- Visibility: hold a fixed panel of queries that had impressions in both periods and compare positions and the share of those queries ranking in the top three and top ten. A fixed panel stops new or lost long-tail queries from distorting the average.
- Click-through: compare CTR within position bands for the same panel. Lower CTR at stable positions points to SERP layout, such as AI Overviews or new features, not rankings.
- Brand versus non-brand: Search Console’s branded queries filter, where your property has it, separates the two. Branded clicks follow marketing, PR and offline demand more than algorithm changes, so they make a rough control for algorithm questions, provided brand activity was similar in both years.
Hypothetical example:
A retailer’s non-brand organic clicks ran about 18% below the previous year through the first half of 2026, then came in about 4% above it for 15 July to 31 August. The team launched rebuilt category pages on 8 July 2026 and credits them with the 22-point swing. But the site lost ground in the June 2025 core update, so from mid-July 2026 its YoY base is last year’s lower, post-update level. The launch happened to land inside the anniversary window, so the YoY line cannot tell the two apart.
Within 2026 alone, the rebuilt category directory is up 11% against the four weeks before launch (10 June to 7 July, after the May 2026 update had finished). Product and blog pages, which were not touched, are up 1 to 2% over the same weeks, which is the seasonal lift. That leaves about 9 to 10 points for the rebuild, concentrated where the work happened and still holding after six weeks. The verdict: the rebuild looks real at roughly +9% on category pages, a few per cent sitewide. The 22-point swing is mostly last year’s loss leaving the base, and it should not appear in the report as the launch result.
What would change the verdict
- The control moved as much as the target: the change is market-wide or algorithmic, not yours.
- The effect disappears when you drop the anniversary weeks from both windows: it was the base effect.
- Impressions moved but clicks and conversions did not: suspect a reporting change before a ranking change.
- The gain fades within a few weeks: treat it as volatility, and check it again after the next update.
For reporting, I would put three numbers side by side with a one-line verdict: the raw YoY change, the like-for-like or before-and-after change, and the target segment against its control. Mark last year’s rollout dates on the chart, with Search Console’s custom annotations or in your dashboard, so the next person reading the YoY line does not rediscover the base effect.
The decision rule: the YoY figure is context, not evidence. Claim a change only when a comparison that excludes last year’s update and a segment-versus-control comparison agree in direction for at least four weeks, and the effect sits where the change was made. When they disagree, report the disagreement and the likely base effect rather than the number that tells the better story.
Need help with this on your own site?
Niraj Raut works with businesses in Nepal, Australia, the UK, Europe and the US on technical SEO, ecommerce SEO, local SEO and AI search.
Add a reply Cancel reply
Replies are open to approved members. Log in or apply to join.