Discussion starter: a content SEO question, posted with an answer from Niraj Raut to open the thread. If you have dealt with this on a site, reply with what you saw, especially where it differs.
- Content SEO
Have content scoring grades correlated with rankings in controlled tests for you, or just with content that looks like everyone else’s?
Disclosure: Niraj Raut, who posted this answer, runs the SEO consultancy linked at the end of it.
1 reply
-
adminAdmin
Controlled public evidence that raising a content grade improves rankings is thin. The published data is observational and weak even on its own terms: an Ahrefs study of five content tools across 20 keywords found only weak correlations between scores and rankings. The correlation also has a built-in bias. A grade is calculated from the pages that already rank, so ranking pages score well partly by construction.
What a grade measures reliably is how closely a draft’s terms and subtopics match the current top results. That makes it useful as a floor, because it catches subtopics searchers expect and a draft missed, and risky as a target, because pushing an already relevant page from 70 to 95 mostly adds vocabulary every competitor already uses. It earns its keep on under-covered pages and adds little on pages that are relevant already and need a reason to be chosen over nine similar results.
What the grade is measuring
The tools differ in detail and keep their weightings private, but the common approach runs like this. Take the top results for a query, extract the terms, entities and headings those pages share, and score a draft on how much of that shared vocabulary it covers, sometimes with length and readability folded in. The reference set is the current SERP.
An exam analogy fits. Imagine writing a marking scheme by listing the points that appear in the ten best-graded answers, then checking whether those ten answers score well against it. They will, and that says nothing about whether the scheme identifies good answers. It also cannot reward an answer that makes a strong point none of the ten made. Content grades have the same structure. The score is fitted to the pages being ranked, so a correlation measured on those same pages is inflated, and a point absent from the reference set earns no credit however useful it is. In modelling terms, the feature has been built from the outcome it is being tested against.
Why grades and rankings correlate even if the grade does nothing
- Circularity. The reference set is the ranking set, and top pages share vocabulary with each other.
- Confounding. Sites that produce thorough, well-researched pages also tend to have links, brand demand, editorial budgets and a track record. The grade picks up the effort, while the rankings may be responding to everything that comes with it. The same trap applies to your own data: the pages you optimised hardest are likely the ones you also promoted and linked to.
- A real relevance threshold. Covering the core subtopics is part of being relevant at all. A page about offset accounts that never explains how the offset reduces interest is likely to struggle, and a low grade can flag that. This is the part of the correlation worth keeping.
- Diminishing returns. Past that threshold, extra terms mostly add length and repetition. Coverage helps a page get into the set of candidates for a query; the order within that set is decided by many other signals.
What the public evidence shows, and what it cannot
The Ahrefs study, published in May 2025, took 20 random keywords, scored the ranking pages with five tools (Surfer, Frase, NeuronWriter, Clearscope and Ahrefs’ own AI Content Helper), and averaged Spearman and Kendall correlations against rankings. Correlations were weak for every tool and described as very weak for Surfer, Frase and Clearscope. NeuronWriter and Ahrefs’ own tool fared best, and the article discloses that Ahrefs makes that tool. The authors still read a weak positive correlation as a sign that scores can help, while acknowledging that correlation may not be causation.
I would read it more cautiously. Twenty keywords is a small sample, the design is observational, the scores share the circularity above, and the study comes from a vendor with a tool in the test. It supports “grades and rankings are weakly associated”. It does not support “raising your grade will lift your rankings”, and it does not test whether high-grade pages are more interchangeable with their competitors, which is the concern in the question.
Before-and-after case studies are weaker still. When they cover a handful of pages with no control group, and bundle the rewrite with other changes that can move rankings on their own (a fresh crawl, a new date, added internal links, a better title), they cannot separate the grade from everything else.
Google’s guidance describes something a coverage grade cannot see. Its helpful content self-assessment asks “Does the content provide original information, reporting, research, or analysis?” and “Does the content provide substantial value when compared to other pages in search results?” It asks whether you are mainly summarising what others have to say without adding much value, and it answers its own question about preferred word counts with “(No, we don’t.)” These are questions for assessing your own content, not a scoring formula. But a grade used as a target rewards convergence on the existing results, which is close to the opposite of what they describe.
If your tool now adds an AI-visibility or “GEO” grade, apply the same scepticism. Google’s guide to its generative AI features (May 2026) says you do not need AI-specific rewording or artificial chunking for those features.
How to use a grade without producing copycat content
What the grade shows What it probably means What to do Low grade; the draft misses subtopics a searcher would expect A real gap Add them, in your own structure Low grade; the draft skips tangents or synonyms competitors happen to use Tool noise Ignore it High grade; the page still does not rank The constraint is elsewhere: intent mismatch, links, site reputation or SERP features Stop editing copy and diagnose the real constraint High grade; the page reads like every other result Relevant, with no reason to be chosen Add material no competitor has - Use the term list during research as a gap check, then close the tool. Writing inside the editor with a live score tends to pull every draft towards the SERP average.
- Set a floor, not a target: within the range of the current top results rather than above them.
- Track a distinctiveness count next to the grade: original data, first-hand tests or photos, worked examples, expert input, specific numbers and decision rules that appear in none of the top ten pages. Make that the editorial target.
- If you have the tooling, measure how similar a draft is to the current top results (embedding similarity is one way). A grade rising alongside similarity means the draft is converging, not improving.
- Take grades out of writer KPIs and bonuses. A metric every competitor can max out is a poor measure of what makes your page the one to choose.
- Watch for grade-driven bloat: length creep, repeated terms and padded FAQ sections.
Hypothetical example:
Two guides on choosing a home battery. Draft A scores 94 by covering every term the top ten share: capacity, depth of discharge, inverter types, warranties, rebates. Draft B scores 78 because it skips some tangents, but it adds a worked payback calculation for three household profiles, a table of warranty exclusions taken from five manufacturers’ documents, and an installer’s list of the site problems that delay installations. The grade prefers A. A searcher comparing ten similar pages has a reason to choose B, and B is the kind of page Google’s self-assessment questions describe.
There is also a plausible but untested argument that consensus pages are the easiest for AI answers to replace, because a summary of the top results already contains everything they say. Treat that as a hypothesis worth tracking, not a finding.
A test that answers the question
With enough comparable pages, you can test this directly. The design needs three arms, because the real question is not “does optimising help?” but “does optimising towards the grade beat adding original material?”
- Population: existing pages of the same type with stable impressions. Avoid new pages, which mix indexing effects into the result. Use dozens of pages per arm at minimum, and more if traffic per page is low.
- Arms: (A) edit to the tool’s target grade only; (B) add distinctive material while holding the grade roughly flat; (C) no change. A fourth arm doing both is useful if you have the pages.
- Assignment: randomise within traffic bands and topic clusters, and spread writers across arms so writer quality does not become the variable.
- Metrics: primary, organic clicks against the control arm. Secondary, rankings for the target queries, the number of queries each page appears for, CTR at comparable positions and conversions. Do not use the tool’s own score as an outcome.
- Timing: ship all arms on the same day with no date changes, internal-link changes or title changes bundled in. Start measuring once most changed pages have been recrawled (check server logs or a URL Inspection sample), and avoid confirmed update windows. Google has documented that smaller core updates happen alongside the announced ones, so a concurrent control arm matters more than a quiet calendar.
- Decision rule: if A beats C and matches B, keep the grade as a publishing gate. If B beats A, demote the grade to a research checklist and set the target on distinctiveness. If A helps only pages that started with low grades, use the grade as a floor for under-covered pages. If nothing beats C, the constraint is not the copy.
A small site will not have the pages for a clean test. Use the grade as a floor, track the distinctiveness count, and compare cohorts of pages published under each approach over a few months, knowing the result is suggestive rather than proof.
Where I land
I would keep the tool for research and gap checks, take the grade out of every KPI, and replace it with an editor-reviewed count of what each page adds that the current top results do not. A grade that every competitor can reach cannot be the thing that separates your page from theirs.
Need help with this on your own site?
Niraj Raut works with businesses in Nepal, Australia, the UK, Europe and the US on technical SEO, ecommerce SEO, local SEO and AI search.
Add a reply Cancel reply
Replies are open to approved members. Log in or apply to join.