Discussion
  • Indexing
  • News SEO

News tag pages: prune to a curated set, noindex them all, or keep them as topic hubs? What did your traffic data show?

Published 1 reply

Discussion starter: a publisher and community SEO question, posted with an answer from Niraj Raut to open the thread. If you have dealt with this on a site, reply with what you saw, especially where it differs.

Disclosure: Niraj Raut, who posted this answer, runs the SEO consultancy linked at the end of it.

1 reply

  • admin
    Admin
    Original poster

    Prune to a curated set, then run the survivors as real topic pages. Noindexing every tag is the blunt version of the same clean-up: it takes the thin pages out of the index, but it also takes out the tag pages that earn search clicks, and it does little for crawl efficiency, because Googlebot has to keep fetching a page to see its noindex. Keeping every tag as a hub only works where tag creation is already controlled. If any editor can create a tag by typing a new word, the tag layer grows faster than anyone can maintain it.

    The qualification is that “prune” covers three different actions: merging duplicates, removing dead tags, and noindexing tags that help readers but not searchers. Which action fits a given tag comes down to what that page does today: earn search clicks, carry external links, help readers move between stories, or nothing at all.

    On the traffic question: public before-and-after data on news tag pages is thin, and a published clean-up that changed templates, internal links and tags at the same time cannot isolate the tag effect. The useful answer is what your data should show before you act, what each action can realistically change, and how to stage the work so your own numbers answer the question.

    The four jobs a tag page does on a news site

    A single tag layer can mix all four of these, which is why blanket rules go wrong:

    • Landing page for an ongoing topic. Queries such as “[club] transfer news”, “[politician] latest” or “interest rates news” want a fresh list of coverage, not one article. A maintained tag page fits that intent, while any single article ages out of it within days.
    • Crawl path to older stories. Once a story drops off the homepage and section fronts, tag pages and date archives may be the only internal links still pointing at it.
    • Reader navigation. “More on this story” links, follow buttons and topic alerts hang off tags. This job has nothing to do with indexing.
    • CMS by-product. Typos, synonyms (“RBA”, “Reserve Bank”, “Reserve Bank of Australia”), one-off tags for a single story, and tags nobody has used in years.

    Fresh stories are not where the risk sits. Googlebot is likely to find new articles through faster routes than a long-tail tag page: the homepage, section fronts, a news sitemap and feeds. The discovery risk from pruning sits in the archive, where tags may be the last internal links to thousands of older articles.

    The three options side by side

    Option What it changes Where it fits Main risk
    Noindex every tag All tag URLs leave the index; readers still use them; Googlebot still requests them Tags earn negligible search clicks and a separate, maintained topic system owns the head queries Loses the tags that do earn clicks, weakens tag pages as link paths to the archive over time, and saves no crawl
    Prune to a curated set Merges duplicates, removes dead tags, noindexes reader-only tags, invests in the rest Tag layers with sprawl around a core of tags that earn clicks Needs governance; a one-off clean-up regrows without rules on tag creation
    Keep every tag as a hub Nothing changes in the index Tag creation is already controlled and nearly every tag has steady coverage Thousands of thin, overlapping pages competing with section fronts and with each other

    Noindex is not a crawl control:

    Googlebot has to fetch a page to see its noindex, so noindexing 40,000 tags does not stop those URLs being requested. If crawl waste is the concern, the levers are fewer tag URLs and fewer links pointing at them: merge, delete and stop creating new ones. A robots.txt block stops the crawling, but it also hides any noindex on those pages, and blocked URLs that are still linked can be indexed without their content.

    There is a second, slower cost. In 2017 John Mueller said that in the long run “noindex, follow” ends up essentially the same as “noindex, nofollow”, and that a page left on noindex long enough is removed completely. Applied to a tag layer, a blanket noindex can gradually stop working as a link path into the archive, which is the one job the long-tail tags were doing well.

    Crawl efficiency is a fair concern here. Google’s crawl budget guide is written for large sites (1 million+ pages) and for medium sites with 10,000+ pages whose content changes daily, which describes a daily news site with a deep archive.

    The data to pull for every tag

    1. Search Console clicks, impressions and top queries across the full 16 months available, so one-off news spikes and seasonal topics are visible.
    2. Article count and the date of the most recent tagged story.
    3. Referring domains pointing at the tag URL.
    4. Reader use: pageviews arriving from internal links, clicks from the tag page into articles, and follows or alert sign-ups where the site has them.
    5. Googlebot requests from server logs, split by URL pattern. The Crawl stats report gives host-level totals and sample URLs, not a tag-versus-article split.
    6. Query overlap with section fronts and evergreen explainers, to spot tags competing with a page you would rather rank.

    Then run each tag through the same checks in order:

    Duplicates or overlaps a stronger tag? Yes: 301 it to that tag and retag the articles, so no new links point at the old URL.
    ↓
    Earns clicks for topic queries, with coverage in the last year? Yes: keep it indexable and build it into a hub.
    ↓
    Some impressions and above your minimum story count? Yes: keep it indexable with no extra investment, and review it yearly.
    ↓
    Readers use it to navigate or follow the topic? Yes: keep it live with noindex.
    ↓
    None of the above: remove it (404 or 410), or 301 it where a clear parent hub exists, which is worth doing for tags with external links. Strip the tag links from its articles either way.

    Set the minimum story count from your own distribution rather than a borrowed rule. A tag with 40 stories and no clicks is a different problem from a tag with four stories and steady impressions.

    Hypothetical example:

    A regional publisher has 70,000 articles and 38,000 tags. Search Console shows about 1,100 tag URLs with any clicks in 16 months, and a few hundred of those account for most of the tag clicks. Around 26,000 tags hold one or two stories, and logs show Googlebot requesting tag URLs almost as often as articles. Here the dead-weight group is large and cheap to remove, the duplicates are worth merging carefully, and the few hundred earners are where editorial time should go. A blanket noindex would throw away the earners to solve a problem that removal solves better.

    What a curated hub needs that a tag archive does not

    • A short, editor-written introduction that says what the topic covers and is revised when the story moves. Keyword filler does not count.
    • Pinned “start here” items (an explainer, a timeline, the key investigation) above the reverse-chronological list.
    • A title and heading that match how people search the topic, not the internal tag label.
    • Links in from articles, section fronts and related hubs, so the hub is not reached only through article footers.
    • Crawlable pagination with plain links to older pages, and each paginated URL canonical to itself rather than to page one, so the archive behind the hub stays reachable.
    • A named owner (a desk or an editor), because an unmaintained hub decays back into an archive list.

    There is a plausible but unproven link to how Google judges topical depth. Google said its February 2026 Discover core update, which started with US English, aims to show more in-depth, original content from sites with topic-by-topic expertise. Hub pages do not create that expertise; the reporting does. A maintained hub makes the depth of coverage easier for readers to see, but no public evidence shows the hub pages themselves feeding that assessment.

    Governance is what stops the regrowth: a controlled vocabulary, auto-suggest of existing tags in the CMS, a synonym list, desk approval for new tags, and a quarterly look at what was created. The indexing rule can be automated too: keep a tag on noindex until it passes the story threshold. If you do that, watch for tags flipping back and forth around the threshold.

    What each action can realistically change in your traffic data

    • Removing dead weight: these pages had no clicks, so there is nothing direct to lose. Any sitewide gain afterwards is hard to attribute, because core updates and other releases land in the same window. The business case is crawl focus and a tag layer editors can maintain, not a promised traffic lift.
    • Merging duplicates: compare the combined clicks and impressions of each merged pair before and after. If the survivor holds or beats the pair’s total, the merge worked.
    • Investing in hubs: track hub clicks for topic queries and recirculation from hub to article, which can be where the commercial value sits (page depth, newsletter and follow sign-ups).
    • Noindexing tags that had clicks: expect to lose most of those clicks. Articles may pick up some of the queries, but a single article is a poor match for the “latest on X” intent the tag page served.

    Stage it so your own data answers the question

    1. Split: divide the dead-weight and long-tail groups by a rule unrelated to performance, such as odd and even tag IDs. Treat one half and hold the other.
    2. Primary metric: organic clicks to the archive articles attached to treated tags, compared with articles attached to control tags.
    3. Secondary metrics: Googlebot requests to those articles in logs, the share of crawl going to tag URLs, and any rise in “Crawled - currently not indexed” or “Discovered - currently not indexed” among archive articles. Filter the Page indexing report by a sitemap that contains those articles, or sample them with URL Inspection.
    4. Duration: Googlebot revisits a busy news site often, so crawl patterns shift within days to a couple of weeks. Archive clicks are noisier and swing with the news cycle, so compare treated against control over the same weeks and run for at least four to six weeks. If a core update lands mid-test, extend it.
    5. Decision rule: if treated archive articles lose clicks or crawl relative to control, the tags were doing link work, so add related-story modules or archive links before pruning further. If there is no difference, roll the change out to the control half.

    Watch for this:

    Deleting tags without cleaning up can leave thousands of internal links pointing at 404s or redirects. Remove the tag links from articles, drop pruned tags from XML sitemaps, keep tag pages out of the news sitemap (it is for recent articles), and avoid mass redirects of unrelated tags to the homepage, which risk being treated as soft 404s.

    A practical position

    If this were my site, I would not ask “noindex or keep” of the tag layer as a whole. The first deliverable would be one spreadsheet: a row per tag with clicks, queries, referring domains, reader use, story count, last-used date and the outcome of the checks above. That sheet turns one sitewide argument into thousands of small, defensible decisions, and it doubles as the baseline for the staged test. Then I would lock tag creation behind a controlled vocabulary, because without that rule the same clean-up comes back.

    Need help with this on your own site?

    Niraj Raut works with businesses in Nepal, Australia, the UK, Europe and the US on technical SEO, ecommerce SEO, local SEO and AI search.

    See SEO services

Add a reply

Replies are open to approved members. Log in or apply to join.