Discussion starter: an international SEO question, posted with an answer from Niraj Raut to open the thread. If you have dealt with this on a site, reply with what you saw, especially where it differs.
- International SEO
At several million URLs, which hreflang method has been easiest to keep accurate: XML sitemaps, HTML head, or HTTP headers?
Disclosure: Niraj Raut, who posted this answer, runs the SEO consultancy linked at the end of it.
1 reply
-
adminAdmin
At several million URLs, I would default to XML sitemaps, on one condition: the sitemaps are generated from the same equivalence data your platform uses to build pages, and the whole output is validated before each publish. The method itself does not change how Google treats the annotations. Google’s documentation says the three methods “are equivalent from Google’s perspective and you can choose the method that’s the most convenient for your site”. So the real question is operational: which method lets you catch and fix errors before they spread.
There are two main exceptions. On an estate where every locale runs on one platform with one database, and pages are purged from cache quickly when a cluster changes, HTML head annotations are a sound choice and easier to see, because they show up in page source and in any crawl. And on templates where URLs appear and expire within hours, head tags reflect the current state better than a sitemap job can, unless you regenerate those files just as often. Google describes HTTP headers as useful for non-HTML files such as PDFs; as the main method for millions of HTML pages, they are the hardest of the three to audit.
Accuracy is a data problem before it is a markup problem
All three methods publish the same information: for each URL, the equivalent URLs in other languages or regions. At this scale that information changes daily. Products are withdrawn in one market but not another, localised slugs get renamed, pages redirect, canonicals move and new locales launch. Whichever method you pick, the annotations are only as accurate as the mapping behind them.
Google’s rules make small mapping errors expensive. Each language version “must list itself as well as all other language versions”, and return links are required: “If two pages don’t both point to each other, the tags will be ignored.” In a 20-locale cluster, one URL that starts redirecting or returning 404 can break all 19 pairs it belongs to. Nothing in Search Console will flag it either, because the hreflang error reporting went away when the International Targeting report was retired in 2022.
A useful way to picture it:
hreflang behaves more like a set of mutual introductions than a label. Each page introduces its siblings, and Google only accepts an introduction when both sides confirm it. Technically, Google pairs URLs whose annotations are reciprocal and ignores pairs where one side is missing, so a mapping error removes pairs quietly instead of raising a visible error.
How each method drifts at several million URLs
Aspect XML sitemaps HTML head HTTP headers Where the annotations live Separate files written by a batch job Every HTML response, built by the page template Every response, set by the application, server or CDN rules What must stay true The job reads current mapping data and its output publishes successfully Every template, on every platform, can look up the full cluster at render time, and caches are purged when a cluster changes The same as HTML head, plus the header passes through every proxy and CDN layer intact How it drifts The job fails or reads stale data and nobody notices; entries keep pointing at URLs that now redirect, return 404 or carry noindex Cached pages keep old alternates; locale sites on different release schedules disagree about the cluster; a tracking snippet in the head template closes the head early Rules maintained outside the CMS fall out of step with it; long headers run into header-size limits in proxies or CDNs How you see errors Only by crawling the sitemap entries and fetching every alternate In page source, rendered HTML and any standard crawl Only in response headers, which page-source checks do not show The head-closing risk is easy to miss on large sites with many tracking snippets. HTML parsers end the head when they meet an element that is not allowed there, such as a tracking image or an iframe snippet pasted into the head template, and link elements after that point end up in the body. Google’s HTML method puts the annotations in the head, so a head that closes early can strand them on some templates and not others. Check the rendered HTML, not the template source.
Why sitemaps hold up better on multi-platform estates
Four properties make sitemaps easier to keep accurate at this size:
- Reciprocity by construction. If each cluster is stored once, keyed on a stable ID such as a SKU or content ID, and the generator writes one
<url>entry per member listing the same full set, return links cannot go missing unless the cluster record itself is wrong. With head tags, each page computes its own view of the cluster at render time, sometimes from different data on different platforms, and that is where one-way pairs come from. - Validation before publishing. You can check every cluster and every alternate URL before the files go live, and keep the last good version if a run fails. With head tags, validation happens after the fact by crawling, and at several million URLs any crawl is a sample.
- Cross-platform joins happen in one place. A central job can join catalogues from different platforms or domains. Head tags need each platform to query the others while rendering pages.
- Ownership without template releases. A data or SEO team can fix mapping bugs without waiting for a front-end release on every locale.
The file limits need planning. Google limits each sitemap to “50MB (uncompressed) or 50,000 URLs”, and each
<url>entry must list every alternate version, including itself.Hypothetical arithmetic:
With 25 locales and about 120 bytes per
xhtml:linkline, each entry is roughly 3KB, so a file reaches the 50MB limit at around 16,000 entries, well before 50,000 URLs. Four million URLs would need about 250 files under sitemap index files. Cap files by bytes rather than URL count, and split them by locale and template so a failed segment is easy to isolate and regenerate.Where sitemaps are weaker
- Lag on high-churn templates. If listings expire hourly and the generator runs nightly, entries can point at dead URLs for up to a day. Regenerate volatile segments more often, or put those templates on head tags and keep sitemaps for the stable ones. Either way, give each URL one source of annotations.
- Nobody sees them by accident. Errors in head tags turn up in routine crawls. Errors in sitemaps only turn up if your crawler reads sitemap annotations and fetches every alternate, so that check has to be scheduled, not assumed.
- Google has to fetch the files. Google describes a sitemap as “merely a hint: it doesn’t guarantee that Google will download the sitemap or use the sitemap for crawling URLs on the site.” Watch the “Last read” dates in the Sitemaps report for the individual files, not only the index.
- Nightly regeneration can damage lastmod. If the job stamps every URL with the run time, lastmod stops meaning anything. Google uses lastmod “if it’s consistently and verifiably (for example by comparing to the last modification of the page) accurate”, so take it from the page’s content history, not from the file’s build time.
On mixing methods, Google’s page says: “While you can use all three methods at the same time, there’s no benefit in Search”, and adds that three implementations can be much harder to manage than one. It does not say how Google resolves annotations that disagree between methods. If you migrate from head tags to sitemaps, generate both from the same table during the switch, compare them on a sample, then remove the head tags.
The checks that matter more than the method
Run these on the full generated output before publishing (sitemaps), or on a large stratified crawl sample (head tags and headers):
- Every alternate returns a 200 directly, with no redirect, 404 or 410.
- Every alternate is indexable and is its own canonical. An annotation pointing at a URL that canonicalises elsewhere sends two conflicting messages.
- Every pair is reciprocal, and every entry lists itself.
- Codes are a valid language code with an optional region code, such as
en-GB.en-UKis invalid because the ISO 3166-1 code for the United Kingdom is GB. - Each hreflang value appears once per cluster, and each URL belongs to one cluster only.
x-default, if used, points to the same URL from every member of the cluster.- URLs are fully qualified, including
https://, as Google requires. - Missing equivalents are omitted, not substituted. If a product does not exist in France, leave fr-FR out rather than pointing it at a category or the home page.
Hypothetical example:
A retailer runs en-GB, en-IE and de-DE on one commerce platform and fr-FR, es-ES and it-IT on another after an acquisition. Head tags call a shared mapping API during rendering, and HTML is cached at the CDN for 24 hours. When 40,000 products are withdrawn from France, pages in the other five locales keep listing French URLs until their cache entries expire, and those French URLs now redirect to categories, so the pairs fail Google’s return-link rule. Moving to sitemaps generated nightly from a SKU-keyed equivalence table removes the cross-platform call, and the pre-publish check catches the redirects before the files go live. The cost: the team now needs a sitemap-aware crawler to see the annotations at all.
How to monitor accuracy when Search Console will not
Measure the process and the outcome separately.
Signal Where it comes from What a bad trend suggests Share of clusters passing validation on each run Your generator or crawl A mapping or data-feed problem, which you can trace by locale and template Wrong-market share: clicks and impressions in a country going to URLs meant for another locale Performance report filtered by country, or the API or bulk export at this scale Broken clusters, canonical folding, or locales too similar for Google to treat as distinct Google-selected canonical for a sample of each locale pair URL Inspection Near-identical regional pages clustered as duplicates, which can look like an hreflang failure in the Performance report Last read date per sitemap file Sitemaps report Files Google has stopped fetching or cannot parse Wrong-market share is the outcome metric I would put in front of stakeholders, because it measures what hreflang is for: the right URL in the right country. Build a baseline per market first. Markets served by near-identical English pages, such as the UK, Ireland and Australia, can carry some cross-market traffic even with correct annotations, so judge the trend rather than the absolute number.
A decision rule for your setup
Condition Lean towards XML sitemaps Lean towards HTML head Platforms Locales on more than one platform, domain group or team One platform and one database for all locales URL churn Catalogue changes daily or slower Listings appear and expire within hours, unless sitemap segments can be regenerated that often Caching Long HTML cache lifetimes without purging on catalogue changes Pages purged promptly when a cluster changes Ownership An SEO or data team can own a generation pipeline The front-end team owns templates and releases often QA tooling A crawler that reads sitemap annotations and fetches every alternate Only standard page crawls available Page weight should not drive the decision. Googlebot fetches up to 2MB of an HTML file, and even 40 link elements add only a few kilobytes. PDFs and other non-HTML files cannot carry head tags, so annotate them with HTTP headers, or in the sitemaps if that is your method for pages.
Where I would start
If I inherited this site, I would not begin by choosing a method. I would build the equivalence table on stable IDs, write the validator, and baseline wrong-market share for each country. Then I would pick the method whose failures that validator can catch before they reach Google: sitemaps on a multi-platform estate, head tags on a single platform with fast cache purging. Either way, keep one source of annotations per URL, and treat a falling validation pass rate as an incident rather than a reporting curiosity.
Need help with this on your own site?
Niraj Raut works with businesses in Nepal, Australia, the UK, Europe and the US on technical SEO, ecommerce SEO, local SEO and AI search.
- Reciprocity by construction. If each cluster is stored once, keyed on a stable ID such as a SKU or content ID, and the generator writes one
Add a reply Cancel reply
Replies are open to approved members. Log in or apply to join.