Discussion starter: an enterprise SEO question, posted with an answer from Niraj Raut to open the thread. If you have dealt with this on a site, reply with what you saw, especially where it differs.
- Enterprise SEO
- Internal Linking
Merchandising and recommendation engines change internal links daily. How do you keep linking stable enough to measure anything?
Disclosure: Niraj Raut, who posted this answer, runs the SEO consultancy linked at the end of it.
1 reply
-
adminAdmin
Separate the internal links into two layers and manage them differently. The stable layer, meaning navigation, breadcrumbs, category and subcategory links, curated editorial links and “related categories” modules, carries the structure you want search engines to learn, and it should change only through deliberate decisions. The dynamic layer, meaning recommendations, trending products and personalised modules, can change as often as merchandising needs, as long as important pages never depend on it for their links. That split lets you measure the effect of changes to the stable layer, while recommendation engines keep doing their commercial job. What you should not do is serve search engines a different set of links from users to make things stable; that is cloaking.
Why daily-changing links make measurement hard
- Crawlers see different links on each visit, so the internal link signals pointing at any product fluctuate.
- Tests become noisy. If you change a category page’s linking to test an effect, recommendation modules are changing hundreds of other links at the same time.
- Important pages may lose links when an algorithm stops recommending them, with no one noticing.
The two-layer model
Layer Examples How it changes Purpose Stable Navigation, breadcrumbs, category listings, curated guides, related categories Deliberately, through review Structure and priority for search and users Dynamic “Customers also bought”, trending, personalised picks Continuously, by algorithm Conversion and discovery for users Implementation choices
- Make sure every important page has stable links. Products should be reachable through category listings and breadcrumbs regardless of what recommendation modules show.
- Consider how dynamic modules render. Server-rendered recommendation modules are crawlable and add changing links; client-rendered ones that load after interaction are mostly invisible to crawlers. Either is acceptable, provided users and crawlers get the same treatment and the stable layer carries the important links.
- Reduce churn for anonymous users where it makes sense. Recommendations recalculated daily for all anonymous visitors, rather than per request, are more stable for everyone, crawlers included, without treating crawlers differently.
- Keep personalisation for identified users. Logged-in recommendations can be as personalised as you like; they do not affect what anonymous visitors, including crawlers, see.
Measuring the link graph over time
Take regular crawl snapshots, weekly or monthly, and record internal link counts and link sources for key pages. That shows whether important pages are losing links over time and separates stable-layer changes from dynamic-layer noise. When testing a linking change, apply it to the stable layer on a set of pages and compare with a matched set, using the snapshots to confirm the dynamic layer did not change differently between groups.
Hypothetical example:
A homeware retailer’s product pages are linked mainly through a server-rendered “you may also like” module that changes on every request. Crawl snapshots show some products appearing in hundreds of internal links one week and a handful the next; new products with no sales history rarely appear at all. The retailer ensures every product appears in its category listings with sensible pagination, adds a stable “related categories” module, and changes recommendations for anonymous visitors to update daily. Internal link counts for products stabilise, new products get crawled faster through categories, and a later test of category page linking produces a clear result instead of noise.
Watch for this:
Detecting search engine crawlers and showing them a fixed set of links while users see recommendations is a form of cloaking. Stability should come from the design of the page, which applies to everyone, not from treating crawlers differently.
Pagination and category listings carry more weight than they seem
When recommendation modules are unstable, category listings become the main reliable path to products. That makes pagination and sorting decisions important. If a category’s default sort changes daily by popularity, products move between listing pages and their link paths shift. Using a stable default order for crawlable category pages, with other sorts available as user choices that do not create new indexable URLs, gives each product a predictable place. Paginated listing pages should be crawlable with ordinary links, so products deep in large categories are still reachable.
Give merchandising teams a way in
Merchandisers need to promote products, and they will use whatever levers exist. Give them curated slots within the stable layer, such as featured products on category pages or links in seasonal guides, with a simple process and clear expectations about how long they stay. That channels promotional linking into places that are both commercially useful and measurable, rather than into constant changes to navigation.
Keep a stable, deliberately managed link layer that gives every important page its links, let recommendations change freely on top of it, treat crawlers and users identically, and use regular crawl snapshots to measure the link graph and run clean tests.
Need help with this on your own site?
Niraj Raut works with businesses in Nepal, Australia, the UK, Europe and the US on technical SEO, ecommerce SEO, local SEO and AI search.
Add a reply Cancel reply
Replies are open to approved members. Log in or apply to join.