Discussion
  • Enterprise SEO

Which automated SEO checks in CI/CD have caught real problems before release, and which just generated noise the developers ignored?

Published 1 reply

Discussion starter: an enterprise SEO question, posted with an answer from Niraj Raut to open the thread. If you have dealt with this on a site, reply with what you saw, especially where it differs.

Disclosure: Niraj Raut, who posted this answer, runs the SEO consultancy linked at the end of it.

1 reply

  • admin
    Admin
    Original poster

    The checks that catch real problems test invariants with a direct link to search outcomes: robots directives and noindex on production templates, robots.txt changes, canonical tags, status codes of key URLs, presence of main content and internal links in the server-rendered HTML, structured data that parses and carries required fields, hreflang reciprocity, and leaked staging hostnames. The checks that generate noise test preferences or scores with high variance: Lighthouse score thresholds, title length warnings, keyword rules, heading hierarchy nitpicks and generic “SEO scores”. The difference is whether a failure means something will definitely go wrong in search, or only that a tool has an opinion.

    Checks that earn their place

    Check What it catches Fail or warn
    Meta robots and X-Robots-Tag on key templates Accidental noindex from staging settings or feature flags Fail
    robots.txt diff against production Disallow rules that block important sections Fail on new disallows affecting key paths
    Canonical present, absolute, on the production host, self-referencing where expected Canonicals pointing to staging, to the homepage or to the wrong variant Fail
    Status codes for a fixed list of key URLs Broken routes, unexpected redirects, soft errors Fail
    Main content and internal links in server-rendered HTML Content or links moved into client-side rendering Fail if below a threshold
    Structured data parses and has required properties Broken JSON-LD, missing prices on products Fail for revenue templates, warn elsewhere
    hreflang reciprocity on a sample Broken language clusters after routing changes Warn or fail depending on markets
    Staging or internal hostnames in output Links, canonicals or assets pointing to non-production hosts Fail
    Redirect map for changed URLs URLs removed without redirects Fail if a URL with traffic disappears
    HTML size budget Pages growing past sensible limits Warn

    Checks that tend to become noise

    • Lighthouse or performance scores as pass/fail. Scores vary run to run and across environments. Use budgets on specific metrics measured consistently, and treat them as warnings.
    • Title and meta description length rules. Useful as editorial guidance, but not worth breaking a build over.
    • Keyword presence or density. Not a meaningful test of search readiness.
    • Heading hierarchy rules. Skipped heading levels rarely matter for search.
    • Missing alt text on every image. Important for accessibility where images are meaningful, but a blanket check flags decorative images and trains developers to write empty alt text to silence it.
    • Generic SEO audit scores. A single number that mixes critical and trivial issues cannot be acted on.

    Design principles that keep checks trusted

    1. Test representative URLs per template, including edge cases such as long pages, out-of-stock products, paginated pages and localised versions.
    2. Run against rendered output with production configuration, because many failures come from environment differences.
    3. Fail only on critical issues. Everything else should warn, with a named owner.
    4. Keep false positives near zero. One noisy check teaches developers to ignore all of them.
    5. Explain every failure in the output: what failed, why it matters, how to fix it.
    6. Run a post-deploy check on production as well, because CDN, caching and configuration can differ from staging.

    Hypothetical example:

    A marketplace adds a CI check that fetches twenty representative URLs from staging after each build and fails if any has a noindex, a canonical to a non-production host, a non-200 status, or fewer than a set number of links to product pages in the server-rendered HTML. In the first quarter it blocks three releases: one carrying a staging noindex into production, one where a new component rendered product links only after scrolling, and one where a routing change returned 404s for a category type. Earlier, a check that failed builds when Lighthouse performance dropped below a threshold had been disabled within a month because it failed randomly.

    Not every release goes through CI

    Many of the changes that hurt search never pass through a code pipeline. Editors change templates in the CMS, marketing tools inject scripts, tag managers add or remove elements, and SEO plugins change settings with a click. A check suite that runs only on code deployments misses all of those. Complement CI checks with a scheduled production monitor that runs the same critical checks against live URLs every day, or every few hours for large sites, and alerts the owner when something changes. It catches configuration changes, CMS edits and third-party script problems, and it also catches the occasional deployment that bypassed the pipeline.

    Who owns the checks

    Checks work when engineering owns the pipeline and SEO owns the rules. SEO defines what must be true, with the reason for each rule, and engineering implements the checks in the tools they already use. When a check fails, the output should point the developer to the rule’s owner. Reviewing the checks quarterly keeps them aligned with the site: templates change, and a check for a template that no longer exists is noise in waiting.

    Watch for this:

    Staging environments often carry a noindex or robots.txt block to keep them out of search. Checks must distinguish between “noindex present because this is staging” and “noindex present in the production build”. Test the build artefact with production configuration, or run the check after deployment, or the most valuable check will fail on every build.

    Automate the invariants whose failure would definitely hurt search, fail builds only on those, warn on the rest, keep false positives close to zero, and treat any check developers learn to ignore as a check to fix or remove.

    Need help with this on your own site?

    Niraj Raut works with businesses in Nepal, Australia, the UK, Europe and the US on technical SEO, ecommerce SEO, local SEO and AI search.

    See SEO services

Add a reply

Replies are open to approved members. Log in or apply to join.