Discussion
  • Indexing
  • Technical SEO

When ‘Discovered - currently not indexed’ suddenly falls, is quality improving, or has Google stopped tracking those URLs? How do you tell?

Published 1 reply

Discussion starter: a technical SEO question, posted with an answer from Niraj Raut to open the thread. If you have dealt with this on a site, reply with what you saw, especially where it differs.

Disclosure: Niraj Raut, who posted this answer, runs the SEO consultancy linked at the end of it.

1 reply

  • admin
    Admin
    Original poster

    A sudden fall in “Discovered - currently not indexed” can mean several very different things, and you tell them apart by looking at where the URLs went. If the URLs moved into “Indexed” or “Crawled - currently not indexed”, Google crawled them, which is progress of some kind. If they left the report without appearing elsewhere, Google may have dropped them from the set of URLs it tracks, which happens when URLs are no longer linked or listed in sitemaps, or when a site removes or blocks them. Reporting changes and sampling can also move the numbers. The checks are straightforward: compare totals across Page indexing categories, sample affected URLs with URL Inspection, and look at logs to see whether Googlebot actually fetched them.

    Possible explanations

    Explanation What you would see
    URLs were crawled and indexed “Indexed” rises; logs show Googlebot fetching them; impressions may appear
    URLs were crawled but not indexed “Crawled - currently not indexed” rises
    Google stopped tracking them Totals fall without corresponding rises elsewhere; URLs no longer linked or in sitemaps
    Your site removed or blocked them Rises in “Not found (404)”, “Blocked by robots.txt” or “Excluded by ‘noindex’ tag”
    Reporting change or data refresh Shifts across many sites at once; no change in logs

    A diagnostic sequence

    1. Compare the Page indexing categories for the same dates. Where did the numbers go?
    2. Sample affected URLs. Export examples from before the drop and inspect a sample with URL Inspection: indexed, crawled not indexed, unknown to Google, or excluded?
    3. Check logs. Did Googlebot request those URLs around the time of the change?
    4. Check your own changes. Sitemap updates, internal linking changes, noindex or robots.txt changes, URL removals.
    5. Check performance data. Did impressions rise for the page groups involved? If so, the change is probably positive.

    Interpreting the outcome

    • Moved to indexed with impressions: quality or linking improvements likely helped; note what changed and do more of it.
    • Moved to crawled not indexed: Google looked and decided not to keep them; the pages may need more value, or may not be worth indexing.
    • Dropped from tracking: check whether you intended that. If the URLs matter, restore links and sitemap entries; if not, it is a useful clean-up.
    • Moved to excluded categories: confirm the exclusion was deliberate.

    Hypothetical example:

    A marketplace sees “Discovered - currently not indexed” fall by a third in two weeks and celebrates. The Page indexing report shows “Indexed” barely changed, while “Crawled - currently not indexed” rose by roughly the same amount. Logs confirm Googlebot crawled the URLs after a sitemap restructuring. Google looked at them and chose not to index them. The team shifts focus from getting URLs discovered to improving or removing the thin listing pages that Google is declining to keep.

    Watch for this:

    Treating a falling “Discovered” count as a win without checking where the URLs went can hide a problem, such as accidental noindex tags or a broken sitemap that caused Google to stop tracking URLs you wanted indexed.

    Why Google might stop tracking URLs

    Google’s list of known URLs is not permanent. URLs that are no longer linked internally, have been removed from sitemaps, have never been crawled after a long time, or belong to patterns that consistently produced low-value pages may drop out of the discovered set. For some sites that is healthy: parameter variants and old faceted URLs that should never have been discovered disappear. For others, it is a warning that pages the site wants indexed have lost the links or sitemap entries that kept them visible. The difference depends entirely on which URLs left, which is why sampling them matters more than the headline number.

    Sitemaps by section make this easier

    Splitting sitemaps by section or template lets you filter the Page indexing report by sitemap, which shows which sections gained or lost in each category. A drop in “Discovered” concentrated in one section, such as old listings, is easy to interpret; the same drop across the whole site is not. For large sites, this structure is worth setting up before the next unexplained change.

    Track the categories together

    Individual Page indexing categories are hard to interpret alone. Track them together over time, for each important section, ideally through separate sitemaps so the report can be filtered by section. Pair them with impressions and crawl data. The useful question is not whether a category went up or down, but whether the pages you want indexed are being indexed and receiving impressions.

    Follow the URLs: compare Page indexing categories, sample with URL Inspection, check logs and your own changes, and judge the change by whether the pages you care about moved into the index and started earning impressions.

    Need help with this on your own site?

    Niraj Raut works with businesses in Nepal, Australia, the UK, Europe and the US on technical SEO, ecommerce SEO, local SEO and AI search.

    See SEO services

Add a reply

Replies are open to approved members. Log in or apply to join.