Skip to content
SEO Community
  • Knowledge
  • Discussions
  • Experiments
  • Case Studies
  • Experts
  • Events
  • About
Log In Apply to Join
SEO Community
  • Knowledge
  • Discussions
  • Experiments
  • Case Studies
  • Experts
  • Events
  • About
Apply to Join Log In
  1. Home
  2. Knowledge
  3. Technical SEO

Search topic

Technical SEO

Practitioner discussions, experiments, case studies and insights about Technical SEO, each attributed to the member who wrote it.

  • All (73)
  • Discussions (73)
Reset
Discussion
  • Technical SEO

X-Robots-Tag headers for PDFs and images at scale: has Google respected them consistently for you, and how did you audit coverage?

Google generally respects X-Robots-Tag when it recrawls a file; most gaps come from set-up. Audit the headers each file URL returns, then check…

Niraj Raut
Niraj Raut
1 replies 2 days ago
Discussion
  • Technical SEO

Programmatic pages built on public datasets like census or government stats: still defensible when everyone uses the same data?

Programmatic pages built on public datasets are defensible only when they add something the source and other publishers do not.

Niraj Raut
Niraj Raut
1 replies 3 days ago
Discussion
  • Crawling

AI crawler requests now exceed Googlebot on some sites. When server cost is the constraint, how do you decide which bots to allow, throttle or block?

Decide bot by bot on value against cost, using verified data. Search engine crawlers are essential and should be served well.

Niraj Raut
Niraj Raut
1 replies 4 days ago
Discussion
  • Technical SEO

User-triggered Google fetchers such as NotebookLM generally ignore robots.txt. In log analysis, do you treat them as visits, as scraping, or ignore them?

Treat them as their own category: neither crawling, nor human visits, nor scraping in the usual sense.

Niraj Raut
Niraj Raut
1 replies 4 days ago
Discussion
  • Crawling

Has launching hundreds of thousands of new templated pages ever slowed crawling of your existing important pages?

It can when crawl capacity is limited and new URLs draw attention from existing pages. Launch in batches, prioritise key pages and gate on quality.

Niraj Raut
Niraj Raut
1 replies 5 days ago
Discussion
  • Indexing

Still running dynamic serving with Vary: User-Agent on a large site? Did caching or indexing problems eventually push you to a responsive build?

Dynamic serving still works, but most large sites using it went responsive, usually over caching and parity. If you stay, prioritise parity and…

Niraj Raut
Niraj Raut
1 replies 5 days ago
Discussion
  • Technical SEO

Product specs load only when a tab is clicked on a WooCommerce store. Split them into separate URLs, or does that just create thin pages?

Do not split the specifications into separate URLs. A page per product's specifications would be thin, duplicate the product's main page and compete…

Niraj Raut
Niraj Raut
1 replies 7 days ago
Discussion
  • Technical SEO

When does Google prefer an aggregator page over the original source it aggregates, based on what you’ve observed?

Google tends to prefer aggregators that compare or cover all options, and originals for entity queries or when the aggregator only repackages them.

admin
adminAdmin
1 replies November 13, 2026
Discussion
  • Indexing

When ‘Discovered - currently not indexed’ suddenly falls, is quality improving, or has Google stopped tracking those URLs? How do you tell?

A sudden fall in "Discovered - currently not indexed" can mean several very different things, and you tell them apart by looking at where the URLs…

admin
adminAdmin
1 replies November 13, 2026
Discussion
  • Crawling

On an enterprise domain with several busy subdomains on shared infrastructure, have you seen a heavy UGC subdomain reduce crawling of the commercial one?

It can, indirectly: crawling is largely per host, but heavy UGC crawling on shared servers can slow responses, so Google crawls the commercial host…

admin
adminAdmin
1 replies November 12, 2026
Discussion
  • Technical SEO

With Google Shopping and AI shopping answering price queries, where is organic traffic still growing for price comparison sites?

Growth has shifted to where Google's own features are weaker: complex service comparisons, price history, alerts, local stock, used goods and expert…

admin
adminAdmin
1 replies November 11, 2026
Discussion
  • Crawling

Calendar and date-archive crawl traps: do you block them in robots.txt, cap the range server-side, or change the links? Which held up long term?

The durable fix is changing how the calendar generates links and returning a 404 for out-of-range dates, with a robots.txt rule kept as a backstop.

admin
adminAdmin
1 replies November 11, 2026
  • 1
  • 2
  • …
  • 7
  • Next page

Related topics

  • AI Search
  • Content SEO
  • Indexing
  • eCommerce SEO
  • Local SEO
  • SEO Analytics
  • Enterprise SEO
  • Crawling
  • Entity SEO
  • Agency
  • Digital PR
  • International SEO
Browse the full knowledge hub
SEO Community

A free community for people who work in SEO and modern search.

Community

  • Knowledge
  • Discussions
  • Experiments
  • Case Studies
  • Insights
  • Experts
  • Events

About

  • About the Community
  • About Niraj Raut
  • Community Guidelines
  • Contact

Legal

  • Privacy
  • Terms
  • Cookie Policy

Account

  • Apply
  • Log In

© 2026 SEO Community. Member contributions remain the work of their authors.

Independent from, and complementary to, nirajraut.com.np.