Key takeaways
- At scale, templates matter more than individual pages. Fix a template once and thousands of URLs improve.
- Control filter URLs so crawlers spend their time on products and categories that sell.
- Prioritize categories by margin, demand and how close they are to page one.
- Handle out-of-stock and discontinued products with clear rules, not case by case.
What makes SEO different when a store has millions of products?
With millions of products, you cannot optimize page by page. The work moves to templates, rules and priorities: how URLs are generated, which pages crawlers should reach, how categories link to products, and which sections deserve human attention first. One wrong setting in a filter or canonical tag can create or hide hundreds of thousands of URLs overnight.
Google's e-commerce SEO best practices cover the basics for any store. This guide is about what changes when the catalog is huge, the kind of work I describe on the e-commerce SEO service page. The general method behind it is on the SEO consultant home page.
How do you stop faceted navigation from wasting crawl budget?
Decide which filters deserve their own indexable page, usually those with real search demand like brand plus category, and block or consolidate the rest. Google suggests several options: disallowing filter parameters in robots.txt, using URL fragments, pointing filtered pages to the unfiltered version with canonicals, or adding nofollow to filter links.
| Option | What it does | When to use it |
|---|---|---|
| robots.txt disallow | Stops crawling of filter URLs | Filters with no search demand, at very large scale |
| URL fragments (#) | Filters change the page without creating new crawlable URLs | When you can change how the filters are built |
| rel="canonical" to the unfiltered page | Consolidates signals; crawling of duplicates drops over time | Filtered pages that are near duplicates of the category |
| Indexable filter pages | A clean, static URL with its own title and copy | Filters people actually search for, like brand plus category |
The source for these options is Google's guide on managing crawling of faceted navigation URLs, which also recommends returning a 404 when a filter combination has no results. On one large store, filter URLs can outnumber real products many times over, so this is usually the first fix.
How should you handle product variants and duplicate products?
Pick one canonical URL for each product and make every duplicate point to it: variant parameters, tracking URLs, the same item listed in several categories and printer-friendly versions. Index separate variant pages only when people search for them specifically, such as a distinct size or part number, and give those pages their own content.
Google explains the signals it uses in how to specify a canonical URL: canonical tags, redirects, sitemaps and internal links should all point to the same version. In large catalogs the most common problem is not a missing tag but contradicting signals, for example a canonical pointing to one URL while menus, sitemaps and breadcrumbs link to another.
Which categories should you work on first?
Start with categories where three things meet: good margin, real search demand and pages already close to page one. Improving a category on page two for a profitable product line usually pays back faster than building a new section from zero. Rank your categories with data from Search Console, analytics and your margin reports.
| Category profile | Priority | Typical action |
|---|---|---|
| High margin, high demand, positions 5 to 20 | First | Unique copy, better filters, internal links, FAQ |
| High margin, low visibility, low competition | Second | Build or restructure the category page |
| Low margin, high demand | Third | Template improvements only |
| Low margin, low demand | Last | Keep healthy, no custom work |
Can product and category copy be templated?
Yes, when the template pulls real, useful data: specifications, compatibility, dimensions, materials and stock. That works for the long tail of products. Your top categories and best sellers deserve unique copy written by a person who understands the buyer. What does not work is mass-produced text that adds nothing, which Google can treat as spam.
Google's spam policies describe scaled content abuse: many pages generated mainly to rank, with little value for users. The line is usefulness. A template that shows which models a part fits is helpful; a paragraph that repeats the product name five times is not.
What should you do with out-of-stock and discontinued products?
Keep temporarily out-of-stock products live, show the status clearly and update availability in your product markup. For discontinued products, redirect to a direct replacement when one exists. When there is no replacement, return a 404 or 410, unless the page still helps people, for example with specifications, manuals or compatible parts.
| Situation | Recommended handling |
|---|---|
| Out of stock, returning soon | Keep the page (200), show the status, offer alerts and alternatives, set availability to OutOfStock |
| Discontinued, direct replacement exists | 301 redirect to the replacement |
| Discontinued, still searched for specs or parts | Keep the page, mark it discontinued, link to compatible products |
| Discontinued, no value left | 404 or 410, and remove it from sitemaps and internal links |
Google's product structured data documentation lists the availability values you can use in the offer, so search results show the right status. Whatever rules you choose, write them down so the whole team applies them the same way.
How do you build internal links across millions of pages?
Build internal links with rules, not by hand: clear category trees, breadcrumbs on every product, links from categories to their best sellers, related-product blocks based on real relationships like compatibility, and links from guides to the categories they discuss. Every important page should be reachable in a few clicks from the home page.
Orphan products, pages that no category or menu links to, are common in large catalogs after imports and migrations. Compare the URLs in your XML sitemaps with the URLs your crawler finds through links; the gap is your orphan list. Remember that each sitemap file has size limits, so large stores split them by section, as explained in Google's sitemap overview.
What should you monitor on a large catalog every week?
Watch indexing and crawling first: the page indexing report in Search Console, crawl stats, sitemap coverage by section, and spikes in 404s or redirects. Then track business numbers: organic revenue and conversions by category. Set alerts for sudden drops, because on a large site one bad release can remove thousands of pages from the index in days.
- Indexed vs submitted URLs, per sitemap section.
- New 404s, redirect chains and server errors after each release.
- Crawl stats: total requests and response times.
- Organic clicks, revenue and conversions for the top 20 categories.
- Product rich result errors and warnings.
- Share of crawled URLs that are filter or parameter URLs.
How I run this with a team
I supervise SEO for 18 US online stores with more than 3 million products in total, leading a team of three SEO specialists and one assistant. What makes it work is structure: one prioritized backlog per store, written rules for filters, variants and discontinued products, template fixes before page fixes, and a weekly review of the numbers above. For store owners, the online store SEO page explains how I work with smaller catalogs too, and the Shopify SEO checklist covers platform specifics. If your catalog is large and your crawl or indexing looks off, start with technical SEO or ask for a free diagnosis.
Frequently asked questions
What is crawl budget?
It is the number of URLs a search engine is able and willing to crawl on your site in a given time. It matters mainly for large sites with many generated URLs.
Should I noindex my filter pages?
Noindex keeps pages out of results but crawlers still request them. For very large stores, blocking or consolidating filter URLs saves more crawling.
Is AI-generated product copy allowed?
Google judges content by its value, not how it was produced. Mass-produced text with little value can break its spam policies, however it was written.
How often should a large store be audited?
Run a full technical audit at least once a year and after every migration or major release, with weekly monitoring in between.
Can one consultant handle a store with millions of products?
A consultant can set the strategy, rules and priorities, but execution at that scale usually needs a team or developers working alongside.
Sources
- Google Search Central, Managing crawling of faceted navigation URLs
- Google Search Central, How to specify a canonical URL
- Google Search Central, Product structured data
- Google Search Central, Spam policies for Google web search
- Google Search Central, What is a sitemap
- Google Search Central, E-commerce SEO best practices