usdigitalmarketingindex.com SEO
How can JavaScript faceted filters expose too many crawlable URL states?
Updated November 15, 2026
By writing every filter combination into the URL and rendering a link to every further combination. A listing with a handful of filters has thousands of possible states, and if each state has its own address and each page links to its neighbors, a crawler following links is led through all of them. Google’s documentation describes the result: faceted navigation based on URL parameters “can generate infinite URL spaces”, and because the URLs look new, crawlers “can’t determine whether the URLs are going to be useful without crawling first” (Google’s guidance on faceted navigation). The answer is to decide which few filter states deserve to be pages, expose those as real links, and keep the rest from becoming crawlable addresses.
Filter state and URL proliferation
A JavaScript filter component has to keep its state somewhere. Each choice has consequences.
| Where state is kept | URLs created | Crawl exposure |
|---|---|---|
| In memory only | None; one URL for the listing | None, and filtered views cannot be shared or bookmarked |
| In the fragment, after a hash | None that crawlers treat as separate | None; Google's guidance says it generally does not support fragments in crawling and indexing |
| In query parameters, updated with the History API | One per combination | Depends on whether links to them are rendered |
| In query parameters, with every option rendered as a link | One per combination | Full; every state is discoverable |
The fourth row is where the problem lives. Count a modest example, a shoe listing with these filters.
| Filter | Options | States including "none selected" |
|---|---|---|
| Brand | 12 | 13 |
| Size | 15 | 16 |
| Color | 10 | 11 |
| Price band | 5 | 6 |
| Sort order | 4 | 4 |
Single-selection combinations: 13 x 16 x 11 x 6 x 4 = 54,912 URLs for one category. Allow several options per filter, let parameters appear in any order, and add pagination, and the figure has no practical ceiling.
Recognizing useful landing states
A small number of filter states match how people search. Those are worth having as pages.
A filter state is a candidate landing page when:
- People search for it by name: waterproof trail shoes, a brand plus a category.
- It contains a meaningful number of products that will remain available.
- It can carry its own title, heading, and introductory content.
- It is a single filter, or at most a stable pair.
A state is not a candidate when it is a size, a price band, a sort order, a combination of three or more filters, or a state with few or no results.
| State | Candidate? | Reason |
|---|---|---|
| Category plus brand | Yes | Searched by name; stable range |
| Category plus a defining attribute, such as waterproof | Yes | A recognized subtype |
| Category plus size | No | Narrows a list; not searched as a page |
| Category plus color plus size plus price | No | A permutation |
| Any state, sorted by price | No | Same items, reordered |
| A state with zero results | No | Empty |
Restricting irrelevant combinations
Treat the two groups differently in the markup.
Candidate states get real, clean URLs and real links.
<a href="/shoes/trail-running/waterproof/">Waterproof</a>
<a href="/shoes/trail-running/brand/northgate/">Northgate</a>
These pages are self-canonical, listed in the sitemap, and given their own titles and headings.
Other states are applied without creating crawl paths.
<button type="button" data-filter="size" data-value="10">Size 10</button>
A button with a handler filters the list for the visitor and offers no href for a crawler to follow. The state can still be written to the URL with the History API, so that the view can be shared.
Rules and trade-offs for the non-candidate states:
| Measure | Effect | Trade-off |
|---|---|---|
| Render filter controls as buttons, not links | No crawl paths are created | The views are not discoverable, which is the intent |
| Keep state in the fragment | Crawlers treat every state as the base URL | Cannot become a landing page later without changing the scheme |
| Disallow the filter parameters in robots.txt | Stops requests for those URLs | Blocked URLs cannot pass a canonical or noindex signal; they may still be listed if linked from elsewhere |
| Canonical tag from filtered URLs to the unfiltered listing | Consolidates signals where the URLs do get crawled | The URLs are still crawled; a hint only |
| Return 404 for combinations with no results | Removes empty states | Needs server-side handling |
| Fix parameter order and reject unknown parameters | Prevents duplicates of the same state | Engineering effort |
Google’s guidance offers the first of the blocking options plainly: “Use robots.txt to disallow crawling of faceted navigation URLs”, observing that there is frequently no good reason to allow crawling of filtered items. The cleanest approach, though, is not to render the links at all, so that there is nothing to block.
Whatever the mechanism, be consistent. Internal links, canonical tags, and the sitemap should all point at the same small set of URLs.
QA of selected facets
Verify both halves: the chosen states are reachable, and the rest are not multiplying.
| Check | How | Pass condition |
|---|---|---|
| Candidate links | Rendered HTML of the category page | Each candidate state is an anchor with a clean path |
| Non-candidate controls | Rendered HTML | Buttons or controls without href |
| Crawl size | Crawl the category with rendering enabled | URL count equals the category, its pagination, and the candidate states |
| Canonicals | Inspect a filtered URL | Points to itself for candidates; to the base listing for others |
| Empty states | Request a combination with no results | Not-found status, or noindex |
| Parameter order | Request the same state with parameters reordered | Redirects to one form, or canonicalizes to it |
| Server logs | Verified Googlebot requests to the category | Requests concentrate on base, pagination, and candidate URLs |
| Search Console | Page indexing report filtered to the category path | No large count of filter URLs in duplicate or not-indexed states |
| Visitors | Use the filters in a browser | Filtering, sharing a filtered view, and the back button all work |
The crawl size check is the decisive one. If a crawl of one category returns tens of thousands of URLs, the filters are exposed.
Sort orders are a particular kind of non-candidate state, since they change nothing but sequence. Their handling is covered in the guide to how sorting parameters produce duplicate ecommerce category URLs.
The rule: a few filter states are pages and get links. The rest are controls and get none.
Add your comment to this answer