Skip to the indexed answers
Digital Marketing Index

usdigitalmarketingindex.com SEO

Why should internal site-search result pages generally be kept out of Google’s index?

Because a site’s search function can produce a page for any string anyone types, and a page made that way has nothing of its own for a searcher on Google to land on. Each is a list assembled by a query. It can duplicate a category page, come back empty, or display whatever text was typed into the address. The set of possible URLs has no limit. Keeping them out of search prevents a large, low-value surface from being crawled and listed. The word generally matters: a page that began as a search and has been turned into a curated, stable landing page is a different thing, and can deserve indexing.

Types of site-search URL

Internal search produces several kinds of address.

Type Example pattern Character
Free-text query /search?q=blue+widgets One URL per string typed; unlimited
Empty result /search?q=asdfgh A page saying nothing was found, returned with a 200 status
Query plus filters and sort /search?q=widgets&sort=price&page=3 Multiplies each query many times
Query matching a category /search?q=drills Duplicates the drills category page
Query text echoed on the page /search?q=any+words+at+all The page displays whatever was typed

The last type is the reason this is more than a tidiness question. If a site shows the typed query on a page that can be indexed, anyone can create pages on that site containing words of their choosing, simply by linking to a search URL.

Quality and spam risks

Three problems follow from leaving search results open.

  • Unlimited URLs. A crawler that finds links to search URLs can be led into a space with no end. Google’s documentation on faceted navigation describes the same mechanism for filters: parameter-based URLs “can generate infinite URL spaces” (Google’s guidance on faceted navigation).
  • Pages with nothing of their own. A results list is a rearrangement of content that lives on other pages.
  • Abuse. Third parties can generate search URLs containing spam phrases and link to them, so that the site appears to host that text.

Google’s guidance for ecommerce sites also notes a related fact about crawling: “Googlebot generally doesn’t try to submit searches into a search box as part of crawling a site” (Google’s guidance on ecommerce site structure). Search result URLs are found through links, on the site or elsewhere. Internal links to search results are therefore the first thing to look for.

Blocking versus noindex tradeoffs

Two mechanisms are available, and they do different things.

Mechanism Effect Tradeoff
Disallow the search path in robots.txt Crawlers do not request search URLs A blocked URL that is linked from elsewhere can still be listed by its address; Google cannot see a noindex on a page it may not fetch
noindex on search result pages Pages can be crawled and are kept out of results Crawl requests are still spent on them

Choose by the situation.

  • If search URLs are not yet indexed and the concern is crawl volume, disallow the path.
  • If search URLs are already indexed, apply noindex and leave them crawlable until they drop out. Blocking first would stop Google from seeing the noindex.
  • Do not apply both at once to pages you want removed.

Whichever is chosen, also return a proper not-found status for searches with no results where the platform allows it, and escape or omit the echoed query text so the page cannot be used to display arbitrary strings.

Maintaining navigation usefulness

Keeping search results out of Google does not mean hiding the search box from visitors. It means not relying on search URLs as part of the site’s structure.

  • Link to categories, hubs, and products in navigation and body content, not to search URLs.
  • If a campaign or menu item currently points to a search URL, replace it with a real page.
  • Use the internal search log as research. Queries that visitors type repeatedly show demand for a page that may not exist.

That last point is where the exception arises. The decision table separates a raw search result from a curated landing page.

Property Raw search result page Curated landing page
How it is created By any query typed By an editor's decision
URL Parameter with the query string A stable, readable path
Title and heading The query echoed Written
Introductory content None Written for the subject
Items shown Whatever matches at the moment Chosen, ordered, and maintained
Number of possible pages Unlimited Fixed and listed
Canonical Unclear, or self-referencing per query Points to itself
Linked from navigation No Yes
Outcome Keep out of the index May be indexed

A site that sees repeated searches for waterproof hiking boots, and has no such category, should build a category or landing page for it with its own URL and content. That page is then a category in every sense, even if a saved search supplies its items. What should not happen is opening the search path as a whole because a few queries are valuable.

Check the result.

  1. Search Google for the site’s search path to see whether result URLs are listed.
  2. Crawl the site and list internal links that point to search URLs.
  3. Review Search Console’s page indexing report for search URLs in any category.
  4. Request a search URL with a nonsense query and confirm what status it returns and what text it displays.

What makes a category page worth a searcher’s visit, as opposed to a bare list, is covered in the guide to the useful information a category landing page should provide.

The criterion: a page created by typing is kept out. A page created by deciding can be let in.

Link trail

Reader comments on this answer

Add your comment to this answer

Your email address will not be published. Required fields are marked *