usdigitalmarketingindex.com SEO
How should a planned maintenance outage communicate temporary unavailability to crawlers?
Updated November 15, 2026
By answering every request during the outage with HTTP status 503, Service Unavailable, ideally with a Retry-After header, and by keeping the robots.txt file reachable with its normal response. A 503 tells a crawler the truth: the server cannot serve the page now, and the condition is temporary. Google’s guidance for disabling a site for a short time recommends exactly this, an informational error page “with a 503 HTTP response status code instead of all content” (Google’s guidance on pausing or disabling a website). The responses to avoid are the ones that misreport the situation: a 200 with a maintenance message, which presents the notice as the page’s content, and a 404 or a redirect, which say the page is gone or has moved.
Expected interruption and duration
Plan the outage as a dated event with a known length.
| Planning item | Detail |
|---|---|
| Scope | The whole site, or particular sections or functions |
| Start and end | Date, time, time zone |
| Duration | As short as the work allows |
| Timing | The period of lowest visits, from analytics |
| Rollback | What happens if the work overruns |
| Owner | Who switches maintenance mode on and off, and who confirms each |
Duration matters to search. Google’s guidance frames the 503 approach as an option for urgently disabling a site “for 1-2 days”. Its documentation on status codes says that with server errors, already indexed URLs “are preserved in the index, but eventually dropped”, and that URLs that persistently return a server error are removed from the index (Google’s documentation on HTTP status codes). A maintenance window of hours is well inside what the 503 is meant for. An outage running into many days is not a maintenance window, and the guidance suggests other arrangements for longer closures.
If only part of the site is affected, limit the 503 to that part. Checkout under maintenance does not require the articles to be unavailable.
Temporary status codes
| Response during outage | What it tells a crawler | Suitable? |
|---|---|---|
| 503 Service Unavailable | Temporarily unable to serve; try later | Yes |
| 200 with a "down for maintenance" page | This is the page's content | No |
| 302 or 301 to a /maintenance page that returns 200 | The page has moved; the destination is a normal page | No |
| 404 or 410 | The page does not exist | No |
| 403 | Access forbidden | No |
| Connection refused or timeout | Server unreachable | Unplanned-looking; avoid |
| 429 | Too many requests | Not the meaning intended |
Google’s guidance is explicit about the error codes not to use for this purpose: “Don’t block the website by returning 403, 404, 410 HTTP status codes”.
The 200 case deserves the emphasis, because it is what a default maintenance plugin or a load balancer’s holding page can produce. If every URL returns 200 with the same short notice, a crawler that visits during the window receives the notice as the content of each page.
Two further instructions from the guidance:
- Keep robots.txt working: “Don’t return a 503 HTTP response status code for the robots.txt file because this blocks all crawling.”
- Do not switch robots.txt to disallow everything for the duration. The guidance warns that returning a robots.txt that disallows all crawling may remove the site’s content from Google Search.
The guidance also notes a limit of the 503 itself: while a page returns 503, Google’s systems cannot refresh its titles, descriptions, or structured data.
Retry-After handling
The Retry-After header states when the client may try again. HTTP defines two forms: a number of seconds, or a date.
HTTP/1.1 503 Service Unavailable
Retry-After: 7200
Content-Type: text/html; charset=utf-8
Cache-Control: no-store
or
HTTP/1.1 503 Service Unavailable
Retry-After: Sat, 14 Nov 2026 06:00:00 GMT
Google’s guidance lists it among the best practices for a 503 page: “Use the retry-after HTTP header with a best effort date or duration”. The words “best effort” are apt. The header is an estimate offered to clients, and this guide does not claim any particular crawler will return at the stated moment.
Other practices from the same list, for the error page itself:
- use static HTML;
- minimize off-page resources;
- give users clear guidance, such as when the site is expected back and how to reach customer service.
Implementation points:
- Serve the 503 from a layer that stays up during the work: the web server, load balancer, or CDN, not the application being maintained.
- Apply it to every URL except robots.txt, and to assets if they are unavailable.
- Prevent caching of the 503 by intermediaries and browsers, so it does not persist after the site returns.
- Check that the CDN does not replace the 503 with a cached 200 of the maintenance page, or serve stale pages in a way that hides the test.
- Allow the team’s own addresses through, so the work can be verified before reopening.
Recovery validation
A scheduled outage checklist.
Before:
| Step | Check |
|---|---|
| Maintenance response configured | 503 with Retry-After on a test URL in staging |
| robots.txt exemption | Returns 200 with normal content during maintenance mode |
| Error page | Static, small, states expected return time and contact route |
| Cache rules | 503 not cached |
| Monitoring | Alerts adjusted so the planned outage is recognized; external check ready to confirm status codes |
| Communication | Notice to customers where appropriate |
At the start:
| Step | Check |
|---|---|
| Switch on | Record the time |
| Verify from outside the network | Request several URLs: status 503, header present |
| Verify robots.txt | 200 |
| Verify no 200 maintenance pages | Request deep URLs, not only the home page |
Google’s guidance shows the check: confirm the 503 locally with curl or a similar tool.
curl -I -X GET "https://www.example.com/"
At the end:
| Step | Check |
|---|---|
| Switch off | Record the time |
| Status codes | Sample URLs of every page type return 200 |
| No lingering 503 | Request through the CDN and directly; clear caches if needed |
| robots.txt | Unchanged from before the outage |
| Content and function | Pages render; key journeys work |
| Tags | Canonical, robots meta, and structured data as before |
| Sitemaps | Reachable |
In the days after:
| Indicator | Source | Expectation |
|---|---|---|
| Server errors | Crawl stats and page indexing reports in Search Console | A spike for the window only |
| Crawl rate | Crawl stats | Returns to its previous level; the documentation says Google gradually increases the crawl rate once the server responds with a 2xx status again |
| Indexed pages | Page indexing report | Unchanged |
| Clicks | Performance report | A dip for the window; recovery after |
Keep a record of the outage: planned and real times, the responses served, and the verification results. If a question about a traffic change arises later, the record shows what crawlers were told and for how long.
This guide concerns downtime that is planned and controlled. Server errors that appear without warning, and intermittently, need diagnosis instead of scheduling, as covered in the guide to how intermittent HTTP 503 errors encountered by Googlebot can be diagnosed.
The rule: 503 on every unavailable URL, a Retry-After estimate, robots.txt left working, and status codes checked from outside at the start and the end.
Add your comment to this answer