Skip to the indexed answers
Digital Marketing Index

usdigitalmarketingindex.com SEO

How can broken internal links returning HTTP 404 be repaired at scale?

Group the broken links by their dead destination, decide once per destination what should happen, and apply that decision to every link that points there. A site with five thousand broken links does not have five thousand separate problems to solve. The links point at a smaller number of dead URLs, each linked from several pages, and some of those links sit in templates that repeat on every page. Working by destination turns thousands of edits into a short list of decisions: restore the page, redirect the address, repoint the links to a replacement, or remove the links. The last matters as much as the others. A link should be repointed only when a relevant destination exists.

Discovering referring pages

Build the inventory from a crawl.

  1. Crawl the whole site and export every internal link whose destination returns 404 or 410, with its source page, anchor text, and position.
  2. Add other sources of evidence: Search Console’s page indexing report lists not-found URLs Google has requested (Search Console Help on the page indexing report), and server logs show which missing URLs visitors and crawlers hit.
  3. Pivot the export by destination. For each dead URL: the number of source pages, and whether the link sits in a template or in content.
  4. Sort by the number of sources.

The top of that list is where the leverage is. A dead link in the footer accounts for one row and every page of the site.

Confirm the 404s are real before acting. Request a sample directly. Some crawlers report errors when a server rate-limits them, and some pages return an error to automated clients and a normal page to browsers.

Classifying missing destinations

Put each dead destination into a class. The class determines the fix.

Class How to recognize it Fix
Typo or malformed URL The address is nearly right: a missing letter, a doubled slash, a relative path that resolved wrongly Correct the link
Moved, redirect missing The content exists at a new address Add a redirect and repoint the links
Deleted by mistake The page should exist Restore it
Removed, with a close replacement A newer page covers the same need Repoint to the replacement; redirect the old address
Removed, no replacement The product, event, or article is gone and nothing covers it Remove the links; edit the sentences
Never existed A link to a planned page that was not built Remove, or build the page
Generated by a template bug A pattern of broken addresses from one component Fix the template

Look for patterns before fixing individual rows. Two hundred dead links that all contain the same wrong path segment are one bug.

Replacement versus removal

The judgment that cannot be automated is whether a replacement is relevant.

Repoint when the new destination gives the reader what the sentence promised. A link reading our guide to tankless installation can move to the current installation guide.

Remove when nothing does. Sending a link about a discontinued model to the home page, or to a vaguely related category, leaves the reader somewhere they did not ask to go. In that case:

  • Delete the link and keep the text, if the sentence stands without it.
  • Rewrite the sentence, if it existed to introduce the link.
  • Delete the sentence or the list item, if it has no purpose now.

Check the anchor text when repointing. An anchor written for the old page may misdescribe the new one.

For the address itself, a redirect is worth adding when the dead URL has links from other sites or still receives visits. Where it has neither and nothing replaces it, the 404 is the correct response and can stay. Fix your own links to it and leave the status alone.

An issue log keeps the decisions and their state. Sample rows:

Dead target Sources Location Class Resolution Applied Recheck
/services/drain-cleanng 1 Content Typo Correct to /services/drain-cleaning/ Yes 200
/about/team.html 1,240 Footer template Moved, no redirect Template updated to /about/team/; redirect added Yes 200
/products/model-x100 37 Content Removed, replacement exists Repointed to /products/model-x200/; anchors reviewed Yes 200
/events/open-day-2023 12 Content Removed, no replacement Links removed; four sentences rewritten Yes No links remain
/blog/category//news 85 Sidebar widget Template bug Widget path fixed Yes 200
/guides/heat-pump-sizing 6 Content Never existed Links removed pending the article Yes No links remain

Re-crawl verification

Apply the fixes in an order that removes the most errors first.

  1. Templates and shared components.
  2. Pattern fixes, by search and replace on exact addresses, after testing on a few pages.
  3. Redirects for moved content.
  4. Individual content edits.

Bulk replacement needs care. Replace full, exact addresses. Review the list of affected pages before running it. Keep a backup or a revision history. A careless pattern can rewrite links that were correct.

Then crawl again and compare.

Check Pass condition
Internal links to 404 or 410 Zero, or a listed set awaiting a decision
Repointed links Destination returns 200 directly, without a redirect
Redirects added One hop to a relevant page
Removed links The surrounding text still reads correctly; sample checked by a person
New errors None introduced by the bulk edits
Log Every row has a resolution and a recheck result

Have a person read a sample of the edited sentences. A crawl confirms that links resolve. It cannot tell whether a paragraph still makes sense after its link was removed.

Keep it from building up again.

  • Run the crawl on a schedule and after each release.
  • When a page is unpublished, the CMS or the process should list the pages linking to it.
  • Give editors a link picker that references content instead of typed addresses.
  • Monitor not-found requests in server logs for addresses that suddenly attract traffic.

One large source of dead internal links is content consolidation, where several articles become one. The remapping needed after such a merge is described in the guide to remapping internal links after several articles are combined into one.

The method: pivot by dead destination, classify, decide once, apply everywhere, crawl again.

Link trail

Reader comments on this answer

Add your comment to this answer

Your email address will not be published. Required fields are marked *