A 404 is not a penalty
This misconception is worth killing first, because it drives a lot of pointless work. Google has been explicit for years that 404s are normal and don't harm the ranking of your other pages. Products get discontinued. Campaigns end. Somebody mistypes a URL. A site live for six years reporting zero 404s is either brand new or lying about its status codes.
What a large count sometimes indicates is a structural fault — a migration that dropped a URL pattern, an export that broke every product link. That's worth investigating. The number is a symptom you check, not a score you optimise.
An audit tool flagging "3,000 broken URLs" in red is selling urgency. Open the list, sort by backlinks. Usually eleven of them matter.
404 vs soft 404 vs 410
Three ways to say a page is gone, and only two of them are honest.
| State | What the server returns | What Google does |
|---|---|---|
| 404 Not Found | 404 status, with or without a helpful page | Drops the URL from the index over time, re-checks occasionally in case it returns. |
| 410 Gone | 410 status — permanently removed, deliberately | Same outcome, marginally faster, and Google re-checks less often. |
| Soft 404 | 200 OK on a page that says the content is missing | Flags it as a soft 404, refuses to index it, and keeps crawling it because it looks alive. |
Why soft 404s are the genuine problem
A soft 404 is a lie your server tells. The page returns 200 — "here is your content" — and the content is the words "page not found". Google can't tell from the status code that the URL is dead, so it keeps fetching it. On a large site that's real crawl capacity spent on nothing.
Two common causes: single-page apps that route client-side and serve 200 for every path, and CMS setups where a deleted item quietly lands on an empty category listing. Both look fine in a browser and both are wrong at the protocol level. Check with curl -I — the browser won't tell you.
Google also applies the label to redirects landing somewhere irrelevant, which is why mass-redirecting old URLs to the homepage doesn't work. See 301 redirect.
The triage rule
Don't redirect 404s as a policy. Redirect the ones carrying something worth keeping. Everything else stays a 404, and that's the correct outcome rather than a compromise.
| The dead URL | Action |
|---|---|
| Has external backlinks | 301 to the closest equivalent. This is why redirect maps exist — you're recovering an asset someone else gave you. |
| Had organic traffic in the last 12 months | Restore the page if the content still holds. Redirect if something replaced it. |
| Discontinued product with a successor | 301 to the successor, or the category if there isn't one. |
| Discontinued product with nothing similar | 404 or 410. A useful not-found page beats a redirect that doesn't answer the request. |
| Expired campaign, event or job listing | 404, with a link to the current equivalent. |
URL that never existed — typos, scraped junk, /wp-login.php on a Next.js site | Leave it. Nothing to fix. |
Finding the ones that matter
- Search Console, Page Indexing, "Not found (404)". Google's own list of dead URLs it has crawled.
- Your backlink tool's broken-pages view. Ahrefs, Semrush and Moz all filter dead URLs by referring domains. Sort descending, stop after the first page.
- Analytics, last 12 months, filtered to your 404 page title. What people are still trying to reach.
- Server logs. The only source catching URLs nobody links to but bots keep requesting. See log file analysis.
Building a 404 page worth landing on
The page must return a 404 status code. That's the non-negotiable part, and the part frameworks get wrong. Everything after it is user experience — a real cost, since a dead end is a lost visitor who had already decided to come to you.
Say what happened in plain language. Offer the search box. Link your three or four most useful pages, not a sitemap dump. Skip the automated "related pages" widget unless it's genuinely accurate; a bad guess is worse than an honest dead end.
And check the status code after every deploy, in the same breath as checking robots.txt. curl -I https://yoursite.com/a-url-that-does-not-exist should return HTTP/2 404. If it returns 200, your framework is serving a shell and every mistyped URL on the internet is now an indexable page on your site.