Broken link scanning

How links are classified

Which answers make a link broken, a warning or blocked, how redirects and soft 404 pages count, and why a 403 or 429 never counts as broken.

4 min read

Every link the scanner checks ends in one of five states. By the end of this page you will know which answers land in each state. You will also know which ones lower your health score.

The health score is the share of checked links not counted as broken. Only the Broken state pulls it down.

Broken

A broken link fails for your visitors too. A link counts as broken in these cases:

  • The server answers with a 4xx or 5xx status, such as 404, 410, 500 or 503. The two exceptions are 403 and 429, covered under Blocked.
  • The host name does not exist. Scan reports show DNS and project pages show No DNS.
  • The site's security certificate fails. Scan reports show SSL and project pages show SSL error.
  • The page is a soft 404.

Soft 404 pages

A soft 404 page loads normally but tells the visitor the page is gone. The server answers 200, so a plain status check misses it. Scan reports mark these links S404.

The scanner reads the page title and the main heading. An error phrase such as "page not found" can fill most of that text. Then the page counts as a soft 404. The scanner knows these phrases in English, German, French, Spanish, Norwegian, Swedish, Danish and Finnish.

A nearly empty page that also asks search engines not to index it counts too. One of those signs alone does not.

Warning

A warning means the scanner could not verify the link. Warnings show in your results, but they never lower the score. Three things cause one:

  • The server did not answer in time. Scan reports show TIMEOUT and project pages show Timed out.
  • The server refused or dropped the connection. Scan reports show CONNECTION and project pages show No answer.
  • The site blocked the scanner, as described under Blocked.

Many servers ignore automated checkers but serve people normally. Open a warning link in your browser to see if it loads. A link to a private or internal address also lands here, because the scanner does not visit those.

Blocked

Blocked is a kind of warning. The site refused the scanner or turned it away.

Scan reports and public report pages list these links in a separate Blocked links note below the table. The Broken links page of a project and its CSV export leave them out.

Why a 403 or 429 is a warning

A 403 answer means "forbidden". Firewalls, country rules and hotlink protection often send it to bots while they serve people fine. A 429 answer means "too many requests". The scanner got slowed down before it learned anything about the page.

Neither answer proves the link is broken. So on any site, a 403 or a 429 lands in Blocked, never in Broken.

Sites that block every checker

Some sites block automated checkers across the board. LinkedIn and Facebook are the best-known examples. When one of these sites refuses the scanner or drops the connection, the link lands in Blocked.

A real 404 from those sites still counts as broken. The scanner only goes easy on answers that say "go away", not on answers that say "not found".

A link that opens a firewall challenge page instead of the real page lands in Blocked too.

Redirect

The scanner follows up to 5 redirects and judges the page at the end. A redirect is never a row of its own.

  • A redirect that ends on a working page is fine. The report does not list it.
  • A redirect that ends on a 404 counts as broken. The status shows the final answer.
  • A redirect to another site that then does not answer lands in Blocked. The redirect itself answered, so the link most likely works.
  • A chain longer than 5 hops stops there, and the report does not list it.

OK

A link is OK when it answers with a status below 400 and the page is not a soft 404. OK links count in links checked and keep your score up. Reports do not list them one by one.

The WordPress plugin counts differently

The Check for Broken Links plugin uses its own, narrower rule. It counts only 404 and 410 answers as broken. So the plugin and your scan report can show different broken counts for the same site.