Broken Link CheckerEvery scan runs in two stages. First it discovers the pages on your site, then it checks the links on those pages. By the end of this page you will know what each stage does and what it leaves out.
Stage one: discovery
The scan opens your start page first. With Follow sitemap.xml on, it also reads your sitemap and queues every page listed there. That way it finds pages even when nothing links to them.
The scanner looks for the sitemap where your robots.txt points. It also tries the usual addresses, such as /sitemap.xml and /wp-sitemap.xml. A sitemap index works too, one level deep.
Next, the scan follows the links between your pages, one level at a time. Internal means the same host, with or without www. Discovery stops when no new pages turn up, or when the scan reaches the page limit of your plan.
Stage two: checking
The scan then checks every distinct link it collected. A link that appears on many pages gets one check. Pages the scan already visited during discovery count as checked, so it does not download them twice.
The scanner follows redirects and judges the page at the end of the chain. How links are classified explains each result.
The scanner reuses recent answers for the same address. So a link you fixed minutes ago can still show as broken for up to 6 hours.
What a scan checks
- Links between the pages of your own site.
- Links to other websites. On a homepage scan, switch off Check external links under Advanced options to skip them.
- Images, with Check images ticked.
- Scripts and stylesheets, with Check JavaScript ticked.
- Documents such as PDF, Word, Excel and PowerPoint files, with Check documents ticked.
Plan: the three image, script and document boxes need an account. Guests see them switched off. A Free account is enough to turn them on.
What a scan skips
- Links to a spot on the same page, such as
#contact. - Email, phone, SMS, JavaScript, data and FTP links.
- Share buttons for social networks, such as a LinkedIn share link or a tweet composer.
- Pages your robots.txt blocks, while Respect robots.txt stays on.
- Pages and links that match your excluded URL patterns.
- Private and internal addresses. The scanner only checks public websites.
Sites like LinkedIn and Facebook block automated checkers. The scan still lists links to them, but as blocked links, not broken ones.
Pages per scan
| Account | Pages per scan | Scans |
|---|---|---|
| No account | 250 | 2 a day |
| Free | 500 | 50 a month |
| Pro Personal | 1,000 | 200 a month |
| Pro Business | 2,500 | Unlimited |
| Pro Agency | 5,000 | Unlimited |
A Single page scan checks one page and the links on it. It never reads the sitemap.
How long a scan takes
Most sites finish in a couple of minutes. Large sites and slow servers take longer. The scanner spaces out its requests to each host, so it never floods your server.
When a scan stops with a message
Some problems end the scan before it can build a report. The message tells you which one you hit.
- "This site asks crawlers to stay out." Your robots.txt blocks the start page itself.
- A bot check message. A firewall challenges bots on the start page, and the message names the service.
- "The site blocked our scanner" with a 403 or 429. The start page refused the scanner.
- "The start page answered" with a status code. Check that the site address is right.
- "Our scanner could not reach this site." Nothing answered. Try again shortly.
- "The scan stopped before it finished." Run the scan again.