Broken Link CheckerA scan that reports one page crawled found nothing else to visit. By the end of this page you will have found the cause. Your next scan will then reach the rest of your site.
Work through the causes below in order. The first ones are the most common.
You picked Single page
A Single page scan checks one page and the links on it, by design. On the homepage, pick Entire site before you click SCAN →. Scans you start from a project always cover the whole site.
The scanner found no sitemap and no links
The scanner finds pages in two ways. It reads your sitemap, and it follows the links between your pages.
- No sitemap. The scanner looks where your robots.txt points, then tries the usual addresses such as
/sitemap.xml. Add a sitemap and keep Follow sitemap.xml on under Advanced options. - No plain links. The scanner follows ordinary links in the page's HTML. A menu that exists only after scripts run gives it nothing to follow.
Your robots.txt blocks the rest of the site
The scanner respects robots.txt while it looks for pages. A rule that blocks most paths leaves only the start page.
For your own site, open Advanced options on the homepage and untick Respect robots.txt. Scans you start from a project always respect robots.txt.
Your pages differ only by URL parameters
Some sites use addresses such as /?p=12 and /?p=40 for different pages. With parameters removed, all of them look like the same page.
- On the homepage, untick Ignore query parameters under Advanced options.
- In a project, open Scan settings and turn off Remove URL parameters.
A firewall turns the scanner away
A firewall or CDN can let the start page through and refuse the next requests. Those pages then land among the blocked links instead of in the crawl.
Allow the BrokenLinkCheckerBot user agent in your firewall. The crawler page shows the exact string. A link shows as blocked covers this case in more detail.
Your pages live on another host
The scanner treats pages on the same host as internal, with or without www. A subdomain such as shop. counts as another site. Scan each subdomain on its own.
An exclusion pattern matches every page
Excluded URL patterns skip every page and link that contains them. A very short pattern, such as a single /, matches your whole site. Open Scan settings and check Excluded URL patterns.
When the scan stops with a message
Some problems end the scan before it can build a report. The message tells you which one you hit.
- "This site asks crawlers to stay out." Your robots.txt blocks the start page itself.
- A bot check message. A firewall challenges bots on the start page.
- "This address cannot be scanned." The site points to a private or internal address. The scanner only checks public websites.
How a scan works lists every stop message and what it means.