- Crawl problems happen when a search engine tries but fails to reach a page on your website. Let's take a closer look at crawling first. Crawling is the technique through which a search engine uses a bot to visit each and every page of your website.
- To begin, Google suggests that you use their Fetch as Google tool to see how Googlebot scans your page. In Search Console, you'll find Fetch as Google.
- You can fetch without rendering if you only need the DNS connection status and need to respond immediately. The delayed Fetch and Render procedure is useful for getting a side-by-side comparison of how Google views your site versus how a visitor sees it.
- Consult your DNS service provider. If Google is unable to correctly fetch and render your website, you will need to take additional steps. Consult your DNS provider to determine the source of the problem. There could be a problem with the DNS provider, or it could be much worse.
- Check for a 404 or 500 error code on your server. Your server should display a 404 (not found) or 500 (server error) code instead of a failed connection. These codes are more precise than a DNS error code.
Read More: Why does it return an error when I add a sitemap file in Google Webmaster Tools?