Using Webmaster Tools To Analyze And Correct Crawl Errors

Posted on July 12, 2013 by

Using Google Sites is a relatively quick and easy way to get a website up and running.  It’s also quite affordable.  As with any service that provides web design and publication of web pages, there are errors that can crop up.  When an error is encountered, users would notice an error message staring them in the face instead your website content.  Not only is it annoying for a user to try and navigate around a website that contains multiple errors, it also causes them to discredit your site.

It may be comforting to know that just by having errors within the links and programming in your website doesn’t necessarily mean they all need to be corrected at once. Some errors crop up for informational reasons, but others can be crippling, so it’s important to determine which ones to fix and which ones to leave alone.  Crawl errors are categorized into several different error types and appear in a report in your Google webmaster tools.

  1. HTTP.  While HTTP errors are not crippling to a website, they do return an error message to users.  It’s common for these errors to pull up a page that shows 403 pages.   Google has a help page that lists all HTTP status codes and what to do about them.  Server Headers 101 infographic is also a useful tool that can help you make meaning out of HTTP errors.
  2. Not Followed.  Redirect errors are responsible for the bulk of “not followed” problems.  In order to avoid them, ensure that redirect chains are minimized, avoid the use of meta refreshes in page headers, and set the redirect timer for short periods of time.  A chain of redirect messages can send the Google crawler bot down a long line of continuous errors, creating an exhausted crawler that simply can’t find content.  Instead, after finding error after error the bot stops searching.  These errors are important to fix.
  3. Not Found.  Errors that display a “page not found” message are usually 404 errors, and they can be caused by a few different things.
  • After deleting a page on your website, you do not redirect it.
  • After changing the name of page on your website, you do no redirect it.
  • A typo in a link on your website exists and because of the typo Google redirects users to sites that don’t exist.
  • Subfolders can become mixed up after moving a website to a new domain, creating not found errors.
  • When someone else sets up link from their website to yours, but the link they created has a typo in it, a not found error can occur.
  1. Timed Out.  This is an error that occurs when webpages take too long to load.  If it takes a long time, the Googlebot will eventually stop trying to call it.  If you experience timed out errors, scour through the reports in your sever logs and determine if there are any problems that you can detect.  It’s also a good idea to see the speed at which timed out pages are loading.  There are basically three types of timed out errors including a DNS lookup timeout, a URL timeout, and a Robots.txt timeout.

While not all errors are severely detrimental to the health and well-being of your website, some are and they need your attention.  In order to give your website users the best browsing experience possible, and to maintain their trust in your site, stay vigilant and use your webmaster tools to fix crawl errors.

Article courtesy of Wpromote, PPC Management and Search Engine Optimization. Follow @wpromote on Twitter for more industry updates and up to date trends in Search Engine Marketing.

Image License: Creative Commons by Kai Hendry

Author :

  • Editor