About Spider Crawl Tests for SEO

There are many hundreds of bits of information on any given website that search engine spiders see when crawling your website. They see things differently than humans do but it’s what they see that is important to search engine rankings. Therefore, it’s important to know what it is they are spying so you can optimize their journey through your website for maximum effect.

Using the Crawl Test Tool

The free Crawl Test tool from SEOmoz offers quick, at-a-glance information to help keep your website optimized. It displays the number of internal and external links, the pages with unique title tags and duplicate title tags, pages lacking iin title tags, pages with unique and duplicate meta descriptions, pages without meta descriptions, the number of pages indexed by Google, Yahoo and Bing, the average Google cache date, and HTTP codes to let you know if there are problems with any pages being indexed.

Find out what search engines read on your webpage by entering your website URL

Screenshot of the Google spider crawl test tool.

 

This tool helps you see how accessible your website is to the search robots and to diagnose issues to be resolved. A more complete and thorough report is available by subscribing for a monthly fee of $79 to $229.