About Spider Crawl Tests for SEO
There are many hundreds of bits of information on any given website that search engine spiders see when crawling your website. They see things differently than humans do but it’s what they see that is important to search engine rankings. Therefore, it’s important to know what it is they are spying so you can optimize their journey through your website for maximum effect.
Using the Crawl Test Tool
The free Crawl Test tool from SEOmoz offers quick, at-a-glance information to help keep your website optimized. It displays the number of internal and external links, the pages with unique title tags and duplicate title tags, pages lacking iin title tags, pages with unique and duplicate meta descriptions, pages without meta descriptions, the number of pages indexed by Google, Yahoo and Bing, the average Google cache date, and HTTP codes to let you know if there are problems with any pages being indexed.
This tool helps you see how accessible your website is to the search robots and to diagnose issues to be resolved. A more complete and thorough report is available by subscribing for a monthly fee of $79 to $229.