How Does the Website Spider Tool Work?

Your page ranking is determined by what the search engine spiders, or robots, are seeing when they crawl your pages to index them. It is therefore important to know how they are seeing your pages. It can be very different than how humans see it. For example, spiders are text readers, ignoring items like Flash and JavaScript that are there to get human attention. It is also important for your spiders to be able to detect your keywords where they count most, at the top. If your page employs frames or tables, your text content may fall low on the spider’s screen.

SEO Book offeres a free Spider Test Tool that simulates search engine spiders views of the page. You can views the pages indexed on Google, Yahoo, MSN, and Ask by clicking the provided links. It displays the text as indexed, number of words on the page, the number of distinct words, keyword counts and density percentages, outbound links, and HTML source code.

Easy to use website spider tool for content analysis

How the content analysis will look once the spider has crawled and picked up the content

A breakdown of the density of certain phrases in the content

screenshot of where to change the code to make effective content changes

What is a Similar Webpage Tool?

Also offered from SEO Books is an accompanying Webpage Similarity Tool that will compare your site to your competition sites. It displays page titles, meta data, and common phrases. Between these two tools, you can get a good view of your site as the spiders see it, along with a side-by-side comparison to other sites.

Find similar sites to yours in terms of title and meta tags

Results of pages similar to yours with similar word phrases and other SEO factors.