Sorry this site requires JavaScript to be enabled in your browser. See the following guide on How to enable JavaScript in Internet Explorer, Netscape, Firefox and Safari. Alternatively you may be blocking JavaScript with an advert-related or developer plugin. Please check your browser plugins.

Michael Gray: When Google took away the supplemental index last year, they killed one of the key diagnostic tools in the SEO’s toolbox, the ability to identify which parts of a site were unimportant (and being infrequently crawled) in a search engines eyes. However with the use of some structured text and clever searches I’m going to give you back that valuable information.
Comments3 Comments  


from HarryWatson00 3509 Days ago #
Votes: 0

Interesting idea, but will the hassle of changing the header or footer every month be worth it?

from JohnWeb 3509 Days ago #
Votes: 0

Obviously you’d want to code it to automatically change the footer. Couple that with actually tracking crawler hits on the pages and you have something as being crawled does not equal being cached does not equal being updated in the index. Frequency of crawl, cache, and indexing graphed gives some real insight as to the health of pages. You never want to ’asssume’ that site and interlinking structure will determine crawl priorities as external sites may deep link messing up the whole top down architecture. Actually logging it and doing analysis will help identify parts of the site low on the priority list and then you can decide if they should be or take corrective actions if they shouldn’t.Needless to say, but I will anyway, this needs to be automated so you can just look at reports and not waste time compiling them.

Avatar Moderator
from graywolf 3509 Days ago #
Votes: 0

yes @harrywatson you absolutely need to use a server side scripting language like PHP to automatically change the date.yes @johnweb this will really tell indexing frequency not crawling frequency and there is a subtle difference between the two, but I chose crawling frequency since readers are going to grasp the concept quicker.

Upcoming Conferences

Search Marketing ExpoSearch Engine Land produces SMX, the Search Marketing Expo conference series. SMX events deliver the most comprehensive educational and networking experiences - whether you're just starting in search marketing or you're a seasoned expert.

Join us at an upcoming SMX event: