Sorry this site requires JavaScript to be enabled in your browser. See the following guide on How to enable JavaScript in Internet Explorer, Netscape, Firefox and Safari. Alternatively you may be blocking JavaScript with an advert-related or developer plugin. Please check your browser plugins.

Sphinn Home » Sphinn Zone
Crawling vs. Indexing
Sphinn Zone News Source: http://sebastians-pamphlets.com
Published: Oct 20, 2008 - 09:02 pm
Story Found By: martinbowling 1200 Days ago
Category: Sphinn Zone
sebastian weighs in on what is Crawling vs what is Indexing. Very apropos discussion at the moment :)
 Comments9 Comments  

Comments

Avatar
from NickWilsdon 1200 Days ago #
Votes: 1

The first post since June 30th! This gets my vote just to encourage Sebastian to post more often ;)

Avatar Moderator
from Sebastian 1200 Days ago #
Votes: 3

Thanks guys :)

Avatar
from ColinCochrane 1200 Days ago #
Votes: 0

Spot on as always Sebastian.

Avatar
from ChelseaBlacker 1200 Days ago #
Votes: 0

Sebastian, you should ask Aaron Wall if he’s interseted in adding these to his SEO dicitonary (expand the current definitions)!  And give you a link for credit.

Avatar
from JohnHGohde 1200 Days ago #
Votes: -4

No!  I would say that sebastian spoke out on the SEOmoz controversey.SEOmoz is NOT that important people.  If I see, let along read, another SEOmoz spinoff I am going to scream!!!New articles are not necessary.  Comments are just fine, Sebastia.

Avatar
from annie7 1200 Days ago #
Votes: 2

I am happy Sebastian is back at last!

Avatar
from mike 1199 Days ago #
Votes: 1

Not to be a pedant, but there are three steps:Crawl.Parser - this takes something and looks for meaning, e.g. title, links etc.Indexing - this puts the new, meaningful data somewhere.One can do any of these steps without tother, and even some stages multiple times (think - multiple parsers).

Avatar
from IncrediBILL 1199 Days ago #
Votes: 0

per Sebastian’s comment:"The meta tag method is nice to have, but not practically, IOW more or less propaganda."The meta tag method is the only way people using hosted CMS and blogs can actually control their sites because most services don’t provide an efficient method such as robots.txt.I think they should provide both.I’m pretty sure they already do since I’m pretty sure I blocked their crawler and know it respects robots.txt, but I’m not 100% sure it’s the only thing in use.

Avatar Moderator
from Sebastian 1199 Days ago #
Votes: 1

> "The meta tag method is nice to have, but not practically, IOW more or less propaganda."<div></div><div>That addresses meta tags for each and every SEO tool out there. In the same comment I said I do use meta tags to steer the indexing of search engines, but can’t be bothered to implement on-the-page indexer directives for everybody.</div><div></div><div>Of course I agree, they should provice both.</div>

Upcoming Conferences

Search Marketing ExpoSearch Engine Land produces SMX, the Search Marketing Expo conference series. SMX events deliver the most comprehensive educational and networking experiences - whether you're just starting in search marketing or you're a seasoned expert.


SMX Web Site » | SMX Difference » | SMX News »

Join us at an upcoming SMX event:

Latest News From Marketing Land
Upcoming Webcasts

Search Marketing Now Learn more about search marketing with our free online webcasts and webinars from our sister site, Search Marketing Now. Upcoming online events include:


Latest News From Search Engine Land