Sorry this site requires JavaScript to be enabled in your browser. See the following guide on How to enable JavaScript in Internet Explorer, Netscape, Firefox and Safari. Alternatively you may be blocking JavaScript with an advert-related or developer plugin. Please check your browser plugins.

Sphinn Home » SEO
Q&A: An undocumented robots.txt crawler directive from Google
SEO News Source: http://sebastians-pamphlets.com
Published: Nov 14, 2007 - 07:05 pm
Story Found By: Sebastian 1550 Days ago
Category: SEO
Update: Googlebot does understand the undocumented Noindex: directive in robots.txt, but Google said it’s not (yet) set in stone so be careful when you play with it.

***

If you ever need a hot tip on how to waste an hour or two with quite funny but completely useless SEO research, read my divulgement of a new and somewhat mysterious robots.txt statement invented by Google:
Noindex: /
 Comments3 Comments  

Comments

Avatar
from igorthetroll 1550 Days ago #
Votes: 0

Bring anti sea-sickness pills, gets dizzy when you read it.Fun stuff.

Avatar
from g1smd 1545 Days ago #
Votes: 0

Is it just a part of the Robustness Principle in operation at Google?Maybe they are supporting syntax that they see a lot of people using in error?

Avatar Moderator
from Sebastian 1544 Days ago #
Votes: 1

Google has confirmed that their crawler understands the Noindex: directive in robots.txt, but it’s an experiment so be careful when you play with it:http://sebastians-pamphlets.com/about-noindex-crawler-directives-in-robots-txt/#comment-710

Upcoming Conferences

Search Marketing ExpoSearch Engine Land produces SMX, the Search Marketing Expo conference series. SMX events deliver the most comprehensive educational and networking experiences - whether you're just starting in search marketing or you're a seasoned expert.


SMX Web Site » | SMX Difference » | SMX News »

Join us at an upcoming SMX event:

Latest News From Marketing Land
Upcoming Webcasts

Search Marketing Now Learn more about search marketing with our free online webcasts and webinars from our sister site, Search Marketing Now. Upcoming online events include:


Latest News From Search Engine Land