Sphinn Home » Google Other
"Google's rolled out a new tool at Google Webmaster Central, a robots.txt generator. It's designed to allow site owners to easily create a robots.txt file, one of the two main ways (along with the meta robots tag) to prevent search engines from indexing content."
6 Comments     

Comments

from garyprice 193 days ago #
Votes: 0 | Vote:
+ -

On a somewhat related note from November, 2007 some robots.txt research from Penn St.

Researching Robots.txt Files: Study shows Google favored over other search engines by webmasters; Say Hello to BotSeer
http://tinyurl.com/2ua27c

Direct to BotSeer tool (cool!)
http://botseer.ist.psu.edu

Since the original post, BotSeer now also offers info about various web crawlers.


from avi 192 days ago # - show/hide this comment
Votes: -1 | Vote:
+ -

Google, have you run out of earth shattering products to release? If one is running a site that gets to the point where they need to block bots, they should have the basic skills to upload a simple notepad file, IMHO. Does anybody feel that this new tool has value to them?

from crystalq 191 days ago #
Votes: 0 | Vote:
+ -

Yep, It stops product thieves and keeps hidden content hidden.

from bwelford 191 days ago #
Votes: 0 | Vote:
+ -

Good one.  It's all pretty basic stuff but I'm sure many will learn something from perusing this.

from zach007 191 days ago #
Votes: 0 | Vote:
+ -

I agreed it is a good one.

Regards,

Rasya Pazka

A team of successful entrepreneurs credited for  www.SelectWealthSystem.com
A new home-based-business marketing system that provides the strategic high ground for internet marketing.
Pro Team Marketing uses an automated marketing system that is currently promoting a cutting-edge young company, entering the early growth stage, that targets the largest consumer base in the United States with their financial educational products.
  http://www.SelectWealthSystem.com

from webdorf 190 days ago #
Votes: 0 | Vote:
+ -

kevgibbo,
thanks for this article. Didn't know about it before. Should be useful service, i think.


Log in to comment or register here.
Search Marketing Expo

Save the date for:
SMX East (New York City) - Oct. 6-8: See the agenda or register today and save!
SMX London - Nov. 4-5: Pre-agenda rate now available. Click here.
SMX West - Feb. 10-12

Search Marketing Now

Learn more about search marketing through free online webcasts and webinars from our sister site Search Marketing Now.

Upcoming Webcasts: