Sphinn - Google Offers Robots.txt Generator

49
Sphinn It!

Went Hot: March 29, 2008 - 2:55 pm
Posted By: kevgibbo 465 days ago
Topic Type: News Story (Jump to http://searchengineland.com)
Category: Google Other

"Google's rolled out a new tool at Google Webmaster Central, a robots.txt generator. It's designed to allow site owners to easily create a robots.txt file, one of the two main ways (along with the meta robots tag) to prevent search engines from indexing content."

6 Comments

Who Desphunn This (0)

Comments

from garyprice 465 days ago #

Votes: 0 | Vote:

+ -

On a somewhat related note from November, 2007 some robots.txt research from Penn St.

Researching Robots.txt Files: Study shows Google favored over other search engines by webmasters; Say Hello to BotSeer
http://tinyurl.com/2ua27c

Direct to BotSeer tool (cool!)
http://botseer.ist.psu.edu

Since the original post, BotSeer now also offers info about various web crawlers.

from avi 463 days ago # - show/hide this comment

Votes: -1 | Vote:

+ -

Google, have you run out of earth shattering products to release? If one is running a site that gets to the point where they need to block bots, they should have the basic skills to upload a simple notepad file, IMHO. Does anybody feel that this new tool has value to them?

from crystalq 462 days ago #

Votes: 0 | Vote:

+ -

Yep, It stops product thieves and keeps hidden content hidden.

from bwelford 462 days ago #

Votes: 0 | Vote:

+ -

Good one. It's all pretty basic stuff but I'm sure many will learn something from perusing this.

from zach007 462 days ago #

Votes: 0 | Vote:

+ -

I agreed it is a good one.

Regards,

Rasya Pazka

A team of successful entrepreneurs credited for www.SelectWealthSystem.com
A new home-based-business marketing system that provides the strategic high ground for internet marketing.
Pro Team Marketing uses an automated marketing system that is currently promoting a cutting-edge young company, entering the early growth stage, that targets the largest consumer base in the United States with their financial educational products.
http://www.SelectWealthSystem.com