Sorry this site requires JavaScript to be enabled in your browser. See the following guide on How to enable JavaScript in Internet Explorer, Netscape, Firefox and Safari. Alternatively you may be blocking JavaScript with an advert-related or developer plugin. Please check your browser plugins.

"Google’s rolled out a new tool at Google Webmaster Central, a robots.txt generator. It’s designed to allow site owners to easily create a robots.txt file, one of the two main ways (along with the meta robots tag) to prevent search engines from indexing content."
Comments6 Comments  

Comments

Avatar
from garyprice 2314 Days ago #
Votes: 0

On a somewhat related note from November, 2007 some robots.txt research from Penn St. Researching Robots.txt Files: Study shows Google favored over other search engines by webmasters; Say Hello to BotSeerhttp://tinyurl.com/2ua27cDirect to BotSeer tool (cool!)http://botseer.ist.psu.eduSince the original post, BotSeer now also offers info about various web crawlers.

Avatar
from avi 2312 Days ago #
Votes: -1

Google, have you run out of earth shattering products to release? If one is running a site that gets to the point where they need to block bots, they should have the basic skills to upload a simple notepad file, IMHO. Does anybody feel that this new tool has value to them?

Avatar
from crystalq 2312 Days ago #
Votes: 0

Yep, It stops product thieves and keeps hidden content hidden.

Avatar
from bwelford 2312 Days ago #
Votes: 0

Good one.  It’s all pretty basic stuff but I’m sure many will learn something from perusing this.

Avatar
from zach007 2311 Days ago #
Votes: 0

I agreed it is a good one. Regards,Rasya PazkaA team of successful entrepreneurs credited for  www.SelectWealthSystem.com A new home-based-business marketing system that provides the strategic high ground for internet marketing.Pro Team Marketing uses an automated marketing system that is currently promoting a cutting-edge young company, entering the early growth stage, that targets the largest consumer base in the United States with their financial educational products.  http://www.SelectWealthSystem.com

Avatar
from webdorf 2311 Days ago #
Votes: 0

kevgibbo,thanks for this article. Didn’t know about it before. Should be useful service, i think.

Upcoming Conferences

Search Marketing ExpoSearch Engine Land produces SMX, the Search Marketing Expo conference series. SMX events deliver the most comprehensive educational and networking experiences - whether you're just starting in search marketing or you're a seasoned expert.



Join us at an upcoming SMX event: