Sorry this site requires JavaScript to be enabled in your browser. See the following guide on How to enable JavaScript in Internet Explorer, Netscape, Firefox and Safari. Alternatively you may be blocking JavaScript with an advert-related or developer plugin. Please check your browser plugins.

Sphinn Home » SEO
‘I Robot’ with SebastianX of ‘Sebastian’s Pamphlets’ > Robots.txt Help
SEO News Source: http://www.hobo-web.co.uk
Published: Jan 10, 2008 - 07:12 am
Story Found By: AndyBeard 1956 Days ago
Category: SEO
Sebastian is an expert on the technical side of SEO, though sometimes his more in depth posts tend to challenge the neural nodes of accomplished experts.
Shaun grabbed his ear, and in this entertaining and educational interview they go into some of the basics of what Robots.txt is, and how it should be used.
 Comments2 Comments  

Comments

Avatar
from CharlesBM 1814 Days ago #
Votes: 0

I realize that your don’t need to use a robot.txt but I found (using Google Webmaster Tools) they it sometimes has trouble reading your site if no file exists.  Once I started putting a file out there Google stopped returning errors.  Wierd but true.

Avatar
from SettlementBoard 1796 Days ago #
Votes: 0

that depends on the type of hosting you have.  this is typical of shared hosting packages

Upcoming Conferences

Search Marketing ExpoSearch Engine Land produces SMX, the Search Marketing Expo conference series. SMX events deliver the most comprehensive educational and networking experiences - whether you're just starting in search marketing or you're a seasoned expert.


SMX Web Site » | SMX Difference » | SMX News »

Join us at an upcoming SMX event:

Latest News From Marketing Land
Upcoming Webcasts

Digital Marketing Depot Learn more about search marketing with our free online webcasts and webinars from our sister site, Digital Marketing Depot. Upcoming online events include:


Webcasts:
Research:
Whitepapers:

See More Topics

Latest News From Search Engine Land