Sorry this site requires JavaScript to be enabled in your browser. See the following guide on How to enable JavaScript in Internet Explorer, Netscape, Firefox and Safari. Alternatively you may be blocking JavaScript with an advert-related or developer plugin. Please check your browser plugins.

Sebastian is an expert on the technical side of SEO, though sometimes his more in depth posts tend to challenge the neural nodes of accomplished experts.
Shaun grabbed his ear, and in this entertaining and educational interview they go into some of the basics of what Robots.txt is, and how it should be used.
Comments2 Comments  

Comments

Avatar
from CharlesBM 2154 Days ago #
Votes: 0

I realize that your don’t need to use a robot.txt but I found (using Google Webmaster Tools) they it sometimes has trouble reading your site if no file exists.  Once I started putting a file out there Google stopped returning errors.  Wierd but true.

Avatar
from SettlementBoard 2136 Days ago #
Votes: 0

that depends on the type of hosting you have.  this is typical of shared hosting packages

Upcoming Conferences

Search Marketing ExpoSearch Engine Land produces SMX, the Search Marketing Expo conference series. SMX events deliver the most comprehensive educational and networking experiences - whether you're just starting in search marketing or you're a seasoned expert.



Join us at an upcoming SMX event: