- 42
- Sphinn It!
Posted By: AndyBeard 178 days ago
Topic Type: News Story (Jump to http://www.hobo-web.co.uk)
Category: SEO
Shaun grabbed his ear, and in this entertaining and educational interview they go into some of the basics of what Robots.txt is, and how it should be used.
2 Comments


Comments
I realize that your don't need to use a robot.txt but I found (using Google Webmaster Tools) they it sometimes has trouble reading your site if no file exists. Once I started putting a file out there Google stopped returning errors. Wierd but true.
that depends on the type of hosting you have. this is typical of shared hosting packages