Published: Nov 26, 2007 - 03:53 pm
Story Found By: Sebastian 1538 Days ago
Category: SEO
Here is my step by step "How to create a smart robots.txt" guide. Adapt. Test. Enjoy.
8 Comments
8 Comments
Search Engine Land produces SMX, the Search Marketing Expo conference series. SMX events deliver the most comprehensive educational and networking experiences - whether you're just starting in search marketing or you're a seasoned expert.
Join us at an upcoming SMX event:
Learn more about search marketing with our free online webcasts and webinars from our sister site, Search Marketing Now. Upcoming online events include:
Comments
That looks very interesting. Thanks for the share.
Hot topic in two hours methinks. This needs to be used with care. There are several ways to catch yourself out if bots catch different versions of the file from the same site at different times. Use with care, and test, test, test and test again.
Well, too frequent changes of robots.txt can irritate search engines regardless whether the robots.txt is cloaked or not. I second your "<b>Use with care, and test, test, test and test again.</b>".
Great tutorial. Youd be a great, and popular, teacher. Rough around the edges...but extremely knowledgeable. Like the Howard Stern of Search.
Ron, thank you, but Howard Stern is in another league.
You dont even know how cool this is. Im in the midst of writing (yes, this is off topic) an SEO friendly CMS and was wondering how to dynamically rewrite URLs using info from the database. I had know idea this could be done. Well, I had some idea because Im a vbseo user, but didnt realize it was this easy....or obvious i mean.
Interesting article there. It has given me plenty to think about and consider. Cheers.
Rough around the edges isnt what makes him unique. How many other SEO guys have a thorough database/development background and include such strategies and tutorials in their postings? Not many. He has carved his identity as such.