Calendar
Network
Feeds
- 49
- Sphinn It!
Went Hot: June 10, 2008 - 4:08 pm
Posted By: Rhea 122 days ago
Topic Type: News Story (Jump to http://janeandrobot.com)
Category: SEO
5 Comments
Went Hot: June 10, 2008 - 4:08 pm
Posted By: Rhea 122 days ago
Topic Type: News Story (Jump to http://janeandrobot.com)
Category: SEO
5 Comments
Top 10 Most Sphunn
In What's New
- Personal Branding is BS (20)
- SEO Canonical URLs And 301 Redirect... (18)
- 25 Ways to Build Your Community (17)
- Why You Should Build a Web Business... (15)
- How to Test Your Site Usability wit... (15)
- Best SEO Practices During A Website... (15)
- Have Gun, Will Telecommute (15)
- An Experiment - SEO Style - What ar... (14)
- Is the Onebox really that great? (13)
- Google Reigns Supreme [Search Marke... (12)
Latest Comments
- If Satan is creeping up from hell, this ripoff place…
- For all the time I spent thinking that Twitter was quite possible the dumbe…
- interesting...I have a Digg profile, but I'm confused. What is exactly the …
- wow! picitup recognizes images for a specific color palette? This could be …
- I don't think Google is so much "mocking suicide" as it is poking fun at th…
- A much needed caution to be taken before taking up a SEO project, though it…
- Alistair, thanks! That sounds logical. But, a lot of work and tracking. I w…
- storytime,You're best bet is to redirect ver…
- - read more -
Save the date for:
• SMX London - Nov. 4-5: Pre-agenda rate now available. Click here.
• SMX West - Feb. 10-12
Learn more about search marketing through free online webcasts and webinars from our sister site Search Marketing Now.
Upcoming Webcasts:
- 5 Ghastly PPC Mistakes to Avoid
October 14, 2008 - SEM Agencies and Reporting: Best Practices for Success
October 23, 2008 - Measuring an Ad's Value: A Forum Discussion
October 28, 2008
Comments
YOU RAWK Vanessa! Thanks... instant classic + EPIC WIN!
Hooray for Vanessa! I'm sending this straight to my development team.
This is great! I just sent the link over to several people I know who were working with this.
Thanks!
One of the best robots posts I have seen. Will definately link to it when the topic comes up. Incidentally I did a post yesterday which could have used it, but rather links to robotstxt.org.
Use Case - Robots.txt - META/ X-Robots-Tag - Other - Supported By
Allow access to your content - Allow - FOLLOW - INDEX - Google/Yahoo/Microsoft
Disallow access to your content - Disallow - NOINDEX - NOFOLLOW - Google/Yahoo/Microsoft
There is a lot of over-simplification here. It might be wise to explain that links on a page can be followed even if the page is not indexed.
Be aware that noindex (meta tag) does not mean "do not crawl the page", it means "do not show in SERPs". Google quite happily takes a copy of the page (they have to, in order to see the meta tag) and will follow links out from it unless you expressly disallow that.