News Source: http://www.mattcutts.com
Published: Feb 25, 2008 - 08:35 am
Story Found By: theGypsy 1550 Days ago
Category: SEO
Matt Cutts Blog - Okay, this post will be colossally boring to some people. But I wanted to give you a peek at debates behind the curtain in Google’s search quality group. Here’s a policy discussion about NOINDEX and how Google should treat the NOINDEX meta tag. First, you’ll want to read this post about how Google handles the NOINDEX meta tag. You may also want to watch this video about how to remove your content from Google or prevent it from being indexed in the first place. Here’s the conclusion from my earlier blog post:
9 Comments
Comments
Reminds me of STOP signs. They say stop but do they really mean stop? There’s the "it’s after 12am" rule, the "I have to pee" rule, the "I’m learning to drive a stick and don’t want to stall" rule, etc...
Missionary zeal rearing its ugly head once again. "Our highest duty has to be to our users, not to an individual webmaster."Wrong. Regarding the use of any specific content (on which, in aggregate, search companies thrive) the first and foremost duty should to be to respect the decision (communicated, among others, through noindex) of the owner, provider, creator of that specific content.He or she may have left the noindex in place by mistake, but it is not Google’s (or MSN’s or Yahoo’s) competence to make an assumption and override this signal out of self-interest. Because at the end of the day it’s just pure self-interest disguised as noble service to users: "If high-profile sites [...] aren’t showing up in Google because of the NOINDEX meta tag, that’s bad for users (and thus for Google)."
Folks, please read this pamphlet before you vote at Matt’s blog:http://sphinn.com/story/30845Thanks!
To be fair, Matt has posted today a comment regarding the reason behind his current post:"Out of interest, why has it suddenly become a hot issue? Is it a ’reputation’ thing - or a realisation that people in other languages may be getting it wrong disproportionately?" Andrew Heenan, it’s not a hot issue, but every month or so we get a small trickle of sites that are shooting themselves in the foot. "Why didn’t Ben Harper’s site show up? Did bmw.dk remove itself on purpose? These Korean sites have dropped from our index." Every time we see a site that appears to have made a mistake, it re-opens the conversation.
Matt, has also graciously replied to my question :===============Matt, Tell us about a middle ground. In the case: Google to not show that page at all, but it still following the links on that page. [meta name="robots" content="noindex, follow" /]=========================And Matt replied:"Harith, I just got to your follow-up comment. In that case, currently Google wouldn’t show the page but would follow the outgoing links."
... and follow is the default action anyway, so it does that even if you don’t state it.
The point is that ugly hubs and such that were made for crawlers not humans and therefore carry a "noindex,follow" robots meta tag actually help with index penetration.
Point taken, but I was trying to stress that people generally know that meta noindex stops the URL appeaing in the SERPs, and that many people mightly then (wrongly) logically infer that the links on the page were not seen or followed... but they are. :-)
Yep, and I’d like to add that’s a totally flawed "logic". NOINDEX and NOFOLLOW are independent indexer directives, defaulting to INDEX and FOLLOW if absent. Even more folks have problems to use the NONE directive properly. I’ve seen lots of robots meta tags with "NONE,INDEX" or "NONE,FOLLOW" which don’t accomplish what the author has intended, or even "NONE" where the Webmaster meant "no restrictions". Folks shouldn’t read so much crap on the boards and look at the standards instead.