Sphinn - Indexing Search Results: Why Google Is Going Against Its Own Advice

48
Sphinn It!

Indexing Search Results: Why Google Is Going Against Its Own Advice

Went Hot: May 5, 2008 - 1:34 pm
Posted By: janecopland 426 days ago
Topic Type: News Story (Jump to http://ciarannorris.co.uk)
Category: Google Searching

It now seems that Google is ignoring its own advice, and the use of noindex, and is indexing search results pages. If you look at Computer Weekly’s robots.txt file, you’ll see their search pages are noindexed. And yet the pages have been indexed.

8 Comments

Who Desphunn This (1)

Comments

from jdevalk 425 days ago #

Votes: 0 | Vote:

+ -

As commented on the site itself: 1. ciaran, and god knows I love you Ciaran, just doesn't get how robots.txt and robots meta tags interact (hint: they don't, the first blocks the latter), and doesn't ask the obvious question of which weird internal link scheme get's 105,000 internal serps indexed...

from ciaran 425 days ago #

Votes: 1 | Vote:

+ -

And Joost, as commented on the site, we may have been naive, but there is no internal linking on the site. They're indexed because Google decided to start indexing them, and nothing anyone does will change that...

from incrediblehelp 425 days ago #

Votes: 0 | Vote:

+ -

joost even if they delete the robots.txt line and even change from get to post, do you think these pages will have a tough time falling out of the index since they have alredy been crawled-->indexed?

from maverick12210 425 days ago #

Votes: 0 | Vote:

+ -

Hi Jane http://ciarannorris.co.uk/2008/05/05/indexing-search-results-why-google-is-going-against-its-own-advice/

The URL of the story is now returning a 404 error

Please look into it

Thanks

Maverick

from annie7 425 days ago #

Votes: 0 | Vote:

+ -

@maverick12210 works with me...

from janecopland 425 days ago #

Votes: 0 | Vote:

+ -

@maverick12210 Ah, it's not 404ing for me... might want to ask the author about it? He commented second on this thread.

from greggorio 422 days ago #

Votes: 0 | Vote:

+ -

Jane, great post. It appears that not only do they do this organically, they also do this in the PPC world when they recommend "non-quality score" methodologies for their adwords clients. Again, not practicing what they preach!

from Jill 422 days ago #

Votes: 0 | Vote:

+ -

That's the way Google has always shown partially indexed pages (PIPs). They know they exist, but they're not fully indexed. In this case, they're not fully indexed because they are excluded via the robots.txt file.

Been this way forever with excluded URLs...nothing new here...