Sorry this site requires JavaScript to be enabled in your browser. See the following guide on How to enable JavaScript in Internet Explorer, Netscape, Firefox and Safari. Alternatively you may be blocking JavaScript with an advert-related or developer plugin. Please check your browser plugins.

Sphinn Home » Searching
Indexing Search Results: Why Google Is Going Against Its Own Advice
Searching News Source: http://ciarannorris.co.uk
Published: May 05, 2008 - 01:34 pm
Story Found By: janecopland 1480 Days ago
Category: Searching
It now seems that Google is ignoring its own advice, and the use of noindex, and is indexing search results pages. If you look at Computer Weekly’s robots.txt file, you’ll see their search pages are noindexed. And yet the pages have been indexed.
 Comments9 Comments  

Comments

Avatar
from jdevalk 1480 Days ago #
Votes: 0

As commented on the site itself: 1. ciaran, and god knows I love you Ciaran, just doesn’t get how robots.txt and robots meta tags interact (hint: they don’t, the first blocks the latter), and doesn’t ask the obvious question of which weird internal link scheme get’s 105,000 internal serps indexed... 

Avatar
from ciaran 1480 Days ago #
Votes: 1

And Joost, as commented on the site, we may have been naive, but there is no internal linking on the site. They’re indexed because Google decided to start indexing them, and nothing anyone does will change that...

Avatar Moderator
from incrediblehelp 1480 Days ago #
Votes: 0

joost even if they delete the robots.txt line and even change from get to post, do you think these pages will have a tough time falling out of the index since they have alredy been crawled-->indexed?

Avatar
from maverick12210 1479 Days ago #
Votes: 0

Hi Jane http://ciarannorris.co.uk/2008/05/05/indexing-search-results-why-google-is-going-against-its-own-advice/The URL of the story is now returning a 404 errorPlease look into itThanksMaverick

Avatar
from annie7 1479 Days ago #
Votes: 0

@maverick12210 works with me...

Avatar
from janecopland 1479 Days ago #
Votes: 0

@maverick12210 Ah, it’s not 404ing for me... might want to ask the author about it? He commented second on this thread.

Avatar
from greggorio 1477 Days ago #
Votes: 0

Jane, great post. It appears that not only do they do this organically, they also do this in the PPC world when  they recommend "non-quality score" methodologies for their adwords clients. Again, not practicing what they preach!

Avatar Moderator
from Jill 1476 Days ago #
Votes: 0

That’s the way Google has always shown partially indexed pages (PIPs).  They know they exist, but they’re not fully indexed. In this case, they’re not fully indexed because they are excluded via the robots.txt file. Been this way forever with excluded URLs...nothing new here...

Avatar
from jdevalk 1047 Days ago #
Votes: 0

not that good, as the comments clearly show Ciaran just doesn’t get what’s causing this... sorry.

Upcoming Conferences

Search Marketing ExpoSearch Engine Land produces SMX, the Search Marketing Expo conference series. SMX events deliver the most comprehensive educational and networking experiences - whether you're just starting in search marketing or you're a seasoned expert.


SMX Web Site » | SMX Difference » | SMX News »

Join us at an upcoming SMX event:

Latest News From Marketing Land
Upcoming Webcasts

Search Marketing Now Learn more about search marketing with our free online webcasts and webinars from our sister site, Search Marketing Now. Upcoming online events include:


Latest News From Search Engine Land