Sorry this site requires JavaScript to be enabled in your browser. See the following guide on How to enable JavaScript in Internet Explorer, Netscape, Firefox and Safari. Alternatively you may be blocking JavaScript with an advert-related or developer plugin. Please check your browser plugins.

Matt Cutts reverses his opinion on pages indexed in Google that are nothing more than copies of auto-generated snippets.
Comments15 Comments  

Comments

Avatar
from Rhea 2047 Days ago #
Votes: 2

Great find! I’m e-mailing all of my clients right now to make sure they allow crawling of their search results pages.

Avatar
from NickWilsdon 2047 Days ago #
Votes: -1

I’m disappointed that this change to the robots.txt seems to be a pointless political gesture. I thought Obama was trying to keep it real and avoid the spin. Are they hoping the uneducated masses see this robots.txt change as supporting Obama’s new open information policy? Apparently yes.

Avatar
from mvandemar 2047 Days ago #
Votes: 0

Actually, Nick, this wasn’t political in the slightest. They wiped the old site and replaced it with a completely new one, which makes perfect sense. All of the printer friendly pages that composed the vast majority of the old robots.txt don’t exist anymore, and in fact none of the new pages even have print versions.What other people, including news sources such as the BBC, think about it is not the Whitehouse’s fault at all.

Avatar
from NickWilsdon 2047 Days ago #
Votes: 0

@MvandemarYep you maybe right, too many years of New Labour spin always has me assuming the worse. Poor reporting/understanding on the side of the BBC could be at fault.

Avatar
from Skitzzo 2047 Days ago #
Votes: 1

Lol it looks like Matt got caught up in Obama mania being out there in DC for the inauguration.Good catch Michael, I’m now off to try and get thousands of search pages indexed!

Avatar
from Harith 2047 Days ago #
Votes: -2

Take it easy, folks :-) Matt Cutts hasn’t Endorsed indexing of search results In Google, at all.Matt was talking about "Crawling" not about "Indexing". Read for yourself what Matt said:"You’ve probably seen this, but I like the new whitehouse.gov robots.txt: http://bit.ly/x4G7 Much more crawling allowed."And as you know there is a difference between "Crawling" and "Indexing" :-)

Avatar
from Skitzzo 2047 Days ago #
Votes: 0

Harith, true, but if the pages are all search pages and duplicate content printer-friendly pages, why would Google want to crawl them unless they intend to index them.Also, how would you keep search pages out of the index asside from using the robots.txt file? That’s pretty much the most efficient method that I can think of.

Avatar
from Harith 2047 Days ago #
Votes: 0

Skitzzo,I understand and appreciate :-)However, not all craweled materials would be indexed in Google. I.e. "Much more crawling allowed"  doesn’t necessary mean to index the whole crawled content. ;-)

Avatar
from mvandemar 2047 Days ago #
Votes: 2

Harith, Matt on multiple occasions has suggested using robots.txt to block this type of  auto-generated content, and suggests that if you don’t do so you are in violation of Google Webmaster Guidelines.The purpose of crawling is to find content to index, and the purpose of blocking it with robots.txt as he suggested is to keep it out of the index.I know he doesn’t actually want those in the serps, and so does everyone else. What Matt did was  he said something that sounded good at the time, instead of something that actually meant anything. The post was making fun of that. :)

Avatar
from Harith 2046 Days ago #
Votes: 0

mvandemar"The post was making fun of that. :)"However, I wouldn’t be surprised to see Matt chiming in:mvandemar, I don’t endorse indexing of search results on Google :-)

Avatar
from DivisiveCottonwood 2045 Days ago #
Votes: -1

In other breaking news: matt cutts fartsAbout somebody will write a blog post about that

Avatar
from MattCutts 2040 Days ago #
Votes: 0

I was only commenting on the fact that the new robots.txt allowed much more crawling. Which is why I used the word "crawling." Google’s position on indexing search results has not changed.

Avatar
from Mickelodian 2040 Days ago #
Votes: 0

Hold on, hold on... last time I checked my payslip had my company name at the top... Matt were you originally saying was that WE the seo’s and marketing people of the world should filter our own content because Google wasn’t up to the job with duplication?...but it is now so we don’t need to???If the engine can’t remove duplicates or has an issue with it then improve the engine... but for every in the know SEO there are a thousand legitimate sites which (for example) hold duplicate copy for print purposes or in PDF’s). You can’t be expecting these people to do the engines filtration work?

Avatar
from mvandemar 2036 Days ago #
Votes: 0

I was only commenting on the fact that the new robots.txt allowed much more crawling. Which is why I used the word "crawling." Google’s position on indexing search results has not changed.Actually, Matt, if you read he post I discussed where you highlighted the fact that the Google Webmaster Guidelines were changed to instruct people to block that kind content using robots.txt:http://www.mattcutts.com/blog/search-results-in-search-results/You did in fact advise people to block that content from getting crawled, and to do so to keep it out of the index.

Avatar
from MarkeD 1876 Days ago #
Votes: 0

Misinterpreting one 140 character tweet by Matt Cutts doesn’t reverse Google policy of many years, I would have to say this is false information

Upcoming Conferences

Search Marketing ExpoSearch Engine Land produces SMX, the Search Marketing Expo conference series. SMX events deliver the most comprehensive educational and networking experiences - whether you're just starting in search marketing or you're a seasoned expert.



Join us at an upcoming SMX event: