News Source: http://smackdown.blogsblogsblogs.com
Published: Jan 22, 2009 - 04:29 pm
Story Found By: mvandemar 1581 Days ago
Category: Vertical Search
15 Comments
Sorry this site requires JavaScript to be enabled in your browser. See the following guide on How to enable JavaScript in Internet Explorer, Netscape, Firefox and Safari. Alternatively you may be blocking JavaScript with an advert-related or developer plugin. Please check your browser plugins.
News Source: http://smackdown.blogsblogsblogs.com
Published: Jan 22, 2009 - 04:29 pm
Story Found By: mvandemar 1581 Days ago
Category: Vertical Search
15 Comments
Upcoming Conferences
Search Engine Land produces SMX, the Search Marketing Expo conference series. SMX events deliver the most comprehensive educational and networking experiences - whether you're just starting in search marketing or you're a seasoned expert.
Join us at an upcoming SMX event:
- SMX Paris - June 6-7 2013
- SMX Advanced - June 11-12, 2013
- SMX East - October 1-3, 2013
- SMX Stockholm - October 14-15, 2013
- SMX Milan - November 7-8, 2013
- SMX Social Media Marketing - November 20-21, 2013
- SMX Israel - January 2014
- SMX West - March 11-13, 2014
- SMX Munich - March 25-26, 2014
- SMX Sydney - April 2014
- SMX London - May 2014
Latest News From Marketing Land
- Marketing Day: May 22, 2013
- Google Analytics Updates Goals Set-Up Flow, Adds New Templates & Offers A Verification Option
- Twitter Adds Two-Factor Authentication Option Amid Recent Twitter Account Hijacks
- Official: Google Confirms “Conversational Search” Is Live In Chrome
- Twitter’s Lead Generation Card Now Live For Selected Brand Advertisers
- - See more news at Marketing Land -
Upcoming Webcasts
Learn more about search marketing with our free online webcasts and webinars from our sister site, Digital Marketing Depot. Upcoming online events include:
- Taking your PLA Campaigns to the Next Level - Jun 4
- Increase Shares, Tweets, Likes, and Pins to Fuel Search and Drive Engagement - May 30
- Take a Closer Look at Links – Why a Site Ranks - May 21
- The Future of Digital Marketing Services: Where Will Agencies be in 2014? - May 15
- Unlock the Key to Successful Forecasting and Conversions with Big Data - Jun 20
Research:
- Enterprise Social Media Management Software 2013
- B2B Marketing Automation Tools 2013: The Buyer’s Guide
- Enterprise SEO Tools 2013: A Buyer’s Guide
Whitepapers:
- PPC Bid Management Guide
- The Marketer’s Guide to Driving ROI From Facebook Advertising
- Collaborative Customer Interactions Management
- How To Create a Data-Driven Dynasty
- Top 10 Ways to Improve Digital Experiences
Latest News From Search Engine Land
- Penguin 4, With Penguin 2.0 Generation Spam-Fighting, Is Now Live
- SearchCap: The Day In Search, May 22, 2013
- DoubleClick Search Tools To Migrate To Enhanced Campaigns Now Out Of Beta
- Site Command Limited To 30 Results? Google Says A Temporary Domain Clustering Bug
- 2013 Doodle 4 Google Winner Showcases Heartfelt Reunion Between A Soldier & His Daughter
- - See more news at Search Engine Land -
Comments
Great find! I’m e-mailing all of my clients right now to make sure they allow crawling of their search results pages.
I’m disappointed that this change to the robots.txt seems to be a pointless political gesture. I thought Obama was trying to keep it real and avoid the spin. Are they hoping the uneducated masses see this robots.txt change as supporting Obama’s new open information policy? Apparently yes.
Actually, Nick, this wasn’t political in the slightest. They wiped the old site and replaced it with a completely new one, which makes perfect sense. All of the printer friendly pages that composed the vast majority of the old robots.txt don’t exist anymore, and in fact none of the new pages even have print versions.What other people, including news sources such as the BBC, think about it is not the Whitehouse’s fault at all.
@MvandemarYep you maybe right, too many years of New Labour spin always has me assuming the worse. Poor reporting/understanding on the side of the BBC could be at fault.
Lol it looks like Matt got caught up in Obama mania being out there in DC for the inauguration.Good catch Michael, I’m now off to try and get thousands of search pages indexed!
Take it easy, folks :-) Matt Cutts hasn’t Endorsed indexing of search results In Google, at all.Matt was talking about "Crawling" not about "Indexing". Read for yourself what Matt said:"You’ve probably seen this, but I like the new whitehouse.gov robots.txt: http://bit.ly/x4G7 Much more crawling allowed."And as you know there is a difference between "Crawling" and "Indexing" :-)
Harith, true, but if the pages are all search pages and duplicate content printer-friendly pages, why would Google want to crawl them unless they intend to index them.Also, how would you keep search pages out of the index asside from using the robots.txt file? That’s pretty much the most efficient method that I can think of.
Skitzzo,I understand and appreciate :-)However, not all craweled materials would be indexed in Google. I.e. "Much more crawling allowed" doesn’t necessary mean to index the whole crawled content. ;-)
Harith, Matt on multiple occasions has suggested using robots.txt to block this type of auto-generated content, and suggests that if you don’t do so you are in violation of Google Webmaster Guidelines.The purpose of crawling is to find content to index, and the purpose of blocking it with robots.txt as he suggested is to keep it out of the index.I know he doesn’t actually want those in the serps, and so does everyone else. What Matt did was he said something that sounded good at the time, instead of something that actually meant anything. The post was making fun of that. :)
mvandemar"The post was making fun of that. :)"However, I wouldn’t be surprised to see Matt chiming in:mvandemar, I don’t endorse indexing of search results on Google :-)
In other breaking news: matt cutts fartsAbout somebody will write a blog post about that
I was only commenting on the fact that the new robots.txt allowed much more crawling. Which is why I used the word "crawling." Google’s position on indexing search results has not changed.
Hold on, hold on... last time I checked my payslip had my company name at the top... Matt were you originally saying was that WE the seo’s and marketing people of the world should filter our own content because Google wasn’t up to the job with duplication?...but it is now so we don’t need to???If the engine can’t remove duplicates or has an issue with it then improve the engine... but for every in the know SEO there are a thousand legitimate sites which (for example) hold duplicate copy for print purposes or in PDF’s). You can’t be expecting these people to do the engines filtration work?
I was only commenting on the fact that the new robots.txt allowed much more crawling. Which is why I used the word "crawling." Google’s position on indexing search results has not changed.Actually, Matt, if you read he post I discussed where you highlighted the fact that the Google Webmaster Guidelines were changed to instruct people to block that kind content using robots.txt:http://www.mattcutts.com/blog/search-results-in-search-results/You did in fact advise people to block that content from getting crawled, and to do so to keep it out of the index.
Misinterpreting one 140 character tweet by Matt Cutts doesn’t reverse Google policy of many years, I would have to say this is false information