Sorry this site requires JavaScript to be enabled in your browser. See the following guide on How to enable JavaScript in Internet Explorer, Netscape, Firefox and Safari. Alternatively you may be blocking JavaScript with an advert-related or developer plugin. Please check your browser plugins.

Has anyone else here heard about this (apart from Danny Sullivan that is)?
Comments22 Comments  

Comments

Avatar
from qwerty 3774 Days ago #
Votes: 2

Sphunn, reddited, stumbled and delicioused. This is pretty important stuff.

Avatar
from skinner 3774 Days ago #
Votes: 0

Yes I agree. There should be a hall of infamy somewhere where we put the details of people who do this stuff.

Avatar
from DanThies 3774 Days ago #
Votes: 1

It’s actually come up in a few places, I’m publishing because it’s not some isolated problem that only happens to a few underlinked MFA sites.

Avatar Moderator
from graywolf 3774 Days ago #
Votes: 1

Nice job with the tips on how to prevent/fix it at the end of the article.

Avatar
from DanThies 3774 Days ago #
Votes: 0

Thanks, Michael. So far, knock on wood, the reverse cloaking has held up for several months with every site.

Avatar
from g1smd 3774 Days ago #
Votes: 3

This has been going on for years, as the article says, but to allude that only "a few people" knew about it "and kept quiet for the good of the community" is plain false I think. It has been discussed in several public forums many times over the last few years. There is much prior information out there; but this article does a very good job of putting all the facts in one place in an orderly manner.

Avatar
from majorbta 3774 Days ago #
Votes: 0

wow

Avatar
from DJLitten 3774 Days ago #
Votes: 0

Wonderful article Dan. I’m looking forward to see what comes of this.

Avatar
from simplepixel 3774 Days ago #
Votes: 0

Great article. I have a question though. The work around is to basically validate that the search engine bots are in fact who they say they are via IPs, and if not include a noindex for proxies. Correct? What is to prevent the proxies from just stripping out the noindex from pages? Especially if they are setup for malicious purposes. I can see it working to prevent the typical proxy, but it seems to me that the major issue is when these are used in the form an attack on a website’s position in the search engines.

Avatar
from DanThies 3774 Days ago #
Votes: 0

Simple, this is addressed in comments on the post as well... but the "advantage" of this exploit for black hats is that it’s hands-off. They aren’t creating their own proxies, they are exploiting someone else’s proxy server(s), and it appears to be a numbers game - you need to use a bunch of proxies to get the job done. To deploy even one (much less hundreds or thousands) of proxy servers that strip meta tags would mean that you need to get them hosted somewhere. Which makes it a lot less "hands off." You could get caught. A guy like Brad Fallon could send lawyers (or worse) after you.

Avatar
from simplepixel 3774 Days ago #
Votes: 0

Thanks Dan, I guess I didn’t make it all the way through the comments.

Avatar
from DanThies 3774 Days ago #
Votes: 0

Can’t say I blame you - a few of us are writing mini-novels over there!

Avatar
from g1smd 3774 Days ago #
Votes: -1

I am not sure what to make of http://www.bradfallon.com/linkrequest.html and I do wonder if any of that is part of his downfall...

Avatar
from DanThies 3774 Days ago #
Votes: -1

Yeah... that’s why he got hacked all right.

Avatar Administrator
from dannysullivan 3774 Days ago #
Votes: 1

Wish I could say I did all the behind the scenes stuff, Dan -- but really, I’ve mainly added to the other voices that have expressed concerns over domain hijacking and the need to understand what’s an "original" site.

Avatar Administrator
from MattMcGee 3774 Days ago #
Votes: 0

If this has truly been ignored or even back-burnered at Google, Yahoo, etc. ... shame on them. Great write-up, Dan -- you made it easy to understand.

Avatar
from DanThies 3773 Days ago #
Votes: 0

Danny, thanks for clearing that up. I guess you creating a platform (the Bot Obedience panels) was good enough. :D

Avatar
from IncrediBILL 3772 Days ago #
Votes: 0

Yup, this is all old news as we covered this topic at Bot Obedience panels in SES San Jose ’06 and SES Chicago ’06 and even in PubCon ’06 with Google’s Vanessa Fox sitting right there. Google did make good on their promise to provide a way to accurately detect Googlebot so we could stop the spoofing when the Googlebot user agent is passed through the proxy but if it’s filtered out you’re still in trouble. I’ve personally not seen hard evidence of that happening until Dan’s article and I’m not sure the user agent was filtered our or they cached copies and then let Googlebot crawl their cache, but the net result is the same. If Google would just properly attribute content to the rightful owner, where they found it in the first place, and not give the second instance encountered ownership, then the proxy issue and full content scraping forcing your content into duplicate penalties would no longer be a problem. The problem is how do you identify it’s truly your content in such a way that someone else stealing your content also couldn’t duplicate that ID. The simplest suggestion I have has always been that Google would have to add a push/pull mechanism to where you register the page and they instantly pull it, just like the AdSense mediabot does currently. Since the first person to PUSH the content wins ownership your blog would have to wait until Google confirmed receipt of the new content before you could publish it. Guess what I’m advocating is kind of like a real-time interactive version of Google sitemaps which would solve this once and for all.

Avatar
from seopractices 3770 Days ago #
Votes: 1

Great article Dan, a little bit late though, anyways you have the credit... Hopefully many other SEO gurus will start revealing their secrets, ha!

Avatar
from loki 3768 Days ago #
Votes: 0

not an easy decision but thanks for thinking it through so thoroughly.

Avatar
from RoseDesRochers 3753 Days ago #
Votes: 0

Great article. Thank you for submitting it.

Avatar
from Carter 3747 Days ago #
Votes: 0

Wow. It’s great to have this information, and...wow. Thanks to all involved.

Upcoming Conferences

Search Marketing ExpoSearch Engine Land produces SMX, the Search Marketing Expo conference series. SMX events deliver the most comprehensive educational and networking experiences - whether you're just starting in search marketing or you're a seasoned expert.



Join us at an upcoming SMX event: