Sphinn - Googlebot Creates Pages Instead of Simply Indexing Them: New FORM Crawling Algo Goes Bad

56
Sphinn It!

Googlebot Creates Pages Instead of Simply Indexing Them: New FORM Crawling Algo Goes Bad

Went Hot: May 23, 2008 - 8:31 am
Posted By: mvandemar 407 days ago
Topic Type: News Story (Jump to http://smackdown.blogsblogsblogs.com)
Category: Google Other

Last month Google revealed a new crawling method that they were testing out, whereby they were filling out forms on sites that they came across, in order to help facilitate the discovery of new pages. Now there appears to be some unintended side effects from this.

5 Comments

Who Desphunn This (0)

Comments

from patrickaltoft 407 days ago #

Votes: 0 | Vote:

+ -

http://tinyurl.com/6j92qq

54 pages they have indexed from blogstorm by filling in the forms.

from mvandemar 407 days ago #

Votes: 0 | Vote:

+ -

Any new links discovered...? And did you know about this already Patrick?

from patrickaltoft 407 days ago #

Votes: 0 | Vote:

+ -

I knew some were being indexed but I didn't know 54 were until I read your post and tried a query. I'm going to leave it and see what happens.

from Gab 405 days ago #

Votes: 0 | Vote:

+ -

Wouldn't this all cause a canonicalization issue? For the mortgage site Mike highlighted, they've got a whole wack of indexed versions of their mortgage calculator page. Hmmm... new best practice for devs (to avoid the possibility of these canonical issues):

Frame any javascript forms on a page that you need to rank and block G from indexing the framed [form/js] page .

What do you guys think?

from mvandemar 405 days ago #

Votes: 0 | Vote:

+ -

Personally, since this has potential to hit sites that have no idea it's coming, I don't think asking webmasters to come up with the fix is fair.