Calendar
Network
Feeds
- 51
- Sphinn It!
Went Hot: August 13, 2008 - 9:44 am
Posted By: UtahSEOpro 113 days ago
Topic Type: News Story (Jump to http://www.searchenginejournal.com) my network
Category: SEO
* moving the site to a new domain;
* changing a CMS;
* optimization process (for the sake of better crawl rate and usability); etc.
47 Comments
Who Sphunn This Topic?
-
stuntdubl
-
graywolf
-
DavidWallace
-
toddmintz
-
pratt
-
beussery
-
UtahSEOpro
-
BrettFromTibet
-
DJLitten
-
97thfloor
-
incrediblehelp
-
jeffquipp
-
lorenbaker
-
SEOinSeattle
-
SearchBuzz
-
krumel
-
drmadcow
-
EricLander
-
paisley
-
SexySEO
-
watkin29
-
mprough
-
rexolio
-
menudoman
-
baiduyou
-
gunjanpandya
-
inkodeR
-
annie7
-
thinkingserious
-
martinbowling
-
crashingflwrgrl
-
rajiwebstar
-
jmaulson
-
dobata
-
alinapopescu
-
Ghennipher
-
jgianoglio
-
MAndy17
-
Sana
-
pancham
-
yojpotter2
-
MarketingMinefield
-
StephenCronin
-
johnsantangelo
-
shocs
-
AndrewSpining
-
seowriter
-
chandi
-
AffiliateDragon
-
alicejingga
-
delcov
-
singh85
Who Desphunn This Topic what is desphinn?
Sugarrae
08/13/2008 08:52am
Again, not trying to be an ass... my long drawn out explanation can be seen in the comments.
streko
08/14/2008 08:58am
i am desphinning this because TheMadHat is an asshole for desphinning in the first place.
NikkiPilkington
08/13/2008 09:11am
I think it's misjudged advice, especially for any newbies out there. No proof of concept.
Comments
Oh come on! Encouraging 404 errors to "look natural and avoid penalty"? This is bad seo advice! There's no evidence presented in favour of "penalties" really existing for good housekeeping by 301 redirects. At the end of the day, sites change, sometimes in a big way. This post skips lightly across a highly technical subject with no evidence or expermentation to prove the hypothesis.
I really dont see any harm in using 301 redirects for 100s or 1000s of pages at all.
No harm doesn't mean it should be done. If we talk about low-quality (extra) pages (otherwise, why would we want to get rid of them?), what's the point in redirecting them? Doesn't it would be more natural to just delete them and clean up the site and that's it? I insist that every method should be based on common sense: if we have the reason to redirect (links, traffic, etc), then do it, if not, then why would you do that?
Ann if the post is just about deleting pages off your website then say so, but the fact is if you had a 1000 pages on your website and they were getting some clicks/traffic/or had some juice you should be redirecting them. The number of redirects shouldnt even be an issue here.
Ok, now I am just going to look like I am picking on Ann... I'm not... but,
"find most powerful pages and 301-redirect them, and let all the rest go 404."
That just doesn't make sense. If you had a change jar at home, would you advise someone to only remove the quarters, dimes and nickels and throw the pennies out?
I obviously agree that there are many possible ways you could handle dead pages:
http://searchengineland.com/080624-144233.php
But I just think the reasoning in this post - a hypothetical guess that it could cause a penalty with no supporting evidence and a correlation to search engines crawling sites with a lot of 404's better (backed up by a statement that "some webmasters" experienced this, with no citing of sources or internal testing).
This post seems like it was simply put together from reading multiple things on 404's without any actual internal testing or additional knowledge from real word implementation to know if the advice is actually true or not. If there are sources or tests or the like to actually back up the advice, it would do well to post it. That would definitely make for an interesting read.
Again, the post is about deleting the pages (most often it must mean, if you delete a page, it was of not much quality/ use) - in this case, a properly managed 404 page is really the best choice, I insist (except for those pages that still managed to accumulate some power which you don't want to lose).
Rae, which method would you choose in this case? Would you really use 301 across the board? Does it make sense, you think?
Everything should be done for a good reason: good page => redirect, bad page => just get rid of it but mind you 404... What's wrong with the approach?
>>>Rae, which method would you choose in this case? Would you really use 301 across the board? Does it make sense, you think?
I've actually had to do this several times... we buy a site and a lot of the pages are crappy (in our opinion) and we want to get rid of them. But, you know what? They get trickle traffic.
Say each page maybe only gets one visitor every two months... but there are 300 crap pages. I certainly am not throwing away 150 visits a month (of whom, like 50% or more will have toolbars sending Google about 75 or so signals of "yes, people visit this site!").
Or lets say there is an inbound link to every thirty pages... that is ten links. I am going to 301 to it's closest parent page or utilize a single 200 page that collects this minor juice (and human visitors) to spread it (or in Google approved fashion, sculpt the juice to flow) to other pages in place of a 404.
A penny on it's own is worth nothing either... it is all about the collective power of these deleted/dead pages in my mind.
Again, if there is actual information citing that these redirects cause a penalty or that a site with lots of 404 pages get crawled more frequently, I'd love to see it (not being a smartass, I simply love to learn and research). I am open to having my opinion changed - but need some type of rebuttal other than theory, to counter what we've seen in practice over and over if that makes sense.
A post from last spring on Google Webmaster Central blog states "...404 (File Not Found) errors can harm the user experience and negatively impact your site's performance in Google search results."
Doesn't seem clear whether the last part of that statement refers to ranking or simply SERP conversion.
Sometimes to gain, you have to lose something. It's not about "who is right", it's about different tactics, methods.
I've seen instances of multiple chained 301 redirects causing problems, but never heard of a large number of single redirects causing any issues. If so I have a lot of work to do.
Ann and Rae...
Ann.. on your article.. my advice would be 404, everything..
for your 404 page, make copy of html sitemap page.
unless you have need for specific redirects...
meaning..
www.domain.com/services.html
needs to point to:
www.domain.com/merger-acquisition-services.aspx
that is a completely different story.
in that case there is a way to do it correctly using refresh tags
(remmeber i said this last time you wrote an article kinda like this?)
where google will pass past page rank (ok, say that 3 times fast) to new pages. i.e. 5 year old page has Toolbar PR of 4 and new page retains same tool bar PR of 4, 6 weeks later.
Rae: i've done the research on 2k pages from www.domain.com(dirname)/ to subdomain.domain.com/same-page-name.html
and about 50 other websites, where either we have gone from .html to .htm (linux to win server), .html to .asp, .html to .php, .html to .aspx, I am also about to do a transfer from a .net domain to the same domain but the .com version, with all the pages and page names intact... and that experiment will begin... (checking DNS) , nope not yet... i'm guessing i will start the experiment on thursday after i get back from Ibiza.
i rarely use 301 until AFTER i've made the PR transfer.. then it's just a dump.
but then again i do my own experiments and that's actually a major part of my job.. SEO research.
Not very knowledgeable about this, but suppose you served [custom] 404s with links to your money pages... wouldn't that be a simple way of redirecting all that juice/visitors?
Gab,
then those would be different..
a 404 page is a default page that is served when the previous page no longer exists or someone typed in www.domain.com/bab.html instead of www.domain.com/GAB.html
a 404 page isn't really an error if you make it a "soft 404" which is a 404 page instead of nothing.
like this..
http://www2.victoriassecret.com/collection/callAppError.html?fp_redirected=1
@Gab, 404 page should be of course customized - valid point, thank you.
Personally, I use 301's and couldn't see myself just deleting a page. I guess it would all depend on the circumstances though.
Loving the in-depth discussion here ;)
>>>it's about different tactics, methods
I agree... but I am disagreeing based on various tests I've run on my own sites and information traded with other SEO's also experimenting... a method is a method providing there is some fact/experience behind it... otherwise, it isn't a method, it's a theory - and having one stated as fact/a proven method is what I took issue with. Ann has answered several other points, except the one asking where her opinions and statements in this article came from.
P.S. Gab, making a mock 404 that serves a 200 OK code to the engines to catch the juice is something I suggested here: http://searchengineland.com/080624-144233.php as a possible solution... but that is not truly a 404 (because it doesn't serve a 404 error code) and is not what is being recommended in the article on SEJ. That said, we've tried it a few times and are seeing some promising results... though we've had to be careful not to have the page optimized and ranking ;-)
FYI... after reading my stuff above, also.. i would read this...
http://googlewebmastercentral.blogspot.com/2008/08/farewell-to-soft-404s.html
also... stay tuned for further vagueness...
"Can your webserver return 404, but send a helpful "Not found" message to the user?
Of course! More info as "404 week" continues!"
actually... umm.. there is an error in the posting from google.. (yeah.. i know.. you would think there wouldn't be. but there is... )
ok... interesting.. a post on google webmaster central conflicts with something that happened last week using the methods i described above.
So either Google post is flawed or stuff changed since last week re: 404s.. but umm.. i don't think that's the case... hrmmm...
FYI.. i'm not looking at this from an SEO perspective.. (yes i do SEO), but i also have clients we built websites and launch on a weekly basis... and since July 1st, I have been in on the launching, (hng on, getting exact number..).. 5 websites that have new designs, page names, (i am an SEO after all), etc.. i've also transferred 3 websites out of CMS systems to static HTML pages (DW contribute as CMS)
So what i have posted above is from my personal experience doing 500 + websites, not stuff i read on the web somewhere.. we buy/negotiate domain purchases, tranfers and google re-insertion, (moving from old domain to new domain, while still keeping rankings and the old domain works, google just doesn't spider it anymore.)
but if the google post is 100% true.. then... hmm.. ohh... i feel an experiment coming on...
be back in a few.. time to set up the parameters, hmm.. anyone have any ideas for controls?
so.. take page.. do 301 on one page..
damn.. how can you do blanket 404.html page (soft error) vs. 404 hard error? on the same domain..
i'm guessing subdomains won't count.. damn..
ok while i think.. anyone??
I can tell you from experience (working with a site that add and removes hundereds of pages daily) that the best and most appropriate way to get pages removed wihtout penalty is a correct 404 or 403 header response. This tells the search engines to stop indexing the page. I belive 301 redirects are missleading if the end user is looking for one thing and lands on a totally different page. On the other hand, I think a javascript redirect is fine as long as the correct header response is given and the user is made aware that the page no longer exists and they are being redirected.
Don't you just love to hear varying opinions on the subject!
I agree with Sugarrae that it comes down to experimentation. I managed a site that went through a big change - a lot of pages replaced but with new names leaving tons of 404 errors. Google doesn't remove those pages from it's index over night. Additionally, assuming that there may be links to those pages both internally and externally, you're not managing the site well if you allow 404's to stand.
Since that change the site was doing poorly in search. When we isolated all of the 404's and 301'ed them on a page by page basis (redirecting them to the new version of the page or something as close to it) it wasn't long before we starting coming back strong in search.
Google knows, either through Analytics or their toolbar (I believe) that your site has these errors. I believe these are true errors and shows a lack of organization and managment. That just can't serve you well in most cases.
And I'd rather take a chance that I'm passing on importance/link juice to a new page and making the best attempt to put the user infront of a relevant page then to do nothing at all.
It's only my opinion. Great conversation!
ok.. i have the experiment..
if anyone wants to try....
Criteria for Experiment:
1. same domain.
2. 4 pages, all must have toolbar pagerank.. (either create and make toolbar pagerank, or use existing ones. I'm going to be using existing ones..)
3. one page has 301 redirect to new version of same page.. as in page.html 301s to page1.html
4. one page uses 404 death error.. say fof.html and create same page content and put it on fof2.html (using fof.html because me thinks Google can pick up 404page.html, etc.)
5. one page uses 404 "page" which only needs to be copy of sitemap with upper left most link that goes to new version of same page...
6. other page will use a refresh tag combination i use with no problems...
You should never let a page go 404 if you can do a 301. That's elementary site management.
Only when you delete content should a 404 display.
end of.
different method: custom 404 error page that contains sitemap (small website) and/or matches via internal search to another page (large website)
"I've seen instances of multiple chained 301 redirects causing problems" -- issues a warning in Webmaster Tools
shaynorulz... according to last google webmaster post on 404s, (the link above) you aren't supposed to do that.. (a soft 404 redirect).. (what i have an issue with the post about)
301 means "moved permanently" - it is even not logical to use it if you mean to just delete the low-quality page. It's more about common sense than experimentation!
Sorry, missed a good part of this party (due to time differences).
@Rae, as far as you insist on my sharing my experience, I can tell you that I never saw any penalties either from using 404 or 301 - tried both in various cases. I've had cases when plenty of 404 popped up in a day and no ranking (or other) change followed. So, no, I don't believe that 404 can be harmful or that it is not user or bot friendly (I mean proper 404).
So yes, you were correct saying I was citing other people's experience (just because I agree that they might have seen something I failed to see) - and I gave the link to the related discussion and emphasized that 301-penalties are just rumors ("might cause penalties").
The solution I offered for the discussed case still seems most reasonable as there should be a good reason behind using 301 (related pages, pages created as substitute, etc).
@paisley: the thread at Google blog came just in time - thank you for link to it here - much food for thought.
>>>301 means "moved permanently" - it is even not logical to use it if you mean to just delete the low-quality page. It's more about common sense than experimentation!
And a nofollow means "I don't trust this link because I can't vouch for it" yet, you're fine with using it to sculpt pagerank (which is not what nofollow MEANS and is simply a way to utilize it for SEO) but you're argument to back up Ann is that we shouldn't utilize a 301 for SEO because that "isn't what it means"?
I guess you need Rand or Matt to say it's ok to use something in a different way than it is intended for you to be fine with it.
SEO is more about common sense than experimentation? At least that statement gives me a feeling of income security.
@Ann You might consider your solution most reasonable, I consider it wasteful for the many reasons I mentioned. I have no problem disagreeing - just that I am disagreeing based on experience, and even though your own experience you just mentioned lines up with mine, the article comes off citing the experience of others as fact... would just rather see it stated as opinion/specualtion and not that "if you don't 404 deleted pages, you're going to get a penalty" based on the article title and content.
@Rae, I am sorry I sounded the way you just pointed out - I didn't mean to. I also have no problem with disageeing at all but I do have problem with people deliberately misinterpreting my words which I hope you didn't.
>>>but I do have problem with people deliberately misinterpreting my words which I hope you didn't
It would seem to me with that statement that you're calling into question whether or not I deliberately "misinterpreted" your statement there. I tried to extend an olive branch in regards to agreeing to disagree on our professional opinions and you smack me in the face with it.
Ann, if you had ever met me, you'd know if I wanted to "call you out" I would and I don't need to disagree with a post as politely as I can to do so.
Your last comment was simply a personal digg where every post prior to it of MINE was a post regarding your opinion on SEO and not you as a human being. And since you don't know me, I find the insinuation bullshit.
rae: i'm thinkin.. she accepted your olive branch.. but then again.. i think rain is wet..
@Rae, my concern was that you "were citing" what I did not write. I did not say "use 404 or get penalized" like you posted above, similarly, I honestly linked to other people's opinions and never said that was the only way to go.
The goal of the post was not only to represent the only possible solution (which is impossible) but also to outline different related opinions because, unlike you, I don't believe there is one definitive answer to the posed question.
I believe you were trying to disagree politely but to me it sounded like misinterpretation (just like to you I sounded like digging).
I think I like Olive Garden. I'm also a lier and a cheat. I agree to disagree with myself.
You know, since we're back to the sphinn name calling BS I figured I'd call myself an asshole.
Asshole.
Back to the topic:
I 301 every page somewhere, never return 404, I don't consider it to be spammy, and finally I've done it many 10's of times and never suffered in the SERP's for it.
Its my body... I mean site damn it! :)
@TheMadHat the olive garden is okay - i actually prefer the "Girls of the Olive Garden" Playboy magazine edition that came out.
And I will also call you an asshole just for kicks.
Asshole.
You titled your post: How to Get Rid of Multiple Subpages & Not Get Penalized
Then you ended the post with: " The best way-out is to try something in-between: find most powerful pages and 301-redirect them, and let all the rest go 404."
Bold your emphasis, not mine. There was no "deliberate" misinterpretation or accidental interpretation of your article gist. Just a logical conclusion.
>>>because, unlike you, I don't believe there is one definitive answer to the posed question.
The point of my posts above seem to have flown over your head or are being purposely ignored. I don't believe there is one definitive answer. Your article leading people to believe they need to 404 a deleted page to avoid penalty based on specualtion is what I took issue with.
And now I am off to do things that make me money for the day.
@Rae, ok, now this is proper citing - out of context but still citing.
As you say there was no deliberate misinterpretation, there wasn't - you know better.
Your points about losing traffic because of 404 are clear - I say in some cases it is against user experience. 301 should preferably be used for highly relevant pages: when I want to see page B and get to an absolutely other (unrelated) page, I get distracted...
Besides, 301 for multiple pages would be a headache (if not impossible) to redirect.
All in all, on the second thought, the title of the post is really misleading (I can admit my mistakes) but the reasoning behind the method I suggested is still valid.
"Besides, 301 for multiple pages would be a headache (if not impossible) to redirect."
huh?
"Besides, 301 for multiple pages would be a headache (if not impossible) to redirect."
I don't want to misinterpret that, but uh, it's called mod rewrite. unless you manually redirect every page which is like, so 1997.
Hate to be a "me too" person, but I agree with everything Rae has said here.
I believe it does the search marketing and web design industries a disservice when people make it sound like Google gives penalties for doing things that are a natural thing to do when you redesign a website, i.e., redirecting any URL that once existed to a new one that is the equivalent, or at least close enough in nature.
If a URL ever existed on the website, it's almost always a good idea to redirect it somewhere else rather than make it a 404, in my opinion. I imagine there may be a few exceptions, but I can't think of any at the moment. And an alleged Google penalty is certainly not on my list of reasons.
yep, logic based (dynamic) 301 redirects can be done of course.
"emphasized that 301-penalties are just rumors"
Internal 301 penalty is a myth. All you're doing is shifting dollar bills from one pocket to the other. Doing that will not make you more money; Google knows that and so doesn't care.
301 redirecting from one domain to another, on the other hand, can trigger a penalty depending on the intent of the redirect. That's one reason Aviva directory got Google-wacked.
In the WMW thread SER post sited, tedster is not talking about internal 301s.
http://www.webmasterworld.com/google/3706937-2-30.htm
One of his responses was to this comment:
"A whole bunch of redirects were sent off another site into the released site to reposition the content correctly. 4 weeks later it disappeared again and has only managed a minus penalty status in recent months."
Note that the poster is only saying A before B, which doesn't prove A caused B.
One of my clients wanted to buy a domain and redirect it to his main site for link building. That's the kind of thing where you're rolling the dice.
This issue is basically a tug-of-war between SEO and user-experience. If all you care about is preserving link juice/traffic, I would 301 redirect urls with backlinks and delete the rest. If user-experience matters more to you, other options open up. Which ever way you choose, Google penalty isn't a concern.
"Use Robots.txt to block non-existent pages if you don’t want them to be crawled"
If Google crawls "non-existent" pages, that generally means there are links to those pages. IMO a better solution is not to disallow them but to find out where the links are coming from. If internal pages are linking in, drop those links; if external sites are linking in, 301 redirect.
"One of my clients wanted to buy a domain and redirect it to his main site for link building. That's the kind of thing where you're rolling the dice."
From my experience the odds are strongly in your favor.
what a wonderfully scientific debate.
i wish we didnt have to experiment like chemists over basics of the process :(