Sorry this site requires JavaScript to be enabled in your browser. See the following guide on How to enable JavaScript in Internet Explorer, Netscape, Firefox and Safari. Alternatively you may be blocking JavaScript with an advert-related or developer plugin. Please check your browser plugins.

Everyday I am amazed by the countless number of pointless tags that are inserted into the header of a website.
Comments6 Comments  

Comments

Avatar
from g1smd 2444 Days ago #
Votes: 2

*** A robots.txt file is always the preferred method to exclude pages and also the method that most search engines will heed. ***That’s not quite true. Pages excluded by robots.txt can still accumulate PageRank, but they cannot pass it on anywhere because the content (and therefore the links) on the page will never be known.The meta tag "noindex" allows for the page to not appear in the index, but for the page to still pass the PageRank on to all of the other pages that it links out to. Every page needs a unique title and meta description. They need to describe exactly what that page is all about.You’ll also need the !DOCTYPE and some sort of CharSet declaration if the page is supposed to be a valid HTML document.The keywords meta tag isn’t much used for anything. Yahoo takes a peek at it, they say. I rarely use it at all.See also:  http://sphinn.com/story/18565

Avatar
from Halfdeck 2443 Days ago #
Votes: 2

"A robots.txt file is always the preferred method to exclude pages"Like g1smd mentioned, this is not true. We discovered recently that robots.txt is the wrong way to deal with duplicate content problems, for example, and supplemental results, which often arise due to PageRank dilution. Pages blocked by robots.txt still accumulate PageRank.Also see Johnmu’s detailed write up on meta tags.

Avatar
from JohnWeb 2443 Days ago #
Votes: 0

"Pages blocked by robots.txt still accumulate PageRank." That fact elludes many.

Avatar
from BartGibby 2438 Days ago #
Votes: 0

does anyone have any actaul strudies they have done on the robots.txt file still passing pagerank? Or an example of one?I have never seen this happen. please orivde proof of fact.I prefer to use the robots.txt file over the meta noindex / nofollow tag(s). But if three of you gusy say so I may have to concure.On the resoruces that were cited I couldn’t find anything talking about the robot.txt file still passing pagrank value.Please due enlighten me.

Avatar
from g1smd 2438 Days ago #
Votes: 0

By chance, Matt Cutts talked about it earlier in the month.Pages that have PageRank, pass some of it on to all of the URLs that they link out to. They do that even for URLs that are disallowed from spidering by robots.txt.Pages that are not able to be fetched, cannot be parsed, and therefore the links on the page cannot be discovered, and the Pagerank cannot be passed on to anywhere else.For pages that have a meta noindex tag, the page is fetched, the links are followed and pass PageRank on. The noindex tag prevents the page from appearing in the SERPs.

Avatar
from g1smd 2438 Days ago #
Votes: 0

The robots.txt file does not pass PageRank on. What we are talking about is URLs that have been disallowed in some way, that can or cannot pass PR on.That all depends on whether bots can see the links on the page or not. With robots.txt exclusion, they cannot.

Upcoming Conferences

Search Marketing ExpoSearch Engine Land produces SMX, the Search Marketing Expo conference series. SMX events deliver the most comprehensive educational and networking experiences - whether you're just starting in search marketing or you're a seasoned expert.



Join us at an upcoming SMX event: