- 59
- Sphinn It!
Posted By: jdevalk 220 days ago
Topic Type: News Story (Jump to http://eduardblacquiere.com)
Category: Google SEO
6 Comments
6 Comments
Save the date for:
SMX Singapore - July 2-3, 2009
SMX São Paulo - August 4-5
SMX East - October 5-7, 2009
SMX Stockholm - 12-13 October, 2009
SMX Mexico - November 11, 2009
Learn more about search marketing through free online webcasts and webinars from our sister site Search Marketing Now.
Comments
Google has multiple indices - the one we all see is full-text final index, not every URL gets there, however they do have complete web-graph (also index) of the web used in PageRank calculations and other things. Any URL they find gets into web-graph and if has enough PR or it's new domain (or whatever other rules are in place) it will be crawled, if successful and some outgoing links on that page are found then it MAY well pass PR without even being included into final full-text index.
Chances are that the pages that don't pass any PR are those that are penalised or had nofollow META tag in them or some other reasons.
You're right, thanks for your comment!
There's a definition problem with "non-indexed". Using Robots.txt or Meta robots to exclude pages from a search engine index DOESN'T keep Google from crawling the page and taking it into account with the PageRank calculation.
Robots.txt is a different situation from meta robots - if the page is excluded via robots.txt, then this prevents search engines from crawling it in the first place. Search engines cannot give any values to links they have never seen.
Ian, an URI never crawled by Google due to a robots.txt block can accumulate pageRank, and can get indexed.
@IanI agree with Sebastian :)
I think Ian probably meant here that if page can't be crawled then those outgoing links on it can't be seen by search engines and thus the page can't pass out PageRank or similiar algorithms.