Curiouser and Curiouser!

Curiouser and Curiouser! on pagerank http://matt.blogs.it/ RSS feed for topic pagerank Copyright 2006 Matt Mower Squib/0.1 self@mattmower.com self@mattmower.com en-gb PageRank and topics Sat, 10 Aug 2002 12:43:20 +0100 <A href="http://tig.nareau.com/2002/06/21.html#a196">The blogging network</A>. Here's a thought: Blogging is different from simple linking, directional or non-directional, for two reasons: firstly, whether one uses blogger and blogthis! or radio with its subscription list and blogrolls, or a personal directory, there is an inherent bias in the initial set, ie, following links is not random, rather its weighted by some initial set of sources. Haveliwala et.al. model topic specific PageRank using a basis set of topics, I wonder how one ought to characterize this bias in the case of the set of favorites, in the sense that links from favorites have a greater weight. Secondly, in blogging we have inclusion of full or part of a page or RSS item, and this can happen recursively in one of the loosely coupled conversations we see. Thus links already in the item reappear and get a larger weight. Such weights can be used to figure out authoritative sources, one would think, from the perspective of each blog.. [<A href="http://tig.nareau.com/">TIG's Corner</A>] <BLOCKQUOTE dir=ltr style="MARGIN-RIGHT: 0px"> » Good item (and thanks to John Robb for leading me to TIG's Corner).  I'm following the Haveliwala reference to a set of papers on PageRanking algorithms.  The math is way beyond me (did I really graduate math?)  but hopefully I can make some sense of it all. I have a strong interest in topics, and topic mapping as a way of structuring group knowledge.  Haveliwala (a Stanford researcher) has a paper <A href="http://dbpubs.stanford.edu/pub/showDoc.Fulltext?lang=en&doc=2002-6&format=pdf&compression=">here</A> on PageRanking mechansims that use a bias towards particular topics as a way of vectoring the PageRank toward results more relevant to those topics.  Interesting stuff.  </BLOCKQUOTE> http://matt.blogs.it/entries/00000295.html Has pagerank run it's course? Tue, 17 Sep 2002 07:23:11 +0100 Daniel Brandt: <A href="http://www.google-watch.org/pagerank.html">Google's Original Sin</A>. [<A href="http://www.scripting.com/">Scripting News</A>] <BLOCKQUOTE dir=ltr style="MARGIN-RIGHT: 0px"> » A good piece.  The main thrust is that Google's reliance on pagerank, far from being democratic, is uniquely autocratic.  Because sites with a high pagerank matter most, they have more power and it is harder for site with a low pagerank to get noticed regardless of the relevance of their onpage content. <BLOCKQUOTE> "In a democracy, every person has one vote. In PageRank, rich people get more votes than poor people, or, in web terms, pages with higher PageRank have their votes weighted more than the votes from lower pages. As Google explains, "Votes cast by pages that are themselves 'important' weigh more heavily and help to make other pages 'important.'" In other words, the rich get richer, and the poor hardly count at all. This is not "uniquely democratic," but rather it's uniquely tyrannical. It's corporate America's dream machine, a search engine where big business can crush the little guy."</BLOCKQUOTE> The remedy? <BLOCKQUOTE dir=ltr style="MARGIN-RIGHT: 0px"> "We feel that PageRank has run its course. Google doesn't have to abandon it entirely, but they should de-emphasize it. The first step is to stop reporting PageRank on the toolbar. This would mute the awareness of PageRank among optimizers and webmasters, and remove some of the bizarre effects that such awareness has engendered. The next step would be to replace all mention of PageRank in their own public relations documentation, in favor of general phrases about how link popularity is one factor among many in their ranking algorithms. And Google should adjust the balance between their various algorithms so that excellent on-page characteristics are not completely cancelled by low link popularity. "</BLOCKQUOTE> Even if I agree, and I not certain that I do, it's hard to see Google give up what they see as a key differentiator.  It's quite possible that they see an advantage for themselves in the tyranny of pageranks and the power of corporate America to wield them!</BLOCKQUOTE> http://matt.blogs.it/entries/00000548.html The PageRank lottery Tue, 17 Sep 2002 18:39:18 +0100 <A href="http://radio.weblogs.com/0108194/2002/09/17.html#a348">Relevance Ranking Discriminates Against the Irrelevant!</A>. <BLOCKQUOTE> <A href="http://radio.weblogs.com/0107808/2002/09/17.html#a391">Has pagerank run it's course?</A>. Ok, no. Pagerank is not "democratic" by the "every webpage is equally valuable" definition. But then, every web page is not equally valuable.</BLOCKQUOTE> The link real-estate on my webpage actually is worth a whole lot less than, say, the link real-estate on about.com or yahoo.com. The algorithm seems to me to be pretty accurate. Consider also that any webpage can become slashdotted. Anyone's site can become more relevant, interesting, well-linked, widely-read, in short, more "important". For instance, I've ego-surfed google (that is, searching on one's own name) for the last 6 months as I've been working on this blog, and have noticed that yes I have risen in the ranks of "Michael Wilson"s. And not because I'm "the rich", just because I post content that people read.Not only that, but any page can fall out of favor and drift back into obscurity. Perhaps an explanation on why it "should" be democratic is in order, 'cause I just don't get it. Seems to me like someone's just pissy that their pagerank isn't very high. [<A href="http://radio.weblogs.com/0108194/">The Universal Church Of Cosmic Uncertainty</A>] <BLOCKQUOTE dir=ltr style="MARGIN-RIGHT: 0px"> » Okay ya got me.  I can't get above 5 no matter how hard I try (I wonder if Bryan Bell has a strippers theme?) I agree that not every page should have equal rank and the whole "democratic vs." argument was probably a blind alley.  But I do think there are three valid points here: <UL> <LI>Unless your page rank is high for a query, you won't get listed.  Regardless of how relevant your on-page content is. <LI>It's very hard to get a high page rank, unless you already have a high page rank. <LI>New players don't have a high page rank.</LI></UL> This, I think, unfairly discriminates against new content providers over the established players. Perhaps a solution could be something like the US Green Card lottery.  For those not familiar this is a lottery for non-US (and non-UK) citizens where the prize is a much sought after green card for around 100,000 people a year. Perhaps Google should randomly bump the pagerank of 100,000 new pages each time it crawls and let nature takes its course... otherwise, but at the same time I think the article highlights the point that it can (if their data is good) be very hard to "break into" the relevant search criteria regardless of how</BLOCKQUOTE> http://matt.blogs.it/entries/00000555.html Genesis of PageRank Tue, 31 Dec 2002 21:15:33 +0000 I've been playing with <A href="http://www.groxis.com/">Grokker</A> Preview Release 2 this evening.  It's a big improvement over PR1 in many ways although I still wouldn't recommend that anyone other than a search tool nut buy it at this point.  However it did lead me to an interesting <A href="http://www-db.stanford.edu/~backrub/google.html">paper</A> by Sergey Brin about Google.  This was written in, I guess, 97/98 i.e. well before Google became the monster it is today. However it does have the best description of the page rank algorithm and how it is calculated that I have seen so far.  I'm guessing it's a good deal more sophisticated these days but this might be of interest for others like myself who wonder about the inner workings. To quote from that paper: <BLOCKQUOTE dir=ltr style="MARGIN-RIGHT: 0px"> <H4>2.1.1 Description of PageRank Calculation</H4> Academic citation literature has been applied to the web, largely by counting citations or backlinks to a given page. This gives some approximation of a page's importance or quality. PageRank extends this idea by not counting links from all pages equally, and by normalizing by the number of links on a page. PageRank is defined as follows: <BLOCKQUOTE>We assume page A has pages T1...Tn which point to it (i.e., are citations). The parameter d is a damping factor which can be set between 0 and 1. We usually set d to 0.85. There are more details about d in the next section. Also C(A) is defined as the number of links going out of page A. The PageRank of a page A is given as follows: PR(A) = (1-d) + d (PR(T1)/C(T1) + ... + PR(Tn)/C(Tn)) Note that the PageRanks form a probability distribution over web pages, so the sum of all web pages' PageRanks will be one.</BLOCKQUOTE> PageRank or PR(A) can be calculated using a simple iterative algorithm, and corresponds to the principal eigenvector of the normalized link matrix of the web. Also, a PageRank for 26 million web pages can be computed in a few hours on a medium size workstation. There are many other details which are beyond the scope of this paper.</BLOCKQUOTE> http://matt.blogs.it/entries/00001023.html