PageRank

Math 123: Mathematical Aspects of Data Analysis – Fall 2026

Published

October 13, 2026

Where we are

So far, our data has been a point cloud: $n$ observations $\bfx_1,\ldots,\bfx_n\in\Rbb^d$, stacked as the rows of a data matrix $\bfX$. PCA asked which subspace the points live near, and regression asked how one feature depends on the others.

This week we change the question in two ways.

Sometimes the data is not a list of points but a list of relationships: which web pages link to which, who is friends with whom, which proteins interact. The natural object is then a graph, and the questions are about its structure: which vertices are important, and which groups of vertices belong together.
Even for point clouds, we can ask for groups instead of directions: can the points be split into a few clusters of similar observations?

The two threads meet later in spectral clustering, where a point cloud is turned into a graph and the graph is split using eigenvectors. Today we cover Section 5.1 of the textbook, PageRank. Next time we turn to $k$-means clustering (Section 5.3), and after that to the basic vocabulary of graphs (Section 5.2).

Throughout, $\mathbf{1}\in\Rbb^n$ is the all-ones vector, $\bfe_i$ is the $i$-th standard basis vector, and $[n]=\{1,\ldots,n\}$.

PageRank

A search engine has to do three things: find web pages, index them by the words they contain, and decide which of the matching pages to show first. PageRank, the algorithm behind early Google, is an answer to the third problem. The idea is that a page should be important if important pages link to it. This sounds circular, and it is: the circularity is exactly what turns ranking into an eigenvector problem.

Ranking by votes

Consider a web of $n$ pages, where page $k$ links to page $j$ if there is an arrow $k\to j$. Figure 1 shows a toy example with $n=4$.

Figure 1: A toy web with four pages (Figure 5.1 of the textbook). An arrow $k\to j$ means page $k$ links to page $j$.

The links into a page are its backlinks. The simplest score is the number of backlinks: here pages $1,2,3,4$ have $2,1,3,2$ backlinks, so page $3$ wins. This treats every link as one vote, which ignores two things.

A link from an important page should count for more than a link from an obscure one. Pages $1$ and $4$ both have two backlinks, but page $1$ is linked from the popular page $3$, while page $4$ is linked from the unpopular page $2$.
A page should not gain influence just by linking to everything. Page $1$ links to three pages, page $3$ to only one; a link from page $3$ is a more deliberate endorsement.

Both are fixed by the following rule. Let $x_k\geq0$ be the score of page $k$, let $n_j$ be the number of outgoing links of page $j$, and let $L_k$ be the set of pages linking to $k$. Each page splits its own score evenly among the pages it links to: \[ x_k=\sum_{j\in L_k}\frac{x_j}{n_j}, \qquad k=1,\ldots,n. \] For the toy web this reads \[ x_1=\frac{x_3}{1}+\frac{x_4}{2}, \quad x_2=\frac{x_1}{3}, \quad x_3=\frac{x_1}{3}+\frac{x_2}{2}+\frac{x_4}{2}, \quad x_4=\frac{x_1}{3}+\frac{x_2}{2}, \] which is the linear system $\bfA\bfx=\bfx$ with \[ \bfA=\begin{pmatrix} 0 & 0 & 1 & \tfrac12\\ \tfrac13 & 0 & 0 & 0\\ \tfrac13 & \tfrac12 & 0 & \tfrac12\\ \tfrac13 & \tfrac12 & 0 & 0 \end{pmatrix}. \] In words: we want an eigenvector of $\bfA$ with eigenvalue $1$.

Definition 1 (Link matrix) For a web of $n$ pages in which every page has at least one outgoing link, the link matrix $\bfA\in\Rbb^{n\times n}$ has entries \[ A_{jk}=\begin{cases}1/n_k&\text{if page $k$ links to page $j$,}\\0&\text{otherwise.}\end{cases} \] Column $k$ describes where page $k$ sends its vote.

Definition 2 (Column-stochastic matrix) A matrix $\bfA\in\Rbb^{n\times n}$ is column-stochastic if all its entries are non-negative and every column sums to $1$, that is, $\mathbf{1}^\top\bfA=\mathbf{1}^\top$.

Every link matrix is column-stochastic, since column $k$ has $n_k$ entries equal to $1/n_k$. Is $1$ always an eigenvalue of such a matrix, and is it the right one to look at? The key tool is the $\ell_1$ norm $\|\bfv\|_1=\sum_i|v_i|$.

Lemma 1 (Column-stochastic matrices do not increase the $\ell_1$ norm) Let $\bfA$ be column-stochastic. For every $\bfv\in\Rbb^n$ (or $\Cbb^n$), \[ \mathbf{1}^\top\bfA\bfv=\mathbf{1}^\top\bfv \qquad\text{and}\qquad \|\bfA\bfv\|_1\leq\|\bfv\|_1. \]

Proof

Proof. The first identity is $\mathbf{1}^\top\bfA=\mathbf{1}^\top$ applied to $\bfv$. For the second, use the triangle inequality, $A_{ij}\geq0$, and then swap the order of summation: \[ \|\bfA\bfv\|_1 =\sum_{i=1}^n\Big|\sum_{j=1}^nA_{ij}v_j\Big| \leq\sum_{i=1}^n\sum_{j=1}^nA_{ij}|v_j| =\sum_{j=1}^n|v_j|\underbrace{\sum_{i=1}^nA_{ij}}_{=1} =\|\bfv\|_1. \]

Proposition 1 (The spectrum of a column-stochastic matrix) Let $\bfA$ be column-stochastic. Then $1$ is an eigenvalue of $\bfA$, and every (possibly complex) eigenvalue $\lambda$ of $\bfA$ satisfies $|\lambda|\leq1$.

Proof

Proof. Column-stochastic means $\bfA^\top\mathbf{1}=\mathbf{1}$, so $1$ is an eigenvalue of $\bfA^\top$ with eigenvector $\mathbf{1}$. A matrix and its transpose have the same characteristic polynomial, since $\det(\bfA-\lambda\bfI)=\det\big((\bfA-\lambda\bfI)^\top\big)=\det(\bfA^\top-\lambda\bfI)$. So $1$ is also an eigenvalue of $\bfA$ (with a different eigenvector, in general).

If $\bfA\bfv=\lambda\bfv$ with $\bfv\neq\mathbf{0}$, then by Lemma 1, \[ |\lambda|\,\|\bfv\|_1=\|\lambda\bfv\|_1=\|\bfA\bfv\|_1\leq\|\bfv\|_1, \] and dividing by $\|\bfv\|_1>0$ gives $|\lambda|\leq1$.

For the toy web, solving $(\bfA-\bfI)\bfx=\mathbf{0}$ and normalizing so the entries sum to $1$ gives \[ \bfx=\frac{1}{31}\begin{pmatrix}12\\4\\9\\6\end{pmatrix}. \] Page $1$, not page $3$, comes out on top! Page $3$ has the most backlinks, but it casts its whole vote for page $1$, and that is enough to lift page $1$ above it. (The other eigenvalues of $\bfA$ are about $-0.36\pm0.41i$ and $-0.28$: complex, but all of modulus less than $1$, as promised.)

Remark (An old idea)

Remark (An old idea). Ranking by eigenvectors is older than the web. Edmund Landau proposed it in the 1890s for ranking players in chess tournaments, where beating a strong player should count for more than beating a weak one. It was his first mathematical paper, written at age 18.

The random surfer

There is a second way to read the equation $\bfA\bfx=\bfx$. Imagine a web surfer who, at each step, clicks one of the links on the current page uniformly at random. If page $k$ has $n_k$ links, the probability of moving from page $k$ to page $j$ is $A_{jk}$. So if $\bfx^{(t)}$ is the probability distribution of the surfer’s location at time $t$, then \[ x^{(t+1)}_j=\sum_{k=1}^nA_{jk}x^{(t)}_k, \qquad\text{that is,}\qquad \bfx^{(t+1)}=\bfA\bfx^{(t)}. \] Write \[ \Delta_n=\Big\{\bfx\in\Rbb^n:\ x_i\geq0\text{ for all }i,\ \sum_{i=1}^nx_i=1\Big\} \] for the set of probability vectors, the probability simplex. By Lemma 1, $\bfA$ maps $\Delta_n$ into itself: entries stay non-negative and the total stays $1$. A vector $\bfx\in\Delta_n$ with $\bfA\bfx=\bfx$ is a stationary distribution: if the surfer’s location has this distribution, it keeps it forever. So the PageRank score of a page is the long-run fraction of time a random surfer spends there.

This view also suggests how to compute $\bfx$ for a web with billions of pages: start anywhere in $\Delta_n$ and repeatedly apply $\bfA$. This is the power method, and each step costs only one multiplication by the sparse matrix $\bfA$. Why does this find an eigenvector? Here is the idea for a symmetric matrix (something similar holds much more generally).

Proof

Proof. Write $\bfv=\sum_ic_i\bfq_i$ in the eigenbasis, so $c_1\neq0$. Each multiplication by $\bfA$ scales the $i$-th coordinate by $\lambda_i$, so \[ \bfA^t\bfv=\sum_{i=1}^nc_i\lambda_i^t\bfq_i =c_1\lambda_1^t\Big(\bfq_1+\sum_{i\geq2}\frac{c_i}{c_1}\Big(\frac{\lambda_i}{\lambda_1}\Big)^t\bfq_i\Big). \] Every ratio $|\lambda_i/\lambda_1|$ with $i\geq2$ is less than $1$, so the sum in parentheses shrinks to $\mathbf{0}$, and what remains points along $\bfq_1$. For a careful proof and convergence rates, see the power iteration section of Fundamentals of Numerical Computation.

The link matrix is not symmetric, but the same idea applies: we will see directly in Theorem 1 why the iterates converge for PageRank. The gap $|\lambda_1|>|\lambda_2|$ matters, as the third example below shows.

What can go wrong

For a real web, the plain link matrix has three problems.

Dangling pages. A page with no outgoing links (a PDF, say) gives a zero column, so $\bfA$ is not column-stochastic. For the web $1\to2\to3$, where page $3$ links nowhere, $\bfA^3=\mathbf{0}$, so $\bfA$ has no eigenvalue but $0$: the surfer falls off the web, and there is no ranking at all. The standard patch is to pretend a dangling page links to every page, replacing its column by $\frac1n\mathbf{1}$.
Non-uniqueness. If the web is two disconnected pieces, say $1\leftrightarrow2$ and $3\leftrightarrow4$, then every vector $(a,a,b,b)$ with $a,b\geq0$ and $2a+2b=1$ is stationary. The eigenvalue $1$ has a two-dimensional eigenspace, and the ranking of page $1$ against page $3$ is meaningless.
No convergence. For the web $1\leftrightarrow2$, $\bfA=\left(\begin{smallmatrix}0&1\\1&0\end{smallmatrix}\right)$ has eigenvalues $1$ and $-1$. Starting from $\bfx^{(0)}=\bfe_1$, the surfer alternates between the two pages forever, and the power method never settles down.

Teleportation

The fix used by Google is to let the surfer get bored. At each step, with probability $\beta$ the surfer follows a random link as before, and with probability $1-\beta$ they teleport to a page chosen uniformly at random.

Definition 3 (Google matrix) Let $\bfA$ be a column-stochastic link matrix (with dangling pages already patched) and fix a damping factor $\beta\in(0,1)$. The Google matrix is \[ \bfG=\beta\bfA+\frac{1-\beta}{n}\mathbf{1}\mathbf{1}^\top. \] The PageRank vector is the stationary distribution of $\bfG$: the $\bfx^\ast\in\Delta_n$ with $\bfG\bfx^\ast=\bfx^\ast$. The value used by Google was reportedly $\beta=0.85$.

Teleportation fixes all three problems at once, and the proof is short. The heart of it is that $\bfG$ shrinks vectors whose entries sum to zero, such as the difference of two probability vectors.

Theorem 1 (PageRank is well defined, and the power method finds it) Let $\bfG$ be a Google matrix with damping factor $\beta\in(0,1)$.

$\bfG$ is column-stochastic, and $\|\bfG\bfv\|_1\leq\beta\|\bfv\|_1$ for every $\bfv\in\Rbb^n$ with $\mathbf{1}^\top\bfv=0$.
There is exactly one $\bfx^\ast\in\Delta_n$ with $\bfG\bfx^\ast=\bfx^\ast$, and every entry satisfies $x^\ast_i\geq(1-\beta)/n>0$.
For every starting point $\bfx^{(0)}\in\Delta_n$, the power iterates $\bfx^{(t)}=\bfG^t\bfx^{(0)}$ satisfy \[ \|\bfx^{(t)}-\bfx^\ast\|_1\leq2\beta^t. \]

Proof

Proof. Part 1. The entries of $\bfG$ are non-negative, and $\mathbf{1}^\top\bfG=\beta\mathbf{1}^\top\bfA+\frac{1-\beta}{n}(\mathbf{1}^\top\mathbf{1})\mathbf{1}^\top=\beta\mathbf{1}^\top+(1-\beta)\mathbf{1}^\top=\mathbf{1}^\top$. If $\mathbf{1}^\top\bfv=0$, the teleportation term vanishes, $\bfG\bfv=\beta\bfA\bfv+\frac{1-\beta}{n}\mathbf{1}(\mathbf{1}^\top\bfv)=\beta\bfA\bfv$, so $\|\bfG\bfv\|_1=\beta\|\bfA\bfv\|_1\leq\beta\|\bfv\|_1$ by Lemma 1.

Uniqueness. Suppose $\bfx,\bfy\in\Delta_n$ are both fixed by $\bfG$. Then $\bfv=\bfx-\bfy$ has $\mathbf{1}^\top\bfv=1-1=0$ and $\bfG\bfv=\bfv$, so part 1 gives $\|\bfv\|_1=\|\bfG\bfv\|_1\leq\beta\|\bfv\|_1$. Since $\beta<1$, this forces $\bfv=\mathbf{0}$. The existence of $\bfx^\ast$ and the bound on its entries are proved in the box below.

Part 3. The error $\bfx^{(t)}-\bfx^\ast=\bfG^t(\bfx^{(0)}-\bfx^\ast)$ starts with entries summing to zero, and $\bfG$ preserves that property ($\mathbf{1}^\top\bfG\bfv=\mathbf{1}^\top\bfv$). Applying part 1 $t$ times, \[ \|\bfx^{(t)}-\bfx^\ast\|_1\leq\beta^t\|\bfx^{(0)}-\bfx^\ast\|_1\leq\beta^t\big(\|\bfx^{(0)}\|_1+\|\bfx^\ast\|_1\big)=2\beta^t. \]

Proof of existence in Theorem 1 (not covered in class)

By Proposition 1, $1$ is an eigenvalue of $\bfG$, so the real matrix $\bfG-\bfI$ is singular and there is a real $\bfv\neq\mathbf{0}$ with $\bfG\bfv=\bfv$. The subtle point is that $\bfv$ might have entries of both signs, and a probability vector cannot. Let $|\bfv|$ denote the vector of absolute values $|v_i|$. Since the entries of $\bfG$ are non-negative, the triangle inequality gives, entry by entry, \[ |v_i|=\Big|\sum_jG_{ij}v_j\Big|\leq\sum_jG_{ij}|v_j|=(\bfG|\bfv|)_i. \] Summing over $i$, the left side totals $\mathbf{1}^\top|\bfv|$ and the right side totals $\mathbf{1}^\top\bfG|\bfv|=\mathbf{1}^\top|\bfv|$ as well. A sum of non-negative gaps $(\bfG|\bfv|)_i-|v_i|$ that equals zero must have every gap zero, so $\bfG|\bfv|=|\bfv|$. Then $\bfx^\ast=|\bfv|/(\mathbf{1}^\top|\bfv|)$ is in $\Delta_n$ and is fixed by $\bfG$. Finally, $\bfA\bfx^\ast$ has non-negative entries, so $\bfx^\ast=\bfG\bfx^\ast=\beta\bfA\bfx^\ast+\frac{1-\beta}{n}\mathbf{1}$ has every entry at least $(1-\beta)/n$.

So the damping factor plays two roles. It guarantees a unique ranking in which every page gets some credit, and it sets the convergence rate of the power method, regardless of the size of the web. With $\beta=0.85$, fifty iterations already give $2(0.85)^{50}<10^{-3}$. The price is that $\bfx^\ast$ describes a slightly modified web; as $\beta\to1$ we recover the original link structure but lose the guarantees.

For the toy web with $\beta=0.85$, the PageRank vector is approximately $(0.368,\,0.142,\,0.288,\,0.202)$: teleportation evens things out a little, but the order is unchanged.

PageRank in NumPy

Never form $\bfG$: it is dense even when $\bfA$ is sparse. For $\bfx\in\Delta_n$ we have $\mathbf{1}^\top\bfx=1$, so $\bfG\bfx=\beta\bfA\bfx+\frac{1-\beta}{n}\mathbf{1}$, which costs one sparse matrix–vector product.

import numpy as np

def pagerank(A, beta=0.85, tol=1e-10, max_iter=1000):
    n = A.shape[0]
    x = np.full(n, 1 / n)
    for _ in range(max_iter):
        x_new = beta * (A @ x) + (1 - beta) / n
        if np.abs(x_new - x).sum() < tol:
            return x_new
        x = x_new
    return x

A = np.array([[0, 0, 1, 1/2],
              [1/3, 0, 0, 0],
              [1/3, 1/2, 0, 1/2],
              [1/3, 1/2, 0, 0]])
print(pagerank(A))  # approx. [0.368, 0.142, 0.288, 0.202]

Remark (Perron–Frobenius)

Remark (Perron–Frobenius). Theorem 1 is a special case of the Perron–Frobenius theorem: a matrix with strictly positive entries has a simple largest eigenvalue, with an eigenvector whose entries are all positive. Teleportation is a way of making every entry of $\bfG$ positive, at least $(1-\beta)/n$. Exercise 5.1 in the textbook explores the more general condition (irreducibility) under which the conclusion survives.

Summary

PageRank turned a circular definition of importance into an eigenvector of a stochastic matrix, the Google matrix $\bfG=\beta\bfA+\frac{1-\beta}{n}\mathbf{1}\mathbf{1}^\top$. Teleportation made that eigenvector unique, with every entry positive, and made the power method converge with error at most $2\beta^t$ in the $\ell_1$ norm, no matter how large the web (Theorem 1). Next time we turn from ranking vertices to grouping points, with $k$-means clustering.