Graphs and the graph Laplacian

Math 123: Mathematical Aspects of Data Analysis – Fall 2026

Published

October 20, 2026

Where we are

Last time we met $k$-means and Lloyd’s algorithm. In the homework you saw where it breaks: on two concentric rings, it returns wedges that cut across both rings, whatever $k$ you choose (figure). A major part of the problem is the assignment step. Each point goes to its nearest center, so two clusters are always separated by a straight line, the set of points equidistant from their two centers, and no cluster can wrap around another. The rings are not far apart as sets, but they are connected differently: each point on the inner ring has close neighbours on the inner ring, and a gap separates it from the outer one. That suggests forgetting the coordinates and keeping only who is close to whom, in a graph.

This page covers Section 5.2 and the start of Section 5.4 of the textbook. Specifically, we study the graph Laplacian, which measures how rough a function on the vertices is. Next time its eigenvectors give the smoothest ways to place the vertices in $\Rbb^m$, and spectral clustering. For a nice alternative presentation of the ideas here, see the tutorial of von Luxburg (Luxburg 2007).

Throughout, $\mathbf{1}\in\Rbb^n$ is the all-ones vector, $[n]=\{1,\ldots,n\}$, and $\mathbf{1}_S\in\Rbb^n$ is the indicator vector of a set $S\subseteq[n]$.

Graphs

The web of PageRank is a directed graph: a link from $k$ to $j$ says nothing about a link from $j$ to $k$. From now on we focus on undirected graphs, where an edge records a symmetric relationship: two people are friends, two proteins interact, two data points are close.

Definition 1 (Graph, weights, and degree) A graph $G=(V,E)$ consists of a set of vertices $V=[n]$ and a set $E$ of edges, each an unordered pair $\{i,j\}$ of distinct vertices. Its adjacency matrix $\bfW\in\Rbb^{n\times n}$ has $W_{ij}=1$ if $\{i,j\}\in E$ and $W_{ij}=0$ otherwise.

More generally, a weighted graph assigns each pair a weight $w_{ij}=w_{ji}\geq0$, with $w_{ij}>0$ exactly when $\{i,j\}\in E$, and its weight matrix is $\bfW=(w_{ij})$. In both cases $\bfW$ is symmetric with zero diagonal.

The degree of vertex $i$ is $\deg(i)=\sum_{j=1}^nw_{ij}$ (for an unweighted graph, the number of neighbours of $i$), and the degree matrix is the diagonal matrix $\bfD=\operatorname{diag}(\deg(1),\ldots,\deg(n))$. Equivalently, the vector of degrees is $\bfd=\bfW\mathbf{1}$. The volume of a set of vertices $S\subseteq V$ is $\operatorname{vol}(S)=\sum_{i\in S}\deg(i)$.

For an unweighted graph, $\operatorname{vol}(V)=\sum_i\deg(i)=2|E|$, since every edge is counted once from each end.

As a running example, take the barbell graph in Figure 1: two triangles joined by a single bridge.

Figure 1: The barbell graph: two triangles $\{1,2,3\}$ and $\{4,5,6\}$ joined by the edge $\{3,4\}$.

Its adjacency matrix and degree vector are \[ \bfW=\begin{pmatrix} 0&1&1&0&0&0\\ 1&0&1&0&0&0\\ 1&1&0&1&0&0\\ 0&0&1&0&1&1\\ 0&0&0&1&0&1\\ 0&0&0&1&1&0 \end{pmatrix}, \qquad \bfd=\bfW\mathbf{1}=\begin{pmatrix}2\\2\\3\\3\\2\\2\end{pmatrix}. \] Anyone looking at Figure 1 sees two clusters. The goal of this unit is to make a computer see them too.

Walks and connectivity

Definition 2 (Walks, paths, and connectivity) A walk of length $m$ from $i$ to $j$ is a sequence of vertices $i=v_0,v_1,\ldots,v_m=j$ with $\{v_{s-1},v_s\}\in E$ for each $s$. A path is a walk with no repeated vertices. A graph is connected if there is a walk between every pair of vertices. In general, $V$ splits into connected components: maximal sets of vertices that are pairwise joined by walks.

Powers of the adjacency matrix count walks.

Proposition 1 (Powers of the adjacency matrix count walks) Let $\bfW$ be the adjacency matrix of an unweighted graph. For every $m\geq1$, $(\bfW^m)_{ij}$ is the number of walks of length $m$ from $i$ to $j$. Consequently, $i\neq j$ lie in the same connected component if and only if $(\bfW^m)_{ij}>0$ for some $1\leq m\leq n-1$.

Proof

Proof. Induct on $m$. For $m=1$, a walk of length $1$ from $i$ to $j$ is a single edge, and $W_{ij}$ is $1$ or $0$ according to whether the edge exists. Suppose the claim holds for $m$. Every walk of length $m+1$ from $i$ to $j$ is a walk of length $m$ from $i$ to some vertex $\ell$, followed by an edge $\{\ell,j\}$. Sorting the walks by $\ell$, their number is \[ \sum_{\ell=1}^n(\text{walks of length $m$ from $i$ to $\ell$})\cdot W_{\ell j} =\sum_{\ell=1}^n(\bfW^m)_{i\ell}W_{\ell j} =(\bfW^{m+1})_{ij}. \] For the second claim, if there is a walk from $i$ to $j$, then a shortest one repeats no vertex (otherwise we could cut out the loop between the repeats), so it is a path and has at most $n-1$ edges.

For the barbell, $(\bfW^2)_{ii}=\deg(i)$ (walk to a neighbour and back), and $(\bfW^2)_{15}=0$ while $(\bfW^3)_{15}=1$, the single walk $1\to3\to4\to5$.

Random walks on graphs

The random surfer of PageRank makes sense on an undirected graph too. From vertex $j$, step to a neighbour $i$ with probability proportional to the edge weight, $w_{ij}/\deg(j)$. In the convention of PageRank, the transition matrix is \[ \bfA=\bfW\bfD^{-1}, \qquad A_{ij}=\frac{w_{ij}}{\deg(j)}, \] which is column-stochastic as long as no vertex is isolated ($\deg(j)>0$ for all $j$), because column $j$ sums to $\deg(j)/\deg(j)=1$.

Proposition 2 (The stationary distribution of a random walk on a graph) Let $G$ be a weighted graph with no isolated vertices. Then $\bfpi=\bfd/\operatorname{vol}(V)$, the vector with entries $\pi_i=\deg(i)/\operatorname{vol}(V)$, lies in the probability simplex $\Delta_n$ and satisfies $\bfW\bfD^{-1}\bfpi=\bfpi$.

Proof

Proof. The entries of $\bfpi$ are non-negative and sum to $\operatorname{vol}(V)/\operatorname{vol}(V)=1$. Since $\bfD^{-1}\bfd=\mathbf{1}$, we get $\bfW\bfD^{-1}\bfd=\bfW\mathbf{1}=\bfd$; divide by $\operatorname{vol}(V)$.

So on an undirected graph, PageRank without teleportation just ranks vertices by degree. The interesting structure of a symmetric relationship is not in the stationary distribution but in how fast and where the random walk mixes. A walker on the barbell tends to stay in one triangle for a long time before crossing the bridge, and that slow mixing is exactly the signature of a cluster.

The matrix $\bfA$ moves probability distributions forward in time. For the rest of this unit, and in Chapter 6, we use its transpose, \[ \bfM=\bfA^\top=\bfD^{-1}\bfW, \qquad M_{ij}=\frac{w_{ij}}{\deg(i)}, \] which is row-stochastic and acts on functions on the vertices. A vector $\bff\in\Rbb^n$ assigns a number $f_i$ to each vertex $i$, and \[ (\bfM\bff)_i=\sum_{j=1}^n\frac{w_{ij}}{\deg(i)}f_j \] is the weighted average of $\bff$ over the neighbours of $i$. So $\bfM$ is an averaging operator, and $\bfM\mathbf{1}=\mathbf{1}$ says that averaging a constant changes nothing. By the same induction as in Proposition 1, $(\bfM^t)_{ij}$ is the probability that a walker started at $i$ is at $j$ after $t$ steps; Chapter 6 builds its diffusion maps out of $\bfM^t$. (The Markov chain example in the linear algebra notes uses this row-stochastic convention too.)

From point clouds to graphs

We can now turn the rings from the homework into a graph. Given $\bfx_1,\ldots,\bfx_n\in\Rbb^d$, two common constructions are:

the Gaussian similarity graph, with weights \[ w_{ij}=\exp\Big(-\frac{\|\bfx_i-\bfx_j\|_2^2}{2\varepsilon}\Big), \qquad i\neq j, \] where the bandwidth $\varepsilon>0$ sets the distance scale below which points count as similar;
the $m$-nearest-neighbour graph, with an edge $\{i,j\}$ whenever $\bfx_j$ is among the $m$ nearest points to $\bfx_i$ or vice versa.

Either way, the two rings become two pieces of a graph joined by only a few weak edges, much like the two triangles of the barbell. Both constructions only use distances between nearby points, so they work just as well when the data are not vectors at all, as long as we can say which items are similar.

If we instead give each nearest-neighbour edge the length $\|\bfx_i-\bfx_j\|_2$, the shortest path between two vertices measures distance travelled along the data rather than straight through empty space. On the rings, two points on opposite sides of the inner ring are close in $\Rbb^2$ but far apart along the ring. This is the starting point of Isomap (Section 6.4).

Clustering the data now means splitting the vertices into groups while cutting as little edge weight as possible, \[ \operatorname{cut}(S)=\sum_{i\in S}\sum_{j\notin S}w_{ij}, \] without making either group too small. Rather than attack this combinatorial problem directly, we change the question: how should we place the vertices in $\Rbb^m$ so that strongly connected vertices land close together?

Functions on graphs and the Laplacian

Definition 3 (Graph Laplacian) The graph Laplacian of a weighted graph is $\bfL=\bfD-\bfW$, that is, \[ L_{ij}=\begin{cases}\deg(i)&i=j,\\-w_{ij}&i\neq j.\end{cases} \]

For the barbell, $\bfL=\operatorname{diag}(2,2,3,3,2,2)-\bfW$. Every row of $\bfL$ sums to zero, so $\bfL\mathbf{1}=\mathbf{0}$. In terms of the averaging operator, \[ \bfL\bff=\bfD(\bff-\bfM\bff), \qquad (\bfL\bff)_i=\deg(i)\big(f_i-(\bfM\bff)_i\big), \] so $\bfL$ measures how much $\bff$ at each vertex differs from the average of its neighbours. The quadratic form of $\bfL$ makes this global.

Proposition 3 (The Laplacian quadratic form) For every $\bff\in\Rbb^n$, \[ \bff^\top\bfL\bff=\sum_{\{i,j\}\in E}w_{ij}(f_i-f_j)^2=\frac12\sum_{i,j=1}^nw_{ij}(f_i-f_j)^2. \] In particular, $\bfL$ is positive semidefinite.

Proof

Proof. Expand the square and use the symmetry of $\bfW$: \[ \frac12\sum_{i,j}w_{ij}(f_i^2-2f_if_j+f_j^2) =\frac12\sum_i\deg(i)f_i^2+\frac12\sum_j\deg(j)f_j^2-\sum_{i,j}w_{ij}f_if_j =\bff^\top\bfD\bff-\bff^\top\bfW\bff. \] The right-hand side is a sum of non-negative terms, so $\bff^\top\bfL\bff\geq0$.

Read $\bff^\top\bfL\bff$ as the roughness of $\bff$ on the graph: it is small exactly when $\bff$ takes nearly equal values across every heavy edge. Indicator vectors connect this to cutting: $(\mathbf{1}_S)_i-(\mathbf{1}_S)_j$ is non-zero exactly when the edge $\{i,j\}$ crosses from $S$ to its complement, so \[ \mathbf{1}_S^\top\bfL\mathbf{1}_S=\operatorname{cut}(S). \] Finding a set with a small cut means finding a smooth $0/1$ function.

Remark (Why “Laplacian”?)

Remark (Why “Laplacian”?). If the vertices are samples from a smooth surface and the weights are a Gaussian kernel, the sum $\sum w_{ij}(f_i-f_j)^2$ is, after rescaling, a Riemann sum for $\int\|\nabla f\|^2$, and $\bfL$ approximates the negative of the Laplace operator $\Delta=\sum_k\partial^2/\partial x_k^2$. Chapter 11 of the textbook makes this precise. The eigenvectors of $\bfL$ play the role that sines and cosines play on an interval: the slowest-oscillating patterns the space supports.

Rayleigh–Ritz: eigenvectors as optimizers

Eigenvectors of symmetric matrices solve optimization problems. We have used one instance already: the top right singular vectors span the best-fitting subspace. Here is the general statement, which we will use now for graphs and again for every embedding method in Chapter 6.

Proposition 4 (Rayleigh–Ritz) Let $\bfB\in\Rbb^{n\times n}$ be symmetric with eigenvalues $\beta_1\geq\cdots\geq\beta_n$ and orthonormal eigenvectors $\bfu_1,\ldots,\bfu_n$. For every $\bfZ\in\Rbb^{n\times m}$ with orthonormal columns, $\bfZ^\top\bfZ=\bfI_m$, \[ \operatorname{tr}(\bfZ^\top\bfB\bfZ)\leq\beta_1+\cdots+\beta_m, \] with equality at $\bfZ=[\bfu_1,\ldots,\bfu_m]$. If moreover $\bfZ^\top\bfu_1=\mathbf{0}$, then $\operatorname{tr}(\bfZ^\top\bfB\bfZ)\leq\beta_2+\cdots+\beta_{m+1}$, with equality at $\bfZ=[\bfu_2,\ldots,\bfu_{m+1}]$.

For $m=1$: $\max_{\|\bfz\|_2=1}\bfz^\top\bfB\bfz=\beta_1$, and $\max\{\bfz^\top\bfB\bfz:\|\bfz\|_2=1,\ \bfz\perp\bfu_1\}=\beta_2$.

Flipping signs, the smallest eigenvalues and their eigenvectors solve the corresponding minimization problems. PCA uses the maximization (directions of most variance); graphs will use the minimization (directions of least roughness).

We prove Proposition 4 next time, where we also see which functions have zero roughness and use Rayleigh–Ritz to find the smoothest ways to place the vertices of a graph in $\Rbb^m$.

Summary

A graph records relationships through its weight matrix $\bfW$ and degree matrix $\bfD$, and the random walk on it gives the averaging operator $\bfM=\bfD^{-1}\bfW$, whose powers track where a walker goes (Proposition 1). A point cloud becomes a graph through a Gaussian similarity or nearest-neighbour construction, and clustering becomes cutting few edges. The Laplacian $\bfL=\bfD-\bfW$ measures the roughness of a function on the vertices (Proposition 3), and its quadratic form on indicator vectors is the cut. Eigenvectors of symmetric matrices solve trace optimization problems (Proposition 4).

References

Luxburg, Ulrike von. 2007. “A Tutorial on Spectral Clustering.” Statistics and Computing 17 (4): 395–416.