Glossary

Theorems and definitions from the lecture and supplementary notes

Every definition, theorem, proposition, and lemma from the published notes, in the order they appear. Click a result to jump to it.

Result Type Where
Observation and feature vector Definition Lecture 1
Data matrix Definition Lecture 1
Point cloud Definition Lecture 1
Euclidean balls and cubes Definition Lecture 1
Volumes of Euclidean balls and cubes Proposition Lecture 1
Volume near the boundary Proposition Lecture 1
Random variable Definition Lecture 1
Distribution and event Definition Lecture 1
Expectation Definition Lecture 1
Linearity of expectation Proposition Lecture 1
Independent and identically distributed Definition Lecture 2
Expectations of products Proposition Lecture 2
Variance and standard deviation Definition Lecture 2
Covariance Definition Lecture 2
Averaging away independent noise Proposition Lecture 2
Law of the Lazy Data Scientist Theorem Lecture 2
Convex hull Definition Lecture 2
Carathéodory’s theorem Theorem Lecture 2
Approximate Carathéodory theorem Theorem Lecture 2
Eigenvalue and eigenvector Definition Lecture 3
Spectral theorem Theorem Lecture 3
Positive semidefinite matrix Definition Lecture 3
PSD matrices have non-negative eigenvalues Proposition Lecture 3
\(\bfA^\top\bfA\) and \(\bfA\bfA^\top\) are PSD Proposition Lecture 3
Singular values Definition Lecture 3
Building the left singular vectors Lemma Lecture 3
Expansion in an orthonormal basis Lemma Lecture 4
Singular value decomposition Theorem Lecture 4
Singular vectors Definition Lecture 4
Full singular value decomposition Theorem Lecture 4
The SVD and eigendecomposition Proposition Lecture 4
Truncated singular value decomposition Definition Lecture 4
Frobenius norm Definition Lecture 4
Spectral norm Definition Lecture 4
Eckart–Young theorem Theorem Lecture 4
Eckart–Young theorem, Frobenius norm Theorem Lecture 4
\(\bfA\) in singular-vector coordinates Lemma Lecture 5
The spectral norm is the largest singular value Proposition Lecture 5
Trace Definition Lecture 5
Properties of the trace Proposition Lecture 5
The Frobenius norm and all the singular values Proposition Lecture 5
Centered data matrix Definition Lecture 5
Orthogonal projection Definition Lecture 5
Pythagorean theorem Theorem Lecture 5
Properties of orthogonal projection Proposition Lecture 5
Reconstruction error Definition Lecture 6
Reconstruction error is a Frobenius norm Proposition Lecture 6
Best-fitting subspaces come from the SVD Theorem Lecture 6
Principal component analysis Definition Lecture 6
Total variance Definition Lecture 6
Minimum error equals maximum retained variance Proposition Lecture 6
Explained variance ratio Definition Lecture 6
Vector in \(\Rbb^n\) Definition Notes: The Algebra of Data
Matrix and matrix–vector product Definition Notes: The Algebra of Data
Matrices define linear maps Theorem Notes: The Algebra of Data
Transpose and symmetry Definition Notes: The Algebra of Data
Matrix multiplication Definition Notes: The Algebra of Data
Subspace Definition Notes: The Algebra of Data
Span Definition Notes: The Algebra of Data
Linear independence Definition Notes: The Algebra of Data
Basis and dimension Definition Notes: The Algebra of Data
Uniqueness of coordinates Theorem Notes: The Algebra of Data
Invertibility Definition Notes: The Algebra of Data
Column space and null space Definition Notes: The Algebra of Data
Rank–nullity Theorem Notes: The Algebra of Data
The four fundamental subspaces Definition Notes: The Algebra of Data
Inner product Definition Notes: The Algebra of Data
Dot product Definition Notes: The Algebra of Data
Induced norm Definition Notes: The Algebra of Data
\(\ell^p\) norms Definition Notes: The Algebra of Data
Cauchy–Schwarz inequality Theorem Notes: The Algebra of Data
Angle and cosine similarity Definition Notes: The Algebra of Data
Orthogonality Definition Notes: The Algebra of Data
Orthogonal sets are linearly independent Proposition Notes: The Algebra of Data
Orthogonal projection Definition Notes: The Algebra of Data
Properties of projection matrices Theorem Notes: The Algebra of Data
Orthogonality of the fundamental subspaces Theorem Notes: The Algebra of Data
Eigenvalue and eigenvector Definition Notes: The Algebra of Data
Characteristic polynomial Proposition Notes: The Algebra of Data
Algebraic and geometric multiplicity Definition Notes: The Algebra of Data
Eigenvectors for distinct eigenvalues are linearly independent Proposition Notes: The Algebra of Data
Diagonalizability Definition Notes: The Algebra of Data
Diagonalizability criterion Theorem Notes: The Algebra of Data
Symmetric matrices have real eigenvalues Proposition Notes: The Algebra of Data
Eigenvectors for distinct eigenvalues are orthogonal Proposition Notes: The Algebra of Data
Spectral theorem for symmetric matrices Theorem Notes: The Algebra of Data
Eigenvalue characterization of PSD and PD Theorem Notes: The Algebra of Data