Concept: Eigenvalues are scalars associated with a linear transformation of a vector space, and eigenvectors are the corresponding vectors that are scaled by the transformation.
Equation: Av=λv, where A is a matrix, v is an eigenvector, and λ is an eigenvalue.
Hermitian Matrix
Concept: A square matrix that is equal to its own conjugate transpose.
Property: Eigenvalues of a Hermitian matrix are real.
Schur's Theorem
Concept: Every square matrix is unitarily similar to an upper triangular matrix.
Property: The diagonal entries of the Schur form of a matrix are its eigenvalues.
Singular Value Decomposition (SVD)
Concept: Factorizing a matrix into three matrices U, Σ, and VT, where U and V are orthogonal matrices and Σ is a diagonal matrix.
Equation: A=UΣVT
Principal Component Analysis (PCA)
Concept: A technique for reducing the dimensionality of a dataset by projecting it onto a lower-dimensional subspace.
Steps: Compute the covariance matrix, find the eigenvalues and eigenvectors, and project the data onto the principal components.
Gradient Descent
Concept: An optimization algorithm for finding the minimum of a function.
Equation: xn+1=xn−γ∇f(xn), where γ is the learning rate and ∇f(xn) is the gradient of the function at xn.
Data Science
Covariance Matrix
Concept: A matrix that contains the covariances associated with several variables.
Equation: Cij=Cov(Xi,Xj)
Projection onto Principal Components
Concept: Projecting data onto the directions of maximum variance.
Equation: x~=xTv, where v is the eigenvector corresponding to the largest eigenvalue.