Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.
Here’s a structured summary of core topics, concepts, principles, and equations essential for the exam, followed by Mermaid knowledge graphs to visualize relationships.
1. Core Topics & Key Concepts
A. Probability & Statistics
Exponential Distribution
CDF: FX(x)=1−e−λx (for λ=1, FX(5)=1−e−5).
PDF: fX(x)=λe−λx.
Gaussian (Normal) Distribution
Conditional Probability: For z=Cy+ϵ where ϵ∼N(0,σ2Id), p(z∣y)=N(Cy,σ2Id).
Linearity of Expectation: E[3X1+2X2]=3E[X1]+2E[X2].
Markov’s Inequality
For non-negative Y, P(Y≥a)≤aE[Y].
Example: P(Y≥100)≤10020=0.2.
B. Linear Algebra & Optimization
Singular Value Decomposition (SVD)
Properties:
Applicable to any matrix (not just square/symmetric).
Singular values are non-negative.
Singular values of A and AT are identical.
Principal Component Analysis (PCA)
Key Ideas:
Principal components = eigenvectors of the covariance matrix.
Maximizes variance along each component.
Basis is orthogonal.
Convexity
Convex Sets:
Linear constraints (e.g., ∑aixi=b) define convex sets.
Lines in R2 are convex; unions of convex sets are not necessarily convex.
Convex Functions:
ex, ex2+2 are convex.
Composition f∘g is not guaranteed convex.
Optimization:
Convex cost functions (e.g., f(x,y)=2x2+xy+y2) have global minima.
KKT Conditions for constrained optimization:
∇L=0 (stationarity).
β(w−δ)=0 (complementary slackness).
β≥0 (dual feasibility).
Gradient Descent
Update Rule: xk+1=xk−η∇f(xk).
Example: For f(x,y)=x2+2y2−6x+8y, ∇f=(2x−6,4y+8). With η=0.2, (x1,y1)=(1.8,−0.4).
C. Calculus & Constrained Optimization
Lagrange Multipliers
Lagrangian: L=f(x)+∑λigi(x).
KKT Conditions:
∇L=0, λigi(x)=0, λi≥0 (for inequalities).
Maximization on Constraints
Example: Maximize f(x,y,z)=5x+7y+z on x2+y2=25 and x+y−z=4.
Use Lagrange multipliers or parameterize the cylinder.
2. Key Equations
Topic
Equation
Exponential CDF
FX(x)=1−e−λx
Gaussian Conditional
p(z∣y)=N(Cy,σ2Id)
Markov’s Inequality
P(Y≥a)≤aE[Y]
SVD
A=UΣVT (singular values σi≥0)
PCA
Covariance matrix eigenvectors = principal components
Convex Function
f(λx+(1−λ)y)≤λf(x)+(1−λ)f(y)
Gradient Descent
xk+1=xk−η∇f(xk)
KKT Conditions
∇L=0, λigi(x)=0, λi≥0
Lagrange Multipliers
L=f(x)+∑λigi(x)
Graph 2: Linear Algebra & Optimization
Graph 3: Calculus & Constrained Optimization
4. Exam Tips
Probability: Focus on CDF/PDF of exponential/Gaussian distributions and conditional probability.
Linear Algebra: SVD properties, PCA (eigenvectors vs. singular values), and convexity.
Optimization: KKT conditions, gradient descent updates, and Lagrange multipliers.
Calculus: Maximization/minimization on constraints (use substitution or Lagrange multipliers).