IIT M FOUNDATION AN EXAM QDF2 03 Aug - 03 Aug 2025

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Summary of Core Topics and Concepts

1. Optimization and Critical Points

• Gradient and Hessian Matrix:

  • Gradient ($\nabla f$): A vector of partial derivatives.
  • Hessian Matrix ($\nabla^2 f$): A matrix of second-order partial derivatives.
  • Critical Point: A point where the gradient is zero ($\nabla f(x^*) = 0$).
  • Positive Definite Hessian: If the Hessian matrix is positive definite at a critical point, that point is a local minimum.

• Quadratic Forms:

  • Positive Definite: $Q(x) > 0$ for all $x \neq 0$.
  • Positive Semi-Definite: $Q(x) \geq 0$ for all $x$.
  • Negative Definite: $Q(x) < 0$ for all $x \neq 0$.
  • Negative Semi-Definite: $Q(x) \leq 0$ for all $x$.

2. Matrix Properties

• Positive Definite Matrices:

  • A matrix $A$ is positive definite if $x^T A x > 0$ for all $x \neq 0$.
  • Example: $A = \begin{pmatrix} 2 & -1 & 0 \\ -1 & 2 & -1 \\ 0 & -1 & 2 \end{pmatrix}$ is positive definite.

• Eigenvalues and Eigenvectors:

  • Eigenvalues: Scalars $\lambda$ such that $A v = \lambda v$ for some vector $v$.
  • Eigenvectors: Vectors $v$ corresponding to eigenvalues.
  • Hermitian Matrices: Matrices equal to their conjugate transpose. Eigenvalues are real, and eigenvectors corresponding to different eigenvalues are orthogonal.

3. Singular Value Decomposition (SVD)

• SVD: A factorization of a matrix $A$ into $A = Q_1 \Sigma Q_2^T$, where $Q_1$ and $Q_2$ are orthogonal matrices, and $\Sigma$ is a diagonal matrix of singular values.

• Principal Component Analysis (PCA):

  • Covariance Matrix: Measures how much two random variables change together.
  • Eigenvalues of Covariance Matrix: Indicate the variance in the direction of the corresponding eigenvectors.
  • Principal Components: Directions of maximum variance in the data.

4. Gradient Descent

• Gradient Descent Algorithm: An iterative optimization algorithm for finding the minimum of a function.

  • Update Rule: $x_{t+1} = x_t - \eta_t \nabla f(x_t)$, where $\eta_t$ is the learning rate.
  • Learning Rate: Step size in the direction of the negative gradient.

5. Complex Matrices

• Hermitian Matrices: Matrices equal to their conjugate transpose.
• Unitary Matrices: Matrices whose inverse is their conjugate transpose.
• Eigenvalues and Eigenvectors: For complex matrices, eigenvalues may not be real, and eigenvectors corresponding to different eigenvalues may not be orthogonal.

Equations

1. Gradient and Hessian:

1. Quadratic Form:

1. Positive Definite Matrix:

1. Eigenvalues and Eigenvectors:

1. SVD:

1. Gradient Descent Update:

1. Covariance Matrix:

1. Principal Components: