IIT M IMPROVEMENT AN EXAM QIC3 03 Aug - 03 Aug 2025

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Summary of Core Topics, Concepts, Principles, and Equations

1. Optimization and Critical Points

• Gradient and Hessian Matrix:

  • Gradient: $\nabla f(x)$
  • Hessian: $\nabla^2 f(x)$

• Conditions for Local Minima/Maxima:

  • If $\nabla f(x^*) = 0$ and $\nabla^2 f(x^*) > 0$, then $x^*$ is a local minimum.
  • If $\nabla f(x^*) = 0$ and $\nabla^2 f(x^*) < 0$, then $x^*$ is a local maximum.
  • If $\nabla^2 f(x^*)$ is indefinite, $x^*$ is a saddle point.

2. Quadratic Forms and Definiteness

• Quadratic Form: $Q(x, y, z) = x^2 + 2xy - 2xz$

• Definiteness:

  • Positive Definite: $Q(x) > 0$ for all $x \neq 0$
  • Positive Semi-Definite: $Q(x) \geq 0$ for all $x$
  • Negative Definite: $Q(x) < 0$ for all $x \neq 0$
  • Negative Semi-Definite: $Q(x) \leq 0$ for all $x$

3. Matrix Properties

• Positive Definite Matrix: A matrix $A$ is positive definite if $x^T A x > 0$ for all $x \neq 0$.

• Eigenvalues and Eigenvectors:

  • For a Hermitian matrix, eigenvalues are real, and eigenvectors corresponding to different eigenvalues are orthogonal.
  • For a matrix $A$, the eigenvalues of $AA^T$ and $A^T A$ are the same.

4. Singular Value Decomposition (SVD)

• SVD Form: $A = Q_1 \Sigma Q_2^T$

  • $Q_1$ and $Q_2$ are orthogonal matrices.
  • $\Sigma$ is a diagonal matrix with singular values.

5. Principal Component Analysis (PCA)

• Covariance Matrix: $C = \frac{1}{n-1} \sum_{i=1}^n (x_i - \bar{x})(x_i - \bar{x})^T$

• Eigenvalues and Eigenvectors of Covariance Matrix:

  • Eigenvalues represent the variance in the direction of the corresponding eigenvectors.
  • The first principal component is the eigenvector corresponding to the largest eigenvalue.

6. Gradient Descent

• Update Rule: $x_{t+1} = x_t - \eta_t \nabla f(x_t)$

  • $\eta_t$ is the learning rate.
  • $\nabla f(x_t)$ is the gradient of the function at $x_t$.