IIT M IMPROVEMENT AN EXAM QIB2 03 Aug - 03 Aug 2025

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Summary of Core Topics, Concepts, Principles, and Equations

1. Optimization and Critical Points

• Gradient and Hessian Matrix:

  • Gradient ($\nabla f$): A vector of partial derivatives.
  • Hessian Matrix ($\nabla^2 f$): A matrix of second-order partial derivatives.
  • Critical Point: A point where the gradient is zero ($\nabla f(x^*) = 0$).
  • Positive Definite Hessian: If the Hessian matrix is positive definite at a critical point, that point is a local minimum.

• Quadratic Forms:

  • Positive Definite: $Q(x) > 0$ for all $x \neq 0$.
  • Positive Semi-Definite: $Q(x) \geq 0$ for all $x$.
  • Negative Definite: $Q(x) < 0$ for all $x \neq 0$.
  • Negative Semi-Definite: $Q(x) \leq 0$ for all $x$.

2. Eigenvalues and Eigenvectors

• Eigenvalues and Eigenvectors of Matrices:

  • For a matrix $A$, eigenvalues ($\lambda$) and eigenvectors ($v$) satisfy $Av = \lambda v$.
  • Hermitian Matrices: Eigenvalues are real, and eigenvectors corresponding to different eigenvalues are orthogonal.
  • Unitary Matrices: Eigenvalues lie on the unit circle in the complex plane.

3. Singular Value Decomposition (SVD)

• SVD: A matrix $A$ can be decomposed as $A = Q_1 \Sigma Q_2^T$, where $Q_1$ and $Q_2$ are orthogonal matrices, and $\Sigma$ is a diagonal matrix of singular values.

  • Eigenvalues of $AA^T$ and $A^T A$: Non-zero eigenvalues of $AA^T$ and $A^T A$ are equal.

4. Principal Component Analysis (PCA)

• PCA: A technique to reduce the dimensionality of a dataset while preserving its variance.

  • Covariance Matrix: Measures the covariance between different dimensions of the data.
  • Eigenvectors of Covariance Matrix: Principal components are the eigenvectors of the covariance matrix.
  • Projection: Data is projected onto the principal components to reduce dimensionality.

5. Gradient Descent

• Gradient Descent Algorithm: An iterative optimization algorithm to find the minimum of a function.

  • Update Rule: $x_{t+1} = x_t - \eta_t \nabla f(x_t)$, where $\eta_t$ is the learning rate.
  • Learning Rate: Can be constant or vary with iterations (e.g., $\eta_t = \frac{1}{t+1}$).

6. Matrix Properties

• Hermitian Matrix: A matrix equal to its own conjugate transpose.
• Unitary Matrix: A matrix whose conjugate transpose is its inverse.
• Eigenvalues and Eigenvectors: Key properties for understanding matrix behavior and transformations.

Equations

1. Gradient and Hessian:

1. Quadratic Form:

1. Eigenvalue Equation:

1. SVD:

1. Gradient Descent Update:

1. Covariance Matrix: