IIT M IMPROVEMENT AN EXAM QIC4 03 Aug - 03 Aug 2025

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Summary of Core Topics, Concepts, Principles, and Equations

1. Optimization and Critical Points

• Gradient and Hessian Matrix:

  • Gradient ($\nabla f$): Vector of partial derivatives.
  • Hessian Matrix ($\nabla^2 f$): Matrix of second partial derivatives.

• Critical Points:

  • If $\nabla f(x^*) = 0$ and $\nabla^2 f(x^*) > 0$, then $x^*$ is a local minimum.
  • If $\nabla f(x^*) = 0$ and $\nabla^2 f(x^*) < 0$, then $x^*$ is a local maximum.
  • If $\nabla f(x^*) = 0$ and $\nabla^2 f(x^*)$ is indefinite, then $x^*$ is a saddle point.

2. Quadratic Forms and Definiteness

• Quadratic Form: $Q(x, y, z) = x^2 + 2xy - 2xz$

• Definiteness:

  • Positive Definite: $Q > 0$ for all non-zero vectors.
  • Positive Semi-Definite: $Q \geq 0$ for all vectors.
  • Negative Definite: $Q < 0$ for all non-zero vectors.
  • Negative Semi-Definite: $Q \leq 0$ for all vectors.
  • Indefinite: $Q$ takes both positive and negative values.

3. Matrix Properties

• Positive Definite Matrix: A matrix $A$ is positive definite if $x^T A x > 0$ for all non-zero vectors $x$.

• Eigenvalues and Eigenvectors:

  • For a Hermitian matrix, eigenvalues are real, and eigenvectors are orthogonal.
  • For a unitary matrix, $A^{-1} = A^*$.

4. Singular Value Decomposition (SVD)

• SVD: $A = Q_1 \Sigma Q_2^T$

  • $Q_1$ and $Q_2$ are orthogonal matrices.
  • $\Sigma$ is a diagonal matrix with singular values.

5. Principal Component Analysis (PCA)

• Covariance Matrix: Measures the covariance between features.
• Eigenvalues and Eigenvectors of Covariance Matrix: Used to determine principal components.
• Projection: Data is projected onto the eigenvectors with the highest eigenvalues.

6. Gradient Descent

• Gradient Descent Algorithm: Iterative method to minimize a function.

  • Update rule: $x_{t+1} = x_t - \eta_t \nabla f(x_t)$
  • Learning rate ($\eta_t$): Step size for each iteration.

Equations

Optimization

1. Gradient: $\nabla f(x) = \left( \frac{\partial f}{\partial x_1}, \frac{\partial f}{\partial x_2}, \ldots, \frac{\partial f}{\partial x_n} \right)$
2. Hessian Matrix: $\nabla^2 f(x) = \begin{bmatrix} \frac{\partial^2 f}{\partial x_1^2} & \frac{\partial^2 f}{\partial x_1 \partial x_2} & \cdots & \frac{\partial^2 f}{\partial x_1 \partial x_n} \\ \frac{\partial^2 f}{\partial x_2 \partial x_1} & \frac{\partial^2 f}{\partial x_2^2} & \cdots & \frac{\partial^2 f}{\partial x_2 \partial x_n} \\ \vdots & \vdots & \ddots & \vdots \\ \frac{\partial^2 f}{\partial x_n \partial x_1} & \frac{\partial^2 f}{\partial x_n \partial x_2} & \cdots & \frac{\partial^2 f}{\partial x_n^2} \end{bmatrix}$

Quadratic Forms

1. Quadratic Form: $Q(x) = x^T A x$

Matrix Properties

1. Positive Definite Matrix: $x^T A x > 0$ for all $x \neq 0$
2. Eigenvalue Equation: $A v = \lambda v$

SVD

1. Singular Value Decomposition: $A = Q_1 \Sigma Q_2^T$

PCA

1. Covariance Matrix: $C = \frac{1}{n-1} X^T X$
2. Projection: $Y = X W$, where $W$ contains the principal components.

Gradient Descent

1. Gradient Descent Update: $x_{t+1} = x_t - \eta_t \nabla f(x_t)$

Graph 1: Optimization and Critical Points

Graph 2: Quadratic Forms and Definiteness

Graph 3: Matrix Properties

Graph 4: Singular Value Decomposition (SVD)

Graph 5: Principal Component Analysis (PCA)

Graph 6: Gradient Descent

These graphs and summaries should help you understand the core topics and their relationships for the exam.