IIT M FOUNDATION AN EXAM QDF3 03 Aug - 03 Aug 2025

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Summary of Core Topics, Concepts, Principles, and Equations

1. Optimization and Critical Points

• Gradient and Hessian Matrix: For a function $f: \mathbb{R}^n \to \mathbb{R}$, the gradient $\nabla f(x^*) = 0$ and the Hessian matrix $\nabla^2 f(x^*) > 0$ indicate a local minimum at $x^*$.
• Quadratic Forms: Understanding positive definiteness, semi-definiteness, and indefiniteness of quadratic forms.

2. Matrix Properties

• Positive Definite Matrices: A matrix $A$ is positive definite if $x^T A x > 0$ for all non-zero vectors $x$.
• Eigenvalues and Eigenvectors: Properties of eigenvalues and eigenvectors, especially for Hermitian and unitary matrices.
• Singular Value Decomposition (SVD): Decomposition of a matrix $A$ into $Q_1 \Sigma Q_2^T$, where $Q_1$ and $Q_2$ are orthogonal matrices and $\Sigma$ is a diagonal matrix of singular values.

3. Principal Component Analysis (PCA)

• Covariance Matrix: Calculation and properties of the covariance matrix for a dataset.
• Eigenvalues and Principal Components: Using eigenvalues to determine the principal components and projecting data onto lower-dimensional spaces.

4. Gradient Descent

• Algorithm: Iterative method to minimize a function using the gradient and a learning rate.
• Learning Rate: The step size $\eta_t$ in the gradient descent algorithm, which can vary with iterations.

5. Distance Minimization

• Distance Formula: Minimizing the distance between points using derivatives and critical points.
• Parabola and Points: Finding the closest point on a parabola to a given point.

Equations

Optimization

1. Gradient: $\nabla f(x) = \left( \frac{\partial f}{\partial x_1}, \frac{\partial f}{\partial x_2}, \ldots, \frac{\partial f}{\partial x_n} \right)$
2. Hessian Matrix: $\nabla^2 f(x) = \begin{bmatrix} \frac{\partial^2 f}{\partial x_1^2} & \frac{\partial^2 f}{\partial x_1 \partial x_2} & \cdots & \frac{\partial^2 f}{\partial x_1 \partial x_n} \\ \frac{\partial^2 f}{\partial x_2 \partial x_1} & \frac{\partial^2 f}{\partial x_2^2} & \cdots & \frac{\partial^2 f}{\partial x_2 \partial x_n} \\ \vdots & \vdots & \ddots & \vdots \\ \frac{\partial^2 f}{\partial x_n \partial x_1} & \frac{\partial^2 f}{\partial x_n \partial x_2} & \cdots & \frac{\partial^2 f}{\partial x_n^2} \end{bmatrix}$

Quadratic Forms

1. Quadratic Form: $Q(x, y, z) = x^2 + 2xy - 2xz$

Matrix Properties

1. Positive Definite Matrix: $x^T A x > 0$ for all $x \neq 0$
2. Eigenvalue Equation: $A v = \lambda v$
3. SVD: $A = Q_1 \Sigma Q_2^T$

PCA

1. Covariance Matrix: $C = \frac{1}{n-1} X^T X$
2. Principal Components: Eigenvectors of the covariance matrix corresponding to the largest eigenvalues.

Gradient Descent

1. Gradient Descent Update: $x_{t+1} = x_t - \eta_t \nabla f(x_t)$
2. Learning Rate: $\eta_t = \frac{1}{t + 1}$

Distance Minimization

1. Distance Formula: $d = \sqrt{(x_2 - x_1)^2 + (y_2 - y_1)^2}$
2. Parabola: $y = x^2$