IIT M DIPLOMA AN EXAM QDD4 03 Aug - 03 Aug 2025

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Summary of Core Topics, Concepts, Principles, and Equations

1. Optimization and Critical Points

• Gradient and Hessian Matrix: For a function $f: \mathbb{R}^n \to \mathbb{R}$, the gradient $\nabla f(x^*) = 0$ and the Hessian matrix $\nabla^2 f(x^*)$ being positive definite indicate a local minimum at $x^*$.
• Quadratic Forms: Understanding the definiteness of quadratic forms (positive definite, positive semi-definite, negative definite, negative semi-definite).

2. Matrix Properties

• Positive Definite Matrices: A matrix $A$ is positive definite if $x^T A x > 0$ for all non-zero vectors $x$.
• Eigenvalues and Eigenvectors: Properties of eigenvalues and eigenvectors, especially for Hermitian and unitary matrices.
• Singular Value Decomposition (SVD): For a matrix $A$, the SVD is given by $A = Q_1 \Sigma Q_2^T$, where $Q_1$ and $Q_2$ are orthogonal matrices, and $\Sigma$ is a diagonal matrix of singular values.

3. Principal Component Analysis (PCA)

• Covariance Matrix: The sample covariance matrix $C$ is used to find the principal components.
• Eigenvalues of Covariance Matrix: The eigenvalues $\lambda_1, \lambda_2, \ldots$ of the covariance matrix indicate the variance in the directions of the corresponding eigenvectors.
• Principal Components: The principal components are the eigenvectors of the covariance matrix, ordered by the magnitude of their corresponding eigenvalues.

4. Gradient Descent

• Gradient Descent Algorithm: An iterative optimization algorithm for finding the minimum of a function. The update rule is $x_{t+1} = x_t - \eta_t \nabla f(x_t)$, where $\eta_t$ is the learning rate.

5. Hermitian and Unitary Matrices

• Hermitian Matrices: A matrix $A$ is Hermitian if $A = A^*$ (conjugate transpose). Hermitian matrices have real eigenvalues and orthogonal eigenvectors.
• Unitary Matrices: A matrix $A$ is unitary if $A^* A = I$. Unitary matrices preserve the norm of vectors.

Equations

Optimization

1. Gradient: $\nabla f(x) = \left( \frac{\partial f}{\partial x_1}, \frac{\partial f}{\partial x_2}, \ldots, \frac{\partial f}{\partial x_n} \right)$
2. Hessian Matrix: $\nabla^2 f(x) = \begin{bmatrix} \frac{\partial^2 f}{\partial x_1^2} & \frac{\partial^2 f}{\partial x_1 \partial x_2} & \cdots & \frac{\partial^2 f}{\partial x_1 \partial x_n} \\ \frac{\partial^2 f}{\partial x_2 \partial x_1} & \frac{\partial^2 f}{\partial x_2^2} & \cdots & \frac{\partial^2 f}{\partial x_2 \partial x_n} \\ \vdots & \vdots & \ddots & \vdots \\ \frac{\partial^2 f}{\partial x_n \partial x_1} & \frac{\partial^2 f}{\partial x_n \partial x_2} & \cdots & \frac{\partial^2 f}{\partial x_n^2} \end{bmatrix}$

Matrix Properties

1. Positive Definite Matrix: $x^T A x > 0$ for all $x \neq 0$
2. Singular Value Decomposition: $A = Q_1 \Sigma Q_2^T$

PCA

1. Covariance Matrix: $C = \frac{1}{n-1} \sum_{i=1}^n (x_i - \bar{x})(x_i - \bar{x})^T$
2. Principal Components: The eigenvectors $v_1, v_2, \ldots$ of the covariance matrix $C$, ordered by the magnitude of their corresponding eigenvalues $\lambda_1 \geq \lambda_2 \geq \ldots$

Gradient Descent

1. Update Rule: $x_{t+1} = x_t - \eta_t \nabla f(x_t)$