IIT M IMPROVEMENT AN EXAM QIB4 03 Aug - 03 Aug 2025

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Summary of Core Topics, Concepts, Principles, and Equations

1. Optimization and Critical Points

• Gradient and Hessian Matrix: For a function $f: \mathbb{R}^n \to \mathbb{R}$, the gradient $\nabla f(x^*) = 0$ and the Hessian matrix $\nabla^2 f(x^*)$ being positive definite indicate a local minimum at $x^*$.
• Quadratic Forms: Understanding positive definiteness, positive semi-definiteness, negative definiteness, and negative semi-definiteness of quadratic forms.

2. Linear Algebra

• Eigenvalues and Eigenvectors: Properties of eigenvalues and eigenvectors, especially for matrices like $AA^T$ and $A^T A$.
• Singular Value Decomposition (SVD): Decomposition of a matrix $A$ into $Q_1 \Sigma Q_2^T$, where $Q_1$ and $Q_2$ are orthogonal matrices and $\Sigma$ is a diagonal matrix of singular values.
• Principal Component Analysis (PCA): Projection of data onto lower-dimensional spaces using PCA, involving the computation of eigenvalues and eigenvectors of the covariance matrix.

3. Matrix Properties

• Hermitian and Unitary Matrices: Properties and identification of Hermitian and unitary matrices.
• Positive Definite Matrices: Identification and properties of positive definite matrices.

4. Gradient Descent

• Algorithm: Iterative method to minimize a function using the gradient and learning rate $\eta_t$.
• Update Rule: $x_{t+1} = x_t - \eta_t \nabla f(x_t)$.

5. Covariance Matrix and PCA

• Covariance Matrix: Calculation and properties of the covariance matrix for a given dataset.
• Eigenvalues and Principal Components: Using eigenvalues to determine the principal components for dimensionality reduction.

Equations

Optimization

1. Gradient: $\nabla f(x) = \left( \frac{\partial f}{\partial x_1}, \frac{\partial f}{\partial x_2}, \ldots, \frac{\partial f}{\partial x_n} \right)$
2. Hessian Matrix: $\nabla^2 f(x) = \begin{bmatrix} \frac{\partial^2 f}{\partial x_1^2} & \frac{\partial^2 f}{\partial x_1 \partial x_2} & \cdots & \frac{\partial^2 f}{\partial x_1 \partial x_n} \\ \frac{\partial^2 f}{\partial x_2 \partial x_1} & \frac{\partial^2 f}{\partial x_2^2} & \cdots & \frac{\partial^2 f}{\partial x_2 \partial x_n} \\ \vdots & \vdots & \ddots & \vdots \\ \frac{\partial^2 f}{\partial x_n \partial x_1} & \frac{\partial^2 f}{\partial x_n \partial x_2} & \cdots & \frac{\partial^2 f}{\partial x_n^2} \end{bmatrix}$

Linear Algebra

1. Eigenvalue Equation: $A v = \lambda v$
2. Singular Value Decomposition: $A = Q_1 \Sigma Q_2^T$
3. Covariance Matrix: $C = \frac{1}{n-1} \sum_{i=1}^n (x_i - \bar{x})(x_i - \bar{x})^T$

Gradient Descent

1. Update Rule: $x_{t+1} = x_t - \eta_t \nabla f(x_t)$

Graph 1: Optimization and Critical Points

Graph 2: Linear Algebra Concepts

Graph 3: Gradient Descent

Graph 4: Covariance Matrix and PCA

These graphs and summaries should help you understand the core topics and their relationships for the exam.