IIT M DIPLOMA AN1 EXAM ETD1 11 Dec 2022 - 11 Dec 2022

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Here’s a structured summary of core topics, concepts, principles, and equations for the "Diploma Level: Machine Learning Foundations" exam, along with Mermaid knowledge graphs to visualize relationships.

1. Core Topics & Key Concepts

A. Mathematics for Machine Learning

##### 1. Calculus & Optimization

• Limits and Continuity:

  • Definition of limits (e.g., Question 3: Does $\lim_{x \to 0} \frac{1}{\sin(1/x)}$ exist? Answer: No).
  • Continuity of functions (e.g., Question 6: Properties of continuous functions like $|f|$, $f+g$, $fg$, and $g \circ f$).

  • Key Equations:

    • Limit definition: $\lim_{x \to a} f(x) = L$ if $\forall \epsilon > 0, \exists \delta > 0$ such that $|f(x) - L| < \epsilon$ when $|x - a| < \delta$.
    • Continuity at $a$: $\lim_{x \to a} f(x) = f(a)$.

• Gradient Descent:

  • Update rule: $w^{t+1} = w^t - \eta \nabla f(w^t)$ (e.g., Questions 11, 12).

  • Key Equations:

    • Gradient of $f(x_1, x_2, x_3) = x_1^3 + x_2^3 + x_3^3 - x_1x_2 - x_2x_3 - x_1x_3$: $\nabla f = [3x_1^2 - x_2 - x_3, 3x_2^2 - x_1 - x_3, 3x_3^2 - x_1 - x_2]$.
    • For linear regression: $\nabla \text{MSE} = \frac{2}{n} X^T(Xw - y)$.

• Lagrange Multipliers:

  • Used for constrained optimization (e.g., Question 4: Maximize area of a rectangle with perimeter 20).

  • Key Equations:

    • Lagrangian: $\mathcal{L}(x, y, \lambda) = f(x, y) - \lambda g(x, y)$.
    • For rectangle area: $\mathcal{L} = xy - \lambda (2x + 2y - 20)$.
    • Solve $\nabla \mathcal{L} = 0$: $y = \lambda$, $x = \lambda$, $2x + 2y = 20$ → $x = y = 5$.
    • $\frac{2 \times \text{Length}}{\text{Breadth}} = \frac{2 \times 5}{5} = 2$.

• Convexity and Hessian Matrix:

  • A function is convex if its Hessian is positive semi-definite (e.g., Questions 9, 14, 16).

  • Key Equations:

    • Hessian of $f(x, y) = x^2 + y^2 + 2xy$: $H = \begin{bmatrix} 2 & 2 \\ 2 & 2 \end{bmatrix}$ (eigenvalues: 0, 4 → not positive definite).
    • For $f(x) = kx_1^2 + x_2^2 - kx_1x_2$, the Hessian is: $H = \begin{bmatrix} 2k & -k \\ -k & 2 \end{bmatrix}$.
    • Positive semi-definite if $0 \leq k \leq 4$ (eigenvalues $\geq 0$).

##### 2. Linear Algebra

• Matrix Properties:

  • Rank, nullity, column/row space (e.g., Questions 7, 8, 10).

  • Key Concepts:

    • For an $n \times n$ matrix: $\text{rank} + \text{nullity} = n$.
    • Orthogonal matrices: $A^T A = I$ → nullity = 0 (Question 10).
    • Hermitian matrices: $A = A^H$ (eigenvalues are real).
    • Unitary matrices: $A^H A = I$ (eigenvalues lie on the unit circle; 0 cannot be an eigenvalue).

  • Key Equations:

    • Column space = $\text{span}(A_1, A_2, ..., A_n)$.
    • Null space = $\{x | Ax = 0\}$.

• Eigenvalues and Quadratic Forms:

  • Positive semi-definite matrices (e.g., Question 14).

  • Key Equations:

    • For $A = \begin{bmatrix} k & -k/2 \\ -k/2 & 1 \end{bmatrix}$, eigenvalues: $\lambda = \frac{k + 1 \pm \sqrt{(k-1)^2}}{2}$ → $\lambda_1 = k$, $\lambda_2 = 1$.
    • Conditions: $k \geq 0$ and $\det(A) = k - k^2/4 \geq 0$ → $0 \leq k \leq 4$.

##### 3. Probability & Statistics

• Maximum Likelihood Estimation (MLE):

  • For normal distribution: $\hat{\mu} = \frac{1}{n} \sum_{i=1}^n x_i$ (e.g., Question 5).

  • Key Equation:

    • MLE for $\mu$: $\hat{\mu} = \text{sample mean} = 143.2$ (for given heights).

• Poisson Distribution:

  • Markov’s inequality: $P(X \geq k) \leq \frac{\lambda}{k}$ (e.g., Question 15).

  • Key Equation:

    • $X \sim \text{Poisson}(\lambda)$ → $E[X] = \text{Var}(X) = \lambda$.

• Uniform Distribution:

  • Expected cost minimization (e.g., Question 13).

  • Key Equation:

    • Expected cost: $E[\text{Cost}] = \int_0^{100} \text{Cost}(x, a) \cdot \frac{1}{100} dx$.
    • Optimal $a$: Solve $\frac{d}{da} E[\text{Cost}] = 0$ → $a = 33.\overline{3}$.

• Joint Distributions:

  • Transformation of random variables (e.g., Question 18).

  • Key Equation:

    • For $Y = W^2$, $Z = WX$, joint density: $g_{YZ}(y, z) = \frac{3z}{2\sqrt{y}}$ for $0 < y < 1$, $0 < z < \sqrt{y}$.

##### 4. Principal Component Analysis (PCA)

• Properties of Principal Components (e.g., Question 19):

  • Orthogonal components.
  • First PC: Direction of maximum variance (eigenvector of covariance matrix with largest eigenvalue).

  • Key Equations:

    • Covariance matrix: $\Sigma = \frac{1}{n} X^T X$.
    • PCs are eigenvectors of $\Sigma$ sorted by eigenvalues.

B. Machine Learning Foundations

##### 1. Linear Regression

• Mean Squared Error (MSE):

  • $\text{MSE} = \frac{1}{n} \sum_{i=1}^n (y_i - f(x_i))^2$ (e.g., Question 2).

  • Key Equation:

    • For $f(x) = x$ and data $[(1,0), (2,2), ..., (5,5)]$: $\text{MSE} = \frac{1 + 0 + 1 + 1 + 0}{5} = 0.6$.

##### 2. Constrained Optimization

• Karush-Kuhn-Tucker (KKT) Conditions:

  • Used for nonlinear constraints (e.g., Question 17).

  • Key Equations:

    • For $\max xy$ s.t. $x + y^2 \leq 2$: $\mathcal{L} = xy - \lambda (x + y^2 - 2)$. KKT: $\nabla \mathcal{L} = 0$ → $y = \lambda$, $x = 2\lambda y$, $x + y^2 = 2$. Solution: $x = \frac{4}{3}$, $y = \sqrt{\frac{2}{3}}$.

##### 3. Linear Programming

• Formulation:

  • Objective: Maximize profit (e.g., Question 20: $\max 12x + 16y$).
  • Constraints: $x + y \leq 1200$, $y \leq x/2$, $x - 3y \leq 600$, $x, y \geq 0$.

Graph 2: Machine Learning Concepts

Graph 3: Probability & Statistics

Graph 4: Linear Algebra Hierarchy

3. Key Equations Summary