IIT M DIPLOMA FN1 EXAM ETD1 11 Dec 2022 - 11 Dec 2022

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Here’s a structured summary of core topics, concepts, principles, and equations essential for the "Diploma Level: Machine Learning Foundations" exam, along with Mermaid knowledge graphs to visualize relationships.

1. Core Topics & Key Concepts

A. Mathematics for Machine Learning

1. Calculus & Optimization

• Limits and Continuity:

  • Key Concept: A function $f(x)$ is continuous at $x = a$ if $\lim_{x \to a} f(x) = f(a)$.

  • Properties:

    • Sum, product, and composition of continuous functions are continuous.
    • Absolute value of a continuous function is continuous.
  • Example: $\lim_{x \to 0} x \sin(1/x) = 0$ (since $\sin(1/x)$ is bounded).

• Lagrange Multipliers:

  • Key Equation: Maximize/minimize $f(x, y)$ subject to $g(x, y) = c$.

  • Steps:

    1. Define Lagrangian: $\mathcal{L}(x, y, \lambda) = f(x, y) - \lambda (g(x, y) - c)$.
    2. Solve $\nabla \mathcal{L} = 0$.

  • Example: For a rectangle with perimeter $P = 20$, maximize area $A = xy$.

    • Constraint: $2x + 2y = 20$.
    • Solution: $x = y = 5$ (square), so $\frac{\text{Length}}{\text{Breadth}} = 1$.

• Gradient Descent:

  • Key Equation: $w^{t+1} = w^t - \eta \nabla f(w^t)$.
  • Example: For $f(x) = x_1^3 + x_2^3 + x_3^3 - x_1x_2 - x_2x_3 - x_1x_3$, compute one iteration with $\eta = 0.5$ and initial guess $(1, 1, 1)$.

2. Linear Algebra

• Matrix Properties:

  • Column Space vs. Null Space:

    • Column space: Span of columns of $A$.
    • Null space: Solutions to $Ax = 0$.
    • Key Fact: For an $n \times n$ matrix, if $\text{rank}(A) = n$, null space is $\{0\}$.

  • Hermitian and Unitary Matrices:

    • Hermitian: $A = A^H$, eigenvalues are real.
    • Unitary: $A^H A = I$, eigenvalues lie on the unit circle.

  • Orthogonal Matrices:

    • $A^T A = I$, nullity is $0$ (full rank).

• Eigenvalues and Hessians:

  • Hessian Matrix: $H_{ij} = \frac{\partial^2 f}{\partial x_i \partial x_j}$.

  • Convexity:

    • If all eigenvalues of $H$ are $> 0$: Local minimum.
    • If all eigenvalues are $< 0$: Local maximum.
    • Mixed signs: Saddle point.
  • Example: For $f(x, y) = x^2 + y^2 + 2xy$, Hessian is indefinite (eigenvalues: $0, 4$).

3. Probability & Statistics

• Maximum Likelihood Estimation (MLE):

  • Key Equation: For normal distribution, MLE of $\mu$ is the sample mean.
  • Example: Given heights $\{149, 143, ..., 180\}$, MLE of $\mu$ is the average $\approx 154.2$.

• Markov’s Inequality:

  • Key Equation: $P(X \geq k) \leq \frac{E[X]}{k}$.
  • For Exponential Distribution: $P(X \geq k) \leq e^{-\lambda k}$ (not directly Markov, but related to Chernoff bounds).

• Uniform Distribution:

  • Expected Cost Minimization:

    • For $X \sim U[0, 200]$, minimize $E[\text{Cost}]$ where:

• Optimal $a$: Solve $\int_0^a 2(a - x) \, dx = \int_a^{200} (x - a) \, dx$.
• Solution: $a = 66.\overline{6}$.

• Joint Distributions:

  • Transformation of Variables:

    • Given $f_{WX}(w, x)$, find $f_{YZ}(y, z)$ where $Y = W^2$, $Z = WX$.
    • Use Jacobian determinant for change of variables.

4. Principal Component Analysis (PCA)

• Key Properties:

  • Principal components are orthogonal.
  • First PC maximizes variance (eigenvector of covariance matrix with largest eigenvalue).
  • Number of PCs $\leq$ dimensionality of data.

B. Machine Learning Foundations

1. Linear Regression

• Mean Squared Loss (MSL):

  • Key Equation: $\text{MSE} = \frac{1}{n} \sum_{i=1}^n (y_i - f(x_i))^2$.
  • Example: For $f(x) = x$ and data $[(1,1), (2,2), ..., (5,5)]$, MSE = $0.4$.

• Gradient Descent for Regression:

  • Update Rule: $w^{t+1} = w^t - \eta X^T (X w^t - y)$.
  • Example: Given $X, y, w^t$, compute $w^{t+1}$ and $|i + j|$.

2. Optimization & KKT Conditions

• Karush-Kuhn-Tucker (KKT):

  • Conditions for Optimality:

    1. Primal feasibility: $g(x) \leq 0$, $h(x) = 0$.
    2. Dual feasibility: $\lambda \geq 0$.
    3. Complementary slackness: $\lambda g(x) = 0$.
    4. Stationarity: $\nabla f(x) + \sum \lambda \nabla g(x) = 0$.
  • Example: Maximize $f(x, y) = xy$ subject to $x + y^2 \leq 2$, $x, y \geq 0$.

3. Convexity & Positive Semi-Definiteness

• Positive Semi-Definite Matrices:

  • Key Condition: All eigenvalues $\geq 0$.

  • Example: For $f(x) = kx_1^2 + x_2^2 - kx_1x_2$, find $k$ such that $f$ is PSD.

    • Hessian eigenvalues: $\lambda = \frac{2k \pm \sqrt{k^2 + 16}}{4}$.
    • Solution: $k \geq 0$ (but stricter condition may apply; here $k = 4$).

2. Key Equations Summary

Graph 2: Machine Learning Foundations

Graph 3: Relationships Between Topics

4. Exam Preparation Tips

1. Focus on Equations:

   • Memorize MSE, gradient descent update, Lagrange multipliers, and KKT conditions.
   • Understand how to derive Hessians and check convexity.

1. Practice Calculations:

   • Compute MSE for simple regression models.
   • Solve optimization problems with constraints (e.g., rectangle area maximization).
   • Perform one iteration of gradient descent for given functions.

1. Understand Proofs:

   • Why is $\lim_{x \to 0} x \sin(1/x) = 0$?
   • Why are eigenvalues of a Hermitian matrix real?

1. PCA and Statistics:

   • Know how to compute MLE for normal distribution.
   • Understand orthogonality in PCA and how eigenvalues relate to variance.

1. Probability:

   • Work through expected cost minimization for uniform distributions.
   • Apply Markov’s inequality to exponential distributions.

This summary covers all question types in the exam. Use the Mermaid graphs to visualize connections between topics, and prioritize equations for problem-solving.