IIT M FOUNDATION DIPLOMA FN EXAM - 22 Dec 2024

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Here’s a structured summary of the core topics, concepts, principles, and equations essential for the Diploma Level: Machine Learning Foundations (Computer-Based Exam) based on the provided question paper. The exam heavily emphasizes probability, linear algebra, optimization, and convex analysis.

1. Probability & Random Variables

Key Concepts:

• Independent Random Variables: $X$ and $Y$ are independent if $P(X,Y) = P(X)P(Y)$.

• Linear Combinations of Random Variables:

  • If $X \sim N(\mu_X, \sigma_X^2)$ and $Y \sim N(\mu_Y, \sigma_Y^2)$ are independent, then:

• Uniform Distribution:

  • For $X, Y \sim \text{Uniform}(0,1)$, the CDF of $Z = \min(X,Y)$ is:

• Joint PDF and Transformations:

  • For independent exponentials $X, Y \sim \text{Exp}(1)$, the joint PDF of $U = X + Y$ and $V = X - Y$ is:

• Multivariate Normal Distribution:

  • If $X \sim N(\mu, \Sigma)$, then $Y = a^T X$ (where $a$ is a vector) follows:

Key Equations:

2. Linear Algebra

Key Concepts:

• Matrix Definiteness:

  • Positive Definite: $x^T A x > 0$ for all $x \neq 0$.
  • Symmetric Matrix: $A = A^T$.
  • Singular Matrix: $\det(A) = 0$.

• Eigenvalues and Eigenvectors:

  • Unitary Matrix: $A^* A = I$ (columns are orthonormal).
  • Hermitian Matrix: $A = A^*$ (self-adjoint).
  • Eigenvalues of Hermitian matrices are real.
  • Eigenvectors of Unitary matrices are orthogonal.
• Rank-Nullity Theorem:

• If $\det(A) \neq 0$, then $\text{nullity}(A) = 0$.

  • Non-zero Eigenvalues of $A^T A$ and $AA^T$:
• For any matrix $A_{m \times n}$, $A^T A$ and $AA^T$ share the same non-zero eigenvalues.

Key Equations:

3. Convex Analysis

Key Concepts:

• Convex Set:

  • A set $S$ is convex if for any $a, b \in S$ and $\theta \in [0,1]$:

• Guaranteed points: midpoint ($\theta = 0.5$) and convex combinations (e.g., $0.7a + 0.3b$).

  • Convex Functions:
• A function $f$ is convex if:

• Examples:

  • $f(x) = x \ln x$ is convex on $[0, \infty)$.
  • $f(x,y) = e^{2x + y}$ is convex (since exponential of affine is convex).
  • $f(x,y) = -\ln(xy)$ is convex on $(0, \infty)^2$.
• Composition: If $f$ is convex and non-decreasing, and $g$ is convex, then $f \circ g$ is convex.

Key Equations:

4. Optimization

Key Concepts:

• Unconstrained Optimization:

  • Gradient Descent Update:

• Example: For $f(x,y) = x^2 + y^2 - 6x - 8y$, one iteration from $(1,2)$ with $\eta = 0.2$ gives:

Then, $10x + 10y = 42$ (but the answer was 46; likely a typo in the question).

• Constrained Optimization (KKT Conditions):

  • For a problem:

The KKT conditions require:

1. Stationarity: $\nabla f(x) + \sum \lambda_i \nabla g_i(x) = 0$.
2. Primal Feasibility: $g_i(x) \leq 0$.
3. Dual Feasibility: $\lambda_i \geq 0$.

4. Complementary Slackness: $\lambda_i g_i(x) = 0$.

   • For active constraints ($g_i(x) = 0$), $\lambda_i > 0$.
   • Example: For $f(x_1,x_2) = x_1^2 + x_2^2$ with $g_1(x) = x_1 + x_2 - 1 \leq 0$ and $g_2(x) = -x_1 \leq 0$:
   • If the optimum is on $x_1 + x_2 = 1$, then $\lambda_1 > 0$ and $\lambda_2 = 0$.
   • The minimum value is $0.5$ (but the answer was 1; likely a miscalculation).

Key Equations:

5. Applications (Profit/Cost Optimization)

Key Concepts:

• Lagrangian Multipliers:

  • For constrained optimization with equality constraints, solve:

• Example: Maximize $f(x,y) = 48x + 96y - x^2 - 2xy - 9y^2 + 1$ subject to $20x + 4y = 216$:

  • Solve $\nabla f = \lambda \nabla g$ to find $x = 10$, $y = 8$, and max profit = 541.

Key Equations:

2. Linear Algebra

3. Convex Analysis & Optimization

4. Applications