IIT M IMPROVEMENT AN EXAM QDD4 22 - 22 Dec 2024

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Here’s a structured summary of core topics, concepts, principles, and equations for the Diploma Level: Machine Learning Foundations (Computer-Based Exam), along with Mermaid knowledge graphs to visualize relationships.

---

Core Topics & Key Concepts

1. Linear Algebra & Matrix Theory

Key Concepts:

• Quadratic Forms & Positive Definiteness

  • A quadratic form: $f(\mathbf{x}) = \mathbf{x}^T A \mathbf{x}$.
  • Positive Definite Matrix: $\mathbf{x}^T A \mathbf{x} > 0$ for all $\mathbf{x} \neq 0$.
  • Eigenvalues: All eigenvalues of a positive definite matrix are positive.
  • Singular Matrix: $\det(A) = 0$ ⇒ at least one eigenvalue is zero.
  • Symmetric Matrix: $A = A^T$.

• Eigenvalues & Eigenvectors

  • For a matrix $A$, $A \mathbf{v} = \lambda \mathbf{v}$.
  • Unitary Matrix: $A^{-1} = A^*$ (conjugate transpose).
  • Hermitian Matrix: $A = A^*$.
  • Orthogonal Eigenvectors: Eigenvectors of a Hermitian matrix are orthogonal.
  • Eigenvalues of Unitary Matrices: Lie on the unit circle (magnitude 1).

• Determinant & Rank

  • $\det(A) = 0$ ⇒ singular matrix ⇒ at least one eigenvalue is zero.
  • Rank-Nullity Theorem: $\text{rank}(A) + \text{nullity}(A) = n$.
  • If $\det(A) \neq 0$, then $\text{nullity}(A) = 0$.
  • Non-zero eigenvalues of $A^T A$ and $A A^T$: Same for any $m \times n$ matrix $A$.

Key Equations:

1. Quadratic Form Expansion:

where

1. Eigenvalue Equation:

1. Rank-Nullity:

---

2. Convex Optimization

Key Concepts:

• Convex Sets

  • A set $S$ is convex if for any $a, b \in S$, $\theta a + (1-\theta)b \in S$ for $\theta \in [0,1]$.

  • Examples of Convex Combinations:

    • Midpoint: $\frac{a + b}{2}$
    • Weighted average: $0.4a + 0.6b$

• Convex Functions

  • A function $f$ is convex if:

• Examples:

  • $f(x) = e^x$ (convex)
  • $f(x) = x \ln x$ (convex for $x > 0$)
  • $f(x) = -\ln x$ (convex for $x > 0$)
  • Composition of convex functions is convex.

• KKT Conditions (Karush-Kuhn-Tucker)

  • Necessary conditions for optimality in constrained optimization.
  • For a problem:

KKT conditions require:

1. Stationarity: $\nabla f(\mathbf{x}) + \sum \lambda_i \nabla g_i(\mathbf{x}) + \sum \mu_j \nabla h_j(\mathbf{x}) = 0$
2. Primal Feasibility: $g_i(\mathbf{x}) \leq 0$, $h_j(\mathbf{x}) = 0$
3. Dual Feasibility: $\lambda_i \geq 0$
4. Complementary Slackness: $\lambda_i g_i(\mathbf{x}) = 0$

Key Equations:

1. Convex Combination:

1. KKT Lagrangian:

1. Gradient Condition:

---

3. Probability & Statistics

Key Concepts:

• Joint & Marginal Distributions

  • Given $f_{X,Y}(x,y)$, find $f_U(u)$ for $U = X + Y$ using transformation techniques.
  • Jacobian Determinant for change of variables.

• Order Statistics

  • For independent $X, Y \sim \text{Uniform}(0,1)$, the CDF of $Z = \min(X,Y)$ is:

• Multivariate Normal Distribution

  • If $\mathbf{X} \sim N(\mu, \Sigma)$, then $Y = a^T \mathbf{X} \sim N(a^T \mu, a^T \Sigma a)$.

• Linear Combinations of Normals

  • If $X \sim N(\mu_1, \sigma_1^2)$, $Y \sim N(\mu_2, \sigma_2^2)$, then:

Key Equations:

1. CDF of Minimum of Uniforms:

1. Linear Combination of Normals:

1. Transformation of Variables:

where $J$ is the Jacobian determinant.

---

4. Optimization Problems

Key Concepts:

• Gradient Descent

  • Update rule:

• Learning Rate ($\eta$): Step size in gradient descent.

• Constrained Optimization

  • Use Lagrange Multipliers or KKT Conditions for inequality constraints.

Key Equations:

1. Gradient Descent Update:

1. Lagrangian for Constrained Optimization:

---

2. Convex Optimization

3. Probability & Statistics

4. Optimization Algorithms

---

Summary of Key Equations

---

Exam Preparation Tips

1. Linear Algebra:

   • Practice constructing matrices from quadratic forms.
   • Know how to check positive definiteness, eigenvalues, and singularity.

1. Convex Optimization:

   • Understand convex sets, functions, and KKT conditions.
   • Solve small optimization problems using KKT.

1. Probability:

   • Master transformations of random variables (Jacobian).
   • Know properties of Uniform and Normal distributions.

1. Optimization Algorithms:

   • Be comfortable with gradient descent updates.
   • Understand constrained optimization (Lagrange multipliers).

---

This summary covers all key topics from the question paper. Focus on equations, definitions, and problem-solving techniques for the exam. Good luck! 🚀