IIT M DIPLOMA AN EXAM QDD3 26 Oct 2025 - 26 Oct 2025

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Here’s a structured summary of core topics, concepts, principles, and equations essential for the Diploma-Level Machine Learning Techniques exam, based on the provided question paper. The exam focuses on clustering (K-means, Lloyd’s algorithm), PCA, Bayesian inference, kernel methods, and basic linear algebra.

1. Core Topics & Key Concepts

A. Clustering (K-means & Lloyd’s Algorithm)

1. Lloyd’s Algorithm (Standard K-means)

   • Steps:

     1. Initialize cluster centers (e.g., randomly or via k-means++).
     2. Assign points to the nearest center (Euclidean distance).
     3. Recompute centers as the mean of assigned points.
     4. Repeat until convergence (centers stabilize).

   • Convergence:

     • Guaranteed to converge (monotonically decreases within-cluster sum of squares, WCSS).
     • May converge to local optima (sensitive to initialization).
   • Sensitivity to Outliers: Yes (outliers can skew cluster means).
   • Optimality: Not guaranteed (depends on initialization).

1. K-means++ Initialization

   • Probability of selecting a point as a center is proportional to its squared distance from existing centers.
   • Reduces sensitivity to initialization.

1. Tie-Breaking in K-means

   • Affects final clusters if points are equidistant to multiple centers.

1. Convergence in 1 Iteration

   • Happens if initial centers are already the final means (e.g., symmetric data).

B. Principal Component Analysis (PCA)

1. Key Ideas:

   • Linear dimensionality reduction via orthogonal projection.
   • Maximizes variance along principal components (PCs).
   • PCs are eigenvectors of the covariance matrix, ordered by eigenvalues (variance).

1. Covariance Matrix (Σ)

   • For a dataset $X \in \mathbb{R}^{n \times d}$ (centered):

• Eigendecomposition: $\Sigma = V \Lambda V^T$, where:

  • $V$: Matrix of eigenvectors (PCs).
  • $\Lambda$: Diagonal matrix of eigenvalues (variances).

1. Number of Non-Zero Eigenvalues:

   • Equals the rank of the data matrix (≤ min(n, d)).
   • If data is d-dimensional but lies in a k-dimensional subspace, only k eigenvalues are non-zero.

1. Variance Explained:

   • Total variance = sum of all eigenvalues.
   • Variance of the i-th PC = i-th eigenvalue ($\lambda_i$).
   • Reconstruction Error: Zero if all PCs are used; otherwise, equals the sum of discarded eigenvalues.

1. Standard PCA Steps:

   1. Center the data (subtract mean).
   2. Compute covariance matrix $\Sigma$.
   3. Eigendecomposition to get PCs and eigenvalues.
   4. Project data onto top-k PCs.

C. Bayesian Inference (Beta Distribution)

1. Beta Distribution:

   • Prior for binomial/proportion data: $\theta \sim \text{Beta}(\alpha, \beta)$.
   • Posterior Mean: For s successes and f failures:

• Special Cases:

  • $\text{Beta}(1,1)$: Uniform prior.
  • $\text{Beta}(0.5, 0.5)$: U-shaped prior.

1. Example Calculation:

   • Given data with 1 success and 24 failures:

     • For $\text{Beta}(1,1)$: Posterior mean = $\frac{1+1}{1+24+1+1} = 0.039$.
     • For $\text{Beta}(0.5,0.5)$: Posterior mean = $\frac{1+0.5}{1+24+0.5+0.5} = 0.034$.

D. Kernel Methods

1. Kernel Trick:

   • Maps data to a higher-dimensional space where linear separation is possible.
   • Kernel function: $K(x_i, x_j) = \phi(x_i)^T \phi(x_j)$.

1. Valid Kernels (must satisfy Mercer’s condition):

   • Linear: $K(x_i, x_j) = x_i^T x_j$.
   • Polynomial: $K(x_i, x_j) = (x_i^T x_j + c)^d$.
   • Gaussian (RBF): $K(x_i, x_j) = \exp(-\gamma \|x_i - x_j\|^2)$.
   • Sigmoid: $K(x_i, x_j) = \tanh(\alpha x_i^T x_j + c)$.

E. Linear Algebra Basics

1. Covariance Matrix:

   • For 2D data $X = \{(x_1, y_1), ..., (x_n, y_n)\}$:

• Variance: $\text{Var}(x) = \frac{1}{n-1} \sum (x_i - \bar{x})^2$.
• Covariance: $\text{Cov}(x,y) = \frac{1}{n-1} \sum (x_i - \bar{x})(y_i - \bar{y})$.

1. Eigenvalues/Eigenvectors:

   • For a 2x2 matrix $A$, solve $\det(A - \lambda I) = 0$.

2. Key Equations

Graph 2: Bayesian Inference & Kernels

Graph 3: Linear Algebra Foundations

4. Exam Tips

1. K-means:

   • Remember it always converges but may not find the global optimum.
   • K-means++ reduces initialization bias.

2. PCA:

   • Number of non-zero eigenvalues = rank of the data.
   • Total variance = sum of all eigenvalues.

3. Bayesian Inference:

   • Posterior mean for Beta prior: plug in successes/failures.

4. Kernels:

   • Valid kernels must be positive semi-definite.

5. Covariance Matrix:

   • Diagonal elements = variances; off-diagonal = covariances.