IIT M DEGREE AN EXAM QDB4 31 Aug 2025 - 31 Aug 2025

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Here’s a structured summary of the core topics, concepts, principles, and equations essential for the Diploma Level: Machine Learning Techniques exam, along with Mermaid knowledge graphs to visualize relationships.

1. Core Topics & Concepts

A. Supervised Learning

1. Perceptron Algorithm

   • Concept: Linear classifier for binary classification. Updates weights iteratively if misclassification occurs.

   • Key Equation:

     • Update rule: $w \leftarrow w + \eta \cdot y_i \cdot x_i$ (if $y_i(w^T x_i) \leq 0$).
     • Convergence: Guaranteed if data is linearly separable (Novikoff’s theorem).
   • Example: Question 2 (convergence analysis), Question 9 (scaling data).

1. Support Vector Machines (SVM)

   • Concept: Maximizes the margin between classes. Hard-margin (no misclassification) vs. soft-margin (allows slack variables $\xi_i$).

   • Key Equations:

     • Hard-margin: $\min_w \frac{1}{2} \|w\|^2$ s.t. $y_i(w^T x_i) \geq 1$.
     • Soft-margin: $\min_{w,\xi} \frac{1}{2} \|w\|^2 + C \sum \xi_i$ s.t. $y_i(w^T x_i) \geq 1 - \xi_i$.
     • Kernel trick: $K(x_i, x_j) = \phi(x_i)^T \phi(x_j)$.
   • Example: Question 3 (kernel transformation), Question 4 (support vectors), Question 8 (soft-margin SVM), Question 10 (decision boundary).

1. Logistic Regression

   • Concept: Probabilistic classifier using sigmoid activation.

   • Key Equation:

     • $P(y=1|x) = \frac{1}{1 + e^{-w^T x}}$.
   • Example: Question 12 (probability threshold).

1. Neural Networks

   • Concept: Multi-layer perceptron (MLP) with activation functions (ReLU, sigmoid).

   • Key Equations:

     • Forward pass: $h = \text{ReLU}(W^{(1)}x)$, $\hat{y} = \sigma(W^{(out)} h)$.
   • Example: Question 6 (probability prediction).

1. Linear Regression & Ridge Regression

   • Concept: Fits a linear model to data. Ridge adds L2 regularization.

   • Key Equations:

     • OLS: $\hat{w}_{ML} = (X^T X)^{-1} X^T Y$.
     • Ridge: $\hat{w}_{Ridge} = (X^T X + \lambda I)^{-1} X^T Y$.
   • Example: Question 15 (coefficient calculation), Question 16 (effect of $\lambda$).

B. Unsupervised Learning

1. K-Means Clustering

   • Concept: Partitions data into $K$ clusters by minimizing within-cluster variance.

   • Key Equation:

     • Objective: $\min \sum_{i=1}^n \|x_i - \mu_{z_i}\|^2$.
   • Example: Question 5 (greedy initialization).

1. Principal Component Analysis (PCA)

   • Concept: Dimensionality reduction via eigen decomposition of covariance matrix.

   • Key Equation:

     • Covariance matrix: $C = \frac{1}{n} X^T X$.
     • Total variance: $\sum \lambda_i$.
   • Example: Question 11 (eigenvalues and variance).

C. Probabilistic Models

1. Naive Bayes

   • Concept: Assumes feature independence given the class. Uses MLE for parameter estimation.

   • Key Equation:

     • $P(y|x) \propto P(y) \prod_i P(x_i|y)$.
   • Example: Question 17 (parameter counting), Question 18 (prediction with Laplace smoothing).

1. Maximum A Posteriori (MAP) Estimation

   • Concept: Bayesian parameter estimation with a prior.

   • Key Equation:

     • $\hat{\lambda}_{MAP} = \arg \max_\lambda P(\lambda|X)$.
   • Example: Question 14 (exponential distribution with uniform prior).

D. Ensemble Methods

1. Random Forest

   • Concept: Aggregates predictions from multiple decision trees via averaging (regression) or voting (classification).
   • Example: Question 13 (probability averaging).

E. Decision Trees

1. Decision Tree Algorithm

   • Concept: Recursively splits data based on feature thresholds. Prone to overfitting if unconstrained.

   • Key Points:

     • Handles numerical/categorical features.
     • Sensitive to data perturbations.
   • Example: Question 7 (properties of decision trees).

2. Key Equations Summary

Graph 2: Unsupervised Learning & Probabilistic Models

Graph 3: Ensemble Methods & Decision Trees

Graph 4: Key Equations Flow

4. Exam-Specific Tips

1. Perceptron/SVM:

   • For convergence (Question 2), cycle through data points and update weights manually.
   • For kernels (Question 3), match terms in $k(x, y)$ to $\phi(x)^T \phi(y)$.
   • Support vectors (Question 4) are points where $y_i(w^T x_i) = 1$.

1. K-Means:

   • Greedy initialization (Question 5) does not guarantee optimality but can achieve $F_{optimal}$ depending on data.

1. Neural Networks:

   • Compute hidden layer outputs step-by-step (Question 6). ReLU: $\max(0, z)$; Sigmoid: $1/(1 + e^{-z})$.

1. Decision Trees:

   • Overfitting (Question 7) is controlled by max depth/pruning. Trees handle mixed feature types.

1. Probabilistic Models:

   • For Naive Bayes (Questions 17–18), count parameters as $(d + 1) \times c$ (where $d$ = features, $c$ = classes).
   • MAP estimation (Question 14) for exponential distribution: $\text{Posterior} \propto \text{Likelihood} \times \text{Prior}$.

1. Regression:

   • Ridge regression (Questions 15–16): As $\lambda \downarrow$, coefficients approach OLS solution.

1. PCA:

   • Total variance (Question 11) is the sum of eigenvalues. Solve the system of equations for $\lambda_i$.