Home / Past papers / MLT End Term / this paper

IIT M DIPLOMA FN EXAM QDD2 13 Apr 2025 - 13 Apr 2025

MLT End Term18 questions50 marks7 SA · 6 MCQ · 5 MSQDiploma / forenoon / QDD2

Elapsed 00:00

Q1MCQ3 marks

For a given dataset, a 1-Nearest Neighbor (1-NN) and a 3-Nearest Neighbor (3-NN) classifier are applied. Which classifier is likely to exhibit a higher Leave-One-Out Cross Validation (LOOCV) error? In case of tie-breaker, assign a positive class (+) to the data point. question figure

1-NN
3-NN
Both have the same error.

Q2MCQ3 marks

Q3MCQ4 marks

Q4MCQ4 marks

Q5MSQ3 marksselect all that apply

Which of the following statements correctly differentiates PCA and Kernel PCA?

PCA maximizes variance in the original feature space, while Kernel PCAmaximizes variance in a higher-dimensional transformed space.
PCA finds principal components using linear transformations in the originalspace, while Kernel PCA uses non-linear transformations to find principal components in a higher-dimensional space.
Kernel PCA can capture non-linear patterns in data, making it useful when PCAfails to represent complex structures in a linear space.
PCA and Kernel PCA always yield identical results regardless of the datasetstructure.

Q6MSQ3 marksselect all that apply

Q7MSQ2 marksselect all that apply

Which of the following statements are true for bagging?

The final model has lesser variance than the individual learners.
The final model has a higher variance than the individual learners.
Estimators in bagging can be trained parallely.
If the number of data points is large, typically two-third of the data pointsremain unselected in bags.

Q8MSQ2 marksselect all that apply

It penalizes large coefficients to reduce overfitting.
It shrinks the coefficients but does not set them to zero.
It forces more coefficients to be exactly zero, performing feature selection.
It has no effect on the regression model.

Q9SA2 marks

For a decision tree, each node has exactly two child nodes (balanced tree). If the tree has a depth of 3, how many leaf nodes are there?

Your numeric answer:

Q10SA3 marks

Your numeric answer:

Q11SA3 marks

Your numeric answer:

Q12SA3 marks

Your numeric answer:

Comprehension passage - the questions below it refer to this

Based on the above data, answer the given subquestions. question figure

Q13SA3 marks

Your numeric answer:

Q14MCQ2 marks

Will the algorithm converge after this update?

Comprehension passage - the questions below it refer to this

Based on the above data, answer the given subquestions. question figure

Q15MCQ2 marks

It is a circle.
It is a straight line.
It is an ellipse.
It is a parabola.

Q16MSQ3 marksselect all that apply

Which of the following training data points are certainly not support vectors?

Comprehension passage - the questions below it refer to this

Based on the above data, answer the given subquestions. question figure

Q17SA3 marks

Your numeric answer:

Q18SA2 marks

Your numeric answer:

Answer key

Q1 (MCQ, 3 marks): A
Q2 (MCQ, 3 marks): C
Q3 (MCQ, 4 marks): B
Q4 (MCQ, 4 marks): B
Q5 (MSQ, 3 marks): A, B, C
Q6 (MSQ, 3 marks): B, D, E
Q7 (MSQ, 2 marks): A, C
Q8 (MSQ, 2 marks): A, C
Q9 (SA, 2 marks): 8
Q10 (SA, 3 marks): 24
Q11 (SA, 3 marks): 20
Q12 (SA, 3 marks): 11
Q13 (SA, 3 marks): 1
Q14 (MCQ, 2 marks): A
Q15 (MCQ, 2 marks): C
Q16 (MSQ, 3 marks): A, C
Q17 (SA, 3 marks): 0.64 to 0.68
Q18 (SA, 2 marks): 1

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Here’s a structured summary of core topics, concepts, principles, and equations for the Diploma-Level Machine Learning Techniques exam, along with Mermaid knowledge graphs to visualize relationships.

1. Core Topics & Key Concepts

A. Supervised Learning

k-Nearest Neighbors (k-NN)
- LOOCV Error: 1-NN often has higher variance (overfits) than 3-NN, leading to higher LOOCV error.
- Tie-Breaker: Assign positive class (+) if ties occur.
- Key Idea: Larger k smooths decision boundaries (reduces overfitting).

Linear Regression (Normal Equation)
- Model: $y = w_1 x_1 + w_2 x_2 + \dots + w_d x_d$
- Normal Equation:

w = (X^T X)^{-1} X^T y

Assumptions: Linear relationship, no multicollinearity, homoscedasticity.

Ridge Regression (L2 Regularization)
- Objective:

\min_w \|y - Xw\|_2^2 + \lambda \|w\|_2^2

Effect of λ:
- High λ: Shrinks coefficients toward zero (underfitting).
- Low λ: Coefficients approach OLS solution (overfitting).
Example: Match $\theta$ vectors to $\lambda$ values (smaller $\theta$ → larger $\lambda$ ).

Lasso Regression (L1 Regularization)
- Objective:

\min_w \|y - Xw\|_2^2 + \lambda \|w\|_1

Effect of λ:
- Increases λ: Forces more coefficients to exactly zero (feature selection).
- Sparsity: Unlike Ridge, Lasso performs automatic feature selection.

Support Vector Machines (SVM)
- Hard-Margin SVM:

\min_w \frac{1}{2} \|w\|^2 \quad \text{s.t.} \quad y_i(w^T x_i) \geq 1

Soft-Margin SVM:

\min_{w,\epsilon} \frac{1}{2} \|w\|^2 + C \sum \epsilon_i \quad \text{s.t.} \quad y_i(w^T x_i) \geq 1 - \epsilon_i, \epsilon_i \geq 0

Effect of C:
- C → 0: Approaches hard-margin (no slack).
- C → ∞: Allows more slack (overfitting).
Kernel Trick: Transforms data to higher dimensions (e.g., polynomial kernel → ellipse decision boundary).

AdaBoost
- Weight Update: Misclassified points get higher weights in next iteration.
- Example: If 1 point is misclassified (out of 4), its weight increases to 1/2, others decrease to 1/6.
- Formula:

D_{t+1}(i) = \frac{D_t(i) \exp(-\alpha_t y_i f_t(x_i))}{Z_t}

where $\alpha_t = \frac{1}{2} \ln \left( \frac{1 - \epsilon_t}{\epsilon_t} \right)$ .

Decision Trees
- Binary Tree: Each node splits into 2 children.
- Leaf Nodes: For depth d, number of leaves = $2^d$ .
- Example: Depth 3 → 8 leaves.

B. Unsupervised Learning

K-Means (Lloyd’s Algorithm)
- Steps:
  1. Initialize centroids (e.g., $(1,2)$ and $(9,10)$ ).
  2. Assign points to nearest centroid.
  3. Update centroids as mean of assigned points.
- Convergence: Stops when centroids stabilize.
- Example: Final centroids $(c_1, d_1)$ and $(c_2, d_2)$ → compute $c_1 \times c_2$ .

Principal Component Analysis (PCA) vs. Kernel PCA
- PCA:
  - Maximizes variance in original feature space.
  - Linear transformation: $X_{new} = X W$ .
- Kernel PCA:
  - Maximizes variance in higher-dimensional space (via kernel trick).
  - Captures non-linear patterns (e.g., circles, ellipses).
- Key Difference: PCA is linear; Kernel PCA is non-linear.

C. Probabilistic Models

Naïve Bayes
- Assumption: Features are conditionally independent given the class.
- Parameter Count:
  - Gaussian features: 2 parameters (μ, σ²) per feature per class.
  - Bernoulli features: 1 parameter (p) per feature per class.
  - Class priors: 1 parameter per class.
- Example: 2 Gaussian features + 1 Bernoulli feature + 2 classes → 11 parameters (not 12; class priors sum to 1).

Maximum Likelihood Estimation (MLE)
- Uniform Distribution: $f(x; \theta) = \frac{1}{\theta}$ for $0 \leq x \leq \theta$ .
- MLE for θ: $\hat{\theta} = \max(X_i)$ .
- Example: Sample $\{10, 15, 12, 20, 17\}$ → $\hat{\theta} = 20$ .

D. Ensemble Methods

Bagging (Bootstrap Aggregating)
- Key Ideas:
  - Trains multiple models on bootstrapped samples.
  - Reduces variance (vs. individual learners).
  - Parallelizable: Estimators are independent.
- Out-of-Bag (OOB) Error: ~33% of data is left out per tree (for large datasets).

Perceptron Algorithm
- Update Rule:

w \leftarrow w + \eta \cdot y_i x_i \quad \text{if} \quad y_i(w^T x_i) \leq 0

Convergence: Guaranteed if data is linearly separable.
Example: After 1 iteration, squared weight length = 1.

E. Neural Networks

Single-Layer Neural Network
- Architecture:
  - Input layer → Hidden layer (2 neurons) → Output layer (sigmoid).
- Forward Pass:

h_1 = \sigma(w_1^{(1)} x_1 + w_2^{(1)} x_2), \quad h_2 = \sigma(w_1^{(2)} x_1 + w_2^{(2)} x_2)

p = \sigma(w^{out}_1 h_1 + w^{out}_2 h_2)

Example: For $x_{test} = [2, 4]$ , compute $p \approx 0.66$ .

2. Key Equations

Topic	Equation
Linear Regression	$w = (X^T X)^{-1} X^T y$
Ridge Regression	$\min_w \\|y - Xw\\|_2^2 + \lambda \\|w\\|_2^2$
Lasso Regression	$\min_w \\|y - Xw\\|_2^2 + \lambda \\|w\\|_1$
SVM (Primal)	$\min_w \frac{1}{2} \\|w\\|^2 + C \sum \epsilon_i$
SVM (Dual)	$\max_\alpha \sum \alpha_i - \frac{1}{2} \sum \alpha_i \alpha_j y_i y_j x_i^T x_j$
AdaBoost Weight Update	$D_{t+1}(i) = \frac{D_t(i) \exp(-\alpha_t y_i f_t(x_i))}{Z_t}$
PCA	$X_{new} = X W$ (where $W$ = eigenvectors of $X^T X$ )
Kernel PCA	$K = \phi(X) \phi(X)^T$ (kernel matrix)
Perceptron Update	$w \leftarrow w + \eta y_i x_i$ (if misclassified)
MLE (Uniform)	$\hat{\theta} = \max(X_i)$
Naïve Bayes (Gaussian)	$P(x\|y) = \frac{1}{\sqrt{2\pi\sigma_y^2}} \exp\left(-\frac{(x-\mu_y)^2}{2\sigma_y^2}\right)$

Graph 2: Unsupervised Learning & Probabilistic Models

Graph 3: Ensemble Methods & Neural Networks

4. Exam Tips

k-NN: Higher k → smoother boundaries → lower LOOCV error.
Ridge/Lasso: Match $\theta$ vectors to $\lambda$ (smaller $\theta$ → larger $\lambda$ ).
SVM: $C \to 0$ → hard margin; $C \to \infty$ → soft margin.
AdaBoost: Misclassified points get higher weights (e.g., 1/2 vs 1/6).
PCA vs Kernel PCA: PCA is linear; Kernel PCA captures non-linear patterns.
Naïve Bayes: Count parameters carefully (class priors sum to 1).
Perceptron: Converges if data is linearly separable.
Neural Networks: Compute hidden layer outputs step-by-step.