Home / Past papers / MLT End Term / this paper

IIT M FOUNDATION FN EXAM QDF2 13 Apr - 13 Apr 2025

MLT End Term18 questions50 marks7 SA · 6 MCQ · 5 MSQFoundation / forenoon / QDF2

Elapsed 00:00

Q1MCQ3 marks

For a given dataset, a 1-Nearest Neighbor (1-NN) and a 3-Nearest Neighbor (3-NN) classifier are applied. Which classifier is likely to exhibit a higher Leave-One-Out Cross Validation (LOOCV) error? In case of tie-breaker, assign a positive class (+) to the data point. question figure

1-NN
3-NN
Both have the same error.

Q2MCQ3 marks

Q3MCQ4 marks

Q4MCQ4 marks

Q5MSQ3 marksselect all that apply

Which of the following statements correctly differentiates PCA and Kernel PCA?

PCA maximizes variance in the original feature space, while Kernel PCAmaximizes variance in a higher-dimensional transformed space.
PCA finds principal components using linear transformations in the originalspace, while Kernel PCA uses non-linear transformations to find principal components in a higher-dimensional space.
Kernel PCA can capture non-linear patterns in data, making it useful when PCAfails to represent complex structures in a linear space.
PCA and Kernel PCA always yield identical results regardless of the datasetstructure.

Q6MSQ3 marksselect all that apply

Q7MSQ2 marksselect all that apply

Which of the following statements are true for bagging?

The final model has lesser variance than the individual learners.
The final model has a higher variance than the individual learners.
Estimators in bagging can be trained parallely.
If the number of data points is large, typically two-third of the data pointsremain unselected in bags.

Q8MSQ2 marksselect all that apply

It penalizes large coefficients to reduce overfitting.
It shrinks the coefficients but does not set them to zero.
It forces more coefficients to be exactly zero, performing feature selection.
It has no effect on the regression model.

Q9SA2 marks

For a decision tree, each node has exactly two child nodes (balanced tree). If the tree has a depth of 3, how many leaf nodes are there?

Your numeric answer:

Q10SA3 marks

Your numeric answer:

Q11SA3 marks

Your numeric answer:

Q12SA3 marks

Your numeric answer:

Comprehension passage - the questions below it refer to this

Based on the above data, answer the given subquestions. question figure

Q13SA3 marks

Your numeric answer:

Q14MCQ2 marks

Will the algorithm converge after this update?

Comprehension passage - the questions below it refer to this

Based on the above data, answer the given subquestions. question figure

Q15MCQ2 marks

It is a circle.
It is a straight line.
It is an ellipse.
It is a parabola.

Q16MSQ3 marksselect all that apply

Which of the following training data points are certainly not support vectors?

Comprehension passage - the questions below it refer to this

Based on the above data, answer the given subquestions. question figure

Q17SA3 marks

Your numeric answer:

Q18SA2 marks

Your numeric answer:

Answer key

Q1 (MCQ, 3 marks): A
Q2 (MCQ, 3 marks): C
Q3 (MCQ, 4 marks): B
Q4 (MCQ, 4 marks): B
Q5 (MSQ, 3 marks): A, B, C
Q6 (MSQ, 3 marks): B, D, E
Q7 (MSQ, 2 marks): A, C
Q8 (MSQ, 2 marks): A, C
Q9 (SA, 2 marks): 8
Q10 (SA, 3 marks): 24
Q11 (SA, 3 marks): 20
Q12 (SA, 3 marks): 11
Q13 (SA, 3 marks): 1
Q14 (MCQ, 2 marks): A
Q15 (MCQ, 2 marks): C
Q16 (MSQ, 3 marks): A, C
Q17 (SA, 3 marks): 0.64 to 0.68
Q18 (SA, 2 marks): 1

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Here’s a structured summary of core topics, concepts, principles, and equations essential for the exam, followed by Mermaid knowledge graphs to visualize relationships.

1. Core Topics & Concepts

A. Supervised Learning

k-Nearest Neighbors (k-NN)
- LOOCV Error: 1-NN often has higher variance (overfits) than 3-NN, leading to higher LOOCV error.
- Tie-Breaker: Assign positive class (+) in ties.

Linear Regression
- Normal Equation:

w = (X^T X)^{-1} X^T y

Ridge Regression (L2 Regularization):

\min_w \|y - Xw\|_2^2 + \lambda \|w\|_2^2

Effect of λ:
- Higher λ → shrinks coefficients (reduces overfitting).
- Lower λ → coefficients closer to OLS.
Example: Match weight vectors (θ) to λ values (Q3).

Logistic Regression & SVM
- Soft-Margin SVM:

\min_{w, \epsilon} \frac{1}{2} \|w\|^2 + C \sum \epsilon_i \quad \text{s.t.} \quad y_i(w^T x_i) \geq 1 - \epsilon_i, \epsilon_i \geq 0

C (Regularization):
- C → 0: Hard-margin SVM (no slack).
- C → ∞: Allows more slack (overfits).
Decision Boundary: Linear in original space, non-linear in kernel space.

AdaBoost
- Weight Update Rule:
  - Misclassified points get higher weights.
  - Correctly classified points get lower weights.
  - Example: After 1st iteration (Q5), weights adjust to focus on errors.

Naïve Bayes
- Conditional Independence Assumption:

P(y|x_1, x_2, ..., x_n) \propto P(y) \prod_i P(x_i|y)

Parameter Count:
- Gaussian features: 2 params (μ, σ) per class per feature.
- Bernoulli features: 1 param (p) per class per feature.
- Class priors: 1 param per class (sums to 1).
- Total: (2 * #Gaussian_features + 1 * #Bernoulli_features) * #classes + (#classes - 1).

B. Unsupervised Learning

K-Means (Lloyd’s Algorithm)
- Convergence: Stops when cluster assignments stabilize.
- Initialization Sensitivity: Final centers depend on initial seeds.
- Example: Final centers for given data (Q11).

PCA vs. Kernel PCA
- PCA: Linear transformation maximizing variance in original space.
- Kernel PCA: Non-linear transformation via kernel trick (e.g., polynomial/RBF), maximizes variance in higher-dimensional space.
- Use Case: Kernel PCA for non-linear patterns.

C. Model Evaluation & Ensemble Methods

Bagging (Bootstrap Aggregating)
- Variance Reduction: Averages multiple models (e.g., decision trees) trained on bootstrapped samples.
- Parallel Training: Estimators are independent.
- Out-of-Bag (OOB) Error: ~37% data points left out per tree (not "two-thirds").

Boosting (AdaBoost)
- Sequential Training: Each model corrects errors of the previous one.
- Weighted Voting: Final prediction combines weak learners weighted by accuracy.

D. Regularization & Feature Selection

Lasso (L1 Regularization)
- Effect of λ:
  - Increases λ: Penalizes large coefficients → sparsity (some coefficients = 0).
  - Feature Selection: Automatically drops irrelevant features.
- Contrast with Ridge (L2): Ridge shrinks but rarely zeros coefficients.

Maximum Likelihood Estimation (MLE)
- Uniform Distribution MLE:

\hat{\theta} = \max(X_i) \quad \text{for} \quad X_i \sim \text{Uniform}[0, \theta]

Example: MLE for θ given samples (Q12).

E. Neural Networks

Single-Layer Perceptron
- Activation: Sigmoid (σ), ReLU, etc.
- Forward Pass:

h = \sigma(w^T x + b), \quad \sigma(z) = \frac{1}{1 + e^{-z}}

Convergence: Guaranteed if data is linearly separable (Q14–15).

Kernel SVM with Polynomial Kernel
- Degree-2 Kernel:

\phi(x) = [1, x_1^2, x_2^2, \sqrt{2}x_1x_2, \sqrt{2}x_1, \sqrt{2}x_2]^T

Decision Boundary: Ellipse (Q16) if $w$ has non-zero quadratic terms.

F. Trees & Decision Boundaries

Decision Trees
- Balanced Binary Tree: Leaf nodes = $2^{\text{depth}}$ (Q10: depth 3 → 8 leaves).
Support Vectors
- Non-Support Vectors: Points not on the margin (Q17).

2. Key Equations

Topic	Equation
Linear Regression	$w = (X^T X)^{-1} X^T y$
Ridge Regression	$\min_w \\|y - Xw\\|_2^2 + \lambda \\|w\\|_2^2$
Lasso Regression	$\min_w \\|y - Xw\\|_2^2 + \lambda \\|w\\|_1$
Soft-Margin SVM	$\min_{w, \epsilon} \frac{1}{2}\\|w\\|^2 + C \sum \epsilon_i$
PCA	$\max_{\\|u\\|=1} u^T X^T X u$ (eigenvectors of $X^T X$ )
Kernel PCA	$\max_{\\|\alpha\\|=1} \alpha^T K \alpha$ (kernel matrix $K$ )
AdaBoost Weight Update	$D_t(i) = \frac{D_{t-1}(i) \exp(-\alpha_t y_i h_t(x_i))}{Z_t}$
MLE (Uniform)	$\hat{\theta} = \max(X_i)$
Naïve Bayes	$P(y\|x) \propto P(y) \prod_i P(x_i\|y)$
Perceptron Update	$w \leftarrow w + \eta \cdot y_i x_i$ (if misclassified)
Sigmoid Activation	$\sigma(z) = \frac{1}{1 + e^{-z}}$

Graph 2: Unsupervised Learning & Dimensionality Reduction

Graph 3: Ensemble Methods

Graph 4: Regularization & Feature Selection

Graph 5: Neural Networks & Kernels

4. Exam Tips

k-NN: Higher k → smoother decision boundaries (lower LOOCV error).
Ridge/Lasso: Match weight vectors to λ (smaller λ → larger weights).
SVM: C=0 → hard margin; C→∞ → overfits.
AdaBoost: Misclassified points get higher weights in next iteration.
PCA vs. Kernel PCA: Kernel PCA for non-linear patterns.
Bagging: Reduces variance; estimators are independent.
Lasso: Increases λ → more coefficients zeroed out.
K-Means: Final centers depend on initialization.
MLE for Uniform: $\hat{\theta} = \max(X_i)$ .
Naïve Bayes: Count parameters carefully (Gaussian/Bernoulli).
Perceptron: Converges if data is linearly separable.
Kernel SVM: Degree-2 polynomial → elliptical boundary.