Home / Past papers / MLT End Term / this paper

IIT M FOUNDATION FN EXAM QDF1 13 Apr - 13 Apr 2025

MLT End Term18 questions50 marks7 SA · 6 MCQ · 5 MSQFoundation / forenoon / QDF1

Elapsed 00:00

Q1MCQ3 marks

For a given dataset, a 1-Nearest Neighbor (1-NN) and a 3-Nearest Neighbor (3-NN) classifier are applied. Which classifier is likely to exhibit a higher Leave-One-Out Cross Validation (LOOCV) error? In case of tie-breaker, assign a positive class (+) to the data point. question figure

1-NN
3-NN
Both have the same error.

Q2MCQ3 marks

Q3MCQ4 marks

Q4MCQ4 marks

Q5MSQ3 marksselect all that apply

Which of the following statements correctly differentiates PCA and Kernel PCA?

PCA maximizes variance in the original feature space, while Kernel PCAmaximizes variance in a higher-dimensional transformed space.
PCA finds principal components using linear transformations in the originalspace, while Kernel PCA uses non-linear transformations to find principal components in a higher-dimensional space.
Kernel PCA can capture non-linear patterns in data, making it useful when PCAfails to represent complex structures in a linear space.
PCA and Kernel PCA always yield identical results regardless of the datasetstructure.

Q6MSQ3 marksselect all that apply

Q7MSQ2 marksselect all that apply

Which of the following statements are true for bagging?

The final model has lesser variance than the individual learners.
The final model has a higher variance than the individual learners.
Estimators in bagging can be trained parallely.
If the number of data points is large, typically two-third of the data pointsremain unselected in bags.

Q8MSQ2 marksselect all that apply

It penalizes large coefficients to reduce overfitting.
It shrinks the coefficients but does not set them to zero.
It forces more coefficients to be exactly zero, performing feature selection.
It has no effect on the regression model.

Q9SA2 marks

For a decision tree, each node has exactly two child nodes (balanced tree). If the tree has a depth of 3, how many leaf nodes are there?

Your numeric answer:

Q10SA3 marks

Your numeric answer:

Q11SA3 marks

Your numeric answer:

Q12SA3 marks

Your numeric answer:

Comprehension passage - the questions below it refer to this

Based on the above data, answer the given subquestions. question figure

Q13SA3 marks

Your numeric answer:

Q14MCQ2 marks

Will the algorithm converge after this update?

Comprehension passage - the questions below it refer to this

Based on the above data, answer the given subquestions. question figure

Q15MCQ2 marks

It is a circle.
It is a straight line.
It is an ellipse.
It is a parabola.

Q16MSQ3 marksselect all that apply

Which of the following training data points are certainly not support vectors?

Comprehension passage - the questions below it refer to this

Based on the above data, answer the given subquestions. question figure

Q17SA3 marks

Your numeric answer:

Q18SA2 marks

Your numeric answer:

Answer key

Q1 (MCQ, 3 marks): A
Q2 (MCQ, 3 marks): C
Q3 (MCQ, 4 marks): B
Q4 (MCQ, 4 marks): B
Q5 (MSQ, 3 marks): A, B, C
Q6 (MSQ, 3 marks): B, D, E
Q7 (MSQ, 2 marks): A, C
Q8 (MSQ, 2 marks): A, C
Q9 (SA, 2 marks): 8
Q10 (SA, 3 marks): 24
Q11 (SA, 3 marks): 20
Q12 (SA, 3 marks): 11
Q13 (SA, 3 marks): 1
Q14 (MCQ, 2 marks): A
Q15 (MCQ, 2 marks): C
Q16 (MSQ, 3 marks): A, C
Q17 (SA, 3 marks): 0.64 to 0.68
Q18 (SA, 2 marks): 1

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Here’s a structured summary of core topics, concepts, principles, and equations for the Diploma-Level Machine Learning Techniques Exam, followed by Mermaid knowledge graphs to visualize relationships.

Core Topics & Key Concepts

1. Supervised Learning

A. k-Nearest Neighbors (k-NN)

Concept: Classifies a data point based on the majority class of its k nearest neighbors.
Key Points:
- 1-NN: High variance (overfits), sensitive to noise.
- Higher k (e.g., 3-NN): Smoother decision boundaries, lower variance, but may underfit.
- Leave-One-Out Cross-Validation (LOOCV): Evaluates model by leaving one data point out as validation.
- Tie-Breaker: Default to positive class (+) if votes are tied.
Equation:
- Distance metric (Euclidean):

d(\mathbf{x}, \mathbf{x}') = \sqrt{\sum_{i=1}^n (x_i - x_i')^2}

B. Linear Regression

Concept: Models the relationship between features (X) and target (y) as a linear equation.
Key Points:
- Normal Equation: Closed-form solution for weights (w).
- Assumptions: Linear relationship, no multicollinearity, homoscedasticity.
Equations:
- Model:

y = w_1 x_1 + w_2 x_2 + \dots + w_n x_n + b

Normal Equation (for squared error loss):

\mathbf{w} = (X^T X)^{-1} X^T \mathbf{y}

Ridge Regression (L2 Regularization):

\min_{\theta} \|\mathbf{y} - X\theta\|_2^2 + \lambda \|\theta\|_2^2

Effect of λ:
- High λ: Shrinks coefficients toward zero (underfitting).
- Low λ: Coefficients remain large (overfitting).

C. Logistic Regression & Perceptron

Perceptron Algorithm:
- Update Rule (for misclassified point):

\mathbf{w} \leftarrow \mathbf{w} + \eta \cdot y_i \mathbf{x}_i

Convergence: Guaranteed if data is linearly separable.

D. Support Vector Machines (SVM)

Concept: Finds the hyperplane maximizing the margin between classes.
Key Points:
- Hard-Margin SVM: No misclassifications allowed.
- Soft-Margin SVM: Allows misclassifications via slack variables (ξ).
- Kernel Trick: Transforms data into higher dimensions for non-linear separation.
Equations:
- Primal Problem (Soft-Margin):

\min_{w, \xi} \frac{1}{2} \|w\|^2 + C \sum_{i=1}^n \xi_i

Subject to:

y_i (w^T x_i + b) \geq 1 - \xi_i, \quad \xi_i \geq 0

Effect of C:
- High C: Strict margin (hard-margin-like).
- Low C: Wider margin (more misclassifications allowed).

E. AdaBoost

Concept: Ensemble method that combines weak learners (e.g., decision stumps) into a strong classifier.
Key Points:
- Weight Update: Increases weights for misclassified points.
- Error Calculation:

\epsilon_t = \sum_{i=1}^n D_t(i) \cdot \mathbb{I}(y_i \neq h_t(x_i))

Weight Update Rule:

D_{t+1}(i) = \frac{D_t(i) \cdot e^{-\alpha_t y_i h_t(x_i)}}{Z_t}

where $\alpha_t = \frac{1}{2} \ln \left( \frac{1 - \epsilon_t}{\epsilon_t} \right)$ .

2. Unsupervised Learning

A. K-Means Clustering

Concept: Partitions data into K clusters by minimizing within-cluster variance.
Key Points:
- Lloyd’s Algorithm:
  1. Initialize centroids.
  2. Assign points to nearest centroid.
  3. Recompute centroids.
  4. Repeat until convergence.
- Sensitive to Initialization: Use k-means++ for better centroids.

B. Principal Component Analysis (PCA) & Kernel PCA

PCA:
- Concept: Linear dimensionality reduction by maximizing variance.
- Steps:
  1. Standardize data.
  2. Compute covariance matrix.
  3. Eigenvalue decomposition.
  4. Project data onto top k eigenvectors.
Kernel PCA:
- Concept: Non-linear PCA using kernel functions (e.g., polynomial, RBF).
- Key Point: Works in higher-dimensional feature space.

3. Ensemble Methods

A. Bagging (Bootstrap Aggregating)

Concept: Reduces variance by averaging multiple models trained on bootstrapped samples.
Key Points:
- Parallel Training: Models are independent.
- Example: Random Forest (bagging + decision trees).
- Out-of-Bag (OOB) Error: Estimates generalization error.

B. Boosting (AdaBoost, Gradient Boosting)

Concept: Sequentially corrects errors of previous models.
Key Points:
- AdaBoost: Focuses on misclassified points.
- Gradient Boosting: Fits residual errors.

4. Neural Networks

Concept: Computational graph with layers of neurons.
Key Points:
- Activation Functions: Sigmoid, ReLU, tanh.
- Forward Propagation:

h = \sigma(W^{(1)} x + b^{(1)}), \quad y = \sigma(W^{(2)} h + b^{(2)})

Backpropagation: Computes gradients via chain rule.

5. Probabilistic Models

A. Naïve Bayes

Concept: Classifies using Bayes’ theorem with feature independence assumption.
Key Points:
- Gaussian Naïve Bayes: Continuous features modeled as normal distributions.
- Bernoulli Naïve Bayes: Binary features.
- Parameters to Estimate:
  - For Gaussian: Mean (μ) and variance (σ²) per class per feature.
  - For Bernoulli: Probability (p) per class per feature.
  - Class priors (P(y)).

B. Maximum Likelihood Estimation (MLE)

Concept: Estimates parameters by maximizing the likelihood function.
Example (Uniform Distribution):
- PDF:

f(x; \theta) = \begin{cases} \frac{1}{\theta}, & 0 \leq x \leq \theta \\ 0, & \text{otherwise} \end{cases}

MLE for θ: $\hat{\theta} = \max(X_1, X_2, \dots, X_n)$ .

6. Model Evaluation & Regularization

A. Bias-Variance Tradeoff

Bias: Error due to overly simplistic model (underfitting).
Variance: Error due to overly complex model (overfitting).
Regularization:
- L1 (Lasso): Penalizes absolute coefficients (sparse solutions).
- L2 (Ridge): Penalizes squared coefficients (smooth solutions).

B. Cross-Validation

LOOCV: Special case of k-fold CV where k = n (number of samples).

7. Decision Trees

Concept: Recursively splits data based on feature thresholds.
Key Points:
- Entropy/Information Gain: Measures split quality.
- Pruning: Reduces overfitting.
- Leaf Nodes: In a balanced binary tree of depth d, number of leaves = 2^d.

Graph 2: Unsupervised Learning & Ensemble Methods

Graph 3: Probabilistic Models & Neural Networks

Graph 4: Model Evaluation & Regularization

Key Equations Summary

Topic	Equation
Linear Regression	$\mathbf{w} = (X^T X)^{-1} X^T \mathbf{y}$
Ridge Regression	$\min_{\theta} \\|\mathbf{y} - X\theta\\|_2^2 + \lambda \\|\theta\\|_2^2$
Perceptron Update	$\mathbf{w} \leftarrow \mathbf{w} + \eta \cdot y_i \mathbf{x}_i$
SVM (Soft-Margin)	$\min_{w, \xi} \frac{1}{2} \\|w\\|^2 + C \sum_{i=1}^n \xi_i$
AdaBoost Weight Update	$D_{t+1}(i) = \frac{D_t(i) \cdot e^{-\alpha_t y_i h_t(x_i)}}{Z_t}$
Gaussian Naïve Bayes	$P(x\|y) = \frac{1}{\sqrt{2\pi\sigma_y^2}} e^{-\frac{(x-\mu_y)^2}{2\sigma_y^2}}$
MLE (Uniform Dist.)	$\hat{\theta} = \max(X_1, X_2, \dots, X_n)$
K-Means Objective	$\min \sum_{i=1}^k \sum_{x \in C_i} \\|x - \mu_i\\|^2$
PCA	$\text{Covariance Matrix} = \frac{1}{n} X^T X$
Kernel PCA	$K(x, x') = \phi(x)^T \phi(x')$ (e.g., polynomial kernel)