Home / Past papers / MLT End Term / this paper

IIT M IMPROVEMENT AN EXAM QIM3 13 Apr - 13 Apr 2025

MLT End Term17 questions50 marks6 MCQ · 6 SA · 5 MSQImprovement / afternoon / QIM3

Elapsed 00:00

Q1MSQ2 marksselect all that apply

What is the role of the regularization term in ridge regression?

It ensures that the dataset is normally distributed.
It penalizes large coefficients to reduce overfitting.
It forces more coefficients to be exactly zero, performing feature selection.
It increases the variance of the model.

Q2MSQ2 marksselect all that apply

Which of the following conditions must be satisfied for a decision tree to be pure at a given node?

The entropy of the node is maximum.
The information gain at the node is negative.
All samples at a node belong to the same class.
The entropy of the node is 0.

Q3MCQ3 marks

Q4MCQ3 marks

Q5MCQ2 marks

Kernel PCA with a polynomial kernel of degree 2.
Kernel PCA with a polynomial kernel of degree 3.
Kernel PCA with a polynomial kernel of degree 4.
Standard PCA

Q6MCQ1 mark

Weak learners used in the ensemble method typically perform slightly better than random guessing.

TRUE
FALSE

Q7MSQ3 marksselect all that apply

Model 1 will have a higher bias than Model 2.
Model 2 will have a lower variance than Model 1.
Model 1 is more likely to underfit.
Model 1 is more likely to overfit.

Q8MSQ3 marksselect all that apply

Q9MSQ4 marksselect all that apply

Q10SA4 marks

Your numeric answer:

Q11SA4 marks

Your numeric answer:

Q12SA4 marks

Your numeric answer:

Q13SA4 marks

The following dataset represents different weather conditions and the corresponding decision on whether a person chooses to play tennis: question figure

The following dataset represents different weather conditions and the corresponding decision on whether a person chooses to play tennis:

Your numeric answer:

Q14SA3 marks

Your numeric answer:

Q15SA3 marks

Your numeric answer:

Comprehension passage - the questions below it refer to this

Based on the above data, answer the given subquestions. question figure

Q16MCQ2 marks

Which classifier has the lower training error?

Q17MCQ3 marks

Answer key

Q1 (MSQ, 2 marks): B
Q2 (MSQ, 2 marks): C, D
Q3 (MCQ, 3 marks): B
Q4 (MCQ, 3 marks): A
Q5 (MCQ, 2 marks): B
Q6 (MCQ, 1 marks): A
Q7 (MSQ, 3 marks): A, C
Q8 (MSQ, 3 marks): B, D, E
Q9 (MSQ, 4 marks): B, E
Q10 (SA, 4 marks): 262.2 to 262.8
Q11 (SA, 4 marks): 0.148 to 0.154
Q12 (SA, 4 marks): 4
Q13 (SA, 4 marks): 0.61 to 0.67
Q14 (SA, 3 marks): 1
Q15 (SA, 3 marks): 1
Q16 (MCQ, 2 marks): A
Q17 (MCQ, 3 marks): B

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Here’s a structured summary of core topics, concepts, principles, and equations for the Diploma-Level Machine Learning Techniques exam, along with Mermaid knowledge graphs to visualize relationships.

1. Core Topics & Key Concepts

A. Supervised Learning

Linear Regression & Regularization
- Ordinary Least Squares (OLS):
  - Minimizes $\sum (y_i - \hat{y}_i)^2$ .
  - Closed-form solution: $w = (X^T X)^{-1} X^T y$ .
- Ridge Regression (L2 Regularization):
  - Loss: $\text{MSE} + \lambda \sum w_j^2$ .
  - Penalizes large coefficients (reduces overfitting).
  - Solution: $w = (X^T X + \lambda I)^{-1} X^T y$ .
- Lasso Regression (L1 Regularization):
  - Loss: $\text{MSE} + \lambda \sum |w_j|$ .
  - Forces sparse solutions (feature selection).
- Key Equation:

\text{Ridge: } w = \arg\min_w \|y - Xw\|_2^2 + \lambda \|w\|_2^2

\text{Lasso: } w = \arg\min_w \|y - Xw\|_2^2 + \lambda \|w\|_1

Logistic Regression
- Sigmoid function: $\sigma(z) = \frac{1}{1 + e^{-z}}$ .
- Decision boundary: $\hat{y} = 1$ if $\sigma(w^T x) > 0.5$ .
- Key Equation:

P(y=1|x) = \sigma(w^T x)

Support Vector Machines (SVM)
- Hard-Margin SVM:
  - Maximizes margin $\frac{2}{\|w\|}$ .
  - Constraints: $y_i(w^T x_i + b) \geq 1$ .
- Soft-Margin SVM:
  - Allows misclassification with slack variables $\xi_i$ .
  - Primal problem:

\min_{w,b} \frac{1}{2}\|w\|^2 + C \sum \xi_i \quad \text{s.t.} \quad y_i(w^T x_i + b) \geq 1 - \xi_i, \xi_i \geq 0

Dual Problem:
- Solve for $\alpha$ (Lagrange multipliers).
- Kernel trick: $K(x_i, x_j) = \phi(x_i)^T \phi(x_j)$ .
Key Properties:
- If $\alpha_i = 0$ : Correctly classified (non-support vector).
- If $0 < \alpha_i < C$ : Lies on margin (support vector).
- If $\alpha_i = C$ : Misclassified or inside margin.

Decision Trees
- Impurity Measures:
  - Entropy: $H(S) = -\sum p_i \log_2 p_i$ .
  - Gini Index: $G(S) = 1 - \sum p_i^2$ .
- Splitting Criteria:
  - Information Gain (IG): $\text{IG} = H(\text{parent}) - \sum \frac{|S_v|}{|S|} H(S_v)$ .
- Stopping Conditions:
  - Node is pure (all samples same class, entropy = 0).
  - Max depth reached.

Ensemble Methods
- AdaBoost:
  - Combines weak learners (slightly better than random guessing).
  - Updates weights: Increase for misclassified samples.
  - Final predictor: $H(x) = \text{sign}(\sum \alpha_t h_t(x))$ .
- Key Equation:

\alpha_t = \frac{1}{2} \ln \left( \frac{1 - \epsilon_t}{\epsilon_t} \right), \quad \epsilon_t = \frac{\sum_{i=1}^n w_i \mathbb{I}(y_i \neq h_t(x_i))}{\sum_{i=1}^n w_i}

Perceptron Algorithm
- Updates weights when misclassified: $w \leftarrow w + \eta (y_i - \hat{y}_i) x_i$ .
- Converges if data is linearly separable.

B. Unsupervised Learning

Clustering (Lloyd’s Algorithm / K-Means)
- Steps:
  1. Initialize centroids randomly.
  2. Assign points to nearest centroid.
  3. Recompute centroids as mean of assigned points.
- Key Equation:

C_k = \frac{1}{|C_k|} \sum_{x \in C_k} x

Kernel PCA
- Projects data into higher-dimensional space using kernel $K(x, x')$ .
- Polynomial Kernel: $K(x, x') = (x^T x' + c)^d$ .
- Key Idea: Use kernel trick to avoid explicit transformation.

C. Probabilistic Models

Naïve Bayes
- Assumes feature independence given class: $P(y|x_1, \dots, x_n) \propto P(y) \prod P(x_i|y)$ .
- Prediction: $\hat{y} = \arg\max_y P(y) \prod P(x_i|y)$ .

Bayesian Inference
- Prior: $\theta \sim \text{Beta}(\alpha, \beta)$ .
- Posterior: $\theta | \text{data} \sim \text{Beta}(\alpha + \text{heads}, \beta + \text{tails})$ .
- Posterior Mean: $E[\theta] = \frac{\alpha + \text{heads}}{\alpha + \beta + n}$ .

D. Neural Networks

Single-Layer Neural Network
- Activation Functions:
  - ReLU: $\text{ReLU}(z) = \max(0, z)$ .
- Forward Pass:
  - Hidden layer: $h = \text{ReLU}(W^{(1)} x + b^{(1)})$ .
  - Output: $\hat{y} = W^{(2)} h + b^{(2)}$ .

E. Dimensionality Reduction

PCA vs. Kernel PCA
- PCA: Linear projection to maximize variance.
- Kernel PCA: Nonlinear projection using kernel functions.

2. Key Equations Summary

Topic	Equation
Ridge Regression	$w = (X^T X + \lambda I)^{-1} X^T y$
Lasso Regression	$w = \arg\min_w \\|y - Xw\\|_2^2 + \lambda \\|w\\|_1$
Logistic Regression	$P(y=1\|x) = \frac{1}{1 + e^{-w^T x}}$
SVM (Primal)	$\min_{w,b} \frac{1}{2}\\|w\\|^2 + C \sum \xi_i$
SVM (Dual)	$\max_\alpha \sum \alpha_i - \frac{1}{2} \sum \alpha_i \alpha_j y_i y_j K(x_i, x_j)$
Entropy	$H(S) = -\sum p_i \log_2 p_i$
Information Gain	$\text{IG} = H(\text{parent}) - \sum \frac{\|S_v\|}{\|S\|} H(S_v)$
AdaBoost Weight	$\alpha_t = \frac{1}{2} \ln \left( \frac{1 - \epsilon_t}{\epsilon_t} \right)$
K-Means Centroid	$C_k = \frac{1}{\|C_k\|} \sum_{x \in C_k} x$
Polynomial Kernel	$K(x, x') = (x^T x' + c)^d$
Beta Posterior Mean	$E[\theta] = \frac{\alpha + \text{heads}}{\alpha + \beta + n}$

Graph 2: Unsupervised Learning

Graph 3: Probabilistic Models

Graph 4: Neural Networks

Graph 5: SVM Key Concepts

4. Exam-Specific Insights

Regularization (Q2, Q4):
- Ridge shrinks coefficients; Lasso performs feature selection.
- Weight vectors:
  - $w_3$ (sparse) → Lasso.
  - $w_2$ (large coefficients) → Linear Regression.
  - $w_1$ (moderate coefficients) → Ridge.

Decision Trees (Q3):
- Pure node: All samples same class or entropy = 0.

Clustering (Q5):
- Recompute centroids after assignment to find new clusters.

Kernel PCA (Q6):
- Relationship $x_4^2 = x_1 x_2 + x_1 x_2 x_3$ suggests degree 3 polynomial kernel.

AdaBoost (Q7, Q17, Q18):
- Weak learners must perform better than random (error < 50%).
- Lower training error → Higher weight in ensemble.

Bias-Variance Tradeoff (Q8):
- Model 1 (linear): High bias, low variance.
- Model 2 (high-degree polynomial): Low bias, high variance.

SVM Dual Problem (Q9):
- $\alpha_i = 0$ : Correctly classified.
- $0 < \alpha_i < C$ : On margin.
- $\alpha_i = C$ : Misclassified or inside margin.

Perceptron (Q10):
- Converges for linearly separable data.
- Final weight vector depends on update order.

Kernel Regression (Q11):
- Use kernel to transform features implicitly.
- Prediction: $y_{\text{pred}} = w^{*T} \phi(x')$ .

Bayesian Inference (Q12):
- Posterior mean: $\frac{\alpha + \text{heads}}{\alpha + \beta + n}$ .

Neural Networks (Q13):
- ReLU: $\text{ReLU}(z) = \max(0, z)$ .
- Output: Linear combination of hidden layer activations.

Naïve Bayes (Q14):
- Compute $P(\text{Yes}|\text{Features}) \propto P(\text{Yes}) \prod P(\text{Feature}_i|\text{Yes})$ .

Logistic Regression (Q15):
- Predict $\hat{y} = 1$ if $\sigma(w^T x) > 0.55$ .

SVM Prediction (Q16):
- Dual solution $\alpha^*$ defines support vectors.
- Prediction: $\text{sign}(\sum \alpha_i y_i K(x_i, x_{\text{test}}) + b)$ .