Home / Past papers / MLT End Term / this paper

IIT M IMPROVEMENT FN EXAM QDD2 22 Dec - 22 Dec 2024

MLT End Term22 questions50 marks12 SA · 7 MCQ · 3 MSQImprovement / forenoon / QDD2

Elapsed 00:00

Q1SA2 marks

NOTE: Enter the answer correct to one decimal place question figure

Your numeric answer:

Q2SA3 marks

Your numeric answer:

Q3SA3 marks

Your numeric answer:

Q4SA3 marks

Consider a logistic regression model trained to detect spam emails. Emails containing harmful phishing links are labeled as spam (1), while regular emails are labeled as not spam (0). A good spam detector should correctly identify almost all emails with phishing links. Even a single phishing email incorrectly classified as not spam could expose users to significant risks. However, the detector may classify some regular emails as spam. This trade-off is acceptable to ensure safety. To ensure user safety, if the threshold is set to a low value to prioritize the detection of phishing emails, type 1. If the threshold is set to a high value to minimize false positives (misclassifying regular emails as spam), type 0.

Your numeric answer:

Q5SA5 marks

Your numeric answer:

Q6MCQ3 marks

3, 5, 1, 4, 2
5, 3, 1, 2, 4
3, 5, 2, 4, 1
5, 3, 1, 4, 2
None of these

Q7MCQ3 marks

Q8MCQ4 marks

Q9MCQ4 marks

Q10MCQ2 marks

If we remove all the non-support vectors from the dataset, what will be the impact on the model using SVM algorithm?

Model will overfit.
The model will not be changed.
Model will underfit.
Accuracy of the model will increase.

Q11MCQ2 marks

In each round of AdaBoost, the weight for a particular training observation is decreased from round t to round t + 1 if the observation was:

classified incorrectly by the weak learner trained in the round.
classified correctly by the weak learner trained in the round.
classified incorrectly by a majority of the weak learners trained up to theround.
classified correctly by a majority of the weak learners trained up to the round.

Q12MCQ1 mark

Is the following statement true or false? For a fixed size of the training and the test set, increasing the complexity of the model always leads to an increment in the test error.

TRUE
FALSE

Q13MSQ3 marksselect all that apply

Consider a supervised ML problem where a model is trained using a training dataset, and its performance is evaluated on both the training and validation datasets. During training, the model’s loss is plotted over time for both the datasets. Assume that the loss L(t) is decreasing as the model learns. The following graph shows two curves, Curve (A) and Curve (B), representing the loss for the training and validation datasets, respectively: question figure

Curve (A) represents the training loss.
Curve (B) represents the training loss.
Curve (A) represents the validation loss.
Curve (B) represents the validation loss.

Q14MSQ3 marksselect all that apply

Which of the following options are correct? Select all that apply.

Underfitted models generally have less bias and high variance.
A decision tree with the maximum possible length may lead to overfitting.
Weak learners are those whose performance is slightly better than therandom classifier.
All the estimators in bagging can be trained parallelly.

Comprehension passage - the questions below it refer to this

Q15SA2 marks

Your numeric answer:

Q16SA2 marks

Your numeric answer:

Q17SA2 marks

Your numeric answer:

Comprehension passage - the questions below it refer to this

Q18SA0.5 marks

Your numeric answer:

Q19SA0.5 marks

Your numeric answer:

Q20SA0.5 marks

Your numeric answer:

Q21SA0.5 marks

Your numeric answer:

Q22MSQ1 markselect all that apply

Which of the following is correct ?

Answer key

Q1 (SA, 2 marks): 2.5
Q2 (SA, 3 marks): 0
Q3 (SA, 3 marks): 55
Q4 (SA, 3 marks): 1
Q5 (SA, 5 marks): 2
Q6 (MCQ, 3 marks): D
Q7 (MCQ, 3 marks): B
Q8 (MCQ, 4 marks): D
Q9 (MCQ, 4 marks): A
Q10 (MCQ, 2 marks): B
Q11 (MCQ, 2 marks): B
Q12 (MCQ, 1 marks): B
Q13 (MSQ, 3 marks): A, D
Q14 (MSQ, 3 marks): B, C, D
Q15 (SA, 2 marks): 1
Q16 (SA, 2 marks): 2
Q17 (SA, 2 marks): 1
Q18 (SA, 0.5 marks): 1
Q19 (SA, 0.5 marks): 0
Q20 (SA, 0.5 marks): 0
Q21 (SA, 0.5 marks): 2
Q22 (MSQ, 1 marks): A, B

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Here’s a structured summary of the core topics, concepts, principles, and equations essential for the exam, followed by Mermaid knowledge graphs to visualize relationships.

Core Topics & Concepts

1. Principal Component Analysis (PCA)

Purpose: Dimensionality reduction while preserving variance.
Steps:
1. Center the data (subtract mean).
2. Compute covariance matrix $\Sigma = \frac{1}{n} X^T X$ .
3. Eigen-decomposition: $\Sigma = U \Lambda U^T$ .
4. Project data onto top $k$ eigenvectors (principal components).
Variance along PC1:
- For a dataset $\{ \lambda_1 u, \lambda_2 u, \dots \}$ , variance along PC1 is the largest eigenvalue $\lambda_1$ .
- Equation:

\text{Variance} = \lambda_1 = \frac{1}{n} \sum_{i=1}^n (x_i - \bar{x})^2

Reconstruction Error:
- If top $k$ PCs are retained, error = sum of discarded eigenvalues.
- Equation:

\text{Error} = \sum_{i=k+1}^d \lambda_i

2. Maximum A Posteriori (MAP) Estimation

Prior + Likelihood → Posterior:

P(\theta | X) \propto P(X | \theta) P(\theta)

Uniform Prior:
- For $\theta \sim \text{Uniform}[0, b]$ , MAP estimate is the maximum observed data point (if $\theta$ is upper-bound of uniform distribution).
- Equation:

\hat{\theta}_{MAP} = \max(X_1, \dots, X_n)

3. Logistic Regression & Class Imbalance

Decision Boundary:

P(y=1) = \sigma(w^T x + b), \quad \sigma(z) = \frac{1}{1 + e^{-z}}

Threshold Adjustment:
- Low threshold → High recall (catches most positives, more false positives).
- High threshold → High precision (fewer false positives, misses some positives).
Trade-off:
- Safety-critical applications (e.g., spam/phishing detection) prioritize recall (low threshold).

4. Cross-Validation (LOOCV)

Leave-One-Out Error:

\text{Error}_{LOO} = \frac{1}{n} \sum_{i=1}^n (y_i - \hat{f}^{-i}(x_i))^2

Steps:
1. For each $i$ , train on $n-1$ points (excluding $(x_i, y_i)$ ).
2. Predict $\hat{y}_i$ and compute squared error.
3. Average errors.

5. K-Means Clustering

Algorithm Steps:
1. Initialize $k$ centroids randomly.
2. Assign points to nearest centroid.
3. Recompute centroids as mean of assigned points.
4. Repeat until convergence.
Correct Sequence: Initialize → Assign → Recompute → Reassign (Option D: 5→3→1→4→2 is incorrect; correct is 1→2→3→4→5).

6. Optimization & Lagrangian

Lagrangian for Constrained Optimization:

L(x, \lambda) = f(x) + \lambda g(x)

For $\min f(x)$ s.t. $g(x) \geq 0$ , $\lambda \geq 0$ .
- Dual Problem:

\max_{\lambda \geq 0} \min_x L(x, \lambda)

Example:
- For $f(x) = x_1^2 + 2x_2$ , $g(x) = x_1 - x_2 - 3 \geq 0$ ,
- $L(x', \lambda) = 1 + 8 + \lambda (1 - 4 - 3) = 9 - 6\lambda$ .
- $\max_{\lambda \geq 0} L(x', \lambda) = +\infty$ if $\lambda \to \infty$ .

7. Gradient Descent

Update Rule:

w_{t+1} = w_t - \eta \nabla f(w_t)

For Linear Regression:
- $f(w) = \|X^T w - y\|^2$ ,
- $\nabla f(w) = 2X(X^T w - y)$ .
Example:
- Given $X, y, w_0, \eta$ , compute $w_1 = w_0 - \eta \nabla f(w_0)$ .

8. Perceptron Algorithm

Update Rule:

w_{t+1} = w_t + \eta (y_i - \hat{y}_i) x_i

$\hat{y}_i = \text{sign}(w_t^T x_i)$ .
- Example:
For $w^t = [0, 1, 2]^T$ , $x_1 = [1, -1, 0]^T, y_1 = 1$ :
- $\hat{y}_1 = \text{sign}(0 \cdot 1 + 1 \cdot (-1) + 2 \cdot 0) = -1 \neq y_1$ .
- $w^{t+1} = [0, 1, 2] + 1 \cdot [1, -1, 0] = [1, 0, 2]$ .

9. Support Vector Machines (SVM)

Key Concepts:
- Support Vectors: Points lying on margin boundaries.
- Slack Variables ( $\xi_i$ ): Measure margin violation.
- Kernel Trick: Non-linear decision boundaries.
Soft-Margin SVM:
- Minimize $\frac{1}{2} \|w\|^2 + C \sum \xi_i$ s.t. $y_i(w^T x_i + b) \geq 1 - \xi_i$ .
Impact of Removing Non-Support Vectors:
- No change to the model (only support vectors define the boundary).

10. AdaBoost

Weight Update:
- Increase weights for misclassified points.
- Decrease weights for correctly classified points.
Weak Learners:
- Slightly better than random guessing (e.g., decision stumps).

11. Bias-Variance Tradeoff

Model Type	Bias	Variance	Error Behavior
Underfitted	High	Low	Poor train/test performance
Overfitted	Low	High	Good train, poor test
Well-generalized	Balanced	Balanced	Good train/test

12. Naive Bayes

Assumption: Features are conditionally independent given the class.
Probability Estimation:

P(x_i | y) = \frac{\text{Count}(x_i, y)}{\text{Count}(y)}

Prediction:

\hat{y} = \arg\max_y P(y) \prod_{i=1}^d P(x_i | y)

Example:
- For binary features, $P(x_i=1 | y) = \frac{\text{Number of 1's in class } y}{\text{Total samples in class } y}$ .

13. Training vs. Validation Loss

Training Loss: Decreases monotonically with training.
Validation Loss:
- Decreases initially, then may increase (overfitting).
- Early Stopping: Stop training when validation loss starts rising.

14. Bagging & Boosting

Technique	Parallel?	Base Learners	Focus
Bagging	Yes	High variance (e.g., deep trees)	Reduce variance
Boosting	No	Weak learners	Reduce bias

Key Equations Summary

Topic	Equation
PCA Variance	$\text{Variance} = \lambda_1 = \frac{1}{n} \sum (x_i - \bar{x})^2$
MAP Estimation	$\hat{\theta}_{MAP} = \arg\max_\theta P(X\|\theta) P(\theta)$
Logistic Regression	$P(y=1) = \sigma(w^T x)$ , $\sigma(z) = \frac{1}{1 + e^{-z}}$
LOOCV Error	$\text{Error} = \frac{1}{n} \sum (y_i - \hat{f}^{-i}(x_i))^2$
Gradient Descent	$w_{t+1} = w_t - \eta \nabla f(w_t)$
Perceptron Update	$w_{t+1} = w_t + \eta (y_i - \hat{y}_i) x_i$
SVM Slack Variable	$\xi_i = \max(0, 1 - y_i(w^T x_i + b))$
Naive Bayes	$\hat{y} = \arg\max_y P(y) \prod P(x_i \| y)$

Graph 2: Unsupervised Learning & Optimization

Graph 3: Model Evaluation

Graph 4: Probabilistic Models

Exam Strategy

PCA: Focus on eigenvalues, variance, and reconstruction error.
MAP Estimation: Remember the prior’s role in Bayesian inference.
Logistic Regression: Understand threshold adjustment for imbalanced data.
SVM: Know slack variables, support vectors, and kernel tricks.
Optimization: Lagrangian, gradient descent, and perceptron updates.
Cross-Validation: LOOCV error calculation.
Ensemble Methods: Bagging vs. boosting, weak learners.
Naive Bayes: Conditional independence assumption.