Home / Past papers / MLT End Term / this paper

IIT M IMPROVEMENT AN EXAM QDD3 22 - 22 Dec 2024

MLT End Term22 questions50 marks12 SA · 7 MCQ · 3 MSQImprovement / afternoon / QDD3

Elapsed 00:00

Q1SA2 marks

NOTE: Enter the answer correct to one decimal place question figure

Your numeric answer:

Q2SA3 marks

Your numeric answer:

Q3SA3 marks

Your numeric answer:

Q4SA3 marks

Consider a logistic regression model trained to detect spam emails. Emails containing harmful phishing links are labeled as spam (1), while regular emails are labeled as not spam (0). A good spam detector should correctly identify almost all emails with phishing links. Even a single phishing email incorrectly classified as not spam could expose users to significant risks. However, the detector may classify some regular emails as spam. This trade-off is acceptable to ensure safety. To ensure user safety, if the threshold is set to a low value to prioritize the detection of phishing emails, type 1. If the threshold is set to a high value to minimize false positives (misclassifying regular emails as spam), type 0.

Your numeric answer:

Q5MSQ3 marksselect all that apply

Select the correct statements from the following for k-means algorithm:

In k-means algorithm, all cluster initializations lead to the same result.
k-means algorithm is sensitive to outliers.
One initialization may converge while another may not.
The initialization of cluster centres may affect the number of iterations k-means takes to converge.

Q6MSQ3 marksselect all that apply

Curve (A) represents the training loss.
Curve (B) represents the training loss.
Curve (A) represents the validation loss.
Curve (B) represents the validation loss.

Q7MCQ4 marks

Q8MCQ4 marks

Comprehension passage - the questions below it refer to this

Q9SA2 marks

Your numeric answer:

Q10SA2 marks

Your numeric answer:

Q11SA2 marks

Your numeric answer:

Q12SA5 marks

Your numeric answer:

Q13MCQ3 marks

Q14MCQ3 marks

Which of the following options are not correct? Select all that apply.

Underfitted models generally have less bias and high variance.
A decision tree with the maximum possible length may lead to overfitting.
Weak learners are those whose performance is slightly better than therandom classifier.
All the estimators in bagging can be trained parallelly.

Q15MCQ2 marks

If we remove all the non-support vectors from the dataset, what will be the impact on the model using SVM algorithm?

Model will overfit.
The model will not be changed.
Model will underfit.
Accuracy of the model will increase.

Q16MCQ2 marks

In each round of AdaBoost, the weight for a particular training observation is decreased from round t to round t + 1 if the observation was:

classified incorrectly by the weak learner trained in the round.
classified correctly by the weak learner trained in the round.
classified incorrectly by a majority of the weak learners trained up to theround.
classified correctly by a majority of the weak learners trained up to the round.

Q17MCQ1 mark

Is the following statement true or false? For a fixed size of the training and the test set, increasing the complexity of the model always leads to an increment in the test error.

True
False

Comprehension passage - the questions below it refer to this

Q18SA0.5 marks

Your numeric answer:

Q19SA0.5 marks

Your numeric answer:

Q20SA0.5 marks

Your numeric answer:

Q21SA0.5 marks

Your numeric answer:

Q22MSQ1 markselect all that apply

Which of the following is correct ?

Answer key

Q1 (SA, 2 marks): 2.5
Q2 (SA, 3 marks): 0
Q3 (SA, 3 marks): 52
Q4 (SA, 3 marks): 1
Q5 (MSQ, 3 marks): B, D
Q6 (MSQ, 3 marks): A, D
Q7 (MCQ, 4 marks): B
Q8 (MCQ, 4 marks): A
Q9 (SA, 2 marks): 1
Q10 (SA, 2 marks): 2
Q11 (SA, 2 marks): 1
Q12 (SA, 5 marks): 0.3 to 0.4
Q13 (MCQ, 3 marks): A
Q14 (MCQ, 3 marks): A
Q15 (MCQ, 2 marks): B
Q16 (MCQ, 2 marks): B
Q17 (MCQ, 1 marks): B
Q18 (SA, 0.5 marks): 1
Q19 (SA, 0.5 marks): 0
Q20 (SA, 0.5 marks): 0
Q21 (SA, 0.5 marks): 2
Q22 (MSQ, 1 marks): A, B

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Here’s a structured summary of core topics, concepts, principles, and equations for the exam, followed by Mermaid knowledge graphs to visualize relationships.

Core Topics & Key Concepts

1. Principal Component Analysis (PCA)

Purpose: Dimensionality reduction while preserving variance.
Steps:
1. Standardize data (mean=0, variance=1).
2. Compute covariance matrix: $\Sigma = \frac{1}{n} X^T X$ .
3. Eigen-decomposition: $\Sigma = U \Lambda U^T$ , where $\Lambda$ contains eigenvalues $\lambda_1 \geq \lambda_2 \geq ... \geq \lambda_d$ .
4. Project data onto top $k$ eigenvectors (principal components).
Variance along PC1: For a dataset $\{ \alpha_1 \mathbf{u}, \alpha_2 \mathbf{u}, ..., \alpha_n \mathbf{u} \}$ , the variance along the first PC is the largest eigenvalue of $\Sigma$ .
- If data is scaled multiples of $\mathbf{u}$ , variance = $\text{Var}(\alpha) \cdot \|\mathbf{u}\|^2$ .
- Equation:

\text{Variance along PC1} = \lambda_1 = \frac{1}{n} \sum_{i=1}^n (\alpha_i - \bar{\alpha})^2 \cdot \|\mathbf{u}\|^2

Reconstruction Error: If you retain $k$ PCs, the error is the sum of discarded eigenvalues:

\text{Error} = \sum_{i=k+1}^d \lambda_i

If data lies in a $k$ -dimensional subspace, error = 0.

2. Maximum A Posteriori (MAP) Estimation

Prior + Likelihood → Posterior:

f(\theta | X) \propto P(X | \theta) \cdot P(\theta)

Uniform Prior: For $\theta \sim \text{Uniform}[0, b]$ , the posterior is:

f(\theta | X) \propto \mathbb{I}_{[0, b]}(\theta) \cdot \prod_{i=1}^n \mathbb{I}_{[0, \theta]}(x_i)

MAP Estimate: Mode of the posterior. For Uniform[0,60] prior and data $X = \{x_1, ..., x_n\}$ , MAP is $\max(x_i)$ if $\max(x_i) \leq 60$ , else 60.

3. Logistic Regression & Class Imbalance

Decision Boundary: $\sigma(\mathbf{w}^T \mathbf{x}) \geq t$ , where $t$ is the threshold.
- Low Threshold (t → 0): High recall (few false negatives), high false positives.
- High Threshold (t → 1): High precision (few false positives), high false negatives.
Trade-off: For spam detection (prioritize recall), set low threshold (answer = 1).

4. k-Means Clustering

Sensitive to:
- Initialization (different centroids → different results).
- Outliers (pulls centroids away).
Convergence: Guaranteed to converge (but may be local minima).
Key Points:
- Initialization affects iterations to converge (Option D).
- Outliers can distort clusters (Option B).

5. Training vs. Validation Loss Curves

Training Loss: Decreases monotonically (model fits training data).
Validation Loss:
- Decreases initially, then may increase (overfitting).
- Curve A: Training loss (lower, smoother).
- Curve B: Validation loss (higher, may rise later).

6. Gradient Descent for Linear Regression

Objective: Minimize $f(\mathbf{w}) = \|X^T \mathbf{w} - \mathbf{y}\|^2$ .
Update Rule:

\mathbf{w}_{t+1} = \mathbf{w}_t - \eta \nabla f(\mathbf{w}_t)

Gradient: $\nabla f(\mathbf{w}) = 2 X (X^T \mathbf{w} - \mathbf{y})$ .
- Example Calculation:
Given $X \in \mathbb{R}^{2 \times 3}$ , $\mathbf{y} \in \mathbb{R}^3$ , $\mathbf{w}_0 = [1, 0.5]^T$ , $\eta = 0.01$ :

\mathbf{w}_1 = \mathbf{w}_0 - 0.01 \cdot 2 X (X^T \mathbf{w}_0 - \mathbf{y})

(Answer: $\mathbf{w}_1 = [0.76, -0.02]^T$ ).

7. Perceptron Algorithm

Update Rule: If $y_i (\mathbf{w}^T \mathbf{x}_i) \leq 0$ , update:

\mathbf{w}_{t+1} = \mathbf{w}_t + \eta y_i \mathbf{x}_i

Example:
- $\mathbf{w}^t = [1, 2, -1]^T$ , $(\mathbf{x}_1, y_1) = ([0, -1, 0]^T, -1)$ :

\mathbf{w}^{t+1} = [1, 2, -1]^T + 1 \cdot [0, -1, 0]^T = [1, 1, -1]^T

Next point $(\mathbf{x}_2, y_2) = ([-1, 0, 2]^T, 1)$ :

\mathbf{w}^{t+2} = [1, 1, -1]^T + 1 \cdot [-1, 0, 2]^T = [0, 1, 1]^T

8. Naïve Bayes Classifier

Assumption: Features are conditionally independent given the class.
Parameters: For binary features $x_i \in \{0,1\}$ , estimate:

\hat{p}_i^y = P(x_i = 1 | y) = \frac{\text{Count}(x_i = 1, y)}{\text{Count}(y)}

Prediction: Choose class $y$ that maximizes:

P(y) \prod_{i=1}^d P(x_i | y)

Example:
- For $y=1$ , $\hat{p}_1^1 = \frac{3}{4}$ (since 3 out of 4 points with $y=1$ have $x_1=1$ ).
- $a + b = P(x_1=1 | y=0) + P(x_1=1 | y=1) = \frac{1}{4} + \frac{3}{4} = 1$ .

9. Leave-One-Out Cross-Validation (LOOCV)

Error:

\text{error}_L = \frac{1}{n} \sum_{i=1}^n (y_i - \hat{f}^{-i}(x_i))^2

Example: For $D = \{(1,2), (2,3), (3,4)\}$ , fit $\hat{f} = \beta x$ :
- Leave out $(1,2)$ : Train on $\{(2,3), (3,4)\}$ , get $\beta = 1$ , predict $\hat{y}_1 = 1$ , error $= (2-1)^2 = 1$ .
- Leave out $(2,3)$ : Train on $\{(1,2), (3,4)\}$ , get $\beta = 1$ , predict $\hat{y}_2 = 2$ , error $= (3-2)^2 = 1$ .
- Leave out $(3,4)$ : Train on $\{(1,2), (2,3)\}$ , get $\beta = 1$ , predict $\hat{y}_3 = 3$ , error $= (4-3)^2 = 1$ .
- Total Error: $\frac{1 + 1 + 1}{3} = 1$ .
(Note: The answer is ~0.33, suggesting a miscalculation; recheck $\beta$ estimates.)

10. Lagrangian & Optimization

Lagrangian for Constraint $g(x) \leq 0$ :

L(x, \lambda) = f(x) + \lambda g(x)

Dual Problem: $\max_{\lambda \geq 0} L(x', \lambda)$ .
- If $g(x') \leq 0$ , maximum is $f(x')$ (at $\lambda = 0$ ).
- If $g(x') > 0$ , $L(x', \lambda) \to \infty$ as $\lambda \to \infty$ .
Example:
- $f(x) = x_1^2 + 2x_2$ , $g(x) = x_1 - x_2 - 3 \leq 0$ , $x' = [1, 4]^T$ :
  - $g(x') = 1 - 4 - 3 = -6 \leq 0$ , so $\max L = f(x') = 1 + 8 = 9$ .

11. Bias-Variance Tradeoff & Model Complexity

Underfitting: High bias, low variance (Option A is incorrect).
Overfitting: High variance, low bias (e.g., deep decision trees).
Bagging (e.g., Random Forest):
- Trains estimators in parallel on bootstrapped samples.
- Reduces variance.
AdaBoost:
- Sequentially trains weak learners, upweights misclassified points.
- Reduces bias.

12. Support Vector Machines (SVM)

Support Vectors: Points on the margin ( $\mathbf{w}^T \mathbf{x}_i = \pm 1$ ).
Slack Variables ( $\xi_i$ ): Measure violation of margin.
- $\xi_i = 0$ if point is correctly classified and outside/on the margin.
- $\xi_i = |\mathbf{w}^T \mathbf{x}_i - y_i| - 1$ if misclassified or inside margin.
Soft-Margin SVM: Minimizes $\|\mathbf{w}\|^2 + C \sum \xi_i$ .
Key Observations:
- Removing non-support vectors does not change the model (Option B).
- $\xi_i$ for points:
  - On the correct side of the margin: $\xi_i = 0$ .
  - Inside margin or misclassified: $\xi_i > 0$ .

13. AdaBoost

Weight Update: Decrease weights for correctly classified points, increase for misclassified.
- Answer: Option B (classified correctly in round $t$ ).

14. Model Complexity & Test Error

False Statement: "Increasing complexity always increases test error."
- Truth: Test error may decrease initially (better fit), then increase (overfitting).

IIT M IMPROVEMENT AN EXAM QDD3 22 - 22 Dec 2024

Answer key

Concepts this paper tests

Core Topics & Key Concepts

1. Principal Component Analysis (PCA)

2. Maximum A Posteriori (MAP) Estimation

3. Logistic Regression & Class Imbalance

4. k-Means Clustering

5. Training vs. Validation Loss Curves

6. Gradient Descent for Linear Regression

7. Perceptron Algorithm

8. Naïve Bayes Classifier

9. Leave-One-Out Cross-Validation (LOOCV)

10. Lagrangian & Optimization

11. Bias-Variance Tradeoff & Model Complexity

12. Support Vector Machines (SVM)

13. AdaBoost

14. Model Complexity & Test Error

Graph 2: Classification Models

Graph 3: Model Evaluation & Optimization

Graph 4: Ensemble Methods

Graph 5: Bayesian Methods