Home / Past papers / MLT End Term / this paper

IIT M FOUNDATION ET1 EXAM QPF2 S2 03 Sep - 03 Sep 2023

MLT End Term17 questions100 marks6 SA · 6 MCQ · 5 MSQFoundation / QPF2

Elapsed 00:00

Q1SA6 marks

Your numeric answer:

Q2SA6 marks

Your numeric answer:

Q3SA6 marks

Hint: Assume that these are only two possible words (that is there are only two features) question figure

Your numeric answer:

Q4MCQ6 marks

Q5MCQ6 marks

Q6MCQ6 marks

Q7MCQ6 marks

Q8MCQ6 marks

Q9MCQ6 marks

The given data points from classes A and B can be linearly separated using aHard-margin SVM.
Points (2, 2) from class 1 and (3, 3) from class 0 will be on the decisionboundary or support vectors.
A Soft-margin SVM would be a more robust choice than a Hard-margin SVM forthis dataset as the dataset is not linearly separable.
A perceptron model and a hard margin SVM will always give the same decisionboundary for this dataset.

Q10MSQ6 marksselect all that apply

Q11MSQ6 marksselect all that apply

Which of the following statements is/are true regarding logistic regression and perceptron algorithm?

Q12MSQ6 marksselect all that apply

Q13MSQ6 marksselect all that apply

Comprehension passage - the questions below it refer to this

Based on the above data, answer the given subquestions. question figure

Q14SA4 marks

Calculate the information gain for the "Age Group" feature.

Your numeric answer:

Q15SA4 marks

Calculate the information gain for the "Income level" feature.

Your numeric answer:

Q16MSQ7 marksselect all that apply

The values of 0-1 loss function will be 3
The values of cross entropy loss will be equals to 3.65
The values of 0-1 loss equals 4.
The values of cross entropy loss will be equals to 6.35

Q17SA7 marks

Your numeric answer:

Answer key

Q1 (SA, 6 marks): 0
Q2 (SA, 6 marks): 0.20 to 0.40
Q3 (SA, 6 marks): 0.98 to 1
Q4 (MCQ, 6 marks): C
Q5 (MCQ, 6 marks): A
Q6 (MCQ, 6 marks): A
Q7 (MCQ, 6 marks): B
Q8 (MCQ, 6 marks): D
Q9 (MCQ, 6 marks): A
Q10 (MSQ, 6 marks): A, B
Q11 (MSQ, 6 marks): A, B, D
Q12 (MSQ, 6 marks): A, C, E
Q13 (MSQ, 6 marks): B, C, D
Q14 (SA, 4 marks): 0.20 to 0.60
Q15 (SA, 4 marks): 0.20 to 0.60
Q16 (MSQ, 7 marks): B, C
Q17 (SA, 7 marks): 0.60 to 0.80

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Here’s a structured summary of core topics, concepts, principles, and equations essential for the exam, followed by Mermaid knowledge graphs to visualize relationships.

Core Topics & Concepts

1. Principal Component Analysis (PCA)

Purpose: Dimensionality reduction while preserving variance.
Steps:
1. Compute the covariance matrix $\Sigma$ of the data.
2. Compute eigenvalues $\lambda_1 \geq \lambda_2 \geq \dots \geq \lambda_d$ and eigenvectors $v_1, v_2, \dots, v_d$ .
3. Project data onto the top $k$ eigenvectors (principal components).
Reconstruction Error:
- If you retain the first $k$ components, the reconstruction error for a point $x$ is the sum of the squared projections onto the discarded components:

\text{Error} = \sum_{i=k+1}^d (v_i^T x)^2

For the given problem (Q2), since the last two eigenvalues are 0, the reconstruction error for any point in the span of the first 3 eigenvectors is 0.

2. Bayesian Estimation (Beta-Binomial Model)

Prior: Beta distribution $\text{Beta}(\alpha, \beta)$ .
Likelihood: Binomial distribution $\text{Binomial}(n, p)$ .
Posterior:
- Update parameters: $\alpha' = \alpha + x$ , $\beta' = \beta + n - x$ .
- Posterior mean:

\mathbb{E}[p | \text{data}] = \frac{\alpha'}{\alpha' + \beta'}

Example (Q3):
- Prior: $\text{Beta}(3, 4)$ .
- Data: $n = 50$ , $x = 12$ .
- Posterior: $\text{Beta}(15, 42)$ .
- Mean: $\frac{15}{57} \approx 0.263$ .

3. Naive Bayes Classifier

Assumption: Features are conditionally independent given the class.
Posterior Probability:

P(y | x_1, \dots, x_n) \propto P(y) \prod_{i=1}^n P(x_i | y)

Example (Q4):
- Classes: "Technology" ( $P(y=T) = 0.6$ ), "Sports" ( $P(y=S) = 0.4$ ).
- Words: "Software" and "Football".
- Compute $P(y=T | \text{Software, Football})$ using:

P(y=T) \cdot P(\text{Software} | T) \cdot P(\text{Football} | T)

Normalize by dividing by $P(\text{Software, Football})$ .

4. Gram Matrix & Positive Semidefiniteness

Gram Matrix: $K = X^T X$ for some data matrix $X$ .
Properties:
1. Symmetric: $K = K^T$ .
2. Positive Semidefinite: $v^T K v \geq 0$ for all $v$ .
Example (Q5):
- Matrices A (non-symmetric), B (negative eigenvalue), and D (non-symmetric) cannot be Gram matrices.

5. Linear Regression

Model: $y = w^T x$ .
Solution: Minimize $\|Xw - y\|^2$ .
- Closed-form: $w = (X^T X)^{-1} X^T y$ .
Example (Q6):
- The dataset is symmetric ( $x_1 = x_2$ ), so the solution $w$ must satisfy $w_1 = w_2$ .

6. Ridge Regression

Objective: Minimize $\|Xw - y\|^2 + \lambda \|w\|^2$ .
Trade-off: Balances fit (MSE) and complexity ( $\|w\|^2$ ).
Example (Q7):
- Compare $\text{MSE} + \|w\|^2$ for $w_1, w_2, w_3$ .
- $w_1$ has the lowest combined score.

7. Perceptron Algorithm

Update Rule:

w \leftarrow w + \eta \cdot y_i x_i \quad \text{if} \quad y_i (w^T x_i) \leq 0

Convergence: Guaranteed if data is linearly separable.
Example (Q8):
- The dataset is not linearly separable, so convergence depends on initialization $w^0$ .

8. Kernel SVM

Decision Function:

f(x) = \sum_{i=1}^n \alpha_i y_i K(x_i, x) + b

Polynomial Kernel (Degree 2):

K(x_i, x) = (x_i^T x + 1)^2

Example (Q9):
- Predicted label: $\text{sign}(f(x_{\text{test}}))$ .

9. Linear Separability & SVM

Hard-Margin SVM: Requires linear separability.
Soft-Margin SVM: Allows misclassification (robust to noise).
Example (Q10):
- The dataset is linearly separable, so a hard-margin SVM works.

10. K-Means Clustering (Lloyd’s Algorithm)

Objective: Minimize within-cluster variance.
Example (Q11):
- Points $[2,2]$ , $[0,0]$ , $[2,0]$ , $[0,2]$ form a cluster centered at $[1,1]$ .
- $[0.5, 0.5]$ and $[2, -1]$ are closer to $[1,1]$ than other centers.

11. Logistic Regression vs. Perceptron

Property	Logistic Regression	Perceptron
Output	Probability $P(y=1)$	Binary label $\text{sign}(w^T x)$
Loss	Cross-entropy	0-1 loss (misclassification)
Convergence	Always (convex optimization)	Only if linearly separable
Nonlinear Data	Works (with kernel trick)	Fails

Example (Q12):
- A, B, D are correct.

12. Soft-Margin SVM

Support Vectors: Points on the margin or misclassified.
Example (Q13):
- Only $F$ and $I$ are support vectors.

13. Random Forest

Feature Subsampling: Reduces correlation between trees.
- Decreasing $p$ (number of features) reduces correlation but may underfit individual trees.
- Increasing $p$ increases variance of the model.
Example (Q14):
- B, C, D are correct.

14. Decision Trees (Information Gain)

Entropy:

H(S) = -\sum_i P(y=i) \log_2 P(y=i)

Information Gain:

\text{IG}(S, A) = H(S) - \sum_v \frac{|S_v|}{|S|} H(S_v)

Example (Q15 & Q16):
- Compute IG for "Age Group" and "Income Level".

15. Loss Functions

Loss	Formula	Example (Q17)
0-1 Loss	$\mathbb{I}(y \neq \hat{y})$	Value = 3
Cross-Entropy	$-\sum_i y_i \log(\hat{y}_i)$	Value ≈ 3.65

16. Neural Networks (Forward Pass)

Sigmoid Activation:

\sigma(z) = \frac{1}{1 + e^{-z}}

Example (Q18):
- Compute output of Neuron 1 in the hidden layer:

z = 0.5 \cdot 0.6 + (-0.2) \cdot 0.3 + 0.8 \cdot 0.8 + 0.1 = 0.89

\sigma(z) \approx 0.71 \quad (\text{between } 0.6 \text{ and } 0.8)

IIT M FOUNDATION ET1 EXAM QPF2 S2 03 Sep - 03 Sep 2023

Answer key

Concepts this paper tests

Core Topics & Concepts

1. Principal Component Analysis (PCA)

2. Bayesian Estimation (Beta-Binomial Model)

3. Naive Bayes Classifier

4. Gram Matrix & Positive Semidefiniteness

5. Linear Regression

6. Ridge Regression

7. Perceptron Algorithm

8. Kernel SVM

9. Linear Separability & SVM

10. K-Means Clustering (Lloyd’s Algorithm)

11. Logistic Regression vs. Perceptron

12. Soft-Margin SVM

13. Random Forest

14. Decision Trees (Information Gain)

15. Loss Functions

16. Neural Networks (Forward Pass)

Graph 2: Unsupervised Learning

Graph 3: Bayesian vs. Frequentist Methods

Graph 4: Model Evaluation