IIT M DIPLOMA AN EXAM QDD2 01 Dec 2024 - 01 Dec 2024

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Summary of Core Topics, Concepts, Principles, and Equations

1. Kernel Regression

• Concept: Kernel regression is a non-parametric technique to fit a model to data using a kernel function.

• Key Equations:

  • Kernel function: $k(x_i, x_j) = (1 + x_i^T x_j)^2$
  • Weight vector: $w = \phi(X)\alpha$
  • Kernel matrix: $K$
  • Prediction: $\hat{y} = w^T \phi(x)$

2. Naïve Bayes Algorithm

• Concept: A probabilistic classifier based on Bayes' theorem with strong independence assumptions between features.

• Key Equations:

  • Probability estimates: $P(y = 1)$, $P(f_1 = 1 \mid y = 0)$, $P(f_2 = 1 \mid y = 0)$, etc.
  • Prediction: $P(y \mid f_1, f_2) \propto P(y) \prod_{i=1}^{n} P(f_i \mid y)$

3. Leave-One-Out Cross Validation (LOOCV)

• Concept: A type of cross-validation where each data point is used as a validation set once.

• Key Equations:

  • Model: $y = w_0 + w_1 x$
  • Weight vector: $w = [w_0, w_1]^T$
  • Norm of weight vector: $\|w\|_2^2$

4. k-Nearest Neighbors (k-NN)

• Concept: A non-parametric method used for classification and regression based on the closest training examples in the feature space.

• Key Equations:

  • Distance metric (e.g., Euclidean distance)
  • Prediction: Majority vote for classification, average for regression

5. Decision Trees

• Concept: A tree-like model of decisions and their possible consequences.

• Key Equations:

  • Entropy: $H(S) = -\sum_{i=1}^{c} p_i \log_2 p_i$
  • Information Gain: $IG(S, A) = H(S) - \sum_{v \in \text{Values}(A)} \frac{|S_v|}{|S|} H(S_v)$

6. Linear Regression

• Concept: A linear approach to modeling the relationship between a dependent variable and one or more independent variables.

• Key Equations:

  • Model: $y_i \mid x_i = w^T x_i + \epsilon_i$
  • Loss function: $L(w) = \sum_{i=1}^{n} (w^T x_i - y_i)^2$
  • Ridge Regression: $\hat{w}_{Ridge} = \arg \min_w \sum_{i=1}^{n} (w^T x_i - y_i)^2 + \lambda \|w\|_2^2$
  • Bayesian Linear Regression: $w \sim \text{Laplace}(0, 2/\lambda)$