IIT M DIPLOMA AN EXAM QDD3 01 Dec 2024 - 01 Dec 2024

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Summary of Core Topics, Concepts, Principles, and Equations

1. Machine Learning Basics

• Supervised vs. Unsupervised Learning
• Classification vs. Regression

2. Data Preprocessing

• Standardization and Normalization

  • StandardScaler, MinMaxScaler

• Handling Missing Values

  • KNNImputer

3. Model Evaluation

• Metrics

  • Precision Score: $\text{Precision} = \frac{TP}{TP + FP}$
  • Mean Squared Error (MSE): $\text{MSE} = \frac{1}{n} \sum_{i=1}^{n} (y_{\text{true},i} - y_{\text{pred},i})^2$
  • R² Score: $R^2 = 1 - \frac{SS_{\text{res}}}{SS_{\text{tot}}}$

• Cross-Validation

  • GridSearchCV

4. Linear Models

• Linear Regression

  • Polynomial Regression: $y = \beta_0 + \beta_1 x + \beta_2 x^2 + \dots + \beta_n x^n$

• Ridge Regression (L2 Regularization)

  • Loss Function: $\text{Loss} = \sum_{i=1}^{n} (y_i - \hat{y}_i)^2 + \alpha \sum_{j=1}^{m} \beta_j^2$

• Lasso Regression (L1 Regularization)

  • Loss Function: $\text{Loss} = \sum_{i=1}^{n} (y_i - \hat{y}_i)^2 + \alpha \sum_{j=1}^{m} |\beta_j|$

• Elastic Net

  • Loss Function: $\text{Loss} = \sum_{i=1}^{n} (y_i - \hat{y}_i)^2 + \alpha \left( (1 - l1\_ratio) \sum_{j=1}^{m} \beta_j^2 + l1\_ratio \sum_{j=1}^{m} |\beta_j| \right)$

5. Support Vector Machines (SVM)

• Kernel Trick

  • Linear Kernel: $K(x_i, x_j) = x_i \cdot x_j$
  • Polynomial Kernel: $K(x_i, x_j) = (\gamma x_i \cdot x_j + r)^d$
  • RBF Kernel: $K(x_i, x_j) = \exp(-\gamma \|x_i - x_j\|^2)$
  • Sigmoid Kernel: $K(x_i, x_j) = \tanh(\gamma x_i \cdot x_j + r)$
• Regularization Parameter $C$: Controls the trade-off between achieving a low training error and a low testing error.

6. Stochastic Gradient Descent (SGD)

• Loss Functions

  • Squared Error: $\text{Loss} = \sum_{i=1}^{n} (y_i - \hat{y}_i)^2$
  • Hinge Loss: $\text{Loss} = \max(0, 1 - y_i \hat{y}_i)$

• Regularization

  • L1 Regularization: $\text{Penalty} = \alpha \sum_{j=1}^{m} |\beta_j|$
  • L2 Regularization: $\text{Penalty} = \alpha \sum_{j=1}^{m} \beta_j^2$
  • Elastic Net: Combination of L1 and L2 regularization.

7. Naive Bayes

• Gaussian Naive Bayes: Assumes features follow a normal distribution.

  • Probability Density Function: $P(x_i | y) = \frac{1}{\sqrt{2\pi\sigma^2}} \exp\left(-\frac{(x_i - \mu)^2}{2\sigma^2}\right)$

8. K-Nearest Neighbors (KNN)

• Distance Metrics

  • Euclidean Distance: $d(x, y) = \sqrt{\sum_{i=1}^{n} (x_i - y_i)^2}$
  • Manhattan Distance: $d(x, y) = \sum_{i=1}^{n} |x_i - y_i|$
• Imputation: Using KNN to fill missing values.

9. Text Processing

• CountVectorizer: Converts a collection of text documents to a matrix of token counts.

  • N-grams: Contiguous sequence of n items from a sample of text.

10. Model Selection and Hyperparameter Tuning

• GridSearchCV: Exhaustive search over specified parameter values for an estimator.
• Pipeline: Sequentially applies a list of transforms and a final estimator.