IIT M DIPLOMA AN EXAM QDD4 13 Apr 2025 - 13 Apr 2025

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Here’s a structured summary of core topics, concepts, principles, and equations essential for the Diploma Level: Machine Learning Practice (Computer-Based Exam), along with Mermaid knowledge graphs to visualize relationships.

1. Core Topics & Concepts

A. Data Preprocessing

Key Concepts:

1. Scaling/Normalization:

   • MinMaxScaler: Scales features to [0, 1] range.

• StandardScaler: Standardizes features to have mean=0 and variance=1.

1. Encoding Categorical Data:

   • One-Hot Encoding: Converts categorical variables into binary vectors.
   • Label Encoding: Assigns a unique integer to each category.

2. Handling Missing Data:

   • df.isnull().sum(): Counts NULL values.
   • df.fillna(): Imputes missing values.

3. Polynomial Features:

   • Generates polynomial combinations of features (e.g., $X_1^2, X_1X_2$).
   • Number of features for degree d and n original features:

1. PCA (Principal Component Analysis):

   • Reduces dimensionality by projecting data onto principal components.
   • n_components: Number of principal components to retain.

Key Equations:

B. Supervised Learning

Key Algorithms:

1. Linear Models:

   • Ridge Regression (L2 Regularization):

• Lasso Regression (L1 Regularization):

• SGDRegressor:

  • Supports penalty='l1' or 'l2'.
  • Learning rate schedules: constant, invscaling, adaptive.
  • invscaling learning rate:

1. Logistic Regression:

   • Decision Boundary: Not always linear (depends on features).
   • Regularization: Controlled by C (inverse of regularization strength).
   • Solvers: liblinear, lbfgs, saga (for small/large datasets).

2. Naïve Bayes:

   • Assumes feature independence given the class.

   • Types:

     • GaussianNB: For continuous data.
     • BernoulliNB: For binary features.
     • MultinomialNB: For discrete counts (e.g., text classification).

3. Support Vector Machines (SVM):

   • Kernel Trick: Transforms data into higher dimensions (linear, poly, rbf, sigmoid).
   • Regularization: C controls trade-off between margin width and classification error.

4. k-Nearest Neighbors (k-NN):

   • Distance Metrics: Euclidean, Manhattan (p=1), Minkowski.
   • Computational Bottleneck: Prediction time (searches entire training set).

5. Ensemble Methods:

   • AdaBoost: Boosts weak learners; learning_rate controls contribution of each tree.

   • Random Forest:

     • max_features: Number of features considered for splits.
     • bootstrap: Whether to sample with replacement.
     • oob_score: Out-of-bag error for generalization estimate.
   • Voting Classifier: Combines predictions from multiple models; weights control influence.

Key Equations:

C. Unsupervised Learning

Key Algorithms:

1. k-Means Clustering:

   • Initialization: k-means++ spreads initial centroids to avoid poor clusters.
   • Objective: Minimize within-cluster sum of squares (WCSS).
   • Elbow Method: Plots WCSS vs. k to find optimal k.
   • Silhouette Score: Measures cluster separation/cohesion.

2. Hierarchical Clustering:

   • Linkage Methods: Single, complete, average, centroid.
   • Distance Metrics: Euclidean, Manhattan, Minkowski.

Key Equations:

D. Model Evaluation

Key Metrics:

1. Classification:

   • Accuracy: $\frac{\text{TP} + \text{TN}}{\text{TP} + \text{TN} + \text{FP} + \text{FN}}$
   • Precision: $\frac{\text{TP}}{\text{TP} + \text{FP}}$
   • Recall (Sensitivity): $\frac{\text{TP}}{\text{TP} + \text{FN}}$
   • F1-Score: $2 \times \frac{\text{Precision} \times \text{Recall}}{\text{Precision} + \text{Recall}}$
   • Confusion Matrix: TP, TN, FP, FN.

2. Regression:

   • MSE (Mean Squared Error): $\frac{1}{n} \sum (y_i - \hat{y}_i)^2$
   • R² Score: $1 - \frac{\text{SS}_{\text{res}}}{\text{SS}_{\text{tot}}}$

3. Imbalanced Data:

   • Accuracy can be misleading; use precision-recall curves or F1-score.

Key Equations:

E. Hyperparameter Tuning

Key Methods:

1. GridSearchCV: Exhaustive search over specified parameter values.
2. RandomizedSearchCV: Random sampling of parameter combinations.
3. Cross-Validation: Splits data into k folds for robust evaluation.

F. Neural Networks (MLP)

Key Concepts:

• Hidden Layers: Specified via hidden_layer_sizes (e.g., (100, 50)).
• Activation Functions: relu, logistic, tanh.
• Solvers: adam, sgd, lbfgs.
• Early Stopping: Halts training if validation score doesn’t improve.

G. Dimensionality Reduction

Key Techniques:

1. PCA:

   • Transforms data into orthogonal components.
   • n_components: Number of principal components.

2. Feature Selection:

   • SelectKBest: Selects top k features based on statistical tests.

Graph 2: Supervised Learning Algorithms

Graph 3: Model Evaluation Metrics

Graph 4: Unsupervised Learning

Graph 5: Hyperparameter Tuning

3. Key Takeaways for the Exam

1. Data Preprocessing:

   • Know when to use MinMaxScaler vs. StandardScaler.
   • Understand OneHotEncoder and PolynomialFeatures.
   • Remember that PCA reduces dimensions but loses interpretability.

1. Supervised Learning:

   • Ridge/Lasso: L2 vs. L1 regularization.
   • SGDRegressor: Learning rate schedules (invscaling, adaptive).
   • k-NN: Manhattan (p=1) vs. Euclidean (p=2) distance.
   • Ensemble Methods: RandomForest uses max_features and bootstrap.

1. Model Evaluation:

   • Imbalanced Data: Recall/F1-score > Accuracy.
   • Confusion Matrix: TP, TN, FP, FN.

1. Unsupervised Learning:

   • k-Means: Use k-means++ for better initialization.
   • Elbow Method/Silhouette Score: For optimal k.

1. Hyperparameter Tuning:

   • GridSearchCV vs. RandomizedSearchCV.
   • Cross-validation is essential for robust evaluation.

1. Neural Networks:

   • MLPRegressor requires hidden_layer_sizes and solver.
   • early_stopping prevents overfitting.

1. Common Pitfalls:

   • Overfitting: Reduce learning_rate in AdaBoost or increase regularization.
   • Scaling: Always scale features for distance-based algorithms (k-NN, SVM, PCA).
   • Imbalanced Data: Avoid accuracy; use precision/recall.