IIT M FOUNDATION DIPLOMA AN EXAM - 22 Dec 2024

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Here’s a structured summary of core topics, concepts, principles, and equations essential for the Diploma Level: Machine Learning Practice (Computer-Based Exam). The summary is organized by topic clusters for clarity, followed by Mermaid knowledge graphs to visualize relationships.

1. Data Preprocessing

Core Concepts

• Handling Missing Data:

  • df.dropna(thresh=N): Drops rows with fewer than N non-null values.
  • SimpleImputer (mean/median/mode) or KNNImputer for advanced imputation.

• Feature Scaling:

  • Standardization (StandardScaler): Transforms features to have mean = 0 and std = 1. Equation:

• Normalization (MinMaxScaler): Scales features to a range (default: [0, 1]). Equation:

• Robust Scaling (RobustScaler): Uses median/IQR (for outliers).

  • Categorical Encoding:

• One-Hot Encoding (OneHotEncoder): Converts categorical features into binary columns. Example: ['apple', 'banana'] → [1, 0] and [0, 1].

  • sparse_output=False returns a dense array.
  • Output shape: (n_samples, n_categories + n_numerical_features).

• Label Encoding (LabelEncoder): Assigns integers to categories (not for nominal data).

  • Polynomial Features (PolynomialFeatures):

  Generates polynomial/combination features (e.g., $x_1, x_2, x_1^2, x_1x_2$). Example: degree=2 for input [a, b] → [1, a, b, a², ab, b²].

  • Feature Selection:
• VarianceThreshold: Removes low-variance features.
• SelectKBest: Selects top-k features using statistical tests (e.g., ANOVA, chi-squared).

Key Equations

2. Supervised Learning

Core Algorithms

Linear Models

• Linear Regression:

  • Predicts continuous targets: $y = w^T x + b$.
  • Loss: Mean Squared Error (MSE).

  • Regularization:

    • Ridge (L2): Penalizes large weights ($\lambda \sum w_i^2$).
    • Lasso (L1): Penalizes weights ($\lambda \sum |w_i|$), can zero out features.
    • Elastic Net: Combines L1 and L2.

• Logistic Regression:

  • Binary classification: $P(y=1) = \sigma(w^T x + b)$, where $\sigma$ is the sigmoid.
  • Loss: Log Loss (Cross-Entropy).
  • Multinomial: For multi-class (softmax activation).

Tree-Based Models

• Decision Trees:

  • Splits data based on feature thresholds (e.g., Gini impurity, entropy).

  • Hyperparameters:

    • max_depth: Limits tree depth.
    • min_samples_split: Minimum samples to split a node.
    • min_samples_leaf: Minimum samples in a leaf.
  • Equation (Gini Impurity):

• Random Forest:

  • Ensemble of decision trees (bagging).

  • Key Parameters:

    • n_estimators: Number of trees.
    • max_features: Features considered per split (default: sqrt(n_features)).

• Gradient Boosting (e.g., GradientBoostingClassifier):

  • Sequentially corrects errors of previous trees.

  • Hyperparameters:

    • learning_rate: Shrinks contribution of each tree.
    • n_estimators: Number of boosting stages.

Support Vector Machines (SVM)

• Linear SVM:

  • Finds hyperplane maximizing margin: $w^T x + b$.
  • Hinge Loss: $\max(0, 1 - y(w^T x + b))$.

• Kernel Trick:

  • Maps data to higher dimensions (e.g., RBF, polynomial).

Neural Networks (MLPClassifier/MLPRegressor)

• Architecture:

  • hidden_layer_sizes=(5, 3): 2 hidden layers (5 neurons → 3 neurons).

• Regularization:

  • alpha: L2 penalty term (higher alpha → more regularization).

• Activation Functions:

  • ReLU, sigmoid, tanh.

k-Nearest Neighbors (KNN)

• Prediction: Majority vote (classification) or average (regression) of k nearest neighbors.
• Distance Metrics: Euclidean, Manhattan.

• Hyperparameters:

  • n_neighbors: Number of neighbors.
  • weights: Uniform or distance-weighted.

Key Equations

3. Unsupervised Learning

Core Algorithms

• k-Means Clustering:

  • Assigns points to k clusters by minimizing within-cluster variance.

  • Steps:

    1. Initialize centroids randomly.
    2. Assign points to nearest centroid.
    3. Update centroids as mean of assigned points.

  • Hyperparameters:

    • n_clusters: Number of clusters.
    • init: Initialization method (e.g., k-means++).
  • Inertia: Sum of squared distances to centroids (lower = better).

• Principal Component Analysis (PCA):

  • Linear dimensionality reduction via singular value decomposition (SVD).
  • Variance Explained: Eigenvalues of covariance matrix.

• t-SNE:

  • Non-linear dimensionality reduction for visualization.

Key Equations

4. Model Evaluation

Metrics

Cross-Validation

• k-Fold CV: Splits data into k folds; trains on k-1 folds, tests on 1.
• StratifiedKFold: Preserves class distribution in folds.
• cross_val_score: Computes metric for each fold.

Hyperparameter Tuning

• Grid Search (GridSearchCV): Exhaustively tests all parameter combinations.
• Random Search (RandomizedSearchCV): Samples random combinations (faster for large spaces).

• Example Parameters:

  param_grid = {
      'n_estimators': [50, 100],
      'learning_rate': [0.1, 0.01],
      'max_depth': [3, 5]
  }

5. Ensemble Methods

Core Concepts

Key Algorithms

• AdaBoost:

  • Weights misclassified samples higher in subsequent iterations.

• Gradient Boosting:

  • Fits new models to residuals of previous predictions.

• Random Forest:

  • Bagging with decision trees + feature randomness (max_features).

6. Model Deployment & Incremental Learning

Core Concepts

• partial_fit:

  • Updates model incrementally (for streaming data).
  • Requires classes parameter for first call (in SGDClassifier).

• Pipelines:

  • Chains preprocessing and modeling (e.g., make_pipeline(StandardScaler(), LinearRegression())).

• Model Persistence:

  • Save/load models with joblib or pickle.

7. Clustering Evaluation

8. Dimensionality Reduction

9. Handling Imbalanced Data

• Resampling:

  • Oversampling minority class (SMOTE).
  • Undersampling majority class.

• Class Weights:

  • class_weight='balanced' in sklearn models.

10. Python Libraries & Functions

Graph 2: Supervised Learning Algorithms

Graph 3: Model Evaluation & Tuning

Graph 4: Unsupervised Learning

Graph 5: Ensemble Methods

Graph 6: Neural Networks

Exam Tips

1. Focus on Equations:

   • Memorize scaling formulas (Standard/Min-Max), loss functions (MSE, Log Loss, Hinge Loss), and metrics (Precision, Recall, F1, R²).

2. Sklearn Syntax:

   • Know how to initialize, fit, and predict with models (e.g., model.fit(X_train, y_train)).
   • Understand hyperparameters (e.g., n_estimators, max_depth, learning_rate).

3. Data Preprocessing:

   • When to use StandardScaler vs. MinMaxScaler.
   • How OneHotEncoder affects feature dimensions.

4. Model Evaluation:

   • Difference between accuracy, precision, recall, and F1.
   • When to use ROC-AUC vs. Log Loss.

5. Ensemble Methods:

   • Bagging vs. Boosting: Bagging reduces variance; boosting reduces bias.
   • Random Forest vs. Gradient Boosting: RF uses bagging; GB is sequential.

6. Clustering:

   • k-Means: How to choose k (elbow method/silhouette score).
   • Inertia: Lower = better clustering.

7. Dimensionality Reduction:

   • PCA: Explained variance ratio.
   • t-SNE: For visualization (non-linear).

8. Incremental Learning:

   • Use partial_fit for streaming data (e.g., SGDClassifier).

9. Pipelines:

   • Chain steps (e.g., Pipeline([('scaler', StandardScaler()), ('model', LogisticRegression())])).

10. Common Pitfalls:

    • Data Leakage: Never scale/test on training data.
    • Overfitting: Use regularization (L1/L2) or cross-validation.
    • Class Imbalance: Use class_weight or resampling.