IIT M IMPROVEMENT FN EXAM QDD2 01 Sep - 01 Sep 2024

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Here’s a structured summary of core topics, concepts, principles, and equations essential for the Diploma Level: Machine Learning Practice (Computer-Based Exam). The exam covers Pandas, Scikit-Learn, ML algorithms, preprocessing, model evaluation, and clustering. ---

1. Core Topics & Concepts

A. Pandas & Data Handling

Key Concepts:

1. DataFrame Operations:

   • Shape: df.shape → (rows, columns).
   • Missing values: df.isna().sum().sum() → Total missing values.
   • Value counts: df.column.value_counts() → Frequency of unique values.
   • Querying: df.query("condition") → Filter rows.
   • Indexing: df.iloc[:, n] → Select column n; df.loc[index, column] → Select by label.
   • Mean/aggregations: df.column.mean() (ignores NaN by default).

1. Handling Missing Data:

   • Drop: df.dropna().
   • Impute: KNNImputer(n_neighbors=k) (for numerical data).

1. Data Types:

   • float64 (default for decimals), int64, object (strings/categorical).
   • Check dtypes: df.dtypes.

Key Equations (Pandas):

• Mean (ignoring NaN):

• Value Counts: Frequency of each unique value in a column.

B. Scikit-Learn Workflow

Key Concepts:

1. Data Preprocessing:

   • Scaling:

     • StandardScaler (standardizes to mean=0, std=1).
     • MinMaxScaler (scales to [0, 1]).

   • Polynomial Features:

     • PolynomialFeatures(degree=d, include_bias=True/False).
     • If include_bias=True, adds a column of 1s (for intercept).

   • Encoding:

     • DictVectorizer (for feature extraction from dictionaries).
     • OneHotEncoder (for categorical variables).

1. Train-Test Split:

   • train_test_split(X, y, test_size=0.3, random_state=42).
   • random_state ensures reproducibility.

1. Pipelines:

   • make_pipeline(step1, step2, ...) → Sequential processing (e.g., impute → scale → model).

   • Example:

     pipe = Pipeline([
         ('imputer', KNNImputer(n_neighbors=3)),
         ('scaler', StandardScaler()),
         ('model', LinearRegression())
     ])

1. Model Evaluation:

   • Regression: R² score (model.score(X_test, y_test)).

   • Classification:

     • precision_score, recall_score, confusion_matrix.
     • DummyClassifier(strategy='most_frequent') → Predicts the majority class.

Key Equations (Preprocessing):

• Standard Scaling:

where $\mu$ = mean, $\sigma$ = standard deviation.

• Min-Max Scaling:

• Polynomial Features: For degree=2 and features $[x_1, x_2]$, output includes $[1, x_1, x_2, x_1^2, x_1x_2, x_2^2]$ (if include_bias=True).

C. Supervised Learning Algorithms

1. Linear Models

Key Notes:

• Regularization:

  • alpha=0 → No regularization (e.g., RidgeClassifier(alpha=0)).
  • Higher alpha → Stronger penalty.
• SGD Stopping: Use max_iter (not max_epoch).

• Logistic Regression:

  • class_weight='balanced' adjusts weights inversely to class frequencies.
  • C controls regularization (smaller C → stronger regularization).

2. Tree-Based Models

Key Notes:

• Decision Trees:

  • min_samples_split=2 (default): Minimum samples to split a node.
  • min_samples_leaf=1 (default): Minimum samples in a leaf.
  • Deeper trees → Higher variance (overfitting).

• Random Forest:

  • bootstrap=True (default): Samples with replacement.
  • max_features='sqrt' (default for classification): Features considered for splits.

• Bagging vs. Boosting:

  • Bagging: Parallel training (e.g., Random Forest).
  • Boosting: Sequential training (focuses on errors, e.g., AdaBoost).

3. Nearest Neighbors

Key Notes:

• Distance Metrics: Default is Euclidean.
• Scaling: Critical (use StandardScaler).

• n_neighbors:

  • Small k → Complex boundaries (overfitting).
  • Large k → Smooth boundaries (underfitting).

4. Support Vector Machines (SVM)

Key Notes:

• Support Vectors: Data points closest to the hyperplane.
• Kernel Trick: Transforms data into higher dimensions (e.g., rbf for non-linear boundaries).
• Scaling: Required (use StandardScaler).

D. Unsupervised Learning

1. Clustering

Key Notes:

• K-Means:

  • inertia: Sum of squared distances to centroids (lower = better).
  • n_init: Number of times to run with different centroid seeds.
  • Limitations: Assumes spherical clusters; sensitive to initialization.

• Agglomerative Clustering:

  • Linkage Methods:

    • ward: Minimizes variance.
    • complete: Maximizes distance between clusters.
  • Outliers: Assigned to the nearest cluster (not ignored).

Key Equations (Clustering):

• Inertia (K-Means):

where $C$ = set of centroids.

E. Model Evaluation & Metrics

1. Classification Metrics

2. Regression Metrics

3. Cross-Validation

Key Notes:

• Dummy Classifier: Baseline model (e.g., strategy='most_frequent').
• Class Imbalance: Use class_weight='balanced' or resampling.
• Overfitting: High training accuracy but low test accuracy.

F. Hyperparameter Tuning

Key Concepts:

1. Grid Search:

   • ParameterGrid generates all combinations of hyperparameters.

   • Example:

     param_grid = {
         'n_estimators': [10, 20, 30],
         'max_features': [2, 3, 4]
     }

   • Total combinations: $3 \times 3 = 9$.

1. Random Search: Samples randomly from parameter distributions.

1. Common Parameters:

   • Random Forest: n_estimators, max_depth, min_samples_leaf.
   • SVM: C, kernel, gamma.
   • KNN: n_neighbors, weights.

G. Special Topics

1. Data Snooping

• Definition: Repeatedly testing hypotheses on the same dataset leads to overfitting.

• Effects:

  • Biased test set estimates.
  • Increased false positives (Type I errors).

2. Feature Scaling Impact

• Affected Algorithms:

  • Distance-based: KNN, SVM, K-Means.
  • Gradient-based: Linear Regression, Logistic Regression, Neural Networks.

• Unaffected Algorithms:

  • Tree-based: Decision Trees, Random Forest.

3. Ordinal vs. Nominal Variables

Graph 2: Unsupervised Learning

Graph 3: Model Evaluation

Graph 4: Scikit-Learn Ecosystem

Graph 5: Tree-Based Models

Graph 6: Clustering Algorithms

3. Key Equations Summary

4. Exam Tips

1. Pandas:

   • Memorize df.shape, df.isna().sum(), df.value_counts().
   • Practice indexing (iloc, loc) and querying (df.query).

1. Scikit-Learn:

   • Know default parameters (e.g., RandomForestClassifier(n_estimators=100)).
   • Understand fit, predict, score methods.
   • Recall preprocessing steps (scaling, imputation, encoding).

1. Algorithms:

   • Linear Models: Focus on regularization (alpha, C).
   • Tree-Based: Know max_depth, min_samples_split, bootstrap.
   • KNN: Remember scaling and n_neighbors.
   • SVM: Kernel trick and C parameter.

1. Evaluation:

   • Classification: Precision, recall, F1, confusion matrix.
   • Regression: R², MSE.
   • Cross-validation: LOOCV vs. k-fold.

1. Clustering:

   • K-Means: Inertia, n_init, scaling.
   • Agglomerative: Linkage methods (ward, complete).

1. Hyperparameter Tuning:

   • Calculate total combinations for ParameterGrid.
   • Example: {'a': [1,2], 'b': [3,4]} → 4 combinations.

1. Special Cases:

   • DummyClassifier: Predicts majority class.
   • DictVectorizer: For dictionary-to-feature matrix.
   • PolynomialFeatures: include_bias adds intercept term.

1. Common Pitfalls:

   • Forgetting to scale data for KNN/SVM.
   • Misinterpreting class_weight='balanced'.
   • Confusing max_iter (SGD) with n_estimators (Random Forest).