IIT M IMPROVEMENT AN EXAM QDD3 01 - 01 Sep 2024

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Here’s a structured summary of core topics, concepts, principles, and equations essential for the Diploma Level: Machine Learning Practice (Computer-Based Exam), along with Mermaid knowledge graphs to visualize relationships.

Core Topics & Concepts Summary

1. Pandas DataFrame Operations

• Key Concepts:

  • Indexing: loc (label-based), iloc (position-based).
  • Handling missing data: dropna(), fillna().
  • Data types: dtypes, apply(type).
  • Selection: Slicing, boolean indexing.

• Equations/Code Snippets:

  df.loc[23:24, 'Maths':'History']  # Label-based selection
  df.iloc[2:4, 1:3]                 # Position-based selection
  df.dropna()                        # Remove rows with NaN
  df['Hindi'].apply(type).nunique()  # Count unique data types

2. Data Preprocessing

• Key Concepts:

  • Normalization/Standardization: MinMaxScaler, StandardScaler.

  • Handling Imbalanced Data:

    • Upsampling (SMOTE), downsampling, synthetic samples.
  • Categorical Data: One-hot encoding (pd.get_dummies), label encoding.
  • Text Data: CountVectorizer, TfidfVectorizer.

• When Preprocessing is Needed:

  • Inconsistent entity names (e.g., "Chennai", "CHENNAI").
  • Mixed data types (e.g., strings, numbers, NaN).
  • Features on different scales.

3. Model Training & Evaluation

• Key Concepts:

  • Train-Test Split: train_test_split(test_size=0.2, random_state=42).
  • Cross-Validation: KFold, StratifiedKFold, cross_val_score.

  • Metrics:

    • Regression: r2, neg_mean_squared_error, mean_absolute_error.
    • Classification: accuracy, precision, recall, f1.

• Equations:

  • Mean Absolute Error (MAE):

• R² Score:

4. Hyperparameter Tuning

• Key Concepts:

  • Grid Search: GridSearchCV.
  • Random Search: RandomizedSearchCV.
  • Learning Curves: learning_curve (plots training/test error vs. sample size).

• Example:

  param_grid = {'max_depth': [2, 3, 4], 'min_samples_split': [2, 3]}
  grid_search = GridSearchCV(DecisionTreeClassifier(), param_grid, cv=5)

5. Algorithms & Models

• Supervised Learning:

  • Linear Models:

    • SGDRegressor (supports l1, l2 penalties).
    • LogisticRegression (for classification).

  • Tree-Based:

    • DecisionTreeClassifier (prone to high variance, low bias).
    • RandomForestClassifier (ensemble of trees).

  • SVM:

    • Hard-margin SVM: Correctly classifies all points if linearly separable.

  • Naive Bayes:

    • Assumes feature independence; GaussianNB supports partial_fit.

  • Neural Networks:

    • MLPClassifier/MLPRegressor (hidden layers, activation functions).

• Unsupervised Learning:

  • K-Means:

    • Inertia (sum of squared distances to centroids).
    • Elbow method to choose n_clusters.
  • Radius Neighbors: Uses all neighbors within a radius (robust to outliers).

• Ensemble Methods:

  • Bagging: BaggingClassifier (reduces variance).
  • Boosting: AdaBoost (learning rate shrinks contributor weights).
  • Voting: VotingClassifier (voting='soft' averages probabilities).

6. Handling Large Datasets

• Key Concepts:

  • Chunking: pd.read_csv(chunksize=1024).
  • Incremental Learning: partial_fit (for SGDClassifier, GaussianNB).
  • OpenML: fetch_openml() to load datasets.

7. Model-Specific Notes

• Decision Trees:

  • max_depth: Controls tree depth (higher → risk of overfitting).
  • min_samples_split: Minimum samples to split a node.

• K-Nearest Neighbors (KNN):

  • Sensitive to scale; use normalization.
  • RadiusNeighborsClassifier: Uses all neighbors in radius.

• SVM:

  • Hard-margin SVM: No misclassifications if data is linearly separable.

• Neural Networks:

  • hidden_layer_sizes=(5, 3): 2 hidden layers (5 neurons → 3 neurons).

8. Imbalanced Data

• Techniques:

  • Upsample minority class.
  • Downsample majority class.
  • Synthetic samples (SMOTE).

9. Evaluation Metrics

• Regression:

  • neg_mean_squared_error, r2.

• Classification:

  • accuracy, precision, recall, f1.

10. Scikit-Learn Utilities

• Pipelines: Chain preprocessing and modeling.

  Pipeline([('scaler', MinMaxScaler()), ('sgd', SGDRegressor())])

• Learning Curves: learning_curve to diagnose bias/variance.

2. Model Training & Evaluation

3. Hyperparameter Tuning

4. Handling Imbalanced Data

5. Ensemble Methods

6. Neural Networks in Scikit-Learn

7. K-Means Clustering

Key Equations (Math-Focused)

1. Mean Absolute Error (MAE):

1. R² Score:

1. Inertia (K-Means):

where $C$ is the set of centroids.

1. SGD Update Rule (Linear Models):

where $\eta$ is the learning rate, and $J$ is the loss function.

1. AdaBoost Weight Update:

where $\alpha_t$ is the weight of the weak learner $h_t$.

Exam Tips

1. Pandas:

   • Master loc/iloc and handling NaN values.
   • Know how apply and dtypes work.

2. Model Selection:

   • Use cross_val_score for robust evaluation.
   • For regression, prefer neg_mean_squared_error or r2.

3. Hyperparameter Tuning:

   • GridSearchCV is exhaustive; RandomizedSearchCV is faster.
   • Use learning_curve to diagnose underfitting/overfitting.

4. Algorithms:

   • Decision Trees: High variance; tune max_depth to avoid overfitting.
   • SVM: Hard-margin SVM works only for linearly separable data.
   • KNN: Always normalize data.

5. Imbalanced Data:

   • Avoid removing minority/majority classes entirely (Options A/C in Q31 are incorrect).

6. Neural Networks:

   • hidden_layer_sizes=(5, 3) means 2 layers (5 neurons → 3 neurons).

7. Clustering:

   • Inertia = 0 when n_clusters = n_samples.