IIT M IMPROVEMENT AN EXAM QDD4 22 - 22 Dec 2024

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Here’s a structured summary of core topics, concepts, principles, and equations essential for the Diploma-Level Machine Learning Practice (Computer-Based Exam), along with Mermaid knowledge graphs to visualize relationships.

1. Core Topics & Concepts

A. Data Preprocessing

Key Concepts:

1. One-Hot Encoding (OneHotEncoder)

   • Converts categorical features into binary (0/1) columns.

   • Example:

     from sklearn.preprocessing import OneHotEncoder
     ohe = OneHotEncoder(sparse_output=False)
     ohe.fit_transform([['apple'], ['banana']])

   • Output Shape: If input has n unique categories, output has n binary columns.

1. Feature Scaling

   • MinMaxScaler: Scales features to [0, 1] range.

     • Formula:

• StandardScaler: Scales features to have mean=0 and std=1.

  • Formula:

• When to Use:

  • MinMaxScaler: When features have bounded ranges (e.g., pixel intensities).
  • StandardScaler: For algorithms assuming Gaussian distribution (e.g., SVM, PCA).

1. Handling Missing Data

   • dropna() in Pandas:

     • thresh=N: Drops rows with fewer than N non-null values.
     • how='any' or how='all': Drops rows with any/all missing values.

1. Polynomial Features (PolynomialFeatures)

   • Generates interaction terms and polynomial features.
   • Example (degree=2):

B. Model Evaluation & Metrics

1. Regression Metrics

   • Mean Absolute Error (MAE):

• Mean Squared Error (MSE):

• R² Score:

1. Classification Metrics

   • Accuracy:

• Precision, Recall, F1-Score:

• ROC-AUC: Area under the ROC curve (trade-off between TPR and FPR).
• Log Loss: Measures probabilistic confidence (lower = better).

1. Cross-Validation (cross_val_score)

   • Evaluates model performance across multiple folds.

   • Example:

     from sklearn.model_selection import cross_val_score
     scores = cross_val_score(model, X, y, cv=5, scoring='accuracy')

C. Supervised Learning Algorithms

1. Linear Models

   • Linear Regression:

     • Predicts continuous output:

• SGDRegressor/Classifier:

  • Optimizes loss using Stochastic Gradient Descent.

  • Key Parameters:

    • learning_rate='constant' (fixed learning rate).
    • eta0: Initial learning rate.
    • penalty: 'l1', 'l2', or 'elasticnet' (regularization).

1. Decision Trees & Ensembles

   • DecisionTreeClassifier/Regressor:

     • Splits data based on feature thresholds (Gini/Entropy for classification, MSE for regression).

     • Key Parameters:

       • max_depth: Maximum tree depth.
       • min_samples_split: Minimum samples required to split a node.
       • min_samples_leaf: Minimum samples required in a leaf.

   • Random Forest:

     • Ensemble of decision trees (reduces variance via bagging).

     • Key Parameters:

       • n_estimators: Number of trees.
       • max_features: Features considered for splits (default=sqrt(n_features)).

   • Gradient Boosting (e.g., GradientBoostingClassifier):

     • Sequentially corrects errors of previous trees.

     • Key Parameters:

       • n_estimators: Number of boosting stages.
       • learning_rate: Shrinkage factor for each tree’s contribution.

1. Bagging & Boosting

   • BaggingClassifier:

     • Trains base estimators on bootstrapped samples.

     • Key Parameters:

       • n_estimators: Number of base models.
       • max_samples: Fraction of samples per base estimator.

   • AdaBoost:

     • Weights misclassified samples higher in subsequent iterations.

     • Key Parameters:

       • n_estimators: Number of weak learners.
       • learning_rate: Contribution weight of each learner.

D. Unsupervised Learning

1. K-Means Clustering

   • Partitions data into k clusters by minimizing within-cluster variance.

   • Key Parameters:

     • n_clusters: Number of clusters.
     • init: Initialization method ('k-means++' or 'random').
     • max_iter: Maximum iterations.
   • Inertia: Sum of squared distances to centroids (lower = better).

1. Dimensionality Reduction

   • PCA (Principal Component Analysis):

     • Projects data into lower-dimensional space while preserving variance.

   • t-SNE:

     • Non-linear technique for visualization (preserves local structure).

   • Lasso Regression:

     • Linear model with L1 regularization (performs feature selection).

E. Model Training & Hyperparameter Tuning

1. Grid Search (GridSearchCV)

   • Exhaustively searches over a parameter grid.

   • Example:

     from sklearn.model_selection import GridSearchCV
     param_grid = {'n_estimators': [50, 100], 'learning_rate': [0.1, 0.01]}
     grid = GridSearchCV(GradientBoostingClassifier(), param_grid, cv=3)
     grid.fit(X, y)

   • Best Parameters: grid.best_params_

1. Randomized Search (RandomizedSearchCV)

   • Samples parameter combinations randomly (faster than grid search).

   • Example:

     from sklearn.model_selection import RandomizedSearchCV
     param_dist = {'alpha': loguniform(1e-4, 1e0), 'penalty': ['l1', 'l2']}
     random_search = RandomizedSearchCV(SGDRegressor(), param_dist, n_iter=10)

1. Partial Fit (partial_fit)

   • Used for online learning (streaming data).

   • Example:

     from sklearn.linear_model import SGDClassifier
     clf = SGDClassifier()
     clf.partial_fit(X_batch, y_batch, classes=[0, 1])

F. Neural Networks (MLPClassifier/MLPRegressor)

• Key Parameters:

  • hidden_layer_sizes: Tuple specifying neurons per layer (e.g., (5, 3) = 2 layers).
  • alpha: L2 regularization strength (higher = more regularization).
  • activation: 'relu', 'tanh', or 'logistic'.

• Effect of alpha:

  • Higher alpha → stronger regularization → reduces overfitting.

G. Clustering Evaluation

• Silhouette Score: Measures cluster cohesion and separation.
• Inertia: Sum of squared distances to centroids (K-Means).

H. Feature Selection

• VarianceThreshold:

  • Removes low-variance features.

  • Example:

    from sklearn.feature_selection import VarianceThreshold
    vf = VarianceThreshold(threshold=0.1)
    X_selected = vf.fit_transform(X)

2. Key Equations

Graph 2: Supervised Learning Algorithms

Graph 3: Model Evaluation Metrics

Graph 4: Unsupervised Learning

Graph 5: Hyperparameter Tuning

Graph 6: Ensemble Methods

Graph 7: Neural Networks (MLP)

4. Exam Tips

1. Focus on Equations:

   • Memorize scaling formulas, error metrics, and probability equations (e.g., Gini, Entropy).

2. Understand Parameter Effects:

   • max_depth in trees → controls overfitting.
   • alpha in SGD/MLP → regularization strength.
   • n_clusters in K-Means → affects inertia.

3. Pandas Operations:

   • dropna(thresh=N), groupby, query, nlargest.

4. Scikit-Learn Methods:

   • fit_transform vs fit + transform.
   • partial_fit for streaming data.

5. Common Pitfalls:

   • K-Means: n_clusters must ≤ samples.
   • SGD: Requires feature scaling.
   • Decision Trees: No need for scaling.

Good luck! This summary covers 90% of the exam content. Review the graphs for visual reinforcement and equations for mathematical questions.