IIT M IMPROVEMENT FN EXAM QDD2 22 Dec - 22 Dec 2024

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Here’s a structured summary of the core topics, concepts, and equations essential for the Diploma Level: Machine Learning Practice (Computer-Based Exam). The summary is organized by key themes, followed by a Mermaid knowledge graph to visualize relationships between topics.

1. Core Topics and Concepts

A. Data Preprocessing

1. Handling Missing Data

   • df.dropna(thresh=N): Drops rows with fewer than N non-NaN values.
   • df.fillna(value): Fills missing values with a specified value.
   • Example: df.dropna(thresh=45) drops rows with >5 missing values (for 50 features).

1. Feature Scaling

   • StandardScaler: Standardizes features to have mean=0 and variance=1.

     • Formula: $z = \frac{x - \mu}{\sigma}$
     • scaler.mean_ returns the mean of each feature before scaling.

   • MinMaxScaler: Scales features to a range (default: [0, 1]).

     • Formula: $x_{\text{scaled}} = \frac{x - x_{\min}}{x_{\max} - x_{\min}}$
     • scaler.data_max_ returns the max value of each feature.

1. Encoding Categorical Data

   • OneHotEncoder: Converts categorical features into binary vectors.

     • Example: ['cat', 'dog'] → [[1, 0], [0, 1]].
     • Output shape: (n_samples, n_categories).

   • PolynomialFeatures: Generates polynomial/combination features.

     • Example: degree=2 for [a, b] → [1, a, b, a², ab, b²].
     • Output shape: (n_samples, 1 + n_features + C(n_features, 2) + ... ).

1. Feature Selection

   • VarianceThreshold: Removes low-variance features.

     • Example: VarianceThreshold(threshold=0.1) removes features with variance < 0.1.

B. Model Training and Evaluation

1. Linear Models

   • LinearRegression:

     • fit_intercept=True: Includes a bias term (intercept).
     • fit_intercept=False: Forces the model through the origin.
     • Score (R²): Higher for fit_intercept=True if data has an intercept.

   • SGDRegressor:

     • learning_rate='constant': Fixed learning rate (eta0).
     • partial_fit: Updates model incrementally (useful for large datasets).
     • Loss functions: squared_loss, huber, epsilon_insensitive.

1. Tree-Based Models

   • DecisionTreeClassifier:

     • Hyperparameters:

       • max_depth: Maximum depth of the tree.
       • min_samples_split: Minimum samples required to split a node.
       • criterion: gini (default) or entropy.

     • Attributes:

       • max_depth: Actual depth of the trained tree.
       • feature_importances_: Importance of each feature.
     • Maximum leaf nodes in a balanced binary tree: $2^{\text{max\_depth}}$.

   • RandomForestClassifier:

     • max_features: Number of features to consider for splits (lower → higher variance).
     • n_estimators: Number of trees in the forest.

1. Ensemble Methods

   • BaggingClassifier:

     • max_samples: Number of samples to draw for each base estimator (default: full dataset).
     • bootstrap: Whether to sample with replacement.

   • AdaBoostClassifier:

     • Key hyperparameters: n_estimators, learning_rate.

   • GradientBoostingClassifier:

     • Hyperparameters: n_estimators, learning_rate, max_depth.

1. Neural Networks

   • MLPClassifier/MLPRegressor:

     • hidden_layer_sizes: Tuple specifying neurons per layer (e.g., (5, 3) → 2 layers).
     • alpha: L2 regularization strength (higher → stronger regularization).
     • partial_fit: Incremental learning (requires warm_start=True for repeated fits).

1. Clustering

   • KMeans:

     • Hyperparameters: n_clusters, init (e.g., k-means++), max_iter.

     • Evaluation:

       • Elbow Method: Plot inertia vs. n_clusters to find the "elbow."

       • Silhouette Score: Measures cluster cohesion/separation (higher = better).

         • Formula: $s = \frac{b - a}{\max(a, b)}$, where $a$ = intra-cluster distance, $b$ = inter-cluster distance.

C. Model Evaluation

1. Metrics

   • Regression:

     • Mean Squared Error (MSE): $\text{MSE} = \frac{1}{n} \sum (y_i - \hat{y}_i)^2$.
     • R² Score: $R^2 = 1 - \frac{\text{SS}_{\text{res}}}{\text{SS}_{\text{tot}}}$.

   • Classification:

     • Confusion Matrix: TN, FP, FN, TP.
     • ROC-AUC: Area under the ROC curve (higher = better).
     • Log Loss: $-\frac{1}{N} \sum y_i \log(\hat{y}_i)$.

1. Cross-Validation

   • KFold: Splits data into k folds (no stratification).
   • StratifiedKFold: Preserves class distribution in each fold.
   • GridSearchCV: Exhaustive search over hyperparameter grid.
   • RandomizedSearchCV: Random sampling of hyperparameter combinations.

1. Learning Curves

   • Plot training/validation loss vs. iterations/epochs.
   • Example: Track log_loss over epochs for SGDClassifier.

D. Handling Large Datasets

1. Chunking:

   • pd.read_csv(chunksize=N): Reads file in chunks of N rows.
   • Example: Incremental learning with SGDRegressor.partial_fit.

2. Incremental Learning:

   • Models supporting partial_fit: SGDClassifier, GaussianNB, PassiveAggressiveClassifier.

E. Dimensionality Reduction

1. PCA: Not covered in the questions, but often used with StandardScaler.
2. Feature Selection: Use VarianceThreshold or model-based importance (e.g., feature_importances_ in trees).

2. Key Equations

Graph 1: Data Preprocessing

Graph 2: Model Training

Graph 3: Model Evaluation

Graph 4: Handling Large Data

Graph 5: Clustering Evaluation

4. Exam Tips

1. Pandas/NumPy:

   • Know df.dropna, df.fillna, df.query, and aggregation methods (mean, median, value_counts).
   • Filtering: df[df['Salary'] > 50000] or df.loc[df['Salary'] > 50000].

1. Scikit-Learn:

   • Commands:

     • fit, transform, fit_transform.
     • predict vs. predict_proba.

   • Hyperparameters:

     • Tree depth, n_estimators, learning_rate, alpha (regularization).

   • Attributes:

     • coef_, intercept_, feature_importances_, labels_ (clustering).

1. Mathematics:

   • Focus on scaling formulas, MSE, R², and Silhouette Score.
   • Understand how fit_intercept affects linear models.

1. Algorithms:

   • Decision Trees: How max_depth and min_samples_split affect overfitting.
   • Ensembles: Bagging vs. boosting (e.g., max_samples in BaggingClassifier).
   • Neural Networks: Role of alpha (regularization) and hidden_layer_sizes.

1. Evaluation:

   • When to use cross-validation (e.g., GridSearchCV).
   • How to interpret learning curves (training vs. validation loss).

1. Large Data:

   • Use chunksize with pd.read_csv and partial_fit for incremental learning.

1. Clustering:

   • KMeans: How to choose n_clusters (Elbow Method, Silhouette Score).
   • Evaluation: Silhouette Score does not require ground truth labels.