IIT M DIPLOMA FN1 EXAM ETD1 11 Dec 2022 - 11 Dec 2022

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Here’s a structured summary of core topics, concepts, principles, and equations essential for the "Diploma Level: Machine Learning Practice" exam, along with Mermaid knowledge graphs to visualize relationships.

1. Core Topics & Concepts

A. Data Loading & Preprocessing

Key Concepts:

• sklearn.datasets: Loading built-in datasets (e.g., load_digits, make_blobs).

  • load_digits(return_X_y=True) returns (X, y); otherwise, returns a Bunch object.
  • make_blobs generates synthetic clusters (parameters: n_samples, centers, n_features, random_state).

• Feature Engineering:

  • Polynomial Features: PolynomialFeatures(degree=n) generates interaction terms up to degree n.

    • Example: For input [a, b], degree=2 generates [1, a, b, a², ab, b²].

  • Text Vectorization:

    • CountVectorizer: Converts text to a matrix of token counts (ignores punctuation by default).
    • TfidfVectorizer: Weights terms by inverse document frequency.
    • HashingVectorizer: Stateless, memory-efficient (no vocabulary storage).

• Scaling:

  • StandardScaler: Standardizes features (mean=0, std=1).
  • MinMaxScaler: Scales features to [0, 1].

Equations:

• Polynomial Expansion: For features x = [x₁, x₂], degree=2:

• TF-IDF:

where TF(t, d) = term frequency in document d, DF(t) = document frequency of term t, N = total documents.

B. Supervised Learning

1. Classification

##### Algorithms:

• Logistic Regression:

  • Loss: Log loss (cross-entropy).
  • Regularization: penalty='l1' (Lasso) or 'l2' (Ridge).

• Support Vector Machines (SVM):

  • Linear SVM: SGDClassifier(loss='hinge') or LinearSVC.

  • Kernel SVM: SVC(kernel='rbf').

    • RBF Kernel: Effective for non-linear, radially symmetric data.

  • NuSVC: Controls support vectors/margin errors via ν (nu).

    • $0 < \nu \leq 1$: Upper bound on margin errors, lower bound on support vectors.

• k-Nearest Neighbors (k-NN):

  • KNeighborsClassifier: Predicts majority class of n_neighbors.
  • Regression: KNeighborsRegressor predicts mean of neighbors.

• Naive Bayes:

  • GaussianNB: Assumes features follow normal distribution.
  • ComplementNB: Suitable for imbalanced data (adjusts weights for complement class).

• Decision Trees:

  • Non-parametric, prone to overfitting if unconstrained.
  • Hyperparameters: max_depth, min_samples_split, min_samples_leaf.
  • Splitting: Always binary in sklearn.

• Ensemble Methods:

  • Bagging: BaggingClassifier (e.g., with KNeighborsClassifier).

    • max_samples: Fraction of samples per bag (e.g., 0.7).
    • bootstrap=True: Sampling with replacement.

  • AdaBoost: AdaBoostClassifier with weak learners (e.g., DecisionTreeClassifier(max_depth=1)).

    • Hyperparameters: n_estimators, learning_rate.
  • Random Forest: Extension of bagging with feature subsampling.

##### Evaluation Metrics:

• Confusion Matrix:

  • Precision: $\frac{TP}{TP + FP}$
  • Recall: $\frac{TP}{TP + FN}$
  • Accuracy: $\frac{TP + TN}{TP + TN + FP + FN}$
  • F1-Score: Harmonic mean of precision/recall.

• ROC-AUC:

  • AUC: Area Under the ROC Curve (1 = perfect, 0.5 = random).
  • Multi-class: Binarize output (e.g., One-vs-Rest).
• Log Loss:

where $y_{i,j}$ is the true label, $p_{i,j}$ is the predicted probability.

##### Key Equations:

• Hinge Loss (SVM):

• Logistic Loss:

• L1/L2 Regularization:

2. Regression

##### Algorithms:

• Linear Regression:

  • LinearRegression: Ordinary least squares (OLS).

  • Ridge/Lasso: Ridge(alpha)/Lasso(alpha) with regularization.

    • RidgeCV: Cross-validated Ridge (input alphas as array).

• SGDRegressor:

  • Supports learning_rate='adaptive' (eta decreases if loss plateaus).
  • Loss options: 'squared_error' (OLS), 'huber' (robust).
• k-NN Regression: Averages targets of n_neighbors.
• Decision Tree Regressor: Predicts mean of leaf nodes.

##### Evaluation Metrics:

• MSE/RMSE:

• R² Score:

3. Model Training & Hyperparameter Tuning

##### Key Concepts:

• Incremental Learning:

  • partial_fit: Updates model weights in batches (e.g., SGDClassifier, Perceptron).
  • warm_start=True: Retains weights between fit calls.

• Grid Search:

  • GridSearchCV: Exhaustive search over hyperparameter grid.

    • Parameters: param_grid, cv, refit=True (trains best model on full data).

  • Example:

    params = {'n_estimators': range(10, 21), 'learning_rate': np.linspace(0.1, 1.0, 10)}
    gs = GridSearchCV(AdaBoostClassifier(), param_grid=params, cv=4)
    gs.fit(X_train, y_train)

• Regularization:

  • Ridge/Lasso: alpha controls strength (higher alpha = more regularization).
  • SVM: C (inverse of regularization; smaller C = stronger regularization).

##### Key Equations:

• SGD Update Rule:

where $\eta$ is the learning rate.

C. Unsupervised Learning

1. Clustering

##### Algorithms:

• k-Means:

  • Minimizes inertia (within-cluster sum of squares).
  • Requires n_clusters as input.

• Hierarchical Clustering:

  • Connectivity-based (agglomerative or divisive).
  • Uses linkage criteria (e.g., ward, complete, average).

##### Evaluation Metrics:

• Inertia:

• Silhouette Score: Measures cohesion/separation.

2. Dimensionality Reduction

##### Algorithms:

• PCA: Linear projection to maximize variance.
• t-SNE: Non-linear, preserves local structure (used for visualization).

D. Model Evaluation & Cross-Validation

Key Concepts:

• Train-Test Split: train_test_split(X, y, test_size=0.2, random_state=42).

• Cross-Validation:

  • ShuffleSplit: Randomizes splits (specify n_splits).
  • cross_validate: Computes multiple metrics (e.g., scoring=['accuracy', 'f1']).
• Dummy Classifier: Baseline model (e.g., strategy='most_frequent').

Equations:

• Cross-Validation Error:

where $K$ = number of folds.

E. Neural Networks (MLP)

Key Concepts:

• MLPRegressor/Classifier:

  • hidden_layer_sizes: Tuple specifying layers (e.g., (150, 100, 50) = 3 layers).
  • solver='sgd': Uses stochastic gradient descent.
  • learning_rate_init: Initial learning rate.
• Activation Functions: ReLU (default), logistic, tanh.

F. Text Processing

Key Concepts:

• Tokenization: Splitting text into words/tokens.
• Stop Words: Removed via stop_words='english' in vectorizers.
• n-grams: ngram_range=(1, 2) includes unigrams and bigrams.

G. Practical Sklearn Usage

Key Points:

1. Shape Mismatches:

   • X must be 2D for sklearn (use reshape(-1, 1) for 1D arrays).
   • y can be 1D for classification/regression.

2. Common Errors:

   • ValueError if X/y shapes are incompatible.
   • AttributeError if method doesn’t exist (e.g., clf.score() without fit).

3. Incremental Learning:

   • partial_fit requires classes for classification (pass classes=np.unique(y)).

4. Pipelines:

   • Chain transformations and estimators (e.g., Pipeline([('scaler', StandardScaler()), ('clf', SVC())])).

Graph 2: Model Evaluation & Metrics

Graph 3: Data Preprocessing

Graph 4: Unsupervised Learning

Graph 5: Neural Networks

Graph 6: Hyperparameter Tuning

3. Key Equations Summary

4. Exam Tips

1. Code Snippets:

   • Pay attention to return types (e.g., load_digits(return_X_y=True) vs False).
   • Check for shape compatibility (e.g., X must be 2D for sklearn).
   • Default parameters: e.g., KNeighborsRegressor averages targets for n_neighbors.

1. Algorithms:

   • SVM: loss='hinge' → linear SVM; kernel='rbf' → non-linear.
   • Naive Bayes: ComplementNB for imbalanced data.
   • Decision Trees: Binary splits in sklearn; prone to overfitting.

1. Evaluation:

   • AUC: Higher = better; 0.5 = random classifier.
   • Log Loss: Lower = better (0 = perfect).
   • Confusion Matrix: Recall row-wise, precision column-wise.

1. Hyperparameters:

   • Ridge/Lasso: alpha (higher = more regularization).
   • SVM: C (lower = stronger regularization).
   • Decision Trees: max_depth, min_samples_split to control overfitting.

1. Text Processing:

   • CountVectorizer ignores punctuation by default.
   • HashingVectorizer is memory-efficient but irreversible.

1. Common Pitfalls:

   • Shape errors: Ensure X is 2D (e.g., X.reshape(-1, 1)).
   • Incremental learning: Use partial_fit with classes for classification.
   • GridSearchCV: refit=True trains the best model on the full dataset.