IIT M FOUNDATION DIPLOMA ENDTERM QPB2 07 Aug 2022 IBA - 07 Aug 2022

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Here’s a structured summary of core topics, concepts, principles, and equations essential for the Diploma Level: Machine Learning Practice exam, along with Mermaid knowledge graphs to visualize relationships.

Core Topics & Concepts

1. Scikit-Learn Fundamentals

Key Concepts:

• Datasets in sklearn.datasets:

  • Loaders vs. Fetchers:

    • Both return a Bunch object (dictionary-like with keys like data, target, feature_names).
    • Loaders: Load small standard datasets (e.g., load_iris(), load_wine()).
    • Fetchers: Download larger datasets (e.g., fetch_california_housing()).

  • Synthetic Datasets:

    • make_blobs(), make_classification(), make_regression().

  • Dataset Attributes:

    • data.shape (not data.shape()), target, feature_names, DESCR.

• Bunch Object:

  • Dictionary-like object with keys: data, target, target_names, feature_names, DESCR.

Example Code Snippets:

from sklearn.datasets import load_wine
data = load_wine()
print(data.data.shape)  # Correct (no parentheses)

2. Data Preprocessing

Key Concepts:

• Scaling:

  • StandardScaler: Standardizes features (mean=0, std=1).

    • scaler.fit(X) computes mean/std.
    • scaler.transform(X) applies scaling.
    • scaler.mean_ gives the mean of features.
  • MaxAbsScaler: Scales by maximum absolute value (each feature divided by its max abs value).
  • MinMaxScaler: Scales to a range (e.g., [0, 1]).

• Polynomial Features:

  • PolynomialFeatures(degree=2, interaction_only=True) generates:

    • Original features + interaction terms (no squared terms if interaction_only=True).
    • Example: [x1, x2] → [1, x1, x2, x1*x2].

• Dummy Features:

  • add_dummy_feature(X) adds a column of 1s (bias term) at the start.

• Encoding:

  • MultiLabelBinarizer: Converts multi-label data to binary matrix.

    • Example: [(1,2), (3,)] → [[1,1,0], [0,0,1]].

Equations:

• StandardScaler:

where $\mu$ = mean, $\sigma$ = standard deviation.

• MaxAbsScaler:

3. Model Training & Evaluation

Key Concepts:

• Linear Models:

  • LinearRegression: Fits $y = X\beta + \epsilon$.
  • LogisticRegression: Binary/multi-class classification (uses logistic or softmax for output).
  • SGDRegressor: Supports shuffle=True to shuffle data every epoch.

• Cross-Validation:

  • LeaveOneOut (LOO): Uses $n-1$ samples for training, 1 for testing (repeated $n$ times).
  • Train-Test Split: train_test_split(X, y, test_size=0.2, random_state=42).

• Metrics:

  • Confusion Matrix: TP, TN, FP, FN.
  • Recall (Sensitivity):

• Precision:

• R² Score:

• Explained Variance (PCA):

  • pca.explained_variance_ratio_ gives the proportion of variance explained by each component.

• ROC Curve:

  • Plots TPR (Recall) vs. FPR (1 - Specificity).
  • Perfect classifier: Step at (0,1).

Example Code:

from sklearn.metrics import confusion_matrix
y_true = [0, 1, 2, 2, 2]
y_pred = [0, 0, 2, 2, 1]
cm = confusion_matrix(y_true, y_pred, labels=[0, 1, 2])
### Recall for class 2 = TP/(TP+FN) = 2/(2+1) = 0.67

4. Dimensionality Reduction

Key Concepts:

• PCA (Principal Component Analysis):

  • Transforms data to orthogonal components maximizing variance.
  • PCA(n_components=2).fit(X) computes components.
  • explained_variance_ratio_ gives variance explained by each component.

Equation:

where $W$ = eigenvectors of covariance matrix.

5. Clustering

Key Concepts:

• K-Means:

  • KMeans(n_clusters=3).fit(X) assigns clusters.
  • Elbow Method: Plot inertia_ (WCSS) vs. number of clusters to find optimal $k$.
  • Dendrogram: Hierarchical clustering visualization (uses scipy.cluster.hierarchy).

6. Neural Networks

Key Concepts:

• MLPClassifier/MLPRegressor:

  • hidden_layer_sizes=(10,5): 2 hidden layers (10 and 5 neurons).
  • n_layers_: Total layers (input + hidden + output).
  • coefs_: List of weight matrices (shape [n_layers-1]).

  • Activation Functions:

    • Output layer: logistic (binary) or softmax (multi-class).
    • Hidden layers: relu, tanh, etc.

Example:

from sklearn.neural_network import MLPClassifier
mlp = MLPClassifier(hidden_layer_sizes=(8,12), random_state=12)
mlp.fit(X, y)
print(mlp.n_layers_)  # 4 (input + 2 hidden + output)

7. Ensemble Methods

Key Concepts:

• VotingClassifier:

  • voting='hard': Majority vote.
  • voting='soft': Predicts class with highest average probability.
  • flatten_transform=True: Flattens output for transform.

• BaggingClassifier:

  • Uses base estimators (e.g., SVC) with bootstrapping.

8. Model Persistence & API Design

Key Concepts:

• Scikit-Learn API Design Principles:

  • Consistency: Uniform interface (e.g., fit, predict, transform).
  • Sensible Defaults: Works out-of-the-box (e.g., random_state for reproducibility).
  • Inspection: Attributes like coef_, feature_importances_.
  • Non-Proliferation of Classes: Avoids unnecessary class hierarchies.

Key Equations Summary

Exam Tips

1. Code Outputs: Focus on:

   • Shape of transformed data (e.g., PolynomialFeatures).
   • Scaler outputs (e.g., StandardScaler.mean_).
   • Model attributes (e.g., coef_, n_features_in_).

1. Equations: Memorize:

   • Scaling formulas (StandardScaler, MaxAbsScaler).
   • Metrics (R², Recall, Precision).
   • PCA variance explained.

1. API Design: Know scikit-learn’s consistency (e.g., fit/predict pattern).

1. Visualizations:

   • Elbow Method (K-Means).
   • Dendrogram (Hierarchical Clustering).
   • ROC Curve (Classification Performance).