IIT M FOUNDATION DIPLOMA ENDTERM QPB1 07 Aug 2022 IBA - 07 Aug 2022

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Here’s a structured summary of the core topics, concepts, principles, and equations essential for the "Diploma Level: Machine Learning Practice" exam, based on the provided question paper. The summary is divided into key areas with clear explanations and equations (where applicable).

1. Scikit-Learn (sklearn) Fundamentals

1.1 Datasets and Utilities

Key Concepts:

• Loaders vs. Fetchers:

  • Both return a Bunch object (dictionary-like with keys: data, target, feature_names, DESCR).
  • Loaders: Load small, built-in datasets (e.g., load_iris(), load_wine()).
  • Fetchers: Download larger datasets (e.g., fetch_california_housing()).

• Synthetic Datasets:

  • make_blobs(): Generates isotropic Gaussian blobs for clustering.

    X, y = make_blobs(n_samples=10, centers=3, n_features=2, random_state=0)
    # Output shape: (n_samples, n_features) → (10, 2)

Key Functions:

1.2 Data Preprocessing

Key Transformers:

Key Equations:

1. Standard Scaling:

• $\mu$: Mean of feature.
• $\sigma$: Standard deviation.

1. Max Absolute Scaling:

1. Polynomial Features (degree=2, interaction_only=False):

• With interaction_only=True: $[1, x_1, x_2, x_1x_2]$.

2. Model Training and Evaluation

2.1 Cross-Validation

Key Methods:

Key Outputs:

• LOO Example (for X = [1, 2, 3, 4]):

  split# 1, [1 2 3] [0]
  split# 2, [0 2 3] [1]
  split# 3, [0 1 3] [2]
  split# 4, [0 1 2] [3]

2.2 Metrics

Key Metrics:

Example Confusion Matrix:

• Recall for Class 2: $\frac{2}{2+1} = 0.67$.

2.3 Model-Specific Concepts

Linear Models:

• SGDRegressor:

  • shuffle=True: Shuffles data every epoch.
  • learning_rate='constant', eta0=1e-2: Fixed learning rate.

• LogisticRegression:

  • Default solver: lbfgs.
  • Output: Probabilities (use predict_proba()).

PCA (Principal Component Analysis):

• Explained Variance Ratio:

  • Sum of ratios = 1.
  • Example: For collinear data X = [[1,1], [2,2]], output is [1., 0.].

Decision Trees:

• Hyperparameters:

  • max_depth: Maximum tree depth.
  • min_samples_split: Minimum samples to split a node.
  • min_samples_leaf: Minimum samples at a leaf.

• Effect of max_depth:

  • Decreasing max_depth may decrease or maintain training score (never guaranteed to increase).

K-Means Clustering:

• Elbow Method:

  • Plot WCSS (Within-Cluster Sum of Squares) vs. number of clusters.
  • Optimal $k$ is at the "elbow" point.

• Example:

  wcss = []
  for i in range(1, 11):
      km = KMeans(n_clusters=i, random_state=10)
      km.fit(X)
      wcss.append(km.inertia_)  # WCSS = inertia

Neural Networks (MLP):

• MLPClassifier/Regressor:

  • hidden_layer_sizes=(a, b, c): 3 hidden layers with a, b, c neurons.
  • n_layers_: Total layers = input + hidden + output.
  • out_activation_: Default is softmax (classification) or identity (regression).

3. Ensemble Methods

3.1 Voting Classifier

• Hard Voting: Majority class label.
• Soft Voting: Argmax of averaged probabilities.

• Example:

  VotingClassifier(
      estimators=[('lr', LogisticRegression()), ('rf', RandomForestClassifier())],
      voting='hard'
  )

3.2 Bagging and Boosting

• BaggingClassifier:

  • base_estimator: Model to bag (e.g., SVC()).
  • n_estimators: Number of base models.

• GridSearchCV:

  • cv: Number of cross-validation folds (must be ≥ 1).
  • return_train_score: If True, includes training scores in cv_results_.

4. Clustering and Dimensionality Reduction

4.1 K-Means

• Inertia: Sum of squared distances to nearest cluster center.
• Labels: Assigns each point to a cluster (0 to n_clusters-1).

• Example Output:

  KMeans(n_clusters=3, random_state=5).fit(X).labels_
  # Possible output: [0 0 1 1 2 2]

4.2 Hierarchical Clustering

• Dendrogram:

  • Visualizes cluster merges.
  • Most similar sample to label 1: The sample merged earliest with label 1 (e.g., sample 0 in the example).

5. Model Evaluation and Interpretation

5.1 ROC Curve

• Perfect Classifier:

  • TPR = 1, FPR = 0 (top-left corner).
  • Example: A single point at (0, 1).

5.2 Dummy Models

• DummyRegressor:

  • strategy="mean": Always predicts the mean of y.
  • Example: For y = [1, 2, 3], predictions = [2, 2, 2].

5.3 Nearest Neighbors

• RadiusNeighborsClassifier:

  • radius=1.0: Considers neighbors within Manhattan distance of 1.
  • p=1: Manhattan distance; p=2: Euclidean.

• KNeighborsGraph:

  • kneighbors_graph(X): Adjacency matrix of neighbors.

6. Sklearn API Design Principles

1. Consistency:

   • All estimators follow fit()/predict()/transform().

2. Sensible Defaults:

   • Models work "out of the box" (e.g., LogisticRegression(solver='lbfgs')).

3. Non-Proliferation of Classes:

   • Avoid redundant classes (e.g., use Pipeline instead of custom wrappers).

4. Composition:

   • Build complex models by combining simple ones (e.g., Pipeline, FeatureUnion).

5. Inspection:

   • Hyperparameters are directly accessible (e.g., model.coef_).

Graph 2: Model Relationships

Graph 3: Clustering and Dimensionality Reduction

Graph 4: Preprocessing Transformers

Graph 5: Ensemble Methods