IIT M FOUNDATION DIPLOMA ENDTERM QPB2 07 Aug 2022 IBA NS - 07 Aug 2022

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Here’s a structured summary of core topics, concepts, principles, and equations essential for the Diploma Level: Machine Learning Practice exam, along with Mermaid knowledge graphs to visualize relationships.

Core Topics & Concepts

1. Scikit-Learn Fundamentals

Key Concepts:

• Datasets:

  • load_* (e.g., load_wine, load_iris) and fetch_* (e.g., fetch_california_housing) return a Bunch object (dictionary-like with keys like data, target, feature_names).

  • make_blobs generates synthetic datasets for clustering/classification.

    • Parameters: n_samples, centers, n_features, random_state.
    • Output shape: (n_samples, n_features).

• Data Preprocessing:

  • Scaling:

    • StandardScaler: Standardizes features (mean=0, std=1).

      • scaler.mean_: Mean of training data.
      • scaler.transform(X): Applies scaling.
    • MaxAbsScaler: Scales by maximum absolute value (preserves sparsity).
    • add_dummy_feature: Adds a column of 1s (bias term) to X.

  • Encoding:

    • MultiLabelBinarizer: Converts multi-label data to binary matrix.

    • PolynomialFeatures: Generates polynomial/interaction features.

      • degree=2, interaction_only=True: Includes only interactions (e.g., $x_1x_2$), not squares ($x_1^2$).

• Model Evaluation:

  • Cross-Validation:

    • LeaveOneOut: Splits data into n_samples folds (each sample is a test set once).
    • train_test_split: Splits data into train/test sets.

  • Metrics:

    • r2_score: Coefficient of determination ($R^2$).
    • confusion_matrix: TP, TN, FP, FN for classification.
    • Recall: $\text{Recall} = \frac{TP}{TP + FN}$.
    • ROC Curve: Plots TPR vs. FPR (perfect classifier: TPR=1, FPR=0).

Equations:

1. Standard Scaling:

where $\mu$ = mean, $\sigma$ = standard deviation.

1. MaxAbs Scaling:

1. Polynomial Features (degree=2):

(If interaction_only=True, excludes squares.)

1. R² Score:

1. Recall:

2. Supervised Learning

Key Models:

• Linear Models:

  • LinearRegression: Fits linear coefficients via OLS.
  • LogisticRegression: Binary/multi-class classification (uses softmax for multi-class).

  • SGDRegressor: Linear regression with stochastic gradient descent.

    • shuffle=True: Shuffles data each epoch.
  • Perceptron: Linear classifier (converges if data is linearly separable).
  • RidgeCV: Ridge regression with built-in cross-validation for alpha.

• Tree-Based Models:

  • DecisionTreeClassifier:

    • Hyperparameters: max_depth, min_samples_split, min_samples_leaf.
    • n_features_in_: Number of features in training data.
  • RandomForestClassifier: Ensemble of decision trees.

• Neural Networks:

  • MLPClassifier/MLPRegressor:

    • hidden_layer_sizes: Tuple specifying neurons per layer (e.g., (8, 12)).
    • activation: 'logistic', 'relu', etc.
    • out_activation_: Always 'softmax' for classification (even if activation='logistic').
    • n_layers_: Total layers = input + hidden + output.

• Support Vector Machines (SVM):

  • SVC(kernel='linear'): Linear kernel.
  • support_vectors_: Training samples defining the decision boundary.

Equations:

1. Linear Regression Prediction:

1. Logistic Regression (Binary):

where $\sigma$ = sigmoid function.

1. Softmax (Multi-Class):

1. Decision Tree Split Criterion (Gini Impurity):

where $p_k$ = proportion of class $k$.

1. SVM Decision Function (Linear):

Support vectors satisfy $y_i(w^T x_i + b) = 1$.

3. Unsupervised Learning

Key Models:

• Clustering:

  • KMeans:

    • n_clusters: Number of clusters.
    • inertia_: Sum of squared distances to nearest cluster center (WCSS).
    • Elbow Method: Plot inertia_ vs. n_clusters to find optimal $k$.

  • Hierarchical Clustering:

    • sch.dendrogram: Visualizes cluster merges.
    • method='ward': Minimizes variance when merging clusters.

• Dimensionality Reduction:

  • PCA:

    • n_components: Number of principal components.
    • explained_variance_ratio_: Proportion of variance explained by each PC.
    • For $X = [[1,1], [2,2]]$, PCA returns [1., 0.] (all variance in 1st PC).

Equations:

1. PCA Transformation:

where $W$ = eigenvectors of $X^T X$.

1. K-Means Objective (WCSS):

where $\mu_j$ = cluster centroids.

4. Model Evaluation & Selection

Key Concepts:

• Cross-Validation:

  • GridSearchCV: Exhaustive search over hyperparameters.

  • VotingClassifier: Combines multiple models (e.g., logistic regression + random forest).

    • voting='hard': Majority vote.
    • voting='soft': Predicts class with highest average probability.

• Metrics:

  • Confusion Matrix: TP, FP, FN, TN.
  • Dummy Regressor: Baseline model (e.g., strategy='mean' predicts the mean of y).

Equations:

1. Accuracy:

1. F1 Score:

5. Scikit-Learn API Design

Key Principles:

• Consistency:

  • All estimators have fit(), predict(), transform(), or score() methods.
  • Hyperparameters are set at initialization (e.g., model = SVC(C=1.0)).

• Composition:

  • Pipeline: Chains transformers and estimators.
  • FeatureUnion: Applies multiple transformers in parallel.

• Inspection:

  • Attributes end with _ (e.g., coef_, feature_importances_).

2. Supervised Learning Workflow

3. Unsupervised Learning

4. Model Evaluation & Selection

5. Scikit-Learn API Design

Key Takeaways for the Exam

1. Scikit-Learn Syntax:

   • load_*/fetch_* return Bunch objects.
   • make_blobs generates synthetic data with shape (n_samples, n_features).
   • Always check shape (e.g., data.data.shape, not data.data.shape()).

1. Preprocessing:

   • StandardScaler centers and scales data.
   • PolynomialFeatures(degree=2, interaction_only=True) generates $[1, x_1, x_2, x_1x_2]$.

1. Models:

   • LogisticRegression uses softmax for multi-class.
   • MLPClassifier’s out_activation_ is always softmax.
   • DecisionTreeClassifier’s n_features_in_ = number of input features.

1. Evaluation:

   • LeaveOneOut generates n_samples splits.
   • explained_variance_ratio_ for PCA sums to 1.
   • Perfect ROC curve: TPR=1 at FPR=0.

1. API Design:

   • Hyperparameters are set at initialization.
   • Use Pipeline for chaining steps, FeatureUnion for parallel transformations.