IIT M DIPLOMA AN1 EXAM ETD1 11 Dec 2022 - 11 Dec 2022

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Here’s a structured summary of core topics, concepts, principles, and equations essential for the "Diploma Level: Machine Learning Practice" exam, followed by Mermaid knowledge graphs to visualize relationships.

Core Topics & Concepts

1. Data Handling & Preprocessing

• Dataset Generation:

  • sklearn.datasets.make_classification:

    • Parameters: n_features, n_informative, n_redundant, n_classes, n_samples.
    • Output shapes: X.shape = (n_samples, n_features), y.shape = (n_samples,).
  • Manual dataset creation with numpy (e.g., np.random.rand).

  • sklearn.datasets.load_wine:

    • Returns a Bunch object (dictionary-like) by default. Use return_X_y=True for (X, y) tuple.

• Feature Engineering:

  • Polynomial Features:

    • PolynomialFeatures(degree=2) generates interactions and quadratic terms.
    • Output shape: (n_samples, 1 + n_features + C(n_features, 2)) for degree=2.
    • Example: Input [a, b] → [1, a, b, a², ab, b²].

  • Text Vectorization:

    • CountVectorizer: Converts text to a matrix of token counts. Output shape: (n_documents, n_unique_words).
    • TfidfVectorizer: Weights terms by importance (TF-IDF). Output: sparse matrix.
    • HashingVectorizer: Stateless (no vocabulary storage), memory-efficient for large datasets.

• Scaling/Normalization:

  • StandardScaler, MinMaxScaler, MaxAbsScaler.
  • Critical for distance-based models (e.g., SVM, KNN) and gradient descent.

2. Supervised Learning

• Linear Models:

  • Logistic Regression:

    • Hyperparameters: penalty (l1, l2), C (inverse of alpha), solver, l1_ratio (for elastic net).
    • LogisticRegression(l1_ratio=0.2) sets L1 penalty to 20% and L2 to 80%.

  • Ridge/Lasso Regression:

    • Ridge(alpha=1): L2 regularization. alpha = regularization strength.
    • Lasso(alpha=0.1): L1 regularization (sparse coefficients).
    • RidgeCV: Cross-validated Ridge with alphas as input candidates.

  • SGDClassifier/Regressor:

    • Supports partial_fit for incremental learning.
    • Hyperparameters: loss (hinge, log, perceptron), penalty, learning_rate.
    • SGDClassifier(loss='perceptron') implements a linear SVM (not perceptron).

• Support Vector Machines (SVM):

  • Kernel Trick: kernel='rbf' (Gaussian RBF) for non-linear boundaries.

  • gamma in RBF kernel:

    • gamma='auto' → 1 / n_features (not 1 / (n_features * X.var())).
    • Controls the "reach" of a single training example.

  • Multi-class SVM:

    • LinearSVC uses "one-vs-rest" (OvR) by default.
    • decision_function shape: (n_samples, n_classes) for OvR.

• Decision Trees:

  • Hyperparameters:

    • max_depth, min_samples_split, max_features.

  • Properties:

    • Non-parametric, prone to overfitting if unconstrained.
    • Splits create 2 children nodes (binary splits).
    • Can handle both numerical and categorical features.

• Neural Networks:

  • MLPClassifier:

    • hidden_layer_sizes=(100,): 1 hidden layer with 100 neurons.
    • Weights shape: (n_features, n_neurons) for the first layer.
    • Solvers: sgd, adam.

• Evaluation Metrics:

  • Confusion Matrix:

    • Precision = TP / (TP + FP), Recall = TP / (TP + FN), Accuracy = (TP + TN) / Total.

  • ROC-AUC:

    • AUC = 1 (perfect classifier), 0.5 (random guessing).
    • ROC curve: TPR (y-axis) vs. FPR (x-axis) (not specificity).
  • Log Loss: Measures probabilistic confidence. Lower = better.

3. Unsupervised Learning

• Clustering:

  • K-Means:

    • Sensitive to outliers, terminates at local optima (SSE).
    • Linear algorithm (assigns points to nearest centroid).

  • Hierarchical Clustering:

    • Connectivity-based (agglomerative/divisive).
  • DBSCAN: Density-based (not covered in questions).

4. Model Evaluation & Selection

• Cross-Validation:

  • ShuffleSplit(n_splits=4): Randomly splits data into train/test n_splits times.
  • cross_validate: Returns multiple metrics (e.g., test_score).

• Hyperparameter Tuning:

  • GridSearchCV: Exhaustive search over param_grid.

  • Example:

    params = {'n_estimators': range(30, 41), 'learning_rate': np.arange(0.2, 2.2, 0.2)}

• Incremental Learning:

  • partial_fit in SGDClassifier, SGDRegressor, MiniBatchKMeans.
  • Useful for large datasets (out-of-memory) or streaming data.

5. Ensemble Methods

• Bagging:

  • BaggingRegressor:

    • max_samples: Fraction of samples per bag (e.g., 0.8).
    • bootstrap=False: Sampling without replacement.
    • max_features: Fraction of features per subset.

• Boosting:

  • AdaBoostClassifier:

    • base_estimator: Default is DecisionTreeClassifier(max_depth=1).
    • n_estimators: Number of weak learners.

6. Specialized Topics

• Imbalanced Data:

  • DummyClassifier(strategy="most_frequent"): Predicts the majority class.

• Multi-label Classification:

  • Binarize outputs for ROC-AUC extension.

• Text Processing:

  • CountVectorizer vs. HashingVectorizer:

    • HashingVectorizer doesn’t store vocabulary (memory-efficient).
    • CountVectorizer requires fit to build vocabulary.

7. Mathematical Equations

2. Unsupervised Learning Relationships

3. Model Evaluation & Selection

4. Data Preprocessing Pipeline

5. Ensemble Methods

6. Incremental Learning