IIT M DIPLOMA ET1 EXAM QPD1 S1 03 Sep - 03 Sep 2023

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Here’s a structured summary of the core topics, concepts, principles, and equations essential for the "Diploma Level: Machine Learning Practice" exam, along with Mermaid knowledge graphs to visualize relationships.

Core Topics and Concepts Summary

1. Data Preprocessing

• Scaling Techniques:

  • StandardScaler: Standardizes features to have mean=0 and variance=1.

    • Formula: $z = \frac{x - \mu}{\sigma}$

  • MaxAbsScaler: Scales features by their maximum absolute value.

    • Formula: $x_{\text{scaled}} = \frac{x}{|x|_{\text{max}}}$

  • MinMaxScaler: Scales features to a specified range (default: [0, 1]).

    • Formula: $x_{\text{scaled}} = \frac{x - x_{\text{min}}}{x_{\text{max}} - x_{\text{min}}}$

• Handling Missing Data:

  • SimpleImputer: Replaces missing values with mean, median, or mode.

    • Example: SimpleImputer(strategy='mean')

• Encoding Categorical Data:

  • LabelEncoder: Converts categorical labels to numerical values (e.g., ['cat', 'dog'] → [0, 1]).
  • OneHotEncoder: Creates binary columns for each category (e.g., ['red', 'blue'] → [[1, 0], [0, 1]]).
  • MultiLabelBinarizer: Converts lists of labels into binary vectors (e.g., [['a', 'b'], ['a']] → [[1, 1], [1, 0]]).

• Polynomial Features:

  • PolynomialFeatures: Generates polynomial and interaction features.

    • Example: PolynomialFeatures(degree=2) creates $x_1, x_2, x_1^2, x_1x_2, x_2^2$.

• Pipelines and FeatureUnions:

  • Pipeline: Sequentially applies transformations and estimators.

    • Example:

      Pipeline([('impute', SimpleImputer()), ('scale', StandardScaler())])

  • FeatureUnion: Combines features from multiple transformers.

    • Example:

      FeatureUnion([('poly', PolynomialFeatures()), ('pca', PCA())])

2. Dimensionality Reduction

• PCA (Principal Component Analysis):

  • Reduces dimensionality by projecting data onto principal components.
  • Example: PCA(n_components=2)

3. Supervised Learning Algorithms

• Linear Models:

  • Linear Regression:

    • Equation: $y = w_0x_0 + w_1x_1 + \dots + w_nx_n + \epsilon$

    • Ridge (L2) and Lasso (L1) Regularization:

      • Ridge: Minimizes $||w||_2^2$.
      • Lasso: Minimizes $||w||_1$.
      • Elastic Net: Combines L1 and L2.

  • Logistic Regression:

    • Equation (sigmoid): $\sigma(z) = \frac{1}{1 + e^{-z}}$
    • Solvers: liblinear (good for small datasets), sag (stochastic average gradient).

• Naive Bayes:

  • GaussianNB: Assumes features follow a normal distribution.

    • Predicts posterior probabilities: $P(y|X)$.

• Support Vector Machines (SVM):

  • Kernels: linear, rbf (Radial Basis Function), poly.
  • Example: SVC(kernel='rbf')

• Decision Trees:

  • Parameters:

    • max_depth: Maximum depth of the tree.
    • min_samples_split: Minimum samples required to split a node.
    • min_samples_leaf: Minimum samples required at a leaf node.

  • Attributes:

    • feature_importances_: Importance of each feature.

• Ensemble Methods:

  • Bagging (e.g., RandomForest):

    • Trains multiple models on bootstrapped samples.
    • Example: RandomForestClassifier(n_estimators=100)

  • Boosting (e.g., AdaBoost, Gradient Boosting):

    • Sequentially corrects errors of previous models.
    • Example: AdaBoostClassifier(base_estimator=DecisionTreeClassifier())
  • Voting Classifier: Combines predictions from multiple models.

• Neural Networks (MLP):

  • MLPClassifier: Multi-layer perceptron for classification.

    • Parameters:

      • hidden_layer_sizes: Number of neurons in hidden layers (e.g., (100,)).
      • activation: relu, tanh, logistic.
      • alpha: L2 regularization strength.
      • solver: adam, sgd.

4. Unsupervised Learning Algorithms

• Clustering:

  • K-Means:

    • Assigns data points to $k$ clusters by minimizing within-cluster variance.
    • Example: KMeans(n_clusters=3)

  • Hierarchical Clustering:

    • Linkage methods: complete, ward, single, average.
  • DBSCAN: Density-based clustering for arbitrary-shaped clusters.

5. Model Evaluation

• Metrics:

  • Classification:

    • Accuracy: $\frac{\text{TP} + \text{TN}}{\text{TP} + \text{TN} + \text{FP} + \text{FN}}$
    • Precision: $\frac{\text{TP}}{\text{TP} + \text{FP}}$
    • Recall: $\frac{\text{TP}}{\text{TP} + \text{FN}}$
    • F1-Score: $2 \times \frac{\text{Precision} \times \text{Recall}}{\text{Precision} + \text{Recall}}$

  • Regression:

    • MSE (Mean Squared Error): $\frac{1}{n} \sum_{i=1}^n (y_i - \hat{y}_i)^2$
    • R² Score: $1 - \frac{\mathrm{SS}_{\text{res}}}{\mathrm{SS}_{\text{tot}}}$
• Confusion Matrix: Visualizes TP, TN, FP, FN.
• Cross-Validation: GridSearchCV for hyperparameter tuning.

6. Nearest Neighbors

• KNeighborsClassifier:

  • Parameters:

    • n_neighbors: Number of neighbors to consider.
    • weights: uniform or distance.
    • metric: euclidean, manhattan.
  • Decision boundary complexity increases with smaller n_neighbors.

7. Feature Extraction (Text Data)

• CountVectorizer:

  • Converts text to a matrix of token counts.
  • Example: CountVectorizer() → outputs (n_documents, n_features).

8. Incremental Learning

• Partial Fit:

  • Used for online learning (e.g., SGDClassifier, PassiveAggressiveClassifier).
  • Example: model.partial_fit(X_batch, y_batch)

9. Handling Imbalanced Data

• Class Weighting:

  • class_weight='balanced' in LogisticRegression or SVC.
• Resampling: Oversampling (SMOTE) or undersampling.

10. Model Persistence

• Save/load models with joblib or pickle:

  from joblib import dump, load
  dump(model, 'model.joblib')
  model = load('model.joblib')

Key Equations

1. Standard Scaling:

1. MinMax Scaling:

1. Linear Regression:

1. Logistic Regression (Sigmoid):

1. Ridge (L2) Regularization:

1. Lasso (L1) Regularization:

1. Precision:

1. Recall:

1. F1-Score:

1. Polynomial Features (Degree 2):

Graph 1: Data Preprocessing Workflow

Graph 2: Supervised Learning Algorithms

Graph 3: Clustering Algorithms

Graph 4: Model Evaluation

Graph 5: Neural Networks (MLP)

Graph 6: Ensemble Methods

Exam Tips

1. Focus on Scikit-Learn APIs:

   • Know how to initialize, fit, and predict with estimators (e.g., model.fit(X, y), model.predict(X_test)).

2. Understand Hyperparameters:

   • Example: C in SVM (regularization), n_estimators in Random Forest.

3. Preprocessing Steps:

   • Always scale features for algorithms like SVM, KNN, and neural networks.

4. Evaluation Metrics:

   • For imbalanced data, prioritize precision/recall/F1 over accuracy.

5. Dimensionality Reduction:

   • Use PCA for high-dimensional data before clustering or classification.

6. Avoid Overfitting:

   • Use regularization (L1/L2), cross-validation, and pruning (for trees).

7. Pipelines:

   • Combine preprocessing and modeling steps (e.g., Pipeline([('scaler', StandardScaler()), ('model', SVC())])).

This summary covers all the essential topics, equations, and relationships needed to excel in the exam. Use the Mermaid graphs to visualize connections between concepts!