IIT M FOUNDATION DIPLOMA FN EXAM QDF1 28 Apr 2024 - 28 Apr 2024

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Here’s a structured summary of core topics, concepts, principles, and equations essential for the Diploma Level: Machine Learning Practices (Computer-Based Exam). The summary is organized by themes (e.g., Supervised Learning, Unsupervised Learning, Model Evaluation, Preprocessing, etc.), followed by a knowledge graph in Mermaid syntax to visualize relationships.

1. Core Topics & Concepts

A. Supervised Learning

1. Classification

• Algorithms:

  • Logistic Regression:

    • Assumes linear decision boundary.
    • Uses sigmoid function for binary classification:

• Loss function: Log Loss (Cross-Entropy):

• Naive Bayes:

  • Assumes feature independence (conditional on class label).
  • Types: Gaussian (continuous features), Multinomial (discrete counts), Bernoulli (binary features).
  • Posterior probability:

• Decision Trees:

  • Splits data based on impurity metrics:

    • Gini Impurity:

• Entropy:

• Hyperparameters:

  • max_depth, min_samples_split, min_samples_leaf, criterion (gini/entropy).
  • Support Vector Machines (SVM):
• Finds the optimal hyperplane to maximize margin.
• Kernels: linear, rbf (Gaussian), poly.
• Hinge Loss for SVMs:

• k-Nearest Neighbors (k-NN):

  • Classifies based on majority vote of k nearest neighbors.
  • Distance metrics: Euclidean, Manhattan.
  • Curse of dimensionality: Performance degrades in high dimensions.

• Perceptron:

  • Linear classifier with step function activation.
  • Update rule:

• Converges only if data is linearly separable.

2. Regression

• Algorithms:

  • Linear Regression:

    • Model: $\hat{y} = \mathbf{w}^T \mathbf{x} + b$.
    • Loss: Mean Squared Error (MSE):

• Closed-form solution (Normal Equation):

• Stochastic Gradient Descent (SGD):

  • Updates weights incrementally:

• Hyperparameters:

  • learning_rate (eta0), max_iter, tol (tolerance for early stopping), shuffle (shuffle data each epoch).

• Warm Start: Reuses previous solution for incremental learning.

  • Regularization:
• L1 (Lasso):

• L2 (Ridge):

• Elastic Net: Combines L1 and L2.
• Purpose: Prevent overfitting by penalizing large weights.

3. Ensemble Methods

• Bagging (Bootstrap Aggregating):

  • Trains multiple models on bootstrapped samples and averages predictions.
  • Example: Random Forest (uses decision trees with feature subsampling).

  • Hyperparameters:

    • n_estimators, max_features (subset of features per tree), bootstrap.

• Boosting:

  • Sequentially corrects errors of previous models.
  • Example: AdaBoost (weights misclassified samples higher).
  • Gradient Boosting: Fits new models to residual errors.
• Stacking: Combines predictions of multiple models using a meta-model.

B. Unsupervised Learning

1. Clustering

• k-Means:

  • Partitions data into k clusters by minimizing inertia (within-cluster sum of squares):

• Initialization: k-means++ (better than random).
• Elbow Method: Plot inertia vs. k to find optimal k.

• Silhouette Score: Measures cluster separation/cohesion.

  • Hierarchical Clustering:
• Agglomerative: Starts with each point as a cluster, merges closest clusters.
• Linkage Methods: Single, complete, average, Ward.
• Dendrogram: Visualizes cluster merges.

2. Dimensionality Reduction

• Principal Component Analysis (PCA):

  • Projects data onto orthogonal axes (principal components) maximizing variance.
  • Explained variance ratio: Proportion of variance captured by each PC.

• Feature Engineering:

  • Polynomial Features: Generates interactions/non-linear terms.
  • One-Hot Encoding: Converts categorical features to binary vectors.

  • Scaling:

    • StandardScaler: $z = \frac{x - \mu}{\sigma}$.
    • MinMaxScaler: $x_{\text{scaled}} = \frac{x - \min(X)}{\max(X) - \min(X)}$.
    • MaxAbsScaler: Scales by maximum absolute value.

C. Model Evaluation

1. Metrics

• Classification:

  • Confusion Matrix: | | Predicted + | Predicted - | |---------------|-------------|-------------| | Actual + | TP | FN | | Actual - | FP | TN |
  • Precision: $\frac{TP}{TP + FP}$.
  • Recall: $\frac{TP}{TP + FN}$.
  • F1-Score: Harmonic mean of precision/recall.
  • Accuracy: $\frac{TP + TN}{TP + TN + FP + FN}$.

• Regression:

  • MSE/RMSE, MAE, R² Score.

2. Cross-Validation

• k-Fold CV: Splits data into k folds, trains on k-1 folds, tests on 1 fold.
• Leave-One-Out (LOO): Extreme case of k-fold where k = N (number of samples).
• Stratified k-Fold: Preserves class distribution in folds.

3. Bias-Variance Tradeoff

• Underfitting: High bias (model too simple).
• Overfitting: High variance (model too complex).

• Solutions:

  • Regularization, cross-validation, ensemble methods, more data.

D. Preprocessing & Data Handling

1. Data Cleaning

• Handle missing values: Drop (dropna) or impute (SimpleImputer).
• Outliers: Detect using IQR or Z-score, then clip/remove.

2. Feature Transformation

• ColumnTransformer: Applies different transformers to different columns.
• Pipeline: Chains preprocessing and modeling steps.

3. Pandas Operations

• Indexing:

  • loc: Label-based.
  • iloc: Integer-based.
• Aggregations: groupby, apply, sum, mean.
• Handling Mixed Data: apply(type) to check data types.

E. Neural Networks (MLP)

• Multilayer Perceptron (MLP):

  • Hidden layers with neurons and activation functions (ReLU, sigmoid).

  • Hyperparameters:

    • hidden_layer_sizes: Tuple specifying neurons per layer (e.g., (100, 50)).
    • activation: relu, tanh, logistic.
    • solver: adam, sgd.
  • Tuning: Grid search or random search over hyperparameters.

F. Special Topics

1. Multi-Label/Multi-Class Classification

• Multi-Label: Each sample can have multiple labels (e.g., animals in a photo).
• Multi-Class: Single label from >2 classes (e.g., digit classification).

2. Initialization Methods

• k-Means++: Better centroid initialization than random.
• Xavier/Glorot: Scales initial weights based on layer sizes (for neural networks).

3. Outlier Handling in Clustering

• Agglomerative Clustering: Assigns outliers to the nearest cluster.

2. Key Equations

Graph 2: Unsupervised Learning Relationships

Graph 3: Model Evaluation & Preprocessing

Graph 4: Data Preprocessing

Graph 5: Ensemble Methods

Graph 6: Neural Networks (MLP)

4. Exam Tips

1. Focus Areas:

   • Algorithms: Know when to use which (e.g., k-means for clustering, logistic regression for binary classification).
   • Hyperparameters: Understand their impact (e.g., max_depth in decision trees, n_estimators in random forests).
   • Preprocessing: Scaling, encoding, and handling missing data are critical.
   • Evaluation: Metrics and cross-validation strategies.
   • Math: Equations for loss functions, regularization, and probability (e.g., Naive Bayes).

1. Common Pitfalls:

   • Overfitting: Regularization, cross-validation, and ensemble methods help.
   • Data Leakage: Ensure preprocessing (e.g., scaling) is done within cross-validation folds.
   • Class Imbalance: Use class_weight='balanced' or resampling techniques.

1. Coding:

   • Know how to implement pipelines, column transformers, and grid search in sklearn.
   • Understand how to interpret confusion matrices and classification reports.

1. Pandas:

   • Master loc/iloc, groupby, apply, and handling mixed data types.

This summary covers 90% of the exam content. Use the knowledge graphs to visualize connections between topics, and focus on equations for mathematical questions. Good luck!