IIT M DIPLOMA FN EXAM QDD2 13 Apr 2025 - 13 Apr 2025

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Here’s a structured summary of core topics, concepts, principles, and equations essential for the exam, followed by Mermaid knowledge graphs to visualize relationships.

Core Topics & Concepts

1. Data Preprocessing

Key Concepts:

• Scaling/Normalization:

  • MinMaxScaler: Scales features to [0, 1] range. Formula:

• StandardScaler: Standardizes features to mean=0, std=1. Formula:

• When to use: Distance-based algorithms (KNN, SVM, K-Means) require scaling.

• Encoding Categorical Data:

  • OneHotEncoder: Converts categorical features into binary columns.
  • LabelEncoder: Converts categorical labels into integers (e.g., ['Yes', 'No'] → [1, 0]).

• Polynomial Features:

  • PolynomialFeatures(degree=n): Generates interaction terms up to degree n. Example: For features [a, b], degree=2 generates [1, a, b, a², ab, b²].

• Pipelines:

  • Pipeline([('scaler', StandardScaler()), ('model', SGDRegressor())]): Chains preprocessing and modeling steps.

Common Pitfalls:

• Data Leakage: Fit preprocessing (e.g., scaling) only on training data, then transform test data.
• Sparse Data: Use sparse=True in OneHotEncoder for memory efficiency.

2. Supervised Learning Algorithms

A. Linear Models

• Linear Regression:

  • Hypothesis: $h_\theta(x) = \theta_0 + \theta_1 x_1 + \dots + \theta_n x_n$.
  • Loss Function (MSE):

• Ridge (L2) vs. Lasso (L1) Regression:

  • Ridge: Penalizes large coefficients (prevents overfitting).

• Lasso: Can shrink coefficients to zero (feature selection).

• SGDRegressor:

  • Optimizes loss using stochastic gradient descent.

  • Key parameters:

    • learning_rate='adaptive' (adjusts learning rate dynamically).
    • penalty='l2' (Ridge), alpha=0.01 (regularization strength).

B. Logistic Regression

• Sigmoid Function:

• Loss Function (Log Loss):

• Multiclass: Use multi_class='multinomial' with solver='lbfgs'.

C. k-Nearest Neighbors (KNN)

• Distance Metrics:

  • Euclidean: $\sqrt{\sum_{i=1}^n (x_i - y_i)^2}$
  • Manhattan: $\sum_{i=1}^n |x_i - y_i|$
• Bottleneck: Prediction time scales with training data size (use KD-Trees/Ball Trees for efficiency).

• Hyperparameters:

  • n_neighbors: Trade-off between bias (small k) and variance (large k).
  • weights='distance': Closer neighbors have more influence.

D. Support Vector Machines (SVM)

• Kernel Trick: Maps data to higher dimensions for linear separation.

  • Linear: $K(x_i, x_j) = x_i^T x_j$
  • RBF: $K(x_i, x_j) = \exp(-\gamma \|x_i - x_j\|^2)$
• Regularization: C controls margin width (small C → wider margin, more regularization).

• SVC vs. SVR:

  • SVC: Classification (hinge loss).
  • SVR: Regression (ε-insensitive loss).

E. Decision Trees & Ensembles

• Decision Tree:

  • Splits based on Gini impurity or entropy:

• Overfitting Risk: Prune with max_depth or min_samples_split.

  • Random Forest:
• Ensemble of trees trained on bootstrapped samples (bootstrap=True).
• Feature Randomness: max_features='sqrt' (classification) or 'auto' (regression).

• OOB Score: Out-of-bag error estimate (oob_score=True).

  • AdaBoost:
• Sequentially trains weak learners (e.g., decision stumps), weighting misclassified samples higher.

• Hyperparameters:

  • n_estimators: Number of weak learners.
  • learning_rate: Shrinks contribution of each learner (default=1.0).
• Overfitting Mitigation: Reduce n_estimators or learning_rate.

F. Neural Networks (MLP)

• Architecture:

  • Input layer → Hidden layers (ReLU/tanh) → Output layer (softmax for classification, linear for regression).

• Key Parameters:

  • hidden_layer_sizes=(100,): Single hidden layer with 100 neurons.
  • activation='relu': Fast convergence, avoids vanishing gradients.
  • solver='adam': Adaptive learning rate.
  • early_stopping=True: Stops training if validation score doesn’t improve.
• Scaling: Critical for MLP (use StandardScaler).

3. Model Evaluation

Metrics:

Confusion Matrix:

              Predicted
            |   0   |   1   |
-----------|-------|-------|
Actual 0   |  TN   |  FP   |
Actual 1   |  FN   |  TP   |

• Recall for Class 1: $\frac{TP}{TP + FN}$.

Imbalanced Data:

• Problem: Accuracy is misleading if classes are imbalanced.

• Solutions:

  • Use F1-score, precision-recall curve, or ROC-AUC.
  • Resample data (oversample minority class or undersample majority class).
  • Class weights (class_weight='balanced').

4. Unsupervised Learning

A. Clustering

• K-Means:

  • Objective: Minimize within-cluster variance (inertia).

• Hyperparameters:

  • n_clusters: Number of clusters (use elbow method).
  • init='k-means++': Better initialization than random.

• Scaling: Required (uses Euclidean distance).

  • Hierarchical Clustering:

• Linkage Methods:

  • Single: Minimum distance between clusters.
  • Complete: Maximum distance.
  • Average: Mean distance.
  • Ward: Minimizes variance.
• Distance Metrics: Euclidean, Manhattan, Cosine.

B. Dimensionality Reduction

• PCA (Principal Component Analysis):

  • Transforms data to orthogonal axes (principal components) maximizing variance.
  • Variance Explained: Choose n_components to retain ≥95% variance.
  • Scaling: Required (PCA is distance-based).
• FeatureUnion: Combines multiple feature transformations (e.g., PCA + polynomial features).

5. Hyperparameter Tuning

Methods:

• GridSearchCV: Exhaustive search over specified parameter values.
• RandomizedSearchCV: Samples random combinations (faster for large spaces).

• Key Parameters:

  • cv=5: 5-fold cross-validation.
  • n_iter=10: Number of random samples (for RandomizedSearchCV).

Example for Ridge Regression:

param_dist = {'ridge__alpha': [0.01, 0.1, 1, 10, 100]}
search = RandomizedSearchCV(pipeline, param_distributions=param_dist, cv=5)

6. Model Interpretation & Debugging

Common Issues:

• Overfitting:

  • Symptoms: High training accuracy, low test accuracy.

  • Solutions:

    • Regularization (L1/L2).
    • Prune trees (max_depth).
    • Early stopping (neural networks).
    • More data or features.

• Underfitting:

  • Symptoms: Low training/test accuracy.

  • Solutions:

    • Increase model complexity (more layers/neurons).
    • Add features/polynomial terms.
    • Reduce regularization.

Debugging Tips:

• Learning Curves: Plot training vs. validation error vs. dataset size.
• Feature Importance: Use model.feature_importances_ (tree-based) or coefficients (linear models).

7. Scikit-Learn Specifics

Key Classes/Methods:

Pitfalls:

• fit vs. transform:

  • fit: Learns parameters (e.g., mean/std for scaling).
  • transform: Applies learned parameters.
  • Always fit on training data only!
• Data Leakage: Avoid scaling/encoding before train-test split.

2. Supervised Learning Algorithms

3. Model Evaluation Metrics

4. Hyperparameter Tuning

5. Handling Imbalanced Data

6. Clustering Algorithms

7. Neural Networks (MLP)