IIT M FOUNDATION ET1 EXAM QPF2 S1 03 Sep - 03 Sep 2023

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Here’s a structured summary of the core topics, concepts, and equations for the Diploma Level: Machine Learning Practice (Computer-Based Exam). The summary is organized by key themes, followed by a Mermaid knowledge graph to visualize relationships.

1. Data Preprocessing

Key Concepts:

• Scaling/Normalization:

  • StandardScaler: Standardizes features to have mean = 0 and variance = 1.

    • Formula:

• MinMaxScaler: Scales features to a range (default: [0, 1]).

  • Formula:

• MaxAbsScaler: Scales by the maximum absolute value (preserves sparsity).

  • Formula:

• Example (Q2):

  from sklearn.preprocessing import MaxAbsScaler
  scaled_data = MaxAbsScaler().fit_transform([[1], [-2], [3]])
  # Output: min(scaled_data) = -1 (since -2/2 = -1)

• Imputation:

  • SimpleImputer: Replaces missing values with mean/median/mode.

    • Example (Q3):

      from sklearn.impute import SimpleImputer
      imp = SimpleImputer(strategy='mean')
      imp.fit_transform([[1], [2], [np.nan], [4]])  # Imputed mean = 3

• Categorical Encoding:

  • LabelEncoder: Converts categories to integers (e.g., ["cat", "dog"] → [0, 1]).

    • Note: Not suitable for non-ordinal data (use OneHotEncoder instead).

    • Example (Q10):

      from sklearn.preprocessing import LabelEncoder
      encoder = LabelEncoder()
      encoded = encoder.fit_transform(["cat", "dog", "cat"])  # Output: [0, 1, 0]

  • OneHotEncoder: Creates binary columns for each category.

  • MultiLabelBinarizer: Converts multi-label data to binary matrix (Q4).

    • Example:

      from sklearn.preprocessing import MultiLabelBinarizer
      mlb = MultiLabelBinarizer(classes=['cold', 'hot', 'large', 'small'])
      mlb.fit_transform([['large', 'cold'], ['small', 'hot']])
      # Output: [[1, 0, 1, 0], [0, 1, 0, 1]]

• Polynomial Features:

  • PolynomialFeatures: Generates interaction terms/polynomials (Q17).

    • Example:

      from sklearn.preprocessing import PolynomialFeatures
      pf = PolynomialFeatures(degree=2, interaction_only=True)
      pf.fit_transform([[1, 2]])  # Output: [1, 1, 2, 1*2] (no x² terms)

• Pipelines:

  • Pipeline: Chains transformers/estimators (Q3).

    • Example:

      from sklearn.pipeline import Pipeline
      pipe = Pipeline([
          ('impute', SimpleImputer(strategy='mean')),
          ('scale', StandardScaler())
      ])

Key Equations:

2. Dimensionality Reduction

Key Concepts:

• PCA (Principal Component Analysis):

  • Reduces dimensionality by projecting data onto orthogonal axes (principal components).
  • n_components: Number of components to keep.

  • Example (Q14):

    from sklearn.decomposition import PCA
    pca = PCA(n_components=5)
    pca.fit_transform(X)  # X.shape = (n_samples, n_features)

• Feature Union:

  • FeatureUnion: Combines multiple feature transformations (e.g., PCA + PolynomialFeatures).

    • Example (Q14):

      from sklearn.pipeline import FeatureUnion
      combined = FeatureUnion([
          ('pca', PCA(n_components=5)),
          ('poly', PolynomialFeatures(degree=2))
      ])

3. Supervised Learning

Key Algorithms:

1. Linear Models:

   • Linear Regression:

     • Equation: $Y = W_0X_0 + W_1X_1 + \dots + W_nX_n + \epsilon$.
     • fit_intercept=False: Forces intercept $W_0 = 0$ (Q24).

     • Example:

       from sklearn.linear_model import LinearRegression
       reg = LinearRegression(fit_intercept=False)
       reg.fit(X_train, y_train)
       print(reg.coef_)  # Weights [W₁, W₂, ...]

   • Logistic Regression:

     • Solvers:

       • liblinear: Supports L1/L2 regularization (good for small datasets).
       • lbfgs/newton-cg: Fast for large datasets (L2 only).
       • sag/saga: Stochastic gradient descent (supports L1).

     • Regularization:

       • C: Inverse of regularization strength (smaller C = stronger regularization).
       • penalty: 'l1' (Lasso) or 'l2' (Ridge).

     • Example (Q18, Q20):

       from sklearn.linear_model import LogisticRegression
       model = LogisticRegression(class_weight='balanced', C=0.5, solver='liblinear')

     • Attributes:

       • coef_: Feature weights.
       • intercept_: Bias term.

1. Naive Bayes:

   • GaussianNB: Assumes features follow a normal distribution.

     • predict_proba: Returns posterior probabilities (Q5).

     • Example:

       from sklearn.naive_bayes import GaussianNB
       clf = GaussianNB()
       clf.fit(X, y)
       proba = clf.predict_proba([[2.0, 3.0]])  # Output: [[P(class=0), P(class=1)]]

1. Support Vector Machines (SVM):

   • Kernels:

     • 'linear': Linear decision boundary.
     • 'rbf': Radial Basis Function (non-linear).
     • 'poly': Polynomial kernel.

   • Example (Q29):

     from sklearn.svm import SVC
     clf = SVC(kernel='rbf')

1. Decision Trees:

   • Hyperparameters:

     • max_depth: Maximum tree depth (None = unlimited).
     • min_samples_split: Minimum samples to split a node.
     • min_samples_leaf: Minimum samples in a leaf.
     • criterion: 'gini' (Gini impurity) or 'entropy' (information gain).

   • Overfitting:

     • Deeper trees (max_depth=None) are prone to overfitting (Q30).

   • Example:

     from sklearn.tree import DecisionTreeClassifier
     tree = DecisionTreeClassifier(max_depth=3, criterion='gini')

1. Ensemble Methods:

   • Bagging:

     • BaggingClassifier: Trains base estimators on bootstrap samples.

       • Parameters:

         • n_estimators: Number of base models.
         • max_samples: Fraction of samples per estimator.
         • bootstrap: Whether to sample with replacement.

       • Example (Q31):

         from sklearn.ensemble import BaggingClassifier
         bag_clf = BaggingClassifier(
             base_estimator=KNeighborsClassifier(n_neighbors=5),
             n_estimators=50,
             max_samples=0.5,
             bootstrap=True
         )

   • Boosting:

     • AdaBoostClassifier: Combines weak learners (e.g., decision stumps) with adaptive weights.

       • Parameters:

         • n_estimators: Number of boosting stages.
         • learning_rate: Shrinks contribution of each estimator.

       • Example (Q34):

         from sklearn.ensemble import AdaBoostClassifier
         ada = AdaBoostClassifier(
             base_estimator=DecisionTreeClassifier(max_depth=1),
             n_estimators=100,
             learning_rate=0.1
         )

   • Random Forest:

     • RandomForestClassifier: Ensemble of decision trees with feature subsampling.

       • Parameters:

         • max_features: Number of features to consider for splits (e.g., 'sqrt').
         • bootstrap: Whether to use bootstrap sampling.

       • Example (Q32):

         from sklearn.ensemble import RandomForestClassifier
         rf = RandomForestClassifier(
             n_estimators=100,
             max_features='sqrt',
             bootstrap=True
         )

1. Neural Networks:

   • MLPClassifier (Multi-Layer Perceptron):

     • Parameters:

       • hidden_layer_sizes: Tuple specifying neurons per layer (e.g., (100, 50)).
       • activation: 'relu', 'tanh', 'logistic'.
       • alpha: L2 regularization strength.
       • solver: 'adam', 'sgd', 'lbfgs'.

     • Example (Q21, Q22, Q23):

       from sklearn.neural_network import MLPClassifier
       mlp = MLPClassifier(
           hidden_layer_sizes=(100,),
           activation='relu',
           alpha=0.01,
           solver='adam'
       )

Key Equations:

4. Unsupervised Learning

Key Algorithms:

1. Clustering:

   • K-Means:

     • Partitions data into k clusters by minimizing within-cluster variance.

     • Parameters:

       • n_clusters: Number of clusters.
       • init: Initialization method ('k-means++').

     • Attributes:

       • labels_: Cluster assignments for each point.
       • cluster_centers_: Coordinates of centroids.

     • Example (Q7, Q35):

       from sklearn.cluster import KMeans
       kmeans = KMeans(n_clusters=3)
       kmeans.fit(X)
       labels = kmeans.labels_  # Cluster assignments

   • Hierarchical Clustering:

     • Linkage Methods (Q8):

       • complete: Maximum distance between clusters.
       • ward: Minimizes variance when merging.
       • single: Minimum distance (can cause chaining).
       • average: Average distance between clusters.

1. Dimensionality Reduction:

   • PCA: See Section 2.

Key Equations:

5. Model Evaluation

Key Metrics:

• Confusion Matrix:

  • Structure:

    [[TN, FP],
     [FN, TP]]

  • Precision: $\frac{TP}{TP + FP}$ (Q25).
  • Recall: $\frac{TP}{TP + FN}$.
  • F1-Score: Harmonic mean of precision and recall.

• Regularization:

  • L1 (Lasso): Penalizes absolute weights ($\lambda \sum |W_i|$).
  • L2 (Ridge): Penalizes squared weights ($\lambda \sum W_i^2$).

  • Elastic Net: Combines L1 and L2 (Q12).

    • Formula:

• Dummy Regressor:

  • Predicts the mean (regression) or most frequent class (classification) (Q15).

Key Equations:

6. Incremental Learning

Key Concepts:

• partial_fit:

  • Updates model with new data without full retraining.

  • Supported by:

    • SGDClassifier/SGDRegressor.
    • PassiveAggressiveClassifier.
    • Perceptron.
    • MultinomialNB (Q11).

  • Example:

    from sklearn.linear_model import SGDClassifier
    clf = SGDClassifier()
    clf.partial_fit(X_batch, y_batch, classes=[0, 1])

7. Text Feature Extraction

Key Techniques:

• CountVectorizer:

  • Converts text to a matrix of token counts.

  • Parameters:

    • ngram_range: Tuple (min_n, max_n) for n-grams.

  • Example (Q6):

    from sklearn.feature_extraction.text import CountVectorizer
    vectorizer = CountVectorizer()
    X = vectorizer.fit_transform(["hello world", "world hello"])
    # X.shape = (2, 2) (vocabulary: {'hello', 'world'})

Graph 1: Data Preprocessing

Graph 2: Supervised Learning

Graph 3: Unsupervised Learning

Graph 4: Model Evaluation & Miscellaneous

Exam Tips

1. Focus on Equations:

   • Memorize formulas for scaling, regularization, precision/recall, and PCA.

2. Sklearn Syntax:

   • Know how to initialize and fit transformers/estimators (e.g., StandardScaler(), Pipeline()).

3. Hyperparameters:

   • Understand the role of C (regularization), n_estimators (ensemble), max_depth (trees), and alpha (neural networks).

4. Common Pitfalls:

   • LabelEncoder is not for one-hot encoding (use OneHotEncoder).
   • fit_intercept=False forces the intercept to 0 in linear models.
   • K-Means is sensitive to feature scales (always scale data first).