IIT M QUIZ 1 FOUNDATION DAD DIPLOMA QPE2 16 Oct 2022 - 16 Oct 2022

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Summary of Core Topics, Concepts, Principles, and Equations

Machine Learning Concepts

1. Supervised vs. Unsupervised Learning

   • Supervised Learning: Algorithms that learn from labeled data (e.g., linear regression, logistic regression).
   • Unsupervised Learning: Algorithms that learn from unlabeled data (e.g., K-means clustering).

1. Clustering Algorithms

   • K-means Clustering: Partitions data into k clusters based on feature similarity.
   • Equation: Minimize the within-cluster sum of squares (WCSS).

1. Regression Algorithms

   • Linear Regression: Models the relationship between a dependent variable and one or more independent variables.

     • Equation: $y = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \ldots + \beta_n x_n + \epsilon$

   • Logistic Regression: Used for binary classification.

     • Equation: $\log\left(\frac{p}{1-p}\right) = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \ldots + \beta_n x_n$

1. Dimensionality Reduction

   • Principal Component Analysis (PCA): Reduces the dimensionality of data while preserving variance.

     • Equation: $X_{PCA} = X \cdot W$, where $W$ is the matrix of eigenvectors.

1. Model Evaluation Metrics

   • Mean Squared Error (MSE): Measures the average squared difference between predicted and actual values.

     • Equation: $MSE = \frac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^2$

   • R-squared (R²): Measures the proportion of variance in the dependent variable that is predictable from the independent variables.

     • Equation: $R^2 = 1 - \frac{\sum_{i=1}^{n} (y_i - \hat{y}_i)^2}{\sum_{i=1}^{n} (y_i - \bar{y})^2}$

1. Data Preprocessing

   • Scaling: Standardizing or normalizing features.

     • Standard Scaling: $x' = \frac{x - \mu}{\sigma}$
     • Min-Max Scaling: $x' = \frac{x - \min(X)}{\max(X) - \min(X)}$

   • Imputation: Handling missing data.

     • Mean Imputation: Replace missing values with the mean of the feature.

1. Polynomial Features

   • PolynomialFeatures: Generates polynomial and interaction features.

     • Equation: For degree=2, $[1, x_1, x_2, x_1^2, x_1 x_2, x_2^2]$

1. Cross-Validation

   • K-Fold Cross-Validation: Splits data into k folds and trains the model on k-1 folds, validating on the remaining fold.
   • Equation: Average performance across all folds.

1. Regularization

   • Ridge Regression: Adds L2 regularization to linear regression.

     • Equation: $\min_{w} \|Xw - y\|_2^2 + \alpha \|w\|_2^2$

   • Lasso Regression: Adds L1 regularization to linear regression.

     • Equation: $\min_{w} \|Xw - y\|_2^2 + \alpha \|w\|_1$

1. Stochastic Gradient Descent (SGD)

   • SGDRegressor: Uses stochastic gradient descent to fit linear models.

     • Equation: $w = w - \eta \nabla Q_i(w)$, where $\eta$ is the learning rate.

Software Engineering Concepts

1. Scikit-Learn API

   • Design Principles: Consistency, nonproliferation of classes, best estimator.

   • Pipeline: Combines multiple steps into a single object.

     • Example: Pipeline([("poly", PolynomialFeatures(degree=2)), ("feature_scaling", MinMaxScaler())])

1. Data Splitting

   • train_test_split: Splits data into training and testing sets.

     • Example: train_test_split(X, y, test_size=0.2, random_state=42)

1. One-Hot Encoding

   • OneHotEncoder: Converts categorical features into a one-hot encoded format.

     • Example: OneHotEncoder(handle_unknown='ignore')

Mathematical Concepts

1. Linear Algebra

   • Matrix Operations: Essential for understanding transformations and feature scaling.
   • Eigenvalues and Eigenvectors: Important for PCA.

1. Probability and Statistics

   • Mean, Variance, and Standard Deviation: Fundamental for data preprocessing and evaluation metrics.
   • Normal Distribution: Basis for many statistical methods and assumptions in machine learning.