Concepts this paper tests
Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.
Summary of Core Topics, Concepts, Principles, and Equations
Core Topics and Concepts
Machine Learning Models and Algorithms
- Ridge Regression: A linear regression technique with L2 regularization.
- Stochastic Gradient Descent (SGD): An optimization algorithm for training linear models.
- Polynomial Regression: Extending linear regression by adding polynomial terms.
- Support Vector Machines (SVM): Supervised learning models for classification and regression.
- Logistic Regression: A classification algorithm for binary and multiclass problems.
- K-Nearest Neighbors (KNN): A non-parametric method used for classification and regression.
- Gaussian Naive Bayes: A probabilistic classifier based on Bayes' theorem.
- Perceptron: A linear classifier for binary classification.
- KNN Imputation: A method to handle missing values using KNN.
Model Evaluation and Metrics
- Mean Squared Error (MSE): A common metric for regression models.
- R² Score: A statistical measure of how well the regression predictions approximate the real data points.
- Precision Score: A metric for classification models, indicating the ratio of correctly predicted positive observations to the total predicted positives.
Data Preprocessing and Feature Engineering
- StandardScaler: Standardizes features by removing the mean and scaling to unit variance.
- Polynomial Features: Generates polynomial and interaction features.
- CountVectorizer: Converts a collection of text documents to a matrix of token counts.
- KNN Imputation: Imputes missing values using the K-Nearest Neighbors algorithm.
Hyperparameter Tuning and Model Selection
- GridSearchCV: Exhaustive search over specified parameter values for a model.
- Regularization Parameters: Parameters like
alpha in Ridge Regression and C in SVM that control the strength of regularization.
Model Interpretation and Analysis
- Coefficients in Linear Models: Interpretation of model coefficients in linear regression and classification models.
- Decision Boundaries: Understanding how decision boundaries change with different parameters in SVM.
Important Equations
- Ridge Regression Cost Function:
J(θ)=2m1i=1∑m(hθ(x(i))−y(i))2+λj=1∑nθj2
where λ is the regularization parameter.
- Mean Squared Error (MSE):
MSE=n1i=1∑n(ytrue,i−ypred,i)2
- R² Score:
R2=1−SSTSSR
where SSR is the sum of squares of residuals and SST is the total sum of squares.
- Precision Score:
Precision=True Positives+False PositivesTrue Positives
- Polynomial Features:
PolynomialFeatures(degree=d)
generates features of degree up to d.
- KNN Imputation:
KNNImputer(n_neighbors=k,weights="uniform")
imputes missing values based on the k nearest neighbors.