IIT M FOUNDATION AN2 EXAM QPF2 16 JULY 2023 - 16 Jul 2023

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Summary of Core Topics, Concepts, Principles, and Equations

1. Kernel Functions and Kernel PCA

• Kernel Function: A function that computes the dot product of the transformed input data in a higher-dimensional space.

  • Example: $k([x_1, x_2]^T, [y_1, y_2]^T) = 1 + x_1 y_1 + x_2 y_2 + x_1^2 y_1^2 x_2 y_2 + x_1 y_1 x_2^2 y_2^2$
• Transformation Mapping ($\phi$): Maps input data to a higher-dimensional space.

• Kernel PCA: Principal Component Analysis (PCA) applied in a higher-dimensional space using kernel functions.

  • Properties:

    • The number of principal components cannot exceed the dimensionality of the dataset.
    • Linear combinations of valid kernels with positive coefficients are also valid kernels.
    • Kernel functions must produce non-negative outputs.

2. K-means Clustering

• Algorithm: Iterative algorithm to partition data into $k$ clusters.

  • Steps:

    1. Initialize centroids.
    2. Assign each data point to the nearest centroid.
    3. Recompute centroids based on the current assignment.
    4. Repeat until convergence.
  • Termination: When centroids no longer change or a maximum number of iterations is reached.
  • Distance Metric: Euclidean distance is commonly used.

3. Principal Component Analysis (PCA)

• Purpose: Reduces the dimensionality of a dataset while preserving as much variability as possible.

  • Steps:

    1. Center the data.
    2. Compute the covariance matrix.
    3. Compute the eigenvalues and eigenvectors of the covariance matrix.
    4. Select the top $k$ eigenvectors (principal components) corresponding to the largest eigenvalues.

  • Properties:

    • The first principal component captures the maximum variance.
    • The first principal component is the eigenvector corresponding to the largest eigenvalue of the covariance matrix.
    • If data lies in a lower-dimensional subspace, the covariance matrix will have fewer non-zero eigenvalues.

4. Gaussian Mixture Models (GMM)

• Purpose: Models data as a mixture of Gaussian distributions.

  • Components:

    • $\pi_i$: Mixing coefficients.
    • $\mu_i$: Mean of the $i$-th Gaussian component.
    • $\sigma_i^2$: Variance of the $i$-th Gaussian component.
  • Probability Calculation: The probability that a point $x$ comes from a component is given by the posterior probability.

5. Bernoulli Distribution and Maximum Likelihood Estimation (MLE)

• Bernoulli Distribution: Models binary data.

  • Parameter: $p$, the probability of success (e.g., $P(x = 1)$).

  • MLE: Estimates the parameter $p$ based on observed data.

    • For a dataset with $n$ observations, if the MLE estimate for $p$ is $\hat{p}$, then the number of zeros is $n(1 - \hat{p})$.

6. Beta Distribution

• Purpose: Used as a prior distribution for the parameter $p$ in a Bernoulli distribution.

  • Parameters: $\alpha$ and $\beta$.
  • Posterior Distribution: Updated based on observed data.