2024 July07: IIT M AN EXAM QDF4 - 07 Jul 2024

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

Summary of Core Topics, Concepts, Principles, and Equations

1. Principal Component Analysis (PCA)

• Concepts: Projection of data onto principal components, reconstruction error, covariance matrix, eigenvalues, and eigenvectors.

• Equations:

  • Reconstruction error: $||x - (x^T w_1) w_1 - (x^T w_2) w_2||^2$
  • Variance along a principal component: $w^T C w$
  • Covariance matrix: $C = \frac{1}{n-1} \sum_{i=1}^{n} (x_i - \bar{x})(x_i - \bar{x})^T$

2. Kernel Methods

• Concepts: Valid kernels, polynomial kernels, kernel PCA, transformation using kernels.

• Equations:

  • Polynomial kernel: $k(x, y) = (x^T y + c)^d$
  • Kernel PCA: $K = \Phi(X)^T \Phi(X)$
  • Largest eigenvalue of the covariance matrix in kernel PCA: $\frac{\lambda}{n}$, where $\lambda$ is the largest eigenvalue of the centered kernel matrix.

3. Gaussian Mixture Models (GMM) and EM Algorithm

• Concepts: Mixture models, EM algorithm, E-step, M-step, posterior probabilities.

• Equations:

  • Posterior probability: $\lambda_i^k = \frac{\pi_k \mathcal{N}(x_i | \mu_k, \Sigma_k)}{\sum_{j=1}^{K} \pi_j \mathcal{N}(x_i | \mu_j, \Sigma_j)}$
  • Probability of a component generating a data point: $\frac{\lambda_p^k \pi_k}{\sum_{j=1}^{K} \lambda_p^j \pi_j}$

4. K-Means Clustering

• Concepts: Initialization, distance metrics, cluster means, convergence.

• Equations:

  • Euclidean distance: $d(x_i, \mu_j) = ||x_i - \mu_j||_2$
  • Probability of choosing a point as the next mean: $\frac{d(x_i, \mu_j)^2}{\sum_{k=1}^{n} d(x_k, \mu_j)^2}$

5. Bayesian Estimation

• Concepts: Prior and posterior distributions, Beta distribution, Bernoulli distribution, point estimation.

• Equations:

  • Beta distribution: $p \sim \text{Beta}(\alpha, \beta)$
  • Posterior distribution: $p | \text{data} \sim \text{Beta}(\alpha + \text{number of successes}, \beta + \text{number of failures})$
  • Expected value of the posterior: $\hat{p} = \frac{\alpha + \text{number of successes}}{\alpha + \beta + \text{number of trials}}$