Machine Learning Foundations 16 Jul 26 - 19 Jul 2026

Concepts this paper tests

Auto-generated summary from the source site. Handy as a checklist; the lessons above explain each idea from scratch.

This summary outlines the essential topics, concepts, principles, and mathematical equations required for the exam, drawn from the provided question paper.

Summary of Core Topics, Concepts, Principles, and Equations

I. Multivariable Calculus

• Topic: Functions of Multiple Variables

• Concepts:

  • Partial Derivatives: The rate of change of a multivariable function with respect to one variable, treating other variables as constants.
  • Gradient Vector: A vector that contains all the first-order partial derivatives of a scalar-valued multivariable function. It points in the direction of the greatest rate of increase of the function.
  • Directional Derivative: The rate at which a function changes at a specific point in a specified direction.
  • Linear Approximation (Tangent Plane Approximation): Approximating a function near a point using its tangent plane. This is the first-order Taylor approximation.

• Equations:

  • Partial Derivatives for $f(x, y)$:

    $$\frac{\partial f}{\partial x}, \quad \frac{\partial f}{\partial y}$$

  • Gradient Vector for $f(x, y)$:

    $$\nabla f(x, y) = \left\langle \frac{\partial f}{\partial x}, \frac{\partial f}{\partial y} \right\rangle$$

  • Unit Vector (Normalization): For a vector $\mathbf{v} = \langle v_1, v_2 \rangle$, its unit vector $\mathbf{u}$ is:

    $$\mathbf{u} = \frac{\mathbf{v}}{||\mathbf{v}||} = \frac{\langle v_1, v_2 \rangle}{\sqrt{v_1^2 + v_2^2}}$$

  • Directional Derivative of $f(x, y)$ in the direction of a unit vector $\mathbf{u}$:

    $$D_{\mathbf{u}}f(x, y) = \nabla f(x, y) \cdot \mathbf{u}$$

  • Linear Approximation of $f(x, y)$ around a point $(a, b)$:

    $$L(x, y) = f(a, b) + \frac{\partial f}{\partial x}(a, b)(x - a) + \frac{\partial f}{\partial y}(a, b)(y - b)$$

II. Linear Algebra

• Topic: Matrices, Vectors, and Vector Spaces

• Concepts:

  • Vector Operations: Dot product (scalar product), calculating vector magnitude (Euclidean norm), and vector normalization.
  • Matrix Operations: Matrix multiplication, transpose of a matrix.

  • Matrix Spaces:

    • Row Space: The span of the row vectors of a matrix.
    • Column Space: The span of the column vectors of a matrix.
    • Null Space (Kernel): The set of all vectors $\mathbf{x}$ such that $A\mathbf{x} = \mathbf{0}$.
    • Left Null Space: The null space of the transpose of a matrix, i.e., the set of all vectors $\mathbf{y}$ such that $A^T\mathbf{y} = \mathbf{0}$.
  • Rank of a Matrix: The dimension of its column space (and row space), representing the maximum number of linearly independent rows or columns.
  • Invertibility of Matrices: A square matrix $A$ is invertible if and only if its determinant is non-zero, its rank equals its dimension, or none of its eigenvalues are zero.
  • Eigenvalues and Eigenvectors: Eigenvectors are non-zero vectors that, when multiplied by a matrix, result in a scalar multiple of themselves (the eigenvalue). They represent the directions along which a linear transformation acts by scaling.
  • Diagonalization: The process of transforming a square matrix into a diagonal matrix using its eigenvectors. A matrix is diagonalizable if and only if for every eigenvalue, its algebraic multiplicity equals its geometric multiplicity.
  • Symmetric Matrices: Real symmetric matrices have real eigenvalues, are always orthogonally diagonalizable, and eigenvectors corresponding to distinct eigenvalues are orthogonal.
  • Projection: Geometrically projecting a vector onto another vector or a subspace.
  • Line Representation: Describing a line in space using vector equations or parametric equations.

• Equations:

  • Dot Product: For $\mathbf{a} = \langle a_1, a_2 \rangle$ and $\mathbf{b} = \langle b_1, b_2 \rangle$:

    $$\mathbf{a} \cdot \mathbf{b} = a_1b_1 + a_2b_2 = ||\mathbf{a}|| \cdot ||\mathbf{b}|| \cdot \cos\theta$$

  • Vector Magnitude (Euclidean Norm): For $\mathbf{a} = \langle a_1, a_2 \rangle$:

    $$||\mathbf{a}|| = \sqrt{a_1^2 + a_2^2}$$

  • Projection Matrix onto a Line: For projecting onto the line spanned by vector $\mathbf{a}$:

    $$P = \frac{\mathbf{a}\mathbf{a}^T}{\mathbf{a}^T\mathbf{a}}$$

    If $\mathbf{u}$ is a unit vector in the direction of $\mathbf{a}$:

    $$P = \mathbf{u}\mathbf{u}^T$$

  • Characteristic Equation (for finding Eigenvalues $\lambda$): For a square matrix $A$:

    $$\det(A - \lambda I) = 0$$

  • Trace and Determinant for a $2 \times 2$ Matrix (in terms of Eigenvalues $\lambda_1, \lambda_2$): For $A = \begin{pmatrix} a & b \\ c & d \end{pmatrix}$:

    $$\text{trace}(A) = a + d = \lambda_1 + \lambda_2$$
    $$\det(A) = ad - bc = \lambda_1 \lambda_2$$

  • Rank-Nullity Theorem: For an $m \times n$ matrix $A$:

    $$\text{rank}(A) + \text{nullity}(A) = n$$

    where $\text{nullity}(A)$ is the dimension of the null space of $A$.

  • Dimension of Left Null Space: For an $m \times n$ matrix $A$:

    $$\text{dim}(\text{Null}(A^T)) = m - \text{rank}(A)$$

  • Line Equation (Vector Form): For a line passing through point $\mathbf{p}_0$ with direction vector $\mathbf{d}$:

    $$\mathbf{r}(t) = \mathbf{p}_0 + t\mathbf{d}$$

III. Machine Learning Foundations

• Topic: Dimensionality Reduction and Autoencoders

• Concepts:

  • Dimensionality Reduction: Techniques to transform data from a high-dimensional space into a lower-dimensional representation while retaining essential information.

  • Autoencoder: A neural network architecture used for unsupervised learning to discover efficient data codings. It consists of an encoder and a decoder.

    • Encoder ($f$): Maps the high-dimensional input $\mathbf{x}$ to a lower-dimensional latent representation $\mathbf{u}$.
    • Decoder ($g$): Maps the latent representation $\mathbf{u}$ back to a reconstructed input $\mathbf{\hat{x}}$.
  • Reconstruction Loss: A metric used to quantify how accurately the decoder reproduces the original input from its compressed representation. A common choice is Mean Squared Error (MSE). Lower reconstruction loss indicates better preservation of information during dimensionality reduction and reconstruction.

• Equations:

  • Reconstruction Loss (Mean Squared Error): For a dataset $\{\mathbf{x}^{(i)}\}_{i=1}^N$ and their corresponding reconstructed outputs $\{\mathbf{\hat{x}}^{(i)}\}_{i=1}^N$:

    $$\text{Reconstruction Loss} = \frac{1}{N} \sum_{i=1}^N ||\mathbf{x}^{(i)} - \mathbf{\hat{x}}^{(i)}||^2$$

    where $||\mathbf{v}||^2$ denotes the squared Euclidean norm of vector $\mathbf{v}$.