Home / Study together / Weeks 1-12 together

MLF + MLT + MLP: weeks 1-12 together

Everything, interrelated: one map, one path, one End Term plan.

Weeks 1-4 together Weeks 5-8 together Weeks 1-8 together Weeks 1-12 together

The whole course as one story

Think of it like this

Imagine learning to cook. First you learn your knives and heat (MLF's maths). Then you learn recipes (MLT's algorithms). Then you actually cook in a real kitchen with a timer (MLP's scikit-learn). Weeks 9 to 12 are the final course: dishes that need everything, like logistic regression, SVMs, ensembles and neural networks. Each of them is a recipe (MLT) built from an optimisation tool (MLF) that you run in code (MLP).

Tip

Exams covered. The End Term of each course covers all twelve weeks, so this page is the End Term plan for MLF, MLT and MLP together. It also contains both quizzes (weeks 1 to 4 and 5 to 8, boundaries inferred from past papers). Weeks 9 to 12 are the third block: the lessons are listed below, and weeks 5 to 8 and weeks 1 to 4 are the first two.

The 12-week map

Week	MLF (tools)	MLT (algorithms)	MLP (code)
1	Intro, vectors	PCA	End-to-end project
2	Calculus	Kernel PCA	Pipelines, tuning
3	Projection, least squares	k-means	Linear regression, GD
4	Eigenvalues	MLE, Bayes, GMM, EM	Polynomial, regularised
5	Symmetric matrices	Regression	Logistic regression
6	SVD and PCA	Ridge, LASSO	Binary classifiers
7	Gradient descent	k-NN, trees	Multiclass
8	Convexity	Naive Bayes	SVMs in sklearn
9	Lagrange, KKT, logistic as optimisation	Perceptron, logistic regression	Decision trees, ensembles
10	Probability for ML	SVMs	Random forests, boosting
11	Exponential family	Bagging, boosting, AdaBoost	Neural networks in sklearn
12	Estimation and EM	Neural networks	Unsupervised in sklearn

The idea

Notice the shifts. The MLP lanes are not in lock-step with MLT: MLP teaches logistic regression (W5) and SVM code (W8) before MLT's theory (W9 and W10), and MLP W9 to W10 (trees, forests) trail MLT W7 and W11. That is why the braided plans pair lessons by idea, not by week number.

Idea map: where each big idea appears in all three courses

Big idea	MLF (the tool)	MLT (the algorithm)	MLP (the code)
Distance and dot product	W1	W2 kernels, W3 k-means, W7 k-NN	`KNeighborsClassifier`, `KMeans`
Derivative $=0$ and gradient	W2, W7	W3 centroid, W4 MLE, W5 least squares, W9 logistic	`SGDRegressor`, `LogisticRegression`
Projection, least squares	W3	W1 PCA, W5 regression	`LinearRegression`, `PCA`
Eigen and SVD	W4, W5, W6	W1 PCA, W2 kernel PCA, W6 ridge	`PCA`, `Ridge`
Convexity	W8	W5, W6, W9, W10 (all convex losses)	choosing solvers, `C`, `alpha`
Constraints, Lagrange, KKT	W9	W10 SVM dual, support vectors	`SVC(C, kernel, gamma)`
Probability and Bayes	W10	W4 Bayes, W8 Naive Bayes, W9 logistic	`GaussianNB`, `predict_proba`
Exponential family, MLE	W11, W12	W4 MLE, GMM	`GaussianMixture`
EM	W12	W4 GMM, EM	`GaussianMixture` (W12)
Combine many weak models	-	W11 bagging, boosting	W9 and W10 forests, boosting
Layers of non-linearity	W2 and W7 gradients	W12 networks, backprop	W11 `MLPClassifier`

The recommended path

Stage A: weeks 1 to 4. The nine-session braided plan. Add MLP weeks 1 and 2.
Stage B: weeks 5 to 8. The sixteen-session plan, which includes MLP weeks 3 to 8.
Stage C: weeks 9 to 12. The plan below. After it, repeat the quick quiz of every week and then sit full End Term papers.

Stage C braided plan (weeks 9 to 12)

#	Session	Read	Why now	Time
1	Constrained optimisation, Lagrange, KKT	MLF week 9	SVMs are exactly a constrained problem	90 min
2	Perceptron and logistic regression	MLT week 9	logistic regression is MLF 9's example; GD is MLF 7	80 min
3	Support vector machines	MLT week 10	uses Lagrange and KKT from session 1 and kernels (MLT 2)	100 min
4	Probability for ML	MLF week 10	the language of every probabilistic model that remains	80 min
5	Exponential family	MLF week 11	Gaussian, Bernoulli and Poisson fit one pattern; logistic regression becomes natural	70 min
6	Estimation and EM	MLF week 12	the full theory behind MLT 4's GMM and EM	90 min
7	Decision trees and ensembles in code	MLP week 9	MLT 7 trees, now with `max_depth`, `n_estimators`	60 min
8	Bagging, boosting, AdaBoost	MLT week 11	explains why forests and boosting work	90 min
9	Forests, boosting and tuning in code	MLP week 10	hyper-parameter counts are a favourite exam question	60 min
10	Neural networks	MLT week 12	stacks logistic regressions; training is MLF 7's gradient descent	100 min
11	Neural networks in code	MLP week 11	shapes, parameters, activation, `adam`	60 min
12	Unsupervised learning in code	MLP week 12	closes the loop with PCA, k-means and GMM from weeks 1 to 4	60 min

Four problems that tie the whole course together

Your turn: 1. Logistic regression is optimisation, probability and code at once (MLF 9, MLT 9, MLP 5)

For one example with label $y=1$ and score $z=\mathbf w^T\mathbf x+b=0$ , the logistic loss is $-\ln\sigma(z)$ with $\sigma(z)=\frac1{1+e^{-z}}$ . (a) Find $\sigma(0)$ and the loss. (b) The derivative of the loss with respect to $z$ is $\sigma(z)-y$ . What is it here? (c) Which way does gradient descent move $z$ ?

Need a hint?

At $z=0$ the model says "50-50".

Show the full solution

(a) $\sigma(0)=\frac12$ , so the loss is $-\ln\frac12=\ln2\approx\mathbf{0.693}$ . (b) $\sigma(0)-1=\mathbf{-0.5}$ . (c) GD moves against the gradient, so $z$ increases: the score for a true positive is pushed up, raising $\sigma(z)$ towards $1$ . In MLP this is LogisticRegression (or SGDClassifier(loss="log_loss")); in MLT it is the discriminative model; in MLF it is "logistic regression as an optimisation problem".

Your turn: 2. The SVM margin from a Lagrange point of view (MLF 9 and MLT 10 and MLP 8)

A separating hyperplane has $\mathbf w=(3,4)$ . (a) Find the margin width $\frac{2}{\|\mathbf w\|}$ . (b) If the optimiser doubles $\mathbf w$ to $(6,8)$ with the same constraints $y_i(\mathbf w^T\mathbf x_i+b)\ge1$ , what happens to the margin? (c) In SVC, what does a larger C do?

Need a hint?

$\|\mathbf w\|=\sqrt{9+16}$ .

Show the full solution

(a) $\|\mathbf w\|=5$ , so the margin is $\frac25=\mathbf{0.4}$ . (b) $\|\mathbf w\|=10$ , margin $=\mathbf{0.2}$ : smaller $\|\mathbf w\|$ means a wider margin, which is why the SVM minimises $\frac12\|\mathbf w\|^2$ under the constraints. The Lagrange/KKT machinery of MLF week 9 solves that, and only the points on the margin (the support vectors) get non-zero multipliers. (c) A larger C penalises margin violations more: a narrower margin that fits the training data harder (less regularisation).

Your turn: 3. The coin, the prior and the bias (MLF 10 to 12 meet MLT 4)

A coin shows 7 heads in 10 flips. (a) Find the MLE of $p$ . (b) With a $\text{Beta}(2,2)$ prior, find the posterior mode (MAP). (c) Why is it between the MLE and $0.5$ ?

Need a hint?

For $\text{Beta}(\alpha,\beta)$ and $h$ heads in $n$ flips, the posterior is $\text{Beta}(\alpha+h,\beta+n-h)$ with mode $\frac{\alpha+h-1}{\alpha+\beta+n-2}$ .

Show the full solution

(a) $\hat p=\frac7{10}=\mathbf{0.7}$ . (b) Posterior $\text{Beta}(9,5)$ , mode $=\frac{2+7-1}{2+2+10-2}=\frac8{12}=\mathbf{0.667}$ . (c) The prior $\text{Beta}(2,2)$ is centred at $0.5$ , so it pulls the estimate from $0.7$ towards $0.5$ . With more data the pull weakens. This is the same "shrinkage" idea as ridge: a prior on the weights is a penalty.

Your turn: 4. Counting skills you meet in MLP (MLT 11 meets MLP 9 and 10)

A grid search tries $n\_estimators\in\{50,100,200\}$ and $max\_depth\in\{3,5\}$ with 5-fold cross-validation and a final refit. (a) How many models are fitted in total? (b) A one-vs-one SVM on 4 classes trains how many classifiers? (c) A network has layers $4\to5\to3$ : how many parameters (weights plus biases)?

Need a hint?

Combinations $\times$ folds ( $+1$ ); $K(K-1)/2$ ; $\sum(mn+n)$ .

Show the full solution

(a) $3\times2=6$ combinations, $6\times5=30$ fits, plus $1$ refit $=\mathbf{31}$ . (b) $\frac{4\cdot3}{2}=\mathbf6$ (one-vs-rest would be $4$ ). (c) $(4\cdot5+5)+(5\cdot3+3)=25+18=\mathbf{43}$ .

How the three courses relate in the exam

	MLF	MLT	MLP
Style	derivations and calculations by hand	algorithm steps and short calculations	reading scikit-learn code, defaults and counts
Biggest weights	linear algebra, optimisation, probability	PCA, regression, SVM, ensembles, EM	pipelines, SGD, metrics, grid-search counts
Where to look	MLF patterns	MLT patterns	MLP patterns

The three patterns pages show which topics have actually repeated in past papers. Use them to decide where extra revision time goes.

Pace options

The idea

12 weeks, one week of everything per week (the table above, read by row).

6 weeks: two weeks per week: A (1 to 4), B (5 to 8) and C (9 to 12) as three blocks of two weeks each.

3-week End Term sprint if you already know weeks 1 to 8: Week 1 the Stage C plan; Week 2 one full End Term paper per course; Week 3 the patterns pages plus the wrong-answer review.

Last 3 days: Day 1 all the formula sheets; Day 2 the real exam questions of weeks you rated weakest; Day 3 one timed paper and the glossary.