Home / MLF / Week 1

Week 1 · What is machine learning, and which maths do we need?

Official syllabus line: Introduction to machine learning

About 50 minQuiz 1 material (see exam radar)BSCS2004 · Harish Guruprasad Ramaswamy, Arun Rajkumar, Prashanth LA

Before you start: Nothing. If you can add, multiply and read a graph with an x-axis and a y-axis, you are ready.

Watch first: the animated story of this weekAbout 1-2 minutes of real 3D animation with a friendly voice. Pause any time.

Read the words of the video

Imagine teaching a young cousin to tell mangoes from bananas. Instead of long rules about shape and color, you just show them many fruits, saying "mango" or "banana" each time. Soon, they learn to identify new fruits on their own! This is just like machine learning.
Computers can't see mangoes directly. They only understand numbers! So, our first big step in machine learning is to turn everything, like a house or a photo, into lists of numbers. These lists are called vectors. Once we have numbers, mathematics can help the computer find hidden rules to learn from.
Let's find the length of a vector like (3, 4). Imagine three steps right, then four steps up. The straight line distance from start to end is the length. Using Pythagoras' theorem, square three and four, add them, then take the square root. That gives us five. Easy!
For your exams, remember the key words like 'loss functions' and 'classification.' Pay attention to details like 'norm' meaning length, not squared length. And know that 'squared loss' is for numbers, while 'zero-one loss' is for categories. Keep practicing turning the world into numbers. You've got this!

The one-minute story

Think of it like this

Imagine you want to teach a little cousin to tell mangoes from bananas. You could write rules: "if it is long and yellow, it is a banana". Soon you hit trouble: green bananas, short mangoes, half-ripe everything.

Or you could do what really works: show your cousin lots of fruit, and say the name each time. After enough examples they just get it, even for fruit they have never seen.

That is machine learning. Instead of writing the rules, we show a computer lots of examples and let it find the rules.

The catch: a computer cannot look at a mango. It only understands numbers. So the first job in this whole course is: turn the world into numbers, and then do clever things with those numbers. That is exactly why we study maths here.

By the end of this week you will be able to:

say what a data point, a feature, a label and a model are;
tell regression, classification, clustering and dimensionality reduction apart;
turn a data point into a vector and measure its length, the distance between two of them, and how much they point the same way (the dot product);
measure how wrong a model is with a loss function.

Words you need

Data point - one example. One house, one student, one photo.
Feature - one measurable fact about a data point. For a house: area, number of bedrooms, age.
Label (or target) - the answer we want to predict for that data point. For a house: its price. For a photo: "cat" or "dog".
Dataset - the whole pile of data points (and labels, if we have them).
Model - a rule, written as a formula, that takes features in and gives a prediction out.
Training - showing the model the examples so it can adjust itself.
Prediction - what the model says for a new data point.
Loss - a number that says how wrong the model is. Smaller is better.

Idea 1: Everything becomes a list of numbers (a vector)

Think of it like this

A house for sale is described on a board outside: 120 square metres, 3 bedrooms, 10 years old. Write that as a list: $(120, 3, 10)$ .

That list of three numbers is a vector. You can picture it two ways, and both are useful:

as a point in space (go 120 along one road, 3 along another, 10 along a third), or
as an arrow from the starting corner to that point.

A vector with $n$ numbers lives in $n$ -dimensional space, written $\mathbb{R}^n$ . Do not panic about 1000 dimensions. We only ever draw two, and every rule we learn on the page works unchanged in 1000.

A photo of $100 \times 100$ pixels is just a vector of $10{,}000$ brightness numbers. A sentence can become a vector of word counts. Once your data is vectors, maths can touch it.

Two data points as arrows from the corner. The dashed line is the distance between them. Over 3 across and 4 up makes a 3-4-5 triangle, so it is exactly 5.

The length of a vector (its norm)

Pythagoras gives the length of an arrow. For $v = (v_1, v_2)$ :

\|v\| = \sqrt{v_1^2 + v_2^2}

In $n$ dimensions you just keep adding squares under the root: $\|v\| = \sqrt{v_1^2 + v_2^2 + \dots + v_n^2}$ . This is called the norm (or length, or Euclidean norm).

The distance between two points

The distance between $a$ and $b$ is the length of the arrow from one to the other:

\text{distance}(a,b) = \|a - b\|

Later, k-means (MLT week 3) will group points by this distance, and k-NN (MLT week 7) will predict from the nearest neighbours. Same formula.

The dot product: do two arrows point the same way?

Multiply matching entries and add:

a \cdot b = a_1 b_1 + a_2 b_2 + \dots + a_n b_n

Think of it like this

Two people pushing a heavy box. If they push in the same direction, the pushes add up: big positive dot product. If they push at right angles, the second person does nothing useful for the first: dot product zero. If they push against each other: negative.

The dot product hides an angle inside it:

a \cdot b = \|a\|\,\|b\|\cos\theta

So perpendicular vectors have dot product 0 (because $\cos 90^\circ = 0$ ). You will use that sentence all course long. And since $\cos\theta$ is never bigger than 1, we always have $|a\cdot b| \le \|a\|\|b\|$ (the Cauchy-Schwarz inequality), with equality only when the arrows are parallel.

A unit vector has length 1. To make one from any vector, divide by its own length: $\hat v = v / \|v\|$ . It keeps the direction and throws away the size.

Idea 2: The four big families of learning

The idea

Family	What you give the computer	What it must produce	Everyday picture
Regression (supervised)	examples with a number as the answer	a prediction that is a number	"Guess this house's price"
Classification (supervised)	examples with a category as the answer	a category	"Is this email spam or not?"
Clustering (unsupervised)	examples with no answers	groups of similar things	"Sort these photos into piles"
Dimensionality reduction (unsupervised)	examples with no answers	the same data using fewer numbers	"Describe a face with 5 numbers instead of 10,000"

Supervised means a teacher gave you the right answers (labels). Unsupervised means no teacher; the computer finds structure by itself.

Watch out - real exam trap

The classic confusion: is the answer a number you can add and average, or a name?

Predicting tomorrow's temperature in degrees: regression (24.3 and 24.4 are close, and "close" matters).
Predicting whether it will rain: classification (rain / no rain are just names).
A rating of 1 to 5 stars is the tricky one: if the order and gaps matter we treat it as regression; if each star level is just a bucket we treat it as classification. Read what the question says.

Idea 3: Learning means making the loss small

Think of it like this

A teacher marks your homework. She does not just say "wrong". She says how wrong: off by 1 mark, off by 20 marks. A loss function is that marking rule for a model. Training is the model repeatedly looking at its marks and adjusting to lose fewer of them.

Say the true values are $y_1,\dots,y_n$ and the model predicts $\hat y_1,\dots,\hat y_n$ (read "y-hat").

Squared loss (used for regression). Square each error so that over- and under-shooting both count, and big mistakes count extra:

\ell(y,\hat y) = (y-\hat y)^2 \qquad\qquad \text{MSE} = \frac1n\sum_{i=1}^n (y_i-\hat y_i)^2

0-1 loss (used for classification). A mistake costs 1, a correct answer costs 0:

\ell(y,\hat y) = \begin{cases}0 & y=\hat y\\ 1 & y\ne\hat y\end{cases}

Averaged over the data, the 0-1 loss is simply the fraction of mistakes.

Reconstruction loss (used in dimensionality reduction). We squash a data point into fewer numbers, then try to rebuild it, and measure how far the rebuilt point $\hat x$ is from the original: $\|x - \hat x\|^2$ . You will meet this again in PCA (MLF week 6, MLT week 1).

Train error, test error, and memorising

Think of it like this

Suppose a student memorises last year's question paper word for word. Give them last year's paper: 100%. Give them a fresh paper: disaster. They memorised instead of learning.

A model can do the same. So we keep some data hidden: train on one part, test on the part the model has never seen. The test score is the honest one. A model that is great on training data and poor on test data has overfit.

Solved examples

Solved example: 1. Length, distance and unit vector

Question. Let $a = (3,4)$ and $b = (6,8)$ . Find $\|a\|$ , the distance between $a$ and $b$ , and the unit vector in the direction of $a$ .

Solution.

Length of $a$ : $\sqrt{3^2+4^2}=\sqrt{9+16}=\sqrt{25}=5$ .
Difference: $a-b=(3-6,\,4-8)=(-3,-4)$ . Its length is $\sqrt{9+16}=5$ . So the distance is 5.
Unit vector: divide $a$ by its length: $\hat a=(3/5,\,4/5)=(0.6,\,0.8)$ . Check: $0.6^2+0.8^2=0.36+0.64=1$ .

Notice $b = 2a$ : $b$ points the same way but is twice as long, so $\|b\|=10$ and the distance between the tips is $10-5=5$ .

Solved example: 2. Dot product and angle

Question. $a=(3,4)$ and $c=(2,1)$ . Find $a\cdot c$ and the angle between them.

Solution.

$a\cdot c = 3\cdot 2+4\cdot 1=10$ .
$\|a\|=5$ and $\|c\|=\sqrt{4+1}=\sqrt5\approx 2.236$ .
$\cos\theta=\dfrac{10}{5\cdot\sqrt5}=\dfrac{2}{\sqrt5}\approx 0.894$ .
$\theta=\cos^{-1}(0.894)\approx 26.6^\circ$ .

Positive dot product, small angle: the arrows lean the same way. Now try $d=(-4,3)$ : $a\cdot d=-12+12=0$ , so $a\perp d$ . They are at exactly $90^\circ$ .

Solved example: 3. Regression or classification? Supervised or not?

Question. Name the type of problem.

(a) Given a person's age, income and city, predict how many rupees they will spend next month. (b) Given a bank customer's record, decide "will repay" or "will default". (c) Given 10,000 shoppers' purchases with no other information, find groups of similar shoppers. (d) Given photos described by 4,096 numbers each, find 20 numbers that capture most of the information.

Solution. (a) regression: the answer is an amount of money. (b) classification: two named outcomes. (c) clustering: no answers given, we want groups. (d) dimensionality reduction: the same photos in fewer numbers. (a) and (b) are supervised; (c) and (d) are unsupervised.

Solved example: 4. Two kinds of loss on the same predictions

Question. A regression model predicts $\hat y=(2,5,9)$ for true values $y=(3,5,7)$ . Find the MSE. Then a classifier predicts (cat, cat, dog, dog) for true labels (cat, dog, dog, cat). Find its 0-1 loss.

Solution.

Errors $y-\hat y=(1,\,0,\,-2)$ . Squares: $1,\,0,\,4$ . Sum $=5$ . MSE $=5/3\approx1.67$ .
Compare label by label: cat=cat right, dog vs cat wrong, dog=dog right, cat vs dog wrong. Two mistakes out of four, so 0-1 loss $=2/4=0.5$ .

The one big miss (error $-2$ ) cost 4 in squared loss, four times what the error of 1 cost. That is what "big mistakes count extra" means.

Your turn

Your turn: 1. Perpendicular or not?

Is $u=(2,-1,4)$ perpendicular to $w=(3,2,-1)$ ?

Show the full solution

Perpendicular means dot product zero. $u\cdot w = 2\cdot3+(-1)\cdot2+4\cdot(-1)=6-2-4=0$ . Yes, they are perpendicular.

Your turn: 2. Distance in three dimensions

Two houses are $a=(100,2,5)$ and $b=(104,2,8)$ (area, bedrooms, age). How far apart are they?

Need a hint?

The first thing to try: subtract the vectors.

Show the full solution

$a-b=(-4,\,0,\,-3)$ . Squares: $16,\,0,\,9$ , sum $25$ , so the distance is $\sqrt{25}=\mathbf{5}$ .

(In real life you would first rescale the features. Area is measured in the hundreds and age in single digits, so area would dominate the distance. That idea comes back in MLP.)

Your turn: 3. Largest possible dot product

$\|a\|=3$ and $\|b\|=5$ . What is the largest value $|a\cdot b|$ can take, and when does it happen?

Need a hint?

Use $a\cdot b=\|a\|\|b\|\cos\theta$ and think about how big $\cos\theta$ can be.

Show the full solution

$|\cos\theta|\le1$ , so $|a\cdot b|\le 3\cdot5=\mathbf{15}$ . It happens when $\cos\theta=\pm1$ , i.e. the vectors are parallel (same or exactly opposite direction). This exact question, "max of $|a\cdot b|$ given the two lengths", appears in real papers.

Your turn: 4. Choose the loss

A model predicts the price of a taxi ride. Another model predicts whether a passenger will give a tip. Which loss suits each, and why?

Need a hint?

Ask first: is each answer a number or a category?

Show the full solution

The price is a number, so use squared loss (MSE): being off by Rs 50 is worse than off by Rs 5, and squaring reflects that. The tip decision is yes/no, a category, so use 0-1 loss: it just counts the mistakes.

Exam radar

From the 53 MLF Quiz 1 papers on the practice site, the summaries mention these week-1 ideas most often: loss functions (21 papers), dimensionality reduction (20), clustering (20), supervised learning (16), regression (15), classification (15), classification vs regression (13), mean squared loss (8), reconstruction loss (6).

The questions are mostly short, conceptual and worth 2 to 3 marks. They reward knowing the vocabulary exactly, so make sure you can say each of those words in your own sentence.

Watch out - real exam trap

A norm is a length, not a squared length. $\|v\|^2$ is the squared norm. If the question asks for the distance and you give the squared distance, the option you pick will be the planted wrong one.
Squared loss is for numbers; 0-1 loss is for categories. Do not mix them up.
Zero dot product means perpendicular only for non-zero vectors; the zero vector has dot product 0 with everything.
"Unsupervised" does not mean "no maths": it means no labels.

Quick quiz

Predicting whether a customer will cancel a subscription (yes/no) is...

regression
classification
clustering
dimensionality reduction

The answer is a category (yes or no), and we have past customers with known outcomes, so it is supervised classification.

Which of these is a unit vector?

(1, 1)
(0.6, 0.8)
(3, 4)
(0.5, 0.5)

Only (0.6, 0.8) has length 1: 0.36 + 0.64 = 1. (1,1) has length root 2; (3,4) has length 5; (0.5,0.5) has length about 0.707.

Two non-zero vectors have dot product 0. What is the angle between them?

0 degrees
45 degrees
90 degrees
180 degrees

a.b = |a||b|cos(theta) = 0 forces cos(theta) = 0, so the angle is 90 degrees.

Which of these are TRUE about squared loss and 0-1 loss? (select all that apply)

Squared loss punishes big errors much more than small ones.
0-1 loss over a dataset equals the fraction of wrong predictions.
0-1 loss is the usual choice for predicting house prices.
Squared loss can be negative.

Squared loss is never negative (it is a square). House prices are numbers, so squared loss is the usual choice there.

Recap

Remember this

ML = learn the rules from examples instead of writing them. Data must become numbers: each data point is a vector.
Norm $\|v\|=\sqrt{\sum v_i^2}$ ; distance $\|a-b\|$ ; dot product $a\cdot b=\sum a_ib_i=\|a\|\|b\|\cos\theta$ ; perpendicular means $a\cdot b=0$ .
Regression = predict a number; classification = predict a category; clustering and dimensionality reduction = no labels.
A loss function measures how wrong the model is: squared loss (regression), 0-1 loss (classification), reconstruction loss (dimensionality reduction).
Judge a model on data it has not trained on.

Formulas to keep

Norm: $\|v\|=\sqrt{v_1^2+\dots+v_n^2}$
Distance: $\|a-b\|$
Dot product: $a\cdot b=\sum_i a_ib_i=\|a\|\|b\|\cos\theta$ ; perpendicular $\iff a\cdot b=0$
Cauchy-Schwarz: $|a\cdot b|\le\|a\|\|b\|$
Unit vector: $\hat v=v/\|v\|$
Squared loss: $(y-\hat y)^2$ ; MSE $=\frac1n\sum(y_i-\hat y_i)^2$
0-1 loss: $1$ if $y\ne\hat y$ , else $0$ ; average = fraction of mistakes

Connected weeks (learn them together)

MLT week 1 - PCA: finding the best view of your data: MLT starts by treating every data point as a vector. Vectors, lengths and dot products are MLF week 1.
MLP week 1 - Your first end-to-end ML project: What 'learning from data' means, then your first end-to-end project.

Practise with real exam questions

These come straight from past papers (with the real figures and the official answer key). Open 16 real questions on this week All Quiz 1 papers