Home / MLP / Week 1

Week 1 · Your first end-to-end ML project

The one-minute story

By the end of this week you can:

• list the stages of an end-to-end ML project and the scikit-learn design principles;
• explore a pandas DataFrame: shape, missing values, summary statistics, groups, filters;
• split data into train and test (random and stratified), and say why you must;
• impute missing values, encode categories and scale features, fitting on the training data only.

Words you need

Idea 1: The stages of a project

Choosing the performance measure. For regression the usual measure is the RMSE (root mean squared error), $\sqrt{\frac1n\sum(y_i-\hat y_i)^2}$, in the units of the target. When there are many outliers, the MAE (mean absolute error) $\frac1n\sum|y_i-\hat y_i|$ is less sensitive: squaring makes big errors count a lot. RMSE is the $\ell_2$ norm of the error vector (divided by $\sqrt n$), MAE the $\ell_1$ norm.

Idea 2: scikit-learn's design principles

Quiz questions often ask: "Scikit-Learn's API is well designed. The main principles are...". Remember five words:

• Consistency: every object has a simple, uniform interface.
• Inspection: all hyperparameters and learned values are public attributes. Learned values end with an underscore (coef_, statistics_, mean_).
• Non-proliferation of classes: data is plain NumPy arrays / SciPy sparse matrices / pandas, not special classes. Hyperparameters are plain strings and numbers.
• Composition: building blocks combine (a Pipeline is itself an estimator).
• Sensible defaults: you can get a working model quickly.

Three kinds of objects:

The golden rule you will meet again and again: fit on the training data only; transform both. Calling fit on the test data leaks information.

Idea 3: A first look with pandas

We will use a tiny table of 8 houses so every output fits on screen.

import numpy as np
import pandas as pd

df = pd.DataFrame({
    "area":  [50, 80, 65, np.nan, 120, 95, 70, 60],
    "rooms": [2, 3, 2, 3, 4, 3, np.nan, 2],
    "city":  ["A", "B", "A", "C", "B", "A", "C", "B"],
    "price": [30, 52, 40, 55, 90, 70, 48, 38],
})

The commands you use first, with the real output for this table:

df.shape                       # (8, 4)   8 rows, 4 columns
df.isna().sum()                # missing values per column
df.describe()                  # count, mean, std, min, quartiles, max (numbers only)
df["city"].value_counts()      # how often each category occurs
df.groupby("city")["price"].mean()
df.dropna().shape              # rows left if we drop every row with a hole

shape: (8, 4)
isna: area 1, rooms 1, city 0, price 0
describe -> count  area 7, rooms 7, price 8
            mean   area 77.14, rooms 2.71, price 52.88
            min    area 50, rooms 2, price 30
            max    area 120, rooms 4, price 90
value_counts: A 3, B 3, C 2
groupby city mean price: A 46.67, B 60.0, C 51.5
dropna().shape: (6, 4)

Notice count is 7 for area and rooms: describe ignores the missing values. And dropna() would throw away 2 of our 8 rows, a quarter of the data. That is wasteful, which is why we impute instead.

Other everyday pandas moves:

df["area"]                      # one column (a Series)
df[["area", "price"]]           # several columns (a DataFrame)
df[df["price"] > 50]            # rows where price > 50 (boolean filter)
df.loc[2, "price"]              # by LABEL: row label 2, column "price"
df.iloc[2, 3]                   # by POSITION: 3rd row, 4th column
df["price_per_m2"] = df["price"] / df["area"]     # a new column
df.sort_values("price", ascending=False).head(3)
df.groupby("city")["price"].agg(["mean", "median", "count"])
df.corr(numeric_only=True)      # Pearson correlation matrix

Idea 4: Hide a test set first

from sklearn.model_selection import train_test_split

X = df.drop(columns="price")      # features
y = df["price"]                    # label
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42)
print(X_train.shape, X_test.shape)   # (6, 3) (2, 3)

• test_size=0.25 gives 2 of 8 rows to the test set. If you omit test_size and train_size, the default test share is 0.25.
• random_state=42 makes the "random" shuffle repeatable, so you and your friend get identical splits.
• By default the split is random. With a small or imbalanced data set, a random split can accidentally mis-represent a class. Stratified sampling fixes this: it keeps the class proportions the same in train and test:

train_test_split(X, y, test_size=0.2, stratify=y, random_state=0)

With labels that are 80% class 0 and 20% class 1, the test set keeps exactly 20% class 1. (Also available: StratifiedShuffleSplit.) For a regression target you can stratify on a binned version (e.g. income categories) of an important feature.

Why do we need a test set at all? (A standard 2-mark question.) It lets us simulate the model's performance on new data by reserving a portion for testing; without it we would only know how well the model fits data it has already seen, hiding overfitting. Risk of not splitting: an over-optimistic estimate of performance.

Idea 5: Cleaning missing values

Three options for a column with holes: drop the rows (dropna), drop the column, or fill the holes. Filling is done with SimpleImputer:

from sklearn.impute import SimpleImputer

imputer = SimpleImputer(strategy="median")
imputer.fit(X_train[["area", "rooms"]])        # learns the medians from TRAIN only
print(imputer.statistics_)                      # the learned fill values
X_filled = imputer.transform(X_train[["area", "rooms"]])

On the full 8-row table the learned medians are [70. 3.] (area 70, rooms 3), and no nan remains afterwards. With strategy="mean" the fill values would be [77.143 2.714].

Also: KNNImputer(n_neighbors=5) fills a hole from the average of the most similar rows; add_indicator=True appends a column that records which values were missing. The learned values live in statistics_ (an attribute with a trailing underscore: inspection in action).

Idea 6: Words to numbers (encoding categories)

Models need numbers. For the city column ("A", "B", "C"):

from sklearn.preprocessing import OneHotEncoder, OrdinalEncoder

OneHotEncoder().fit_transform(df[["city"]])      # 8x3 SPARSE matrix of 0/1
OrdinalEncoder().fit_transform(df[["city"]])      # 0.,1.,2. integers in one column

OneHotEncoder first rows ->  A: [1,0,0]   B: [0,1,0]   A: [1,0,0]
OrdinalEncoder           ->  [0, 1, 0, 2, 1, 0, 2, 1]

Which one? An ordinal code says "A < B < C" and that B is "twice" A, which is only meaningful for ordered categories (small < medium < large). For plain nominal categories like city names use one-hot, which gives each category its own 0/1 column with no fake order. One-hot returns a sparse matrix by default (.toarray() to see it); handle_unknown="ignore" stops an error when the test data contains a category never seen in training; drop="first" removes one column to avoid redundancy.

LabelEncoder is for targets only (it turns ["cat","dog","cat","bird"] into [1, 2, 1, 0] with classes_ = ['bird', 'cat', 'dog']).

Idea 7: Feature scaling

If one feature is in the hundreds (area) and another in single digits (rooms), distance-based and gradient-based methods get dominated by the big one. Two standard fixes:

from sklearn.preprocessing import StandardScaler
scaler = StandardScaler().fit(X_train[["area"]])
print(scaler.mean_, scaler.scale_)      # learned from train: attributes end with _

StandardScaler is much less affected by outliers than MinMaxScaler, which a single extreme value can squash. Tree-based models do not need scaling; linear models with regularisation, k-NN, SVMs and neural networks do.

Solved examples

Your turn

Exam radar

Quick quiz

Recap

Practise with real exam questions

These come straight from past papers (with the real figures and the official answer key). Open 60 real questions on this week   All Quiz 1 papers