Home / MAD2 / Week 11

Week 11 · Performance: measuring and caching

The one-minute story

By the end of this week you can:

• name the numbers that describe speed (latency, throughput, percentiles, TTFB, page load) and the tools that measure them;
• find a slow spot: N+1 queries, a missing index, a heavy page;
• use browser caching headers, Flask-Caching with Redis, @cache.cached and @cache.memoize, and predict response times in exam code;
• design cache keys, choose an expiry and decide when to invalidate;
• speed up a Vue front end (lazy loading, compression, debounce) and add pagination;
• run a before and after measurement.

Words you need

Idea 1: What to measure, and with what

"The site is slow" is not a measurement. Pick numbers.

Apache Bench example (100 requests, 10 at a time):

ab -n 100 -c 10 http://127.0.0.1:5000/api/books

Read Requests per second (throughput), Time per request (latency) and the Percentage of the requests served within a certain time table (percentiles).

A tiny stopwatch in Python:

import time

def timed(fn, *a):
    t = time.perf_counter()
    result = fn(*a)
    return result, round((time.perf_counter() - t) * 1000, 1)   # milliseconds

Idea 2: Finding the bottleneck

Do not guess. Measure the layers in order: network (dev tools), server (profiling, logs), database (query count and time), front end (scripts).

N+1 queries

You load 3 authors, then for each author you touch author.books. That is 1 query for the list plus 3 more: N+1. With 1000 authors it is 1001 queries.

from sqlalchemy import select
from sqlalchemy.orm import selectinload

# slow: lazy loading, one query per author
for a in db.session.scalars(select(Author)):
    print(a.name, len(a.books))

# fast: load all books for all authors in one extra query
stmt = select(Author).options(selectinload(Author.books))
for a in db.session.scalars(stmt):
    print(a.name, len(a.books))

I counted the SQL statements with SQLAlchemy's before_cursor_execute event on 3 authors with 2 books each: the first loop ran 4 queries, the second only 2. A related fix is a SQL JOIN.

Missing indexes

Without an index the database reads every row (a "full table scan") to find WHERE author_id = 7. An index on that column lets it jump straight to the rows.

class Book(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    author_id = db.Column(db.Integer, db.ForeignKey("author.id"), index=True)
    title = db.Column(db.String(100))

In SQLite you can ask for the plan: EXPLAIN QUERY PLAN SELECT * FROM book WHERE author_id = 7; shows SEARCH ... USING INDEX when an index is used and SCAN when it is not. Indexes speed up reads but slow down writes a little and take space, so index the columns you filter or join on, not everything.

Other usual suspects

• Returning too much data (a full table instead of one page): use pagination.
• Heavy work in the request: sending mail, generating a PDF, a CSV of 100000 rows. Move it to a Celery task (week 10).
• Big images and scripts on the front end.
• Repeating the same expensive answer for every user: cache it.

Idea 3: Cache layers

A request may be answered at several layers. The earlier, the cheaper.

A CDN is a group of servers that delivers web content fast, usually from a server near the client (usually, not always). A shared cache may be used for many users, so it must not hold personalised responses. A private cache is tied to one client. A reverse proxy (such as Nginx) can cache responses for similar later requests. A forward proxy protects clients, not servers, so it is a different thing.

The main benefits of caching: lower latency, lower load on the origin server, less bandwidth, better handling of traffic spikes (scalability). It does not give encryption, DDoS protection or remove the need for the database altogether.

Browser caching headers

The server tells the browser how to cache with response headers.

Revalidation: the browser sends If-None-Match: <etag> (or If-Modified-Since). If nothing changed, the server replies 304 Not Modified with an empty body, and the browser reuses its copy. Cheap, but still one round trip.

from flask import Flask, make_response, request, jsonify
app = Flask(__name__)

@app.get("/data")
def data():
    r = make_response(jsonify(x=1))
    r.headers["Cache-Control"] = "public, max-age=60"
    r.add_etag()
    return r.make_conditional(request)

I tested this with the Flask test client: the first request returned 200 with an ETag; repeating it with If-None-Match: <that etag> returned 304 and an empty body.

Related facts that appear in exams: localStorage has more space than cookies and keeps data after the browser closes; sessionStorage is cleared when the tab or browser closes. A hard refresh bypasses the cache for the page's files but does not delete the whole browser cache of the site. Browsers can cache JavaScript files, and normally only GET requests (without a body) are cached.

Idea 4: Application caching with Flask-Caching and Redis

pip install flask-caching redis

import time
from flask import Flask
from flask_caching import Cache

app = Flask(__name__)
app.config.update(
    CACHE_TYPE="RedisCache",                 # or "SimpleCache" (in-process memory) for tests
    CACHE_REDIS_URL="redis://localhost:6379/1",
    CACHE_DEFAULT_TIMEOUT=300,               # seconds, applies when a decorator gives no timeout
)
cache = Cache(app)

SimpleCache keeps entries in the Python process (lost on restart, not shared between processes), good for learning. RedisCache stores them in Redis: shared between workers, survives an app restart.

@cache.cached caches the whole view

@app.get("/time")
@cache.cached(timeout=5)
def get_time():
    return str(time.time())

The first request runs the function and stores the response. Requests within 5 seconds get the stored copy: the function is skipped. After 5 seconds the entry has expired and the function runs again.

The key matters: cached vs memoize

calls = []

@cache.memoize(timeout=60)
def square(n):
    calls.append(n)
    return n * n

square(3); square(3); square(4)
print(calls)                       # [3, 4]  (the second square(3) was a hit)
cache.delete_memoized(square, 3)   # invalidate only square(3)
square(3)
print(calls)                       # [3, 4, 3]

I ran exactly this and got [3, 4] and then [3, 4, 3].

@app.route('/compute/<int:x>/<int:y>')
@cache.cached(timeout=60, key_prefix='compute')
def compute(x, y):
    return jsonify({'result': x * y})

@cache.memoize(timeout=300)
def compute_data(user_id):
    time.sleep(5)
    return f"Data for user {user_id}"

A more careful look at how long entries live

• The timeout in the decorator overrides CACHE_DEFAULT_TIMEOUT.
• A memoize timeout of 180 s with visits at 0, 150 s and 210 s: the first is a miss, the second a hit, the third is only 60 s after the second but 210 s after the store, so it is a miss. (TTL counts from when the entry was stored, not from the last visit.)
• Expiry is a time to live and does not refresh on a hit.

Idea 5: Cache keys, expiry and invalidation

Key design

A good key is unique for every different answer and identical for every repeat of the same answer.

Bad keys: too coarse (all users share one entry), or including something that changes every time (a timestamp), which gives a hit ratio of zero.

Expiry (TTL)

Pick it by how stale the data may be. A dashboard that can be 5 minutes old: TTL 300. A social feed or stock price: short TTL. Caching a feed for a long time is a bad idea (the question that calls it "recommended" is false). Rarely changing data (list of countries): long TTL.

Invalidation strategies

@app.post("/books")
def add_book():
    # ... save to the database ...
    cache.delete("view//books")       # or cache.delete_memoized(list_books)
    return {"ok": True}, 201

To find the key of a cached view: with the default settings it is view/ plus the path, for example view//books (the double slash is correct, as the path starts with /). Easier and safer: use cache.delete_memoized(function) for memoised functions, or give views an explicit key_prefix.

Hit ratio and stampede (concept)

Hit ratio = hits / (hits + misses). 90 hits and 10 misses is 0.9. A low ratio means bad keys or a TTL that is too short.

Cache stampede: a popular entry expires and 500 requests arrive at the same moment. All see a miss and all hit the database together. Remedies at concept level: one request recomputes while the others wait (a lock), refresh a little before expiry, add a small random amount to each TTL so entries do not all expire together, or warm the cache from a background job.

@cache.cached(timeout=100)
def home():
    time.sleep(30)
    return "hello"

Idea 6: Front-end performance and pagination

Most of the delay a user sees is in the browser: many requests, big files, and heavy scripts.

Lazy-loaded route in Vue 2 + Vue Router 3 (this needs a build setup such as Vue CLI; it will not work from a CDN script tag):

const routes = [
  { path: "/", component: Home },
  { path: "/reports", component: () => import("./views/Reports.vue") }   // downloaded on first visit
];

Debounce, so typing "abc" in a search box makes one API call, not three. I ran this in Node:

function debounce(fn, ms) {
  let t;
  return (...args) => {
    clearTimeout(t);
    t = setTimeout(() => fn(...args), ms);
  };
}
const search = debounce((q) => console.log("search", q), 100);
search("a"); search("ab"); search("abc");   // quick typing
setTimeout(() => search("abcd"), 300);     // later

It printed search abc and then search abcd. The first three calls collapsed into the last one.

Pagination

Never send 50000 rows. Send a page at a time.

@app.get("/api/books")
def books():
    page = request.args.get("page", 1, type=int)
    p = db.paginate(select(Book).order_by(Book.id), page=page, per_page=20, error_out=False)
    return jsonify(items=[b.title for b in p.items], page=p.page, pages=p.pages, has_next=p.has_next)

With 6 books and per_page=4, page 2 holds 2 items, pages is 2, has_next is False (I ran it). Always add an order_by so pages are stable.

Async jobs for heavy work

Anything slower than a second or two (PDF, CSV export, sending mail to everyone) goes to a Celery task: the API replies at once with a task id, and the page polls the status or is told when it finishes (see week 10).

Exam radar

Quick quiz

Viva: say it out loud

Recap

Practise with real exam questions

These come straight from past papers (with the real figures and the official answer key). Open 60 real questions on this week   All End Term papers