Skip to main content

System Design

Can two fast-looking dashboards hide a slow checkout?

Rebuild a checkout latency report from small request samples, then measure how much traffic meets a time budget.

Try the 5-minute exercise

Two checkout servers report their 95th-percentile latency. One shows 100 milliseconds, the other 900. Someone averages the tiles and reports 500 ms for the shop. Could 500 ms describe a request no customer actually experienced? Use a small batch where you can inspect every observation.

Reconstruct the customer population

Server A handled 98 requests in 100 ms each. Server B handled two in 900 ms each. For this exercise, define p95 by nearest rank: sort the observations and take the one at rank ceil(0.95 × count), counting from one. Real dashboards can use other percentile conventions, so name the one you intend to test.

from math import ceil

def p95(samples):
    if not samples:
        return None
    ordered = sorted(samples)
    return ordered[ceil(0.95 * len(ordered)) - 1]

a = [100] * 98
b = [900] * 2
assert (p95(a), p95(b)) == (100, 900)
assert p95(a + b) == 100
assert p95([]) is None

Position 95 in the combined list still belongs to server A. Now move eight requests from A to B, keeping each server’s individual durations unchanged. There are 90 fast requests and 10 slow ones. Both server tiles still show exactly the same values, but the combined p95 jumps to 900 ms. Even knowing both percentile tiles is insufficient to reconstruct the fleet percentile.

Ask a question with a countable answer

If the target is “finish successfully within 200 ms”, count successes within the limit and divide by all attempted checkouts in the same window. In the original batch, assuming all completed successfully, 98 of 100 qualify. If three more attempts time out, the fraction becomes 98 of 103. Dropping the timeouts would make a worse customer experience invisible.

For a large service, compatible histogram buckets can be combined across instances. If the latency budget is exactly a bucket boundary, its cumulative count tells you how many observations met the time limit. If p95 falls somewhere inside a wide bucket, the bucket counts alone only give you a range. Keep the bucket boundaries and time window consistent when comparing releases. Prometheus histogram guidance

Put it into practice · 5 minutes

Find the request at the 90th percentile

Try the exercise and get feedback on your answer.

Try the exercise