query.lab
all tracks

~/learn/ab

A/B Testing curriculum

Experiment design, statistics and the classic pitfalls of product experimentation.

hypothesise → size → randomise → check → analyse → decide

0 / 15 lessons completed

Start with The experimentation lifecycle

Milestone 1 · Foundations

Beginner

Start here — the syntax and mental models everything else builds on.

  1. 1

    Foundations & metric selection

    ~13 min · 3 lessons

    Design a clean experiment with the right metrics and unit.

    hypothesisguardrailsrandomization
  2. 2

    Sample size, power & duration

    ~14 min · 3 lessons

    Work out how many users you need and how long to run.

    α / βpowerMDEduration

Milestone 2 · Real-world data

Intermediate

Combine data from several places and reshape it into answers.

  1. 1

    Hypothesis testing & inference

    ~15 min · 3 lessons

    Pick and read the right statistical test for any metric.

    z-testWelch's tbootstrapCI

Milestone 3 · Interview-ready

Advanced

The techniques that show up in real interviews and real pipelines.

  1. 1

    Diagnostics & classic pitfalls

    ~15 min · 3 lessons

    Spot broken experiments before they mislead you.

    SRMpeekingspilloverswitchback
  2. 2

    Decisions & modern techniques

    ~15 min · 3 lessons

    Make and defend a ship decision with less noise.

    noveltySimpson'smultiple testingCUPED