~/learn/ML System Design · Retrieval & ranking systems
The two-stage recommendation funnel
Why every large system narrows millions of items to a handful in stages.
why this matters
Feeds, search and recommendations questions all expect you to introduce the funnel.
You cannot run a heavy neural network on 100 million videos for every request within 200 ms. So large systems — YouTube, Instagram, TikTok, Amazon, LinkedIn — use a funnel: cheap models filter many items, expensive models score few.
The funnel
100M items │ candidate generation (embeddings, ANN, rules) ~10 ms ▼ ~1,000 candidates │ ranking (heavy model, rich features) ~50 ms ▼ ~100 ranked │ re-ranking (diversity, freshness, policy) ~5 ms ▼ 10–20 shown to the user
Candidate generation optimizes recall: don't miss good items. It usually merges several sources — embedding similarity, 'trending now', 'from accounts you follow', 'similar to what you just watched'.
Ranking optimizes precision at the top: score each candidate with a model that uses hundreds of user, item and context features plus their interactions.
Interview trap
Proposing a single model that scores every item. Always ask about catalogue size and latency, then introduce the funnel.
Pro tip
Mention that each stage is evaluated separately: recall@1000 for retrieval, NDCG@10 for ranking.
key takeaways
- Cheap models over many items, heavy models over few.
- Retrieval optimizes recall.
- Ranking optimizes top precision.
check your understanding
What does the candidate generation stage mainly optimize?