~/learn/ML System Design · Retrieval & ranking systems
Embeddings, two-tower models & vector search
Represent users and items as vectors, then find neighbours fast.
why this matters
Two-tower retrieval with vector search is the default answer for large-scale retrieval.
An embedding is a list of numbers (e.g. 128 of them) that represents a user or item, learned so that similar things end up close together. A two-tower model has one network that turns a user into a vector and another that turns an item into a vector; relevance is their dot product.
The key trick: item vectors don't depend on the user, so you compute them all offline and index them. At request time you compute only the user vector, then search for the nearest item vectors.
Two-tower
user features ─► [user tower] ─► u ┐
├─ score = u · v
item features ─► [item tower] ─► v ┘
item vectors precomputed → vector index → top-K nearest to uExact nearest-neighbour search over 100M vectors is too slow, so we use approximate nearest neighbour (ANN) indexes: HNSW (graph-based, very accurate), IVF (cluster then search a few clusters), product quantization (compress vectors to save memory). Libraries: FAISS, ScaNN; vector databases: Pinecone, Milvus, pgvector.
Interview trap
ANN is approximate: you trade a small loss in recall (say 95% of the true neighbours) for 100× speed. Say this tradeoff explicitly.
Pro tip
Two-tower models can't capture fine user-item interactions (they only meet at the dot product). That's exactly why a heavier ranking stage follows.
key takeaways
- Item vectors are precomputed and indexed.
- Only the user vector is computed at request time.
- ANN trades a little recall for big speed.
check your understanding
Why do two-tower models scale to huge catalogues?