~/learn/Machine Learning · Production ML & modern concepts
Embeddings & vector search
Dense representations, cosine vs Euclidean similarity, and approximate nearest neighbours.
why this matters
Search, recommendations and RAG systems all run on embeddings — a hot topic in ML and AI engineering interviews.
An embedding maps an item — word, product, user, document — to a dense vector so that similar items are close together. They come from neural networks (word2vec, transformers, two-tower recommenders) trained so that related pairs end up near each other.
cosine(a, b) = (a · b) / (‖a‖ · ‖b‖) angle only, ignores length euclidean = ‖a − b‖ angle and length
For normalized vectors, cosine and Euclidean give the same ranking. Searching millions of vectors exactly is slow, so production systems use approximate nearest-neighbour indexes (HNSW, IVF) that trade a little recall for big speedups. This powers semantic search, recommendations and retrieval-augmented generation (RAG).
Interview trap
Embeddings from different models (or versions) live in different spaces — you can't compare them. Re-embed the corpus when you upgrade the model.
key takeaways
- Embeddings place similar items close together.
- Cosine compares direction; normalized vectors make it match Euclidean.
- ANN indexes trade a little recall for speed.
check your understanding
You upgraded your embedding model. What must you do with stored document vectors?