Curiosity · Concept
Embeddings
An embedding is a learned dense vector — typically 384 to 4096 floating-point numbers — that represents a piece of content. Close vectors mean similar meaning. Embeddings are the bridge between unstructured data and anything that needs similarity search, classification, clustering, or retrieval.
Quick reference
- Proficiency
- Beginner
- Also known as
- vector embeddings, dense representations, text vectors, semantic embeddings
- Prerequisites
- Basic linear algebra (vectors, dot product)
Frequently asked questions
What are embeddings?
Embeddings are dense numerical vectors that represent meaning. Two texts with similar meaning produce vectors that are close in high-dimensional space, which lets you search, cluster, or classify by semantic similarity instead of keyword match.
How are embeddings different from LLM outputs?
An embedding model produces a single fixed-size vector summarizing an input. An LLM produces text tokens. Embeddings are used for search and similarity; LLMs are used for generation. Many RAG systems use both — embeddings to find relevant chunks, an LLM to answer from them.
What similarity metric should I use?
Cosine similarity is the most common and ignores vector magnitude. Dot product is faster when vectors are normalized. Euclidean distance is rarely used for text embeddings. The model's documentation specifies which metric it was trained with.
What are Matryoshka embeddings?
Matryoshka Representation Learning (MRL) trains a single embedding model so you can truncate the vector (e.g. use only the first 256 of 1536 dimensions) and still get reasonable quality, letting you trade storage and speed for accuracy at query time.
Sources
- Mikolov et al. — word2vec — accessed 2026-04-20
- OpenAI — New embedding models — accessed 2026-04-20
- Hugging Face MTEB leaderboard — accessed 2026-04-20