An embedding is a way of turning information—words, sentences, images, audio clips, or even user click histories—into a list of numbers called a vector. The list is usually a few hundred to a few thousand numbers long, and each number is a learned feature that captures some property of the input. The key idea is that the model is trained so that items with similar meaning end up with similar vectors, and unrelated items end up far apart.
Because every input becomes a point in the same mathematical space, computers can finally do things like add, subtract, and measure distance between meanings instead of just matching letters. That is why embeddings are the backbone of modern semantic search, retrieval-augmented generation (RAG), recommender systems, clustering, and classification.
How embeddings work
Under the hood, an embedding is produced by a neural network called an encoder. During training, the model sees huge amounts of data and adjusts its weights so that inputs appearing in similar contexts (for example, the words king and queen, or a photo of a golden retriever and the caption "yellow dog") are mapped to vectors that point in similar directions. The resulting coordinates are not hand-designed; they emerge from the model's objective of predicting neighbors, masked words, or related items.
A simple way to picture this: imagine a 3-D map of words. After training, king, queen, prince, and princess form one cluster; cat, dog, and hamster form another; and happy, joyful, and elated form a third. Real embeddings live in much higher dimensions (often 768, 1,536, or 3,072), but the principle is identical—proximity in vector space corresponds to semantic similarity, usually measured with cosine similarity or Euclidean distance. To learn more about how this is trained, see the original word2vec paper by Mikolov et al. and OpenAI's embeddings guide.
Why embeddings matter
Embeddings turn messy, unstructured data into a form that algorithms can reason over efficiently. A search engine can rank documents by meaning rather than by exact keyword overlap, so a query for "how to fix a leaky faucet" can match an article titled "repairing a dripping tap." A recommendation system can find products similar to the one a user just browsed, even when the catalog has no shared tags. And in retrieval-augmented generation (RAG), an LLM grounds its answers in private or up-to-date documents by retrieving the chunks whose embeddings are closest to the user's question.
The same trick works for images (CLIP-style models), audio, code, and structured records, which is why embeddings have become a universal interchange format between data and AI.
Key types of embeddings
- Word embeddings — fixed vectors per word, as in word2vec and GloVe.
- Sentence and document embeddings — one vector per passage, produced by models such as Sentence-BERT and OpenAI's
text-embedding-3. - Image embeddings — vectors from vision encoders like CLIP, ResNet, or DINOv2, enabling cross-modal search.
- Multimodal embeddings — shared spaces where text, images, and audio live together, so a photo can be retrieved with a caption and vice versa.
- Graph and entity embeddings — vectors for nodes in knowledge graphs, used in recommendation and fraud detection.
Once you have embeddings, you typically store them in a vector database such as Pinecone, Weaviate, Milvus, or pgvector, and query it with k-nearest neighbors (k-NN) or approximate nearest neighbors (ANN) search to find the closest matches at scale.
Embeddings are the quiet workhorse of contemporary AI: by translating meaning into geometry, they let machines compare, retrieve, and reason about the world in ways that were impractical before deep learning made vector representations both cheap and remarkably accurate.