What Are Vector Embeddings? | Understanding Things

Vector embeddings are the piece of AI engineering infrastructure most likely to be used without being understood. Teams wire up an embeddings API, drop the vectors into an index, and get retrieval that mostly works — until it doesn’t, and there’s no mental model to debug from. This article builds that model.

The pipeline

Text
  ↓
Embedding model
  ↓
Vector representation
  ↓
Vector space
  ↓
Similarity
  ↓
Retrieval

Each stage is doing something specific:

Text → embedding model. A neural network — usually a variant of the same transformer architecture behind LLMs — reads a string and produces a fixed-length list of numbers, typically a few hundred to a few thousand floats. Unlike a language model’s output, this isn’t a distribution over tokens; it’s a single dense vector meant to summarize the input.

Vector → vector space. That vector isn’t meaningful in isolation. It’s meaningful as a point, positioned relative to every other vector produced by the same model. The model is trained so that inputs it considers semantically related end up as points that are close together, and unrelated inputs end up far apart.

Space → similarity. “Close together” is a literal geometric claim, usually measured with cosine similarity (the angle between two vectors) or Euclidean distance. This is why the field talks about “semantic search”: you’re not matching keywords, you’re measuring geometric proximity between points that a model has arranged according to meaning as it learned to represent it.

Similarity → retrieval. Given a query, embed it with the same model, then find the stored vectors nearest to it. Those nearest neighbors are your candidate results. This is the mechanism underneath everything people call “semantic search,” and it’s the retrieval half of retrieval-augmented generation.

What “semantically similar” actually means

It’s tempting to read “semantic similarity” as “the model understands meaning.” What’s actually true is narrower and more useful: the model was trained on a specific objective — typically to pull together text pairs a training process labeled as related (matching questions to answers, matching paraphrases, matching a document to its summary) and push apart pairs labeled as unrelated. The resulting space encodes whatever notion of relatedness that training objective rewarded, not some universal, objective concept of meaning.

This has concrete consequences:

An embedding model trained mostly on question-answer pairs will place a question and its answer close together, even though a human wouldn’t call a question and its answer “similar” in the everyday sense. That’s a feature, not a bug, for a retrieval system built to answer questions — but it will surprise you if you assumed the space reflects surface-level semantic overlap.
Two sentences that share almost no words can be extremely close in vector space, and two sentences sharing most of their words can be far apart, if the training signal cared about the distinction that separates them (sentiment, negation, entity identity).
Embedding quality is domain-dependent. A general-purpose embedding model may cluster legal or medical text less usefully than one trained or fine-tuned on that domain, because “related” in that domain doesn’t match the general-purpose training signal.

What embeddings are not

Embeddings are not a database, not an index, and not a search algorithm by themselves — they are the numeric representation those systems operate on. Storing, indexing, and efficiently querying large collections of embeddings is a separate concern, covered in What Is a Vector Database?

Embeddings also do not carry provenance, structure, or exact-match guarantees. A vector has no built-in notion of “this came from section 4.2 of document X, published on date Y” — that metadata has to be tracked deliberately alongside the vector by the system storing it. Treating an embedding as if it were the document, rather than a lossy geometric summary of it, is a common source of retrieval systems that quietly lose the ability to cite or verify what they return.

Why this framing matters for architecture

Once you see embeddings as points positioned by a specific, opinionated training objective, several practical decisions follow directly:

Evaluate retrieval quality against your actual task, not a generic benchmark — the space was shaped by someone else’s notion of relatedness, and it may not match yours.
Re-embedding is required whenever you change embedding models; vectors from different models are not comparable, because they were placed in unrelated spaces by unrelated training runs.
Similarity is not correctness. A retrieved document being the nearest neighbor to a query says nothing about whether that document actually answers the query — it says the model’s training process associated them. Downstream reasoning still has to check that assumption.

That last point is worth sitting with, because it’s the exact seam where a lot of RAG systems fail silently.