Embeddings: Understanding and Choosing Your Model
Every RAG system in this series relies on embeddings without explaining them. Here is what an embedding actually is, how to pick a model, and when fine-tuning is worth it.
Every RAG article on this blog assumes you know what an embedding is and moves straight to chunking and retrieval. This one fills that gap,because the embedding model you choose quietly determines the ceiling on your entire RAG system's quality.
What an embedding actually is
An embedding is a vector,a list of numbers,that represents the meaning of a piece of text. Texts with similar meaning end up as vectors that are close together in that vector space, which is exactly what makes similarity search possible: find the nearest vectors to a query vector, and you find the most semantically relevant text.
Choosing a model: the real trade-offs
- ✓OpenAI text-embedding-3: strong general-purpose quality, pay-per-token, zero infrastructure to manage.
- ✓Cohere embed: strong multilingual support,useful when your content spans several languages.
- ✓Open-source (sentence-transformers, BGE, E5): self-hosted, zero per-call cost, but you manage the serving infrastructure and quality varies by model.
Dimensionality: bigger is not always better
Higher-dimensional embeddings (3072 vs 1536, for example) capture more nuance but cost more to store and search. Many modern models support Matryoshka-style truncation, letting you shrink the vector to a smaller dimension with limited quality loss when storage or search speed matters more than maximum precision.
from openai import OpenAI
client = OpenAI()
response = client.embeddings.create(
model="text-embedding-3-small",
input="Migrating a Laravel monolith to microservices",
)
vector = response.data[0].embedding
print(len(vector)) # 1536 dimensionsWhen to fine-tune embeddings
Fine-tuning an embedding model only pays off with a sizeable labeled dataset of your domain's query-document pairs. Most teams get more value improving chunking strategy and retrieval logic first,fine-tuning is a late-stage optimization, not a starting point.
Do not switch embedding models casually once in production. Changing models means re-embedding your entire corpus, since vectors from different models are not comparable,plan it as a migration, not a config change.
Embeddings are the foundation every retrieval decision downstream depends on. Getting the model choice right early saves a costly re-embedding migration later.
Need help with this topic? AI & RAG Integration
Discover this service →