Embeddings and Rerankers

Embeddings turn text into vectors for semantic search and RAG. Rerankers re-score retrieval candidates for better RAG quality. The local FOSS options are now competitive with closed APIs (OpenAI text-embedding-3-large, Voyage, Cohere) for most use cases. For RAG workflows that use these see ai-selfhost-rag-local; for general models see models overview.

Embedding models

★ ★ BGE family (BAAI / Beijing Academy)

• ★ BGE-large-en-v1.5 (1024-dim) — long-running default for English.
• ★ BGE-M3 — multilingual, multi-vector (dense + sparse + ColBERT-style late interaction); strong all-rounder.
• License: MIT.
• Available in Ollama (bge-m3), HF, sentence-transformers.

★ ★ E5 (Microsoft / Liang Wang)

• ★ intfloat/multilingual-e5-large — 100+ languages.
• ★ intfloat/e5-mistral-7b-instruct — 7B Mistral-based instruction-tuned; SOTA-class on English when used carefully.
• License: MIT.

★ ★ Nomic Embed

• ★ nomic-embed-text-v1.5 (768-dim, Matryoshka — can be truncated to 256 / 384 etc.).
• License: Apache 2.0; training data is fully open.
• Available in Ollama (nomic-embed-text).

★ Jina Embeddings v3 / v4

• Multilingual; "task-specific" prompting.
• Apache 2.0 weights for v2; v3+ has commercial restrictions on some sizes.

★ GTE (Alibaba)

• gte-large, gte-Qwen2-7B-instruct.
• Apache 2.0.

★ Mixedbread (mxbai-embed-large)

• 1024-dim; competitive with closed APIs.
• Apache 2.0.

Stella

• Top of MTEB English leaderboard for parts of 2024–25.

Rerankers

Rerankers re-score top-K retrieval results before sending to the LLM. Massive RAG-quality lift; small compute cost.

• ★ ★ BGE-reranker-v2-m3 — multilingual, MIT.
• ★ bge-reranker-large — English; older but solid.
• ★ Jina-reranker-v2-multilingual — strong multilingual.
• ★ mxbai-rerank-large-v1 — Apache 2.0.

Choosing dimensions

• 384–768 dims — fast, lightweight, fine for most RAG.
• 1024+ dims — slightly better quality, bigger storage.
• Matryoshka models (Nomic, mxbai) — train one, truncate at query time. Best of both.

Multilingual: when to care

• English only: bge-large-en-v1.5 or nomic-embed-text are fine.
• Multilingual content: bge-m3, multilingual-e5-large, Jina-v3.

Long context

Most embeddings cap at 512 tokens; some go to 8K+:

• bge-m3 — 8K.
• Jina v2/v3 — 8K.
• Nomic v1.5 — 8K.

Inference

• Ollama runs many embedders (bge-m3, nomic-embed-text, mxbai-embed-large).
• sentence-transformers Python library — the canonical SDK.
• Infinity (github.com/michaelfeil/infinity) — high-throughput dedicated embedding server (FastAPI + ONNX / TensorRT).
• vLLM also serves embeddings on supported models.
• fastembed (Qdrant) — ONNX-runtime-based; very low latency.

Vector DB pairing

For where to store the vectors see databases-orms and vector-databases. Quick picks:

• pgvector — Postgres extension; the simplest path.
• Qdrant — Rust-based; great filtering; strong default.
• Chroma — Python-friendly; good for prototyping.
• Milvus — high-scale.
• LanceDB — file-based; good for local apps.

Pick this if…

• One default embeddings pick: ★ ★ bge-m3 (multilingual, long context) or nomic-embed-text-v1.5 (Matryoshka).
• Best English-only: bge-large-en-v1.5 or Stella.
• One default reranker pick: ★ ★ bge-reranker-v2-m3.
• Production embeddings server: Infinity or vLLM.
• Local Apache-2.0 only: Nomic Embed + bge-reranker.
• Truly multilingual app (50+ languages): bge-m3 + Jina-reranker-v2.