Role in the AI Workflow

Retrieval is the first step in every AI workflow. Before an application can rank results, generate an answer, recommend content, or execute an agent task, it must first identify the most relevant information. Every search engine, RAG application, recommendation system, and AI agent depends on retrieval to build the candidate set used for downstream ranking and inference.

Why AI Retrieval Matters

Every AI application begins with retrieval.

Before an application can rank results, generate an answer, recommend products, or complete an agent task, it first has to find the right information. That initial retrieval step determines everything that follows. If the relevant data isn't retrieved, no ranking model or large language model can recover it later.

The goal is to retrieve the best possible candidate set while keeping latency, infrastructure costs, and downstream inference under control. In practice, retrieval must balance two competing priorities:

  • High recall: Retrieve enough relevant information to maximize answer quality.
  • High efficiency: Limit the candidate set so ranking and AI inference remain fast and cost-effective.

Modern AI applications place far heavier demands on retrieval than traditional search. A single request may need to combine keyword matching, semantic similarity, structured filters, permissions, freshness, personalization, and business rules before returning candidates.

A single request may need to combine keyword matching, semantic similarity, sparse retrieval, structured filters, permissions, freshness, personalization, and business rules before returning candidates.

Different AI workloads place different demands on retrieval:

  • Search: Lexical search, structured filtering, facets
  • RAG: Semantic retrieval, metadata filtering, permissions
  • Recommendations: Similarity search, user preferences, behavioral signals
  • Agentic AI: Semantic retrieval, structured constraints, tool selection, contextual memory

Supporting each method on its own is not the hard part. The hard part is combining them into one retrieval pipeline that returns the highest-quality candidates without adding latency, cost, or operational complexity.

Vespa combines lexical search, vector search, sparse retrieval, structured filtering, and custom retrieval logic run inside one engine, over one index, as one query. Instead of orchestrating multiple systems, you retrieve one candidate set and use it as the foundation for search, recommendations, RAG, and agentic AI.

How Retrieval Works in Vespa

1. Document Preparation: Chunking and Indexing

Before retrieval begins, long documents can be divided into smaller passages that are easier to embed, match, and return as context. Chunking defines the units retrieval and ranking can evaluate, but it does not require every chunk to become its own document.

Vespa supports two common models. In one, each chunk is stored as a separate searchable document, which simplifies passage retrieval but duplicates metadata and breaks some of the connection to the source document. In the other, chunks remain part of the source document, with chunk text and embeddings stored as fields within it. This preserves document-level context and metadata while still allowing Vespa to rank and return the most relevant passages.

Learn more about chunking

Configuring Retrieval in Vespa

Every application has different retrieval requirements. An ecommerce search experience may prioritize lexical retrieval with inventory filters, while a RAG application may rely on semantic retrieval combined with document metadata and access controls. 

Vespa exposes configurable controls throughout the retrieval pipeline so you can tune retrieval behavior without changing the underlying architecture.

Configuration options include:

  • Retrieval strategy: Choose which retrieval operators that participate in a query.
  • Query composition: Define retrieval logic using YQL, including operators, filters, and query parameters.
  • Candidate generation: Tune candidate set sizes to balance recall, latency, and computational cost.
  • Filtering and constraints: Apply structured filters during retrieval using indexed fields such as category, language, geography, permissions, inventory, timestamps, tenant, and document type to improve retrieval efficiency and relevance.
  • Index configuration: Configure inverted indexes, HNSW indexes, and attributes for different retrieval workloads.
  • Query-time parameters: Adjust query terms, embeddings, nearest-neighbor parameters, filters, and execution settings for individual requests.

Because retrieval is broken up into configurable components,you can refine candidate generation over time without changing downstream ranking models or redesigning the search pipeline.

Learn with Vespa

Learn how to build search, recommendation, and RAG applications with Vespa through a free, self-paced course that combines hands-on exercises with links to the documentation.

Start the free course

Frequently Asked Questions

Need more than a quick answer?

If these FAQs don't answer your question, there are several ways to continue:

Learn the fundamentals with our free online training at learn.vespa.ai.

Experience Vespa yourself with a free Vespa Cloud trial.

Watch the Getting Started with Vespa AI Search YouTube video

Contact our team to discuss your application or migration project.
What is AI retrieval?
AI retrieval is the process of selecting the most relevant candidate documents from an index before ranking or AI inference begins. It is the first stage of every AI workflow, including search, retrieval-augmented generation (RAG), recommendations, and agentic AI. Retrieval quality sets the ceiling for every stage that follows, because a document that is not retrieved cannot be ranked or used as context by an LLM.
What is the difference between retrieval and ranking?
Retrieval selects a candidate set of potentially relevant documents, while ranking scores and orders that set with more expensive relevance models. Retrieval optimizes for recall and efficiency across a large index; ranking optimizes for precision over a much smaller set. In Vespa, retrieval produces the candidate set that the multi-phase ranking pipeline then refines.
What is hybrid retrieval?
Hybrid retrieval combines keyword (lexical) search and vector (semantic) search in a single query. Keyword search matches exact terms, identifiers, and rare tokens, while vector search captures meaning and paraphrase, so using both improves quality over either alone. Vespa runs hybrid retrieval in one pass over one index, rather than running two searches and merging their result lists.
Can lexical and vector search run in the same query in Vespa?
Yes, Vespa runs lexical search, vector nearest-neighbor search, sparse retrieval, and structured filters together in a single YQL query. All methods execute inside the same distributed engine over the same index, which removes the need to query separate systems and reconcile their results.
How does filtering work with vector search in Vespa?
In Vespa, structured filters are applied during vector retrieval rather than before or after it as a separate step. Pre-filtering alone can be slow and post-filtering can collapse recall on selective filters, so Vespa's query planner estimates filter selectivity and picks the strategy that protects recall: it constrains the HNSW graph traversal for most filters and falls back to exact search over the filtered set when the filter is highly selective. The result is that enough genuine matches reach ranking even under restrictive filters.
How does AI retrieval work for RAG?
In RAG, retrieval selects the passages an LLM uses as context, usually by combining semantic retrieval with document metadata and access controls. Because the model can only reason over what retrieval returns, accurate and permission-aware retrieval directly improves answer quality and reduces hallucination. Vespa retrieves this context by combining vector, lexical, and structured filtering in one query, then passes the top candidates through ranking before generation.
What retrieval operators does Vespa support?
Vespa supports retrieval operators including nearestNeighbor for semantic vector search, weakAnd for efficient top-k lexical retrieval, wand for weighted-set (learned-sparse) retrieval, contains for keyword matching, and boolean operators (and, or, not) for query logic. Each operator is optimized for a specific strategy and can be combined with others in the same query. All operators execute within one retrieval pipeline over a single index.
Why use a unified retrieval engine instead of a separate search engine and vector database?
A unified retrieval engine runs lexical, vector, sparse, and structured retrieval in one query over one index, so there are no separate systems to keep in sync. Separate systems force you to maintain two indexes, apply filters to each sub-search by hand, and merge and reconcile result lists in the application layer, on top of duplicated infrastructure and a consistency window between the stores. Vespa builds retrieval into its core engine, which lowers operational complexity while improving retrieval quality and developer control.