Retrieve Better Context for AI

Why AI Retrieval Matters

Every AI application begins with retrieval.

Before an application can rank results, generate an answer, recommend products, or complete an agent task, it first has to find the right information. That initial retrieval step determines everything that follows. If the relevant data isn't retrieved, no ranking model or large language model can recover it later.

The goal is to retrieve the best possible candidate set while keeping latency, infrastructure costs, and downstream inference under control. In practice, retrieval must balance two competing priorities:

• High recall: Retrieve enough relevant information to maximize answer quality.
• High efficiency: Limit the candidate set so ranking and AI inference remain fast and cost-effective.

Modern AI applications place far heavier demands on retrieval than traditional search. A single request may need to combine keyword matching, semantic similarity, structured filters, permissions, freshness, personalization, and business rules before returning candidates.

A single request may need to combine keyword matching, semantic similarity, sparse retrieval, structured filters, permissions, freshness, personalization, and business rules before returning candidates.

Different AI workloads place different demands on retrieval:

• Search: Lexical search, structured filtering, facets
• RAG: Semantic retrieval, metadata filtering, permissions
• Recommendations: Similarity search, user preferences, behavioral signals
• Agentic AI: Semantic retrieval, structured constraints, tool selection, contextual memory

Supporting each method on its own is not the hard part. The hard part is combining them into one retrieval pipeline that returns the highest-quality candidates without adding latency, cost, or operational complexity.

The Cost of Stitching Systems Together

Most teams solve this by stitching together multiple systems. Traditional search engines excel at lexical retrieval and filtering but often treat vector search as an extension. Vector databases provide fast semantic similarity search but typically depend on another system for keyword search, filtering, ranking, and serving. As the application scales, that split turns into a standing operational tax:

• Data synchronization: The same documents live in two stores, so every write has to reach both. Any lag opens a consistency window where the two systems disagree about what exists.
• Duplicated infrastructure: Two indexes, two capacity plans, two scaling stories, two failure modes, for one corpus.
• Reconciliation logic in the app: Separate systems return separate result lists, so the application has to merge and re-rank them. Filters have to be applied to every sub-search by hand. Miss one, and that half of the query quietly ignores the filter and returns out-of-scope results.

Vespa takes a different approach. Lexical search, vector search, sparse retrieval, structured filtering, and custom retrieval logic run inside one engine, over one index, as one query. Instead of orchestrating multiple systems, you retrieve one candidate set and use it as the foundation for search, recommendations, RAG, and agentic AI.

How the Approaches Compare

The difference is less about which methods each system supports and more about how they combine.

How AI Retrieval Works in Vespa

Every query passes through the same retrieval pipeline. Vespa organizes retrieval into four stages: query processing, retrieval execution, structured filtering, and candidate generation.  Each stage has a distinct job, and each can be tuned independently for efficiency, scalability, and retrieval quality.

1. Query Processing: One Language for Every Method

Every retrieval request begins with a query written in Vespa Query Language (YQL).

Rather than treating keyword search, vector search, and filtering as separate operations, YQL provides a single language for defining how candidates should be retrieved.

A query can combine retrieval operators, boolean logic, filters, and query parameters into one execution plan. You describe retrieval declaratively, and Vespa executes the pipeline. That keeps complex strategies easier to express, maintain, and evolve than coordinating several APIs by hand.

2. Retrieval execution: Operators over the Right Index

Retrieval operators determine how candidate documents are selected. 

Each operator is optimized for a specific retrieval strategy and can be combined with others within the same query. This allows applications to combine exact matching, semantic retrieval, sparse retrieval, and business constraints within one retrieval pipeline.

Common operators include:

• nearestNeighbor for dense vector (semantic) retrieval.
• weakAnd for efficient top-k lexical retrieval (Vespa's WAND-family operator for OR-style keyword queries).
• wand for weighted-set retrieval, used for learned-sparse representations and weighted-term recommendation.
• contains for keyword and field matching.
• Boolean operators (and, or, not) for precise query logic.

Each operator uses the index structure best suited to its method, and all index types participate in the same distributed engine:

• Inverted indexes for lexical search.
• HNSW indexes for approximate nearest neighbor search. This is the same real-time, mutable HNSW index Vespa updates in place, so vector retrieval reflects fresh data without a rebuild.
• Attributes for fast lookups, filtering, sorting, and grouping.
• Structured document fields for metadata and field-based retrieval.

The engine selects the right index per operator and executes the whole query through one distributed runtime, so different retrieval strategies work together without extra infrastructure.

3. Structured Filtering: Constraints that Protect Recall

Structured filters participate directly in retrieval, instead of being applied after candidate selection. Because filtering is part of the retrieval model, you can combine structured constraints with lexical, semantic, and sparse retrieval in a single query.

Common examples include:

• Show only products that are in stock.
• Retrieve only documents a user is authorized to access.
• Limit results to a specific language or region.
• Search only documents published within the last year.

Filtering alongside vector search is where naive approaches break. Pre-filtering (filter first, then search) guarantees the filter is honored, but running an unaided scan over the filtered subset can be slow. Post-filtering (search first, then drop non-matches) is fast, but on a selective filter it collapses recall: you ask for 100 results and get a handful, because most of what the vector search found gets discarded.

Vespa avoids forcing that choice. When you combine nearestNeighbor with a filter, the query planner estimates how many documents the filter matches and selects a strategy automatically. For most filters it traverses the HNSW graph and skips non-matching neighbors as it goes, so the search keeps moving until it has enough real matches. For highly selective filters it switches to an exact search over just the filtered set. Either way, the goal is to expose the target number of genuine candidates to ranking, so applying constraints during retrieval improves efficiency and relevance instead of trading one for the other.

4. Candidate Generation

Candidate generation defines which documents continue into the ranking pipeline, and how many. You set the target size of the retrieved set (for example with targetHits) independently from ranking, so retrieval can be tuned for recall, latency, and cost on its own.

From there, ranking is multi-phase, and the distinction matters. Vespa's first phase runs a cheap scoring expression across every matched document, not a pre-truncated top-k. It then re-ranks a smaller set per content node with a more expensive second-phase model, and can apply a final, most expensive model to the merged top set in a global phase. Retrieval bounds the match set; multi-phase ranking then spends compute where it pays off. 

As requirements change, you refine candidate generation without redesigning the downstream ranking models.

Configuring Retrieval in Vespa

Every application has different retrieval requirements. An ecommerce search experience may prioritize lexical retrieval with inventory filters, while a RAG application may rely on semantic retrieval combined with document metadata and access controls. 

Vespa exposes configurable controls throughout the retrieval pipeline so you can tune retrieval behavior without changing the underlying architecture.

Configuration options include:

• Retrieval strategy: Choose which retrieval operators that participate in a query.
• Query composition: Define retrieval logic using YQL, including operators, filters, and query parameters.
• Candidate generation: Tune candidate set sizes to balance recall, latency, and computational cost.
• Filtering and constraints: Apply structured filters during retrieval using indexed fields such as category, language, geography, permissions, inventory, timestamps, tenant, and document type to improve retrieval efficiency and relevance.
• Index configuration: Configure inverted indexes, HNSW indexes, and attributes for different retrieval workloads.
• Query-time parameters: Adjust query terms, embeddings, nearest-neighbor parameters, filters, and execution settings for individual requests.

Because retrieval is broken up into configurable components,you can refine candidate generation over time without changing downstream ranking models or redesigning the search pipeline.