Efficient Ranking at Any Scale

The Right Amount of Compute, at the Right Stage

Search, recommendations, and RAG systems are under pressure to evaluate more signals than ever: lexical relevance, vector similarity, freshness, popularity, personalization, business rules, model scores, even LLM-based relevance judgments. Running all of that against every matched document doesn't scale: it's too slow, and too expensive.
Vespa's multi-phase ranking solves this by ranking in stages instead of all at once. Fast ranking first, richer ranking later, and expensive models only on the best candidates. The practical effect: every dollar of compute goes toward results that actually have a shot at the top of the page, not toward scoring documents the user will never see.
That staging is what makes it possible to:

• Cut serving cost without cutting relevance. Expensive models only ever process a few hundred candidates rather than millions, so you get cross-encoder-level quality without cross-encoder-level infrastructure spend.
• Hit sub-second latency at scale. Cheap signals filter the field early; the system never pays the cost of heavy inference on documents that were never going to rank well anyway.
• Improve relevance without a re-architecture. Add a new model or signal at the phase where it belongs without redesigning the retrieval layer underneath it.
• Tune the cost/quality tradeoff yourself. Controls like rerank-count let you decide exactly how many candidates get the expensive treatment, per application, per query, even per experiment.

How Multi-phase Ranking Works

A fast first-phase ranking narrows millions of candidates down to a manageable set. The second phase reranks the strongest candidates on each content node. An optional global phase reranks the final top results after they've been merged across the cluster. The result: you can use cross-encoders, ONNX models, and other expensive ranking logic exactly where they add the most value, without paying for them everywhere else.

Without phase ranking, teams are stuck choosing between two bad options: run a cheap model everywhere and leave relevance on the table, or run an expensive model everywhere and pay for it in latency and infrastructure costs. Multi-phase ranking removes that tradeoff. You get the accuracy of the expensive model and the cost profile of the cheap one because each one runs only where it's earned its keep.

Configuring Multi-Phase Ranking

Multi-phase ranking is defined through rank profiles in the Vespa application schema. A single application can define multiple rank profiles for different use cases, markets, or experiments, and select between them at query time.
This isn’t a fixed pipeline. It's a set of controls your team owns:

• Define ranking expressions: Define custom ranking logic directly in rank profiles, so relevance logic lives in your schema, not buried in application code.
• Combine hybrid signals: Combine text, vector, tensor, metadata, and business signals into a single pipeline, rather than stitching results from separate systems.
• Deploy machine learning models: Drop ONNX and tree-based models into second- or global-phase, where the candidate set is already small enough that inference is fast and affordable to run.
• Control ranking at query-time controls: Select a rank profile and pass query-time inputs, so ranking behavior changes by user, market, or experiment without a redeploy or downtime.
• Profile ranking: See exactly where ranking costs are going by phase, so tuning decisions are based on data rather than guesswork.