WEbinar

Why the Best Reranker Can't Fix Bad Retrieval

Most teams solve slow, imprecise AI search the same way: retrieve a large candidate set and send it through a bigger model to rerank it into shape. It works, until the dataset, the traffic, and the model bill all grow at once, and the fix that used to work becomes the most expensive line in the stack. The deeper problem sits earlier in the pipeline. No reranker, however capable, can recover a relevant result that retrieval never surfaced in the first place. If the right passage isn’t in the candidate set, ranking it well was never on the table, whether the thing reading the results is a person or an agent deciding what to do next.

In this session, Bonnie Chase and Kristian Aune from Vespa.ai walked through how to design retrieval as a funnel instead of a single expensive pass: efficient candidate generation first, progressive narrowing through lexical, vector, and hybrid retrieval, and the most expensive ranking models reserved for the candidates where they can actually change the outcome. You will find out where teams get this boundary wrong in production and how the tradeoffs between latency, compute, and relevance actually get decided at each stage.