What Breaks in Your Retrieval Layer

A retrieval layer serving human queries is one thing. A retrieval layer serving hundreds of concurrent agents, each potentially retrieving, reasoning, reformulating, and retrieving again, is a very different system. This isn’t a conventional concurrency problem; agent workloads multiply retrieval demand while raising the bar for freshness, relevance, and latency all at once. Most retrieval architectures weren’t built for this operating model. This session explores what happens when they encounter it, and why “add more caching” or “throw a bigger vector database at it” doesn’t solve the underlying problem.

Whit Walters, author of GigaOm’s Defeating the Integration Tax, and Bonnie Chase from Vespa.ai walked through the specific failure modes that show up when retrieval has to serve agent workloads instead of human ones: latency stacking, stale context, relevance drift under concurrent load, and the operational overhead of a fragmented stack trying to keep up. You’ll find out what a retrieval architecture actually looks like when it’s under that kind of pressure and what changes when it’s built as a unified layer instead of a bolted-together pipeline.

If you’re building agent systems and haven’t hit this wall yet, you will. We’ll show you where it is before you find it in production.