Unlock the Future of eCommerce

Session 3: Vespa Technical Deep Dive

Piotr Kobziakowski, Senioar Principal Solutions Architect

Session Summary

This session explored the advanced personalization and ranking capabilities of Vespa.ai, particularly in the context of e-commerce applications. It detailed how Vespa enables low-latency, scalable operations by breaking data and system silos, supporting personalized search, product recommendations, and real-time model-driven decisions. The presentation highlighted Vespa’s architecture, including multi-phase ranking, tensor operations, vector-based search, and integration with multimodal models. A live demo illustrated Vespa’s application in personalized car search and visual matching using PDF images. The session concluded with pointers to sample applications and resources for getting started with Vespa.

Transcript

Let’s think about the topics we’ve discussed before. Some elements are already present in vintage implementations. Breaking down silos is extremely important. Vespa excels at this—not just in ranking, as you’ve seen, but in personalized search, query suggestions, and customized category pages like the homepage or any specific section.

Vespa makes these capabilities easy to implement and offers excellent performance in terms of low latency. Beyond this, it handles personalized data sharing efficiently. For example, if you have models with partial updates for user behavioral representation, you can share that outside your core systems—with advertising platforms, marketing platforms, or even inventory tracking.

This works well in Vespa due to its extremely fast updates. You can track changes easily. And when you colocate components, you gain further improvements in latency and speed across personalization engines, clickstreams, and model training pipelines. These can all run together—including things like investment card management and cart data—which are also well-supported by Vespa.

The benefits of this approach are clear: consistent customer profiles and touchpoints, a single source of behavioral data, and improved accessibility across systems. If your data and applications run within Vespa, there are no network delays. Operations run in memory, not over the network—reducing bandwidth costs and improving operational efficiency.

Many teams report heavy gains in operational efficiency when they have one central location for application configurations, data management, and issue investigation. These tasks become significantly easier than in distributed systems. By destroying data and system silos, you’re not only reducing latency in search performance, but also improving application access times and overall responsiveness.

E-commerce Use Cases and Personalization

Let’s now look at what Vespa addresses in e-commerce platforms. I’ll also take this opportunity to announce that we’ll be running another webinar focused on building specific e-commerce components using Vespa. We’ll post the date on our website soon.

In that session, we’ll dive into query personalization and suggestions—something that’s trending right now. These aren’t just based on behavioral popularity or common queries, but truly personalized suggestions. You’ll feel like the system understands you. You start typing, and the system predicts your intent with a high degree of relevance.

We also support personalized product search using lexical, semantic, and visual understanding—combining image embeddings with semantic meaning and text descriptions like titles or product details. This combination improves recall and powers a multi-phase ranking pipeline. Lighter models are applied early, while heavier, more resource-intensive models like cross-encoders are reserved for later ranking stages—achieving both performance and accuracy.

By incorporating personalized vector embeddings based on A/B testing or behavioral models, you can deliver powerful search results. This also applies to personalized category pages. Like in TikTok or other social apps, the more you engage, the more tailored the homepage becomes. It’s not just cool—it directly boosts upsell potential and business outcomes.

“You might also like” recommendations are another layer. These can be based on items viewed or purchased, including cross-category discovery. Relationships between products can be easily modeled and extended to advertising—reusing the same signals and datasets.

Trending product identification is also a strength. You can monitor orders, clicks, and ratings. Reviews and sentiment scores can even be incorporated into Vespa’s ranking profiles. Other use cases include customer engagement, fraud detection, and counterfeit detection. Vespa supports all of this natively.

You don’t need to start with everything at once. Start with search and ranking, and expand from there as you grow more familiar with Vespa.

Personalization at Scale and Vespa Platform Capabilities

When it comes to personalization at scale, today’s systems rely heavily on neural methods. This means tensor operations are fundamental. Vespa supports this out of the box. You can store individual or cohort-level embeddings (as vectors or tensors) and use them directly in ranking. Inference can be executed on GPU or CPU, depending on model complexity. Vespa supports both seamlessly.

This gives you consistent performance regardless of catalog size, user volume, or QPS. Vespa is extremely scalable.

Now, about the platform: there’s often a misconception about what Vespa is. Some think it’s a vector database, others a search engine. But it’s actually a platform. It includes hybrid search (vector + lexical), multi-phase ranking, model inference, data transformation pipelines, and distributed storage.

You can define how each field is stored—on disk, in memory, embedded index, or HNSW graph. Vespa allows you to model datasets across multi-cloud or multi-cluster configurations. One content cluster might serve fast filtering; another might serve complex semantic retrieval.

We wrote about this in our blog, explaining how we built separate content clusters for different workloads. This flexibility supports all the services we’ve discussed. It’s especially relevant for Retrieval-Augmented Generation (RAG) applications.

Take Perplexity, for example. They index 10 billion pages into Vespa and use it to power real-time search for chat interfaces. Their platform showcases what’s possible.

Whether it’s chatbots, virtual assistants, or agentic applications, Vespa’s multi-stage ranking helps extract accuracy from massive knowledge bases. It also supports recommendation systems and advanced search use cases across e-commerce and beyond.

Many platforms are already experimenting with RAG and agent-style architectures. We expect this will become the standard for e-commerce soon.

Vespa Architecture and Operational Principles

Now let’s talk about Vespa’s architecture and core principles. Vespa prioritizes local execution. Model inference and operations like MaxSim run where the data lives—on the same node—so data doesn’t have to move across the network. That’s highly efficient.

Ranking expressions are also executed locally, and computation is distributed across clusters. This ensures stable and predictable performance.

Everything in Vespa is packaged via the application package. It defines configurations, schemas, document types, ranking logic, semantic/lexical handling, field inference (like binarization), and embedded ML models.

Models are bundled and loaded into the Vespa cluster for local inference. The system separates stateless Java containers (handling business logic) from stateful C++ nodes (handling storage and compute). This dual-layer design ensures performance and scalability.

Vespa is a distributed system. You don’t have to worry about manual sharding—it handles that. Computation happens where the data resides, which lowers total cost of ownership and improves performance.

Low latency is essential for advanced use cases like fraud detection, analytics, and payment monitoring. Real-time updates mean you can prevent issues as they happen.

Vespa handles real-time ingestion. Documents are searchable as soon as they’re written—no refreshes or forced merges. It supports partial updates of any field. This is particularly valuable for inventory, carts, or payment flows, where fast updates are critical. Vespa can handle over 100,000 updates per second per node.

Retrieval, Ranking, and Tensor Capabilities

In terms of retrieval and ranking, Vespa supports distributed multi-stage pipelines. Early stages filter with lightweight models; later stages use heavier models. You can choose what model or ranking expression runs at each stage.

This approach allows you to scale accurately and efficiently. For example, in the first stage you might filter 1 million documents down to 1,000 using fast filters. Then apply heavier models only on those 1,000.

You can also control memory usage by paging vectors to disk and loading them only when needed for final ranking.

Vespa supports tensors—scalars, vectors, matrices, maps, and map of vectors. You can mix operators and build expressive ranking functions with dot products, normalization, etc. All of this can be demoed easily.

Live Demo: Personalized Search and Visual Understanding

That brings us to the demo: a personalized car search.

Imagine you want to buy a car. You input preferences like “I like Audi and Mercedes,” “I dislike diesel and Skoda,” and “I can’t drive a manual.” Vespa converts this into a preference vector, then uses it in a ranking profile. Results are personalized instantly.

You can also layer in filters and use additional models to represent emotional preference, taste, or intent. These models work together to deliver hyper-personalized results.

In the next webinar, we’ll show how to extend this to a full e-commerce platform.

We also demoed visual understanding using screenshots from PDFs. Our system breaks images into 32×32 blocks, embeds them, and uses late interaction to match vectors against queries like “child jumping.”

This enables retrieval not only by text or layout but by the actual content of an image. It’s incredibly powerful for visually rich documents.

Resources and Final Thoughts

To wrap up, we encourage everyone to explore the Vespa Python API and our sample apps. These include album recommendation engines, billion-scale vector search, and ColBERT-style late-stage re-rankers.

These are available as open resources to help you build your own production-grade systems. If you’d like to go deeper, please get in touch—we’d be happy to assist.