Powering large-scale RAG, search and recommendation systems where speed, accuracy, and scalability are critical.

Vespa Architecture

The Information Retrieval Foundation for AI

Vespa is a full-stack platform that is purpose-built for AI-powered search and retrieval. It combines hybrid search, real-time ingest, multistage ranking, and large-scale vector and tensor operations—all in one system. Designed for scale and flexibility, Vespa supports advanced personalization, in-place model inference, and seamless integration with LLMs, making it ideal for high-performance AI applications.

Everything You Need for AI-Driven Search

Vespa is not just a search engine or a vector database—it’s a full-stack serving platform optimized for large-scale inference and retrieval. Key architectural features include:

• Hybrid search: Combine structured filters, full-text retrieval, and vector similarity in a single query.
• Personalized results: Use context and user behavior signals for tailored ranking.
• Scalable vector and tensor search: Fast nearest-neighbor search via HNSW with support for billions of vectors.
• Multistage ranking: Execute multi-pass ranking with custom logic and ML model integration.
• Built-in embedding inference: You can choose between passing embeddings to Vespa or letting Vespa embed locally or by calling external services.
• Tensor-based logic: Express complex ranking functions and matching criteria natively.
• Streaming ingest: Real-time ingestion and partial document updates with no refresh cycles.
• LLM integration: Enrich documents or generate responses by invoking LLMs running locally or remotely.

Flexible Data Storage and Indexing

Vespa lets you define a document containing any number of fields, which can be:

• Structured fields (primitives, structs, maps, collections), indexed using database-type indexes, and stored as column fields, in memory or paged to disk.
• Vectors and tensors, indexed in HNSW graphs, 
• Traditional indexes or vector-friendly HNSW graphs, and stored as column fields, in memory or paged to disk.
• Full-text fields, indexed using positional posting lists and stored on disk.

You can deploy multi-cluster, multi-cloud Vespa applications with different content clusters optimized for different data types. For example, one cluster can be for shared, public data, and another for private data sets using streaming, improving efficiency and control at scale.

Architected for Performance: Shared-Nothing and Local Execution

Distributed by Design – Vespa follows a shared-nothing architecture with compute-local execution. This enables:

• Parallel scoring and inference directly on the content nodes
• No network bottlenecks—all ranking, filtering, and model execution happen where the data lives
• Low latency and predictable throughput, even under load

This architecture is essential for workloads like search, personalization, recommendation and RAG, where speed and accuracy directly affect user experience or business outcomes.

Core Components

Vespa provides a modular architecture designed for real-time, AI-powered applications. From defining document schemas and ranking logic in the application package, to processing queries in stateless container clusters and storing data in scalable content clusters—Vespa handles the full lifecycle of search and inference. With millisecond ingest-to-query performance and support for high-throughput updates, it’s built for use cases that demand fresh, fast answers.