Tensor Formalism

Beyond vector representations

Vespa provides a formal tensor framework that unifies dense and sparse data through named dimensions and a consistent set of tensor operations. This lets developers express sophisticated retrieval, ranking, and machine learning computations in a single distributed serving engine, without stitching together specialized systems.

Vectors are a fundamental building block of AI search, but a single embedding cannot always capture the structure and relationships needed for sophisticated relevance. Vespa's tensor framework extends beyond individual vectors, preserving multiple embeddings, features, and their relationships within a unified representation.

With tensors, you can:

Score at token or image-patch level using late-interaction models.
Combine dense and sparse representations with lexical relevance signals.
Express sophisticated ranking logic using tensor operations.
Incorporate personalization signals directly into relevance calculations.
Execute tensor computations where the data resides, at production scale.

The result is a more expressive foundation for retrieval, ranking, and machine learning inference, without sacrificing the performance required for large-scale AI applications.

A unified model for tensor computation

AI search combines dense embeddings, sparse features, and structured data. Vespa’s tensor formalism represents dense, sparse, and mixed tensors in a single model, using named dimensions and a consistent set of operations. Three foundational principles underpin this approach:

Named dimensions and strong typing

Tensor dimensions have meaningful names and explicitly defined types, making it easier to represent complex data, validate operations, and understand how different dimensions interact.
Dense, sparse and mixed tensors

A common representation supports dense numerical arrays, sparse data, and combinations of both. Developers work with different data structures through the same tensor framework rather than relying on separate computational models.
A consistent set of tensor operations

A small set of foundational operations provides a general language for tensor computation. Developers compose more sophisticated calculations from these operations, simplifying development while enabling efficient execution and optimization.

Together, these principles provide a flexible mathematical foundation for representing and computing over complex data. In Vespa, this foundation is built directly into retrieval, ranking, and machine learning inference, letting sophisticated relevance models run in the same distributed serving engine.

Named dimensions and strong typing

Vespa's tensor framework uses named dimensions and strong typing to make complex tensor computations more expressive, predictable, and maintainable. Unlike conventional multidimensional arrays that rely on positional dimensions, Vespa assigns meaningful names and explicit types to tensor dimensions. This lets developers represent structured relationships, combine tensors without relying on dimension order, and validate type compatibility.

For example, a document containing multiple token embeddings can be represented as:

tensor<float>(token{}, embedding[384])

This combines a variable number of tokens with a fixed 384-dimensional embedding for each token, preserving their individual representations within a single tensor. Named dimensions make these relationships explicit, while strong typing helps detect incompatible operations.

Dense, sparse and mixed tensors

AI search applications increasingly combine dense embeddings, sparse features, structured attributes, and multiple representations of the same content. Managing these through separate data models and computational frameworks introduces unnecessary complexity.

Vespa unifies dense, sparse, and mixed tensors in a single framework, letting developers represent and compute over heterogeneous data with consistent tensor expressions. This provides a flexible foundation for sophisticated retrieval and ranking, including late-interaction models, multimodal search, and personalization, without requiring separate systems to manage different tensor representations.

Tensor operations

Vespa's tensor framework uses a small set of composable operations to manipulate, transform, and compute across dense, sparse, and mixed tensors. Vespa's ranking expressions support both scalar and tensor computations, letting developers combine tensor math with other ranking features. This provides a consistent approach to implementing complex relevance models, including similarity calculations, feature interactions, and multi-stage ranking.

The framework is integrated directly into Vespa's distributed serving engine, allowing tensor computations to execute alongside retrieval and ranking, without moving data between separate systems.

Tensor computation in AI search

Vespa's unified tensor framework lets developers build sophisticated retrieval and ranking models that combine multiple representations, relevance signals, and machine learning computations.

Applications include:

Late-interaction retrieval and ranking: Compute relevance across individual token embeddings rather than relying solely on document-level vector similarity.
Hybrid relevance models: Combine vector similarity, lexical matching, structured features, and learned ranking signals in a unified scoring expression.
Multimodal search: Represent and compute across multiple embeddings and features derived from text, images, and other modalities.
Personalized ranking: Incorporate user preferences, behavioral signals and contextual features into relevance calculations.

Explore the tensor formalism

Vespa's tensor framework is grounded in a formal mathematical model that unifies tensor representations and operations. For a deeper understanding of its foundations, explore the original white paper, A Tensor Formalism for Computer Science, which describes the mathematical principles and foundational operations underlying Vespa's implementation.Developers ready to apply these capabilities can find tensor syntax, operations, and implementation examples in Vespa's technical documentation.

Read the white paper Explore the tensor documentation

Ready to Put Tensors to Work?

Whether you're building semantic search, recommendations, RAG, AI agents, or multimodal search, we'd be happy to discuss how tensor representations fit into your AI search architecture and answer any technical questions.

Talk to the Vespa team

How does Vespa's tensor framework differ from traditional tensor frameworks?

Vespa uses a unified tensor model with named dimensions, strong typing, and consistent operations across dense, sparse, and mixed representations. Unlike general-purpose machine learning frameworks, Vespa integrates tensor computation directly into its distributed retrieval and ranking architecture. This lets developers run sophisticated relevance calculations where the data lives, without needing a separate system for query-time ranking.

Can Vespa combine dense, sparse, and mixed tensors in the same ranking expression?

Yes. Vespa's ranking expressions support computations across dense, sparse and mixed tensors, alongside scalar ranking features. Developers can combine vector similarity, token-level interactions, structured attributes and other relevance signals within a single ranking expression. Tensor operations provide the mechanisms for transforming, joining and aggregating these representations.

How does Vespa support late-interaction models such as ColBERT?

Vespa can represent documents using multiple token-level embeddings rather than a single document vector. Tensor operations allow query token embeddings to interact with document token embeddings, enabling fine-grained relevance calculations such as MaxSim. These calculations can be incorporated into Vespa's ranking pipeline, applying more computationally intensive late-interaction scoring to selected candidates rather than the entire corpus.

Can Vespa execute tensor computations without exporting data to another system?

Yes. Vespa executes tensor computations within its distributed serving architecture, alongside retrieval and ranking. This lets applications combine stored tensor representations, query-time inputs, and ranking features without sending candidate documents to a separate system for scoring. It also supports multi-phase ranking, applying increasingly sophisticated computations as the candidate set narrows.

Does Vespa support tensor-based inference as well as tensor operations?

Yes. Vespa supports machine learning inference within its ranking framework, including models imported through ONNX. Tensor expressions can be used to prepare model inputs, combine inference outputs with other relevance signals, and implement additional ranking logic. This allows learned models and explicit tensor computations to participate in the same relevance pipeline.

Are Vespa tensors limited to search and ranking applications?

No. Vespa's tensor framework provides a general mathematical model for representing and computing over multidimensional data. Its integration into Vespa is particularly valuable for retrieval, ranking, recommendations and other applications requiring sophisticated computations over continuously updated data at low latency.

Resources