GEM brings secondhand and vintage clothing from multiple marketplaces into one search experience. Shoppers can explore a vast catalog without searching each marketplace separately.
That inventory changes constantly as items arrive, sell, and disappear. With more than 200 million listings and around 15 million listing updates each day, GEM needs search infrastructure that can keep pace with both its catalog and its shoppers.
The Challenge
For secondhand shoppers, finding the right item often comes down to timing. A photo can capture exactly what someone wants, but a suitable item may not be available in their size or budget when they search.
GEM already offered keyword-based search in two forms: shoppers could search the current inventory interactively, or save their searches and receive alerts when new matching listings arrived. Elasticsearch served both workloads, using its Percolator capability to match incoming listings against stored keyword searches.
GEM then introduced image search, allowing shoppers to find visually similar items using a reference photo. While interactive image search worked, it introduced two challenges.
First, vector search performance fell short of expectations at GEM's scale. Achieving the desired performance would have required substantially more hardware, raising concerns about efficiency and cost.
Second, unlike keyword searches, image searches could not be saved for ongoing alerts because Elasticsearch's Percolator did not support vector similarity matching at the time.
Enabling saved image-search alerts required inverse image search combining vector similarity with structured filters such as size and price. Each incoming listing needed to satisfy the saved searches’ similarity thresholds and filter conditions, returning every qualifying saved search, not just the top-K most similar ones. With around 15 million listing updates a day, that matching had to keep pace with GEM’s inventory.
The evaluation
GEM brought in Searchplex to assess its existing search implementation and determine the best path forward. With experience helping teams build and modernize search systems across Elasticsearch, OpenSearch, Solr, and Vespa, Searchplex brought a broader perspective on the architectural choices and tradeoffs.
Searchplex began with a search-stack audit, examining image-search performance, continuous matching against saved searches, and the characteristics of GEM's workloads, including its growing inventory and update volume. The evaluation also considered GEM's broader requirements for hybrid search and future personalization capabilities.
GEM had initially been exploring how to address these challenges within its existing Elasticsearch architecture. Based on the evaluation, Searchplex introduced Vespa as an alternative and recommended it as a strong fit for scalable hybrid search across both interactive and inverse-search workloads, with support for future personalization and recommendation capabilities.
Rather than committing immediately to a full platform migration, GEM and Searchplex chose to start with a focused production workload: saved image-search alerts. This would deliver a new customer-facing capability while providing a practical way to validate Vespa against GEM's requirements.
The Solution
Following Searchplex's recommendation, GEM selected Vespa for the saved image-search alerts implementation because its retrieval and ranking capabilities could support several key requirements:
Searchplex designed and built a Percolator-style inverse-search solution using Vespa, matching each incoming listing to saved searches only when it meets both the image-vector similarity threshold and structured filter conditions, such as size and price.
The implementation returns every qualifying saved-search identifier to GEM's alert-processing workflow. This fetch-all requirement is essential: limiting results to the top-K most similar matches could exclude shoppers whose saved searches also qualify for notification.
Searchplex validated matching correctness and benchmarked scalability, infrastructure requirements, and cost implications. Testing included qualifying and non-qualifying matches, as well as larger projected workloads to understand how resource needs would change as demand grew.
Combining vector similarity, filter conditions, and fetch-all matching at scale in an inverse search was a challenge I hadn’t encountered before. Elasticsearch’s Percolator couldn’t support the vector component. Vespa unlocked it, and the performance really flies.Ravindra HarigeFounder, Searchplex
For a deeper technical discussion of the architecture, correctness requirements, match cardinality, and scaling tradeoffs, see Searchplex founder Ravindra Harige's Berlin Buzzwords talk, The Three-Body Problem of Inverse Hybrid Search
The Results
With saved image-search alerts running on Vespa, GEM can now notify shoppers when matching items arrive, without requiring them to repeatedly search the marketplace. This extends GEM's existing saved-search experience to images, a capability its previous search architecture could not practically support.
The ability for users to save image searches is something that would not exist at all if we hadn’t started with Vespa.Sampo KarjalainenCo-founder, GEM
The successful production deployment also gave GEM practical experience with Vespa and greater confidence in adopting it for additional search workloads.
What's next
Following the saved image-search alerts deployment, GEM has been working to migrate its primary, user-initiated image and keyword search workloads from Elasticsearch to Vespa. Further ahead, GEM sees opportunities to use Vespa for recommendations, personalization, and discovery experiences. The goal is to help shoppers find a specific item, discover pieces that suit their style, and come across something they might never have thought to search for.
Searchplex helped us understand our options and built our first production solution on Vespa. Their expertise continues to be valuable as we expand our use of Vespa.Sampo KarjalainenCo-founder, GEM
GEM’s experience offers several takeaways for teams pursuing similar capabilities:
For GEM, partnering with Searchplex and building on Vespa turned a product requirement into a production capability, while also laying the groundwork for GEM’s broader ambitions in search, recommendations, and personalized discovery.
Searchplex is an AI-native search and retrieval engineering consultancy and an official Vespa.ai partner. It helps organizations evaluate, design, build, migrate, and optimize production AI search and discovery systems across industries.
From Elasticsearch to Vespa: Rebuilding the Kleinanzeigen Homepage Feed
Agent-driven intelligence for private equity, built on Vespa Cloud
Delivering AI-powered content discovery and personalization for DeviantArt, one of the world's largest online art communities
