Vector search is the new black for enterprise databases

Software slingers from Redis to Teradata are bolting on smarts to stay relevant in GenAI era

About two years ago, popular cache database Redis was among a wave of vendors that added vector search capabilities to their platforms, driven by the surge of interest in generative AI.

Vector embeddings are produced by foundational models such as OpenAI's ChatGPT and are used to represent chunks of language – like words or phrases. The idea is to add vector search to a database, which allows enterprise systems to store, index, and search millions of vectors extending beyond the foundational models to enterprise data.

Since 2023, so many database systems have announced vector search as a core feature that it hardly distinguishes them from the competition. For example, MongoDB, Cassandra, PostgreSQL, Snowflake, and SingleStore all announced the same feature in the same year.

Vendors are now trying to differentiate themselves with more in-depth features in the race to become indispensable in the modern AI stack.

For example, Redis announced LangCache. The fully managed REST service is designed to reduce expensive and latency-prone calls to LLMs by caching previous responses to semantically similar queries.

"It sits between your application and whatever LLMs you're using, and you pass in the query, and we see if there's a response that matches, and then we just hand it back to you instead of hitting the LLM inferencing engine," Redis CEO Rowan Trollope told The Register earlier this month.

YugabyteDB, the PostgreSQL-compatible database with a distributed back end, is also on a mission to more deeply adapt its software to support AI. With Paramount, GM, and Kroger among its customers, the vendor said that integrating vector indexing libraries including Usearch, HNSWLib, and Faiss would boost the performance of the PostgreSQL vector search extension, pgvector.

Founder and co-CEO Karthik Ranganathan told The Register that Yugabyte had replicated and auto-sharded these indexes on its distributed back end to accelerate the performance of pgvector. He argued that while pgvector had the massive community of PostgreSQL behind it, it was reimplementing vector search in the longstanding relational system. It needs performance enhancements to keep up with specialist vector databases.

"Everything in the AI world is going really fast, so we started directly interfacing with some of these open source libraries," he said.

Gartner said last year that "moonshot" AI projects in business were seeing high failure rates. Expectations of what GenAI could achieve in business would come down, and 2025 was set to be the "year of the slide."

In other research, Gartner warned that a lack of preparedness in data infrastructure was hampering progress in bringing GenAI to business problems. It predicted that in 2025 and 2026, organizations would abandon 60 percent of AI projects because their data wasn't ready.

Roxane Edjlali, senior director analyst at Gartner, said that including vector data stores was one of the ways organizations could help extend and improve data management to support new use cases such as GenAI.

"Remember that AI-ready data is not 'one and done.' Think of it as a practice where the data management infrastructure needs constant improvement based on existing and upcoming AI use cases," she said.

She recommended that organizations invest in AI, develop an AI-ready data practice, and ensure continued investment and ongoing maturity in metadata management, data observability, analytics, and AI governance.

Redis and Yugabyte are transactional systems looking to add AI support. The dominant transactional database, in terms of deployments, is Oracle, which analysts describe as an early leader in bringing natural language vector search capabilities to the relational data in business systems.

Yet analytics system vendors are also striving to update their technology to meet the requirements of GenAI.

Teradata has a 40-year history in business intelligence and data warehousing and counts HSBC, Unilever, and American Airlines among its customers. Last year, it announced it was integrating Nvidia NeMo and NIM microservices into the Vantage cloud platform to accelerate AI workloads and support the development of foundation and customized large language models (LLMs) and retrieval-augmented generation (RAG) applications.

Further back, Teradata began investing in machine learning about the time it bought specialist analytics vendor Aster for $263 million.

Martin Willcox, VP of analytics and architecture at Teradata, told The Register that demand for analytics and BI services is rising as businesses explore AI agents and LLM-powered customer interactions. He said Teradata's Massively Parallel Processing (MPP) shared-nothing architecture is capable of running inference for AI models in the 150 to 250 million parameter range. Teradata is, we're told, also improving its API integration with hyperscalers to support their LLMs.

At the same time, clients are creating large stores of unstructured data – images, audio, PDFs, or email, for example – and employing vector search on top to make sense of their own information in natural language, in combination with LLMs.

"Current vector database technologies essentially come in two flavors," Willcox said. "There are special-purpose technologies which perform well for small datasets, but don't always scale and quite often lack enterprise fit and finish. Secondly, there are Model-View-Presenter (MVP) frameworks like Apache Spark, which can scale linearly for these tasks, but often perform pretty poorly for them. And so we think there's a gap in the middle for a vector store that both scales and performs and has all the traditional fit and finish that you would expect of an enterprise-class database management system.

"When you see what our customers do [with AI agents] that also means more BI, more predictive machine learning. Agents by themselves are very sophisticated models, but entirely ignorant of organizational context. In order to make any kind of worthwhile decisions at all, they have to ask lots of questions of back-end database systems."

SingleStore – formerly known as MemSQL – has built a database that supports both OLTP and OLAP in a single system and counts Uber, Kellogg's, and engineering giant GE among its customers. It too has anticipated the demand on databases to support GenAI and has supported vector search since 2017 using an exact-nearest-neighbor approach. Last year, it previewed indexed approximate-nearest-neighbor (ANN) search, a move it claimed offers orders-of-magnitude faster vector search, and makes it easier for developers to build applications.

Technical evangelist Akmal Chaudhri said SingleStore has 400 customers globally, and around 45 use the company's GenAI capabilities to some degree, including vector search. "They cover a wide range of verticals, many of them in highly regulated industries as well. They build chatbots specifically for that environment," he said.

"A lot of them are using the vector capabilities simply because it gives you more than just doing sort of metadata filtering – using SQL, for example, or just searching on individual words. It's really trying to get more context out of what we have. Lots of organizations today have huge amounts of data and don't know what to do with it, and so this gives them the opportunity to kind of interrogate the data."

According to a survey by Gartner, 63 percent of organizations say they either do not have or are unsure if they have the right data management practices for AI. It has also pointed out that despite the current obsession with GenAI, there are other kinds of machine learning that can solve business problems. Simulation, for example, is better suited to planning and forecasting than GenAI, Gartner pointed out (see its slide below).

A Gartner-supplied heatmap of AI use-cases v techniques

At the same time, it reckons the overall DBMS market – an already mature software category – is expected to grow by 16 percent in 2025, to hit a value of around $137 billion. The vendors are likely to continue to chase customer spending by attempting to show how they can help AI solve business problems. ®