It’s a time of seismic adjustment in the database universe. The Silicon Valley data behemoths (there’s a well-known company that starts with the letter O, but there are others too) are busy working to apply autonomous management advantages to their existing stacks to automate-away the sysadmin/DBA headaches of the past. But all the while, a new breed of time series and vector database technologies are emerging, so could we end up witnessing another ‘African mobile leap effect’ here (the continent went straight to mobile with no landline history in many places) with new enterprises skipping the traditional in place of the more transitional?
KX doesn’t use the emerging markets mobile network deployment analogy to describe the potential impact of its vector-native real-time high-volume time-series multi-model data analytics platform, but it could – and yes, that was x5 hyphens in a row on the bounce, such is the new engine power on offer here.
With a new early access program for KDB.AI its multi-model vector database, the company is offering privileged access data services and support for a technology that aims to be part of the evolution of transformative generative AI applications.
At the heart of KX is its kdb+ time series and vector database, which can can process and analyse time series, historical and vector data at speed and scale. The company lists its primary users as developers, data scientists and data engineers, all of which will be looking to build high-performance data-driven applications and turbo-charge their favourite analytics tools in the cloud, or in on-premises deployments, or indeed at the IoT edge.
NOTE: As KX reminds us, vectors are sequences, arrays, lists of mathematical information, data presented as a two-dimensional array that has a size of 1-by-N. Vector processing involves the computation of vectors in single operations. Vector processing tends to be computationally faster and more efficient than running for loops, or querying data row by row.
The software soothsayers at McKinsey estimate that generative AI is forecast to add ‘trillions’ to the global economy – in whichever world currency you choose to measure this leap. This, suggests the analyst firm and others, means that demand for resilient vector databases to support high-performance generative AI and semantic search applications is snowballing.
Temporal reasoning & inference
According to KX CEO Ashok Reddy and team, current vector databases and AI applications, however, exhibit severe limitations. These shortfalls include lack of ‘statefulness’, insufficient processing of high-volume numerical calculations, an inability to adapt in real-time and problems replicating sophisticated risk models. These issues obstruct AI’s potential for temporal reasoning and inference, limiting its effectiveness.
As an additional explanatory note on statefulness – any given stateful application, service or indeed complete software program is one that is capable of saving ‘client’ data (which in this case means ‘user data’ and client as in ‘email client data’ – such out Outlook or Thunderbird – in the same context) between one user session and another. That saved data is the information entity that defines the application’s state i.e. the data in the app is what state the app is in.
By using the KDB+ engine time series vector native database, KX with KDB.AI claims to be revolutionising AI technology, or perhaps (more modestly, at least helping to underpin this new era of use cases). The company talks about its technology’s ability to provide contextual understanding to work on ultra-high-dimensionality problems with causal reasoning and an ability to offer explainability, fairness and sustainability.
In other words, this is something rather more than the relational database management systems of pre-millennial times.
“Moreover, KDB.AI fills a significant gap in the real-time processing and streaming of vectors, a limitation profoundly felt in the AI field today. Seamlessly handling both structured and unstructured data, KDB.AI empowers real-time processing, enabling sliding window search for time-series-based vector search. Its stateful approach caters to various applications, providing native support for large language models (LLMs) and machine learning workflows,” notes the company, in a technical product sheet.
Hyperscaler selection
Available for deployment on all hyperscaler cloud platforms including Azure, AWS, GCP and also for on-premise use, KDB.AI offers universal compatibility with any infrastructure setup.
Key implementation markets include: financial services – it enables real-time processing of vast time-series and alternate datasets; manufacturing: KDB.AI can improve operational efficiency by enhancing quality control, predictive maintenance, and supply chain processes; plus also life sciences and aerospace and defence.The early access program offers access to the latest features and updates with unlimited usage and dedicated support. There’s also a developer sandbox, based on Jupyter Notebook technology.
An African mobile leap situation? Maybe so, perhaps not, who knows… but worth dialling up and listening in to for sure.
Free image (above): Wikimedia Commons.
2 days ago
