Confluent opens pipe on Streamhouse data architecture

Confluent opens pipe on Streamhouse data architecture

Confluent used its Current 2026 user/practicioner conference in San Francisco this week to introduce its Streamhouse architecture to power AI with real-time business context. Not a service, not a platform, not a paradigm, Confluent’s notion of a Streamhouse (think data lakehouse, but for the streaming needs of real-time agentic AI services, get it?) is a data architecture that helps capture, process, govern and serve the current state of a business, giving decisions and actions a common foundation. Let’s pack digital oars, paddles, canoes (insert water boating analogy of your choice) and see how this channel flows.

The organisation says that a Streamhouse is a data architecture approach that lets AI agents and applications “access and act on a governed view of the current state of the business”, all at machine speed.

Detailed at Current 2026 itself was news of Confluent’s Data Streaming Platform now adding new Query capabilities to become what the company claims is “a complete and unified platform” for building a Streamhouse.

Lightning Tables, at your service

Also key here, Lightning Tables (technology built by Confluent as part of its Real-Time Context Engine architecture that works to materialise Apache Kafka topic data into low-latency, queryable tables optimised for sub-second point lookups and SQL scans) lets software application developers and data science practitioners query live streaming data directly in Confluent with low latency, reducing the need to copy data into a separate database to serve applications.

NOTE: Apache Kafka topic data is a continuous, immutable stream of timestamped key-value records organised into named categories.

With Lightning Tables now generally available on AWS, Confluent completes its five-pillar Data Streaming Platform, bringing capabilities to Stream, Connect, Process, Govern and now Query data together in a unified platform for building a Streamhouse. 

Generally available on AWS, Lightning Tables on Confluent Cloud is a read-only serving layer that durably stores and materialises Apache Kafka data into tables, keeping them current as changes arise.

EMEA perspectives

According to Richard Timperlake, SVP for EMEA region at Confluent, “Across EMEA, organisations need to make decisions across markets, systems and regulatory environments, often without replacing the infrastructure they already depend on.”

Timperlake tells us that a Streamhouse adds a real-time operational layer “alongside existing warehouses and lakehouses”, keeping trusted business context current for applications, analytics and AI agents. 

“Teams can respond to changing customer and operational conditions from a shared view of what is happening now, with governance carried through as data moves across the business. It’s a practical way to build on existing investments and give AI the relevant context it needs to act,” added Timperlake.

Why did IBM buy Confluent?

IBM bought Confluent because its data strategy combines real-time data streaming with IBM’s integration, governance and AI capabilities across hybrid environments.

Data lakes, warehouses and lakehouses were built primarily with reporting or other batch jobs in mind. As AI agents take on more work across the enterprise, they must make decisions using information spread across systems that update at different times. 

When those systems have different views of the current state of the business, agents can act on outdated or conflicting information. Those actions can carry into other workflows.

Confluent says a Streamhouse addresses this architectural problem by continuously capturing, processing, governing and serving current business context across agents, applications and analytics. 

A working example of Streamhouse

The company asks us to consider the following illustrative example of an airline managing a cancelled flight:

  • Today, the airline has different systems holding conflicting views of the same operation. Flight operations track where the aircraft is, scheduling tracks when it will depart, and reservations tracks which seats are available. 
  • When those systems update at different times, an autonomous rebooking agent can assemble an itinerary that looks valid in each system but does not work in practice: a seat is available, but the aircraft cannot arrive in time for the passenger’s connection. 
  • Customer-service teams and notification systems can then act on those same conflicting views, spreading the disruption.

With a Streamhouse, each change is capable of updating a shared picture of the airline’s operations. When an aircraft’s location changes, a departure is delayed, or a seat is booked, the source system sends that update into a continuous flow of events. 

“A Streamhouse combines those updates into current business context and makes it available to rebooking agents, customer-service teams and applications as they work. Instead of piecing together separately updated information, they can check aircraft location, departure time and seat availability together before offering a passenger a new itinerary,” explained Confluent in its conference literature this year.

What makes a Streamhouse different

A Streamhouse is built on three architectural principles: current, mission-critical and decentralised. It maintains continuously updated business context, is engineered to production service levels, and meets data where it lives without requiring consolidation first. It works alongside existing data warehouses and lakehouses, adding a real-time operational layer for decisions and actions.

“The next era of enterprise AI will be won by the agents that know what’s happening in the business the moment they act,” said Shaun Clowes, general manager, Confluent and data, IBM. “That’s what a Streamhouse can deliver. With Lightning Tables, our Data Streaming Platform gives each agent and application a live, governed view of the business. This means companies can act on fraud signals sooner, serve customers in the moment and run operations on what’s true now versus what was true yesterday.”

Clowes and team further state that Confluent’s Data Streaming Platform (DSP) provides a “complete, unified foundation” for building a Streamhouse. 

Apache Kafka at the core

At its core is Apache Kafka, which captures and streams business events. 

On its own, Kafka leaves teams to assemble everything else needed to turn those events into current business context. The DSP brings that together in one platform, now adding Query as its fifth pillar alongside Stream, Connect, Process and Govern. 

Core capabilities and latest additions to each platform pillar include the ability to stream. Built on Apache Kafka and the Kora engine (Confluent’s cloud-native, multi-tenant event streaming engine re-architected from Apache Kafka to deliver automated elasticity, sub-millisecond latency, and high availability), this stream capability ​​provides what the organisation says is the “durable, scalable foundation” for capturing and streaming business events across systems with sub-millisecond latency in Confluent internal testing; actual latency varies by configuration and workload.

Other key features here include:

  • Connect, with 150+ fully managed and custom connectors, brings data from enterprise systems into Kafka and delivers it downstream. Hybrid connector management extends how teams manage those connections.
  • Process ​​uses Apache Flink to combine, enrich and derive business context close to the source as data changes. Additional batch execution capabilities help initialise streaming workloads from existing data.​​​
  • Govern ​​helps teams discover, classify and secure data through lineage, catalogue, quality, encryption, schema and access-control capabilities.​​​​​
  • Query serves current business context to analytics, agents and applications.

Query puts streaming data to work for analytics, agents and applications, each through the interface that fits the job. Confluent Tableflow makes streaming data available to lakehouses and warehouses as open table formats, ready for analytics. Real-Time Context Engine serves current business context to AI agents through the Model Context Protocol (MCP). Lightning Tables gives applications low-latency SQL and API access to the current state of the business, so they can respond as conditions change.

Confluent also notes that Lightning Tables provides fast access to the current state of the business and it makes that same current state of the business available for low-latency queries.

It features data freshness and query performance i.e. in Confluent internal testing, approximately 100 milliseconds from new data creation to serving,  with query latency of 10-100 milliseconds and throughput of 1,000+ queries per second. Multiple query interfaces are supported, so users can query through SQL and a direct API. There is also unified governance to apply schemas, lineage and access policies from data capture through to the point of serving.

What should developers think next?

We might suggest here that, looking forward, developers (and their data science and Ops team counterparts) should treat Confluent’s Streamhouse architecture as a fundamental shift from traditional batch warehouses to real-time operational state serving. 

By unifying Stream, Connect, Process, Govern, and the new Query pillar (and we can remember that that’s all powered by Lightning Tables) Confluent claims to eliminate the need to maintain complex downstream ETL sync pipelines and secondary databases just to query live Kafka data. 

As IBM now continues to integrate Confluent across enterprise environments, developers might also think about the need to prioritise upstream data governance and standardised SQL/API interfaces, ensuring applications react instantly to current business realities without taking on extra infrastructure overhead. 

Founded on open source Apache Kafka, Confluent GM Clowes used his press and analyst briefing session to say that “IBM is a company founded on openness and so IBM is a perfect home for an open platform company like Confluent, just like Red Hat” going forward, so if that’s not a promise to watch and track in the future, then nothing is.