Apache Kafka 3.0 is now available, making way for the event processing platform to leave the ZooKeeper project behind for good. It is, therefore, not surprising that the highlights of the release come in the context of Kraft, Kafka’s consensus mechanism, which is still in preview.
The successor to the ZooKeeper now provides Kraft Controllers and Brokers to generate, replicate and load snapshots of the metadata topic partition _cluster_metadata, which is where Kafka Cluster stores and replicates cluster metadata.
The Kafka team redesigned the tool’s metadata record types, making the Kafka Controller responsible for generating producer IDs in both ZooKeeper and Kraft mode for easier transitioning to Kraft once it reaches a stable release.
Tuning the platform
There was also an agreement that producers should enable the strongest message delivery guarantee by default, which is why such instances will now come with idempotence and acknowledgement of delivery by all enabled replicas.
Aside from that, the Kafka team tried to tweak the platform to fit current use cases that were a bit harder to achieve before.
For instance, monitoring tasks often produce some overhead. This is because fetching offsets for multiple consumer groups weren’t allowed. This has been fixed in the OffsetFetch API in version 3.0.
Cleaning up
Because this is a major release, there are some changes to the old code that isn’t working properly. The APIs that used the 24-hour default grace period in Streams have been replaced with explicit new methods that either set the period to zero or allow a custom value to specify the grace period duration.
The Kafka team is also cleaning up. v3.0. It has deprecated support for Java 8 and Scala 2.12, with plans to drop them completely in v4.0.
Find more details about the release here, in the Kafka post.
4 hours ago
