Developers of the Apache Kafka distributed event streaming platform have released version 2.7. The new version comes with new cool features.
For instance, in this version, the developers tackle problems surrounding the create and delete functions which affect the load on the controller, by bringing in resource level quotas.
The feature will allow the system to reject all operations that could overload it. Another protection comes in the form of broker-wide and per-listener connection creation rates. The solutions will help with many problems that were left open in the last version.
Efficiency improvements
Users will now be made aware that they have deleted a source topic from a running application using Kafka Streams. The app will now show a MissingSourceTopicException notification, instead of shutting down the app with no clear reason.
Streams now have a reverse state store iteration that will make it easy to retrieve the latest records efficiently.
In Version 2.7, developers now have trace level end-to-end latency metrics like min, max, and avg, to ensure architectural choices are not a problem. Security was also addressed by adding support for the PEM format, to store private keys and SSL certificates.
The controller is the sole king
The Kafka Project is now working to remove ZooKeeper dependencies to make metadata management scalable. For this, the devs team added an Alterlsr API. The new interface gives the controller, (broker responsible for state management of partitions and replicas,) the right to update the state of partition leaders and ISR (in-sync replica) sets.
The earlier versions made the ability shared between the controller and the partition leader, which meant there were delays and the ISR state reflected incorrectly.
To gain access to all the changes made, read Kafka release notes.