For ten years, Klarrio has been building enterprise data platforms on open source. A lot has changed during that time, according to CTO Bruno De Bus. License changes, a shift toward Chinese contributors, and supply chain attacks have changed the rules of the game. Rather than blind trust, indispensable open source requires control and informed choices.
Since its founding, Klarrio has focused on building custom data platforms for large, complex enterprise environments. Choosing to do this primarily with open-source components rather than proprietary stacks is deliberate. The main goal is to prevent vendor lock-in. By using open source, they make their clients more independent. The code developed for a specific project always remains the property of the client.
When the company started about ten years ago, its work consisted mainly of implementing the data stacks released by major tech companies. Technologies such as Hadoop (Yahoo), Kafka (LinkedIn), and RocksDB (Facebook) formed the foundation. Klarrio’s task was to integrate these robust but often unpolished components into demanding business environments. Today, however, that landscape looks different. According to De Bus, the barrier to entry for starting a similar company is now considerably higher. It’s an advantage that Klarrio has had a decade to grow organically alongside the increasing risks and complexity of the ecosystem.
The end of blind trust
One major change is the approach to security. “Ten years ago, security was almost an afterthought, and you could trust open source almost blindly,” says De Bus. “Now, probably half of what we do consists of security work.” That work focuses primarily on hardening, maintaining, and securing the open-source building blocks.
The days of blind trust are therefore over. Because virtually the entire global IT infrastructure now runs on open source, the ecosystem has become an attractive target for malicious actors. Attacks on the software supply chain were virtually nonexistent ten years ago, but now pose one of the greatest threats.
As attackers operate more sophisticatedly, it’s difficult to tell whether an open-source project is genuine or driven by malicious intent. Sometimes attackers infiltrate a project to plant a backdoor. Others take over abandoned projects, breathe new life into them, and hide malware within them. And then there’s the variant where someone takes existing code, puts their own name on it, and distributes it with malware embedded inside.
Impact of AI on the software supply chain
The rise of generative AI does not make assessing software reliability any easier. Previously, the size of an open-source project was often a reliable indicator of the creators’ dedication. A large codebase implied a significant time investment, which increased the likelihood of long-term maintenance. But that’s no longer the case. “If you come across a large project today, you don’t know where it came from. It could effectively have been cobbled together in just two or three days,” explains De Bus. This creates uncertainty about whether the creator will be able and willing to keep maintaining the project in the future.
To address these threats, Klarrio now follows a strict “security-by-design” philosophy. Whereas a component from Hadoop or Kafka would previously have been adopted without hesitation, the company now rigorously vets every dependency in advance. To this end, the company uses tools from OpenSSF, including Scorecard. This tool automatically checks a project’s health by examining factors such as branch protection, code reviews, signed releases, and fuzzing, and assigns a score. In addition, Klarrio uses Software Bills of Materials (SBOMs) to maintain transparency across the entire supply chain and deploys its own security scanners to rapidly scan large amounts of code for hidden vulnerabilities. When Klarrio detects suspicious code, it is immediately reported to OpenSSF’s Malicious Packages Repository, ensuring the broader community is warned in a timely manner.
Tip: Security by design prevents higher bills
Open source became source-available
Another major shift in the landscape is legal and commercial. U.S. software vendors are regularly moving away from traditional open-source licenses. Initially, this trend began subtly. Enterprise features then disappeared behind a paywall, while only the product’s core remained open source. According to De Bus, this strategy often served more as a marketing tool to retain the “open” label. Without license fees, however, the software could no longer be widely used.
In recent years, the line has become clearer. Large, popular projects such as MongoDB, Elasticsearch, and Terraform have switched to licenses such as the Server Side Public License (SSPL) and the Business Source License (BSL). These licenses, often called “source-available” rather than “open source,” prohibit companies from offering the software as a commercial cloud service.
Klarrio’s CTO understands this shift. According to him, the pressure came primarily from hyperscalers. For example, AWS offered Elasticsearch as a managed service, causing the code’s original developers to see their own revenue model evaporate. In response, Elastic replaced the open Apache 2.0 license with the SSPL in 2021. It wasn’t until 2024 that Elastic added the AGPLv3 license, making Elasticsearch usable again under a license recognized by the Open Source Initiative (OSI). HashiCorp followed a similar path with Terraform in 2023.
These license changes immediately led to forks (branches of the original code). This is how OpenSearch emerged as an alternative to Elasticsearch and OpenTofu as a response to the changes made to Terraform. De Bus, however, offers a critical perspective on this. He points out that Amazon also has a significant say in these forks. Moreover, forking projects leads to fragmentation: it duplicates effort for developers and splits the community in two, even though that collective intellectual power forms the core of open-source success.
For Klarrio, this fragmented market requires a case-by-case approach. With Terraform, the company switched to the OpenTofu fork. For search technology, however, it chose to leave the entire Elastic/OpenSearch debate behind and migrate to alternative products with a licensing model that better aligns with its vision. “For us, that could even be the fairly restrictive license. It essentially says, ‘If you build something with it, you have to submit it all back.’ That’s more acceptable than actually saying you’re not allowed to use it at all without a license,” explains De Bus.
Geopolitical shifts
A third factor shaping today’s ecosystem is the code’s origin. Innovation within the open-source world used to be primarily an American affair. Today, the landscape looks more diverse. Anyone looking at the latest projects within the Apache Foundation, especially in the data domain, will see that much of the innovation comes from large Chinese tech companies. A substantial portion of individual contributors also have a Chinese background these days.
Technically, De Bus says there’s nothing wrong with that. The communities are generally very open to external contributions. Yet in practice, Klarrio observes that European and American enterprise clients are less likely to trust software from China. In addition, practical barriers remain: when the documentation, code comments, and pull requests are written exclusively in Chinese, it becomes particularly difficult for Western developers to contribute to the project. Klarrio does not exclude Chinese components, but this is clearly a shift in the market. As a result, selecting software requires a bit more research, De Bus observes.
The Linux Foundation, the Apache Software Foundation, and the Cloud Native Computing Foundation (CNCF) play a crucial role. Although they cannot provide ironclad guarantees against vulnerabilities, they do offer an essential governance structure. They ensure multiple parties maintain and govern projects. Klarrio therefore strongly prefers projects under such a foundation. This demonstrates commitment and ensures transparency. If a project is discontinued and no longer maintained, the foundation announces this in a timely manner.
Community is key
Ultimately, it is not the technology itself, but the community surrounding it that determines whether a project stands the test of time. A clear example from the past decade is the battle over container orchestration. Ten years ago, Kubernetes, now the undisputed industry standard, was far from the obvious choice. The world ran on VMs, and Apache Mesos was a promising project. Mesos, like the data framework Spark, emerged from the academic research community at the University of California, Berkeley. Even Apple ran deployments of thousands of nodes on Mesos.
Nevertheless, within five years, Mesos had virtually disappeared from the scene, and even the commercial company behind Mesos switched to its competitor, Kubernetes. According to De Bus, the community was the deciding factor in this shift. At its core, Kubernetes was more accessible for external developers to contribute to than the highly academic Mesos. During this transition phase, Klarrio managed infrastructure that ran half on Mesos and half on Kubernetes, but now everything runs entirely on Kubernetes.
A similar mechanism explains Kafka’s dominance. The streaming technology has now well and truly overcome its teething problems. Early versions completely lacked basic security features such as multi-user support or TLS encryption. Kafka is so big today simply because alternatives have never attracted a comparable number of developers.
However, that immense popularity also has a downside. According to De Bus, Kafka is regularly used, out of ignorance, for purposes it wasn’t designed for, such as a simple message-passing interface for lightweight applications. In such cases, a traditional message queue is much more efficient. “Ten years ago, I had to convince clients to start using Kafka. Now I more often advise them to stop using it,” De Bus notes. The same is true for Kubernetes. It’s sometimes set up to run just two containers, even though it’s a complex system that requires in-depth expertise.
The data evolution
Data architecture itself has changed a lot over the past ten years. When Klarrio started, Hadoop dominated the market. As the Hadoop era ended, a highly fragmented landscape of standalone tools emerged. Today, however, convergence is once again on the rise, driven by the emergence of lakehouse architecture. Two dominant storage formats have emerged: Apache Iceberg and Delta Lake.
Competition between Snowflake and Databricks is driving adoption of these formats. To gain market share, they are actively pushing their technologies to the open-source community. This is also evident in metadata catalogs, where they compete as well. Databricks made Unity Catalog open source (which supports Delta Lake, Iceberg, and Hudi) after it had been a proprietary component of its paid platform for years. That move came nine days after Snowflake announced it would make Polaris (with a pure Iceberg focus) open source within 90 days. A little over a month and a half later, Polaris’s source code appeared on GitHub. In the data world, this period is also known as the “Catalog Wars.”
As with Kafka back then, De Bus still sees functional gaps in these new lakehouse components. These gaps are particularly evident in advanced security models. However, the active community is expected to fill in these features over time. Klarrio itself contributes to this effort, often by writing code that enables components to function in unusual, complex contexts.
What does the future hold?
Finally, Klarrio’s technological journey reflects the cloud’s maturation. Ten years ago, the company often introduced the cloud to customers, a process that required developing fundamental cloud policies in consultation with the CISO. Five years later, the motto was “everything to the cloud.” Not long after, the inevitable financial hangover led to the rise of FinOps (cloud cost management) and the development of shared multi-tenant platforms to streamline spending.
The current trend is once again the exact opposite. Organizations want to bring their workloads back from the public cloud or move them to sovereign European clouds. Because Klarrio has always embraced open source, it can facilitate these shifts more easily. Open source is becoming an exit strategy for moving away from proprietary cloud providers.
This experience also shapes how the company views the future. Klarrio prefers not to get involved in technologies that have only just emerged; the risk of a project failing within a few months is too high. For the current lakehouse architecture, however, critical mass has now been reached. De Bus’s advice to companies is therefore: stop building your own data lake from scratch and instead make use of the standardized ecosystem.
Looking ahead to the next five years, De Bus sees the following challenge on the horizon: hosting generative AI on-premises or under the company’s own management. Although the market isn’t quite ready for this yet and the supply of suitable, efficient models is still lacking, the direction is clear. Within the next five years, Klarrio expects to build platforms that let organizations run and train advanced AI models completely independently and securely within their own infrastructure; true to its philosophy, this will be based on open source and self-management.