To develop modern applications quickly and properly, you need a modern way of working. The cloud makes demands on developers, who are not necessarily specialists when it comes to setting up the underlying infrastructure. Pulumi wants to lend a helping hand with open-source Infrastructure-as-Code (IaC).
A 2020 survey by the Cloud Native Computing Foundation (CNCF) (the CNCF Survey from that year) revealed that about 50 percent of developers had to wait a month to access infrastructure. To develop a modern application, this is virtually indispensable.
When we talk about infrastructure in this context, it’s basically about the prerequisites to run an application. Think about setting up AWS Fargate for the compute needed for applications, for example. You have to configure that infrastructure for your application. Developers are not necessarily specialists in this area and probably don’t really want to become that either.
There are several Infrastructure-as-Code tools on the market that help set up such an infrastructure. By far the best known is Terraform from HashiCorp. However, according to Joe Duffy, founder and CEO of Pulumi, the existing offerings are far from simple and easy to use. He and his co-founder wanted to change that. Hence, they founded Pulumi in 2017.
Pulumi now employs over 100 people, has had a Series B investment round that raised $57.5 million, and it operates in six countries, including Germany, the UK, the Netherlands and Belgium. Well-known major customers include BMW and Mercedes-Benz, but Snowflake and Atlassian also use Pulumi’s IaC product. Duffy says the company already has more than 1,500 customers. In addition to its open-source IaC base, Pulumi also offers a commercial product from which it generates revenue. That is called Pulumi Service.
From code to cloud
The goal of Pulumi’s IaC platform is to get developers from code to cloud faster, as Duffy sums it up. That means the developer has to think about infrastructure as little as possible, at least about the lower levels of that infrastructure. The Pulumi platform is best thought of as an abstraction layer, allowing you to configure everything below it at once. So we’re talking about declarative programming here, not imperative. That is, you program based on the outcomes you want to see, and are not necessarily concerned with how it should be programmed.
The Pulumi Architecture Templates are a good example of the way Pulumi works. These are basic blueprints for the most commonly used cloud architectures, which include the eight most commonly used patterns for each service. So as a developer, you don’t have to think about them. You simply specify which one you want to use and have (part of) the infrastructure set up. Even things like a load balancer are created if you use the blueprints.
Note that there is not an Architecture Template for everything. With this, Pulumi focuses on the most common cloud architectures. For each service you set up on these, the company has included the eight most commonly used so-called patterns in the templates. Ultimately, rigging up additional abstraction layers is all about offering the best practices in the market. That doesn’t remove all the complexity, Duffy realizes. But it does make more possible. “The idea is to make the simple things simple, but more importantly make the difficult things possible,” according to him.
Multicloud and multilanguage
An abstraction layer like the one Pulumi builds for developers can only work if there is sufficient support for the different environments and tools developers use. Otherwise, these will still have to program at lower levels. That support seems to be there, judging by the picture below.
Pulumi Universal Infrastructure-as-Code, as the open source platform is called in full, supports more than 100 clouds and cloud services. The well-known public clouds, of course, but also the services that run on them. As an example, Duffy cites the combination of Amazon EKS, CloudFlare and Snowflake. The infrastructure needed to add that to an application can be set up on Pulumi’s platform. In addition, the platform also supports all known languages, from Python, Java and Go to YAML. Finally, there is support for all known DevOps and CI/CD tooling.
Within Pulumi’s open-source development environment, you can set up the infrastructure for your applications in a relatively simple way. You can compare this somewhat to how low-code/no-code platforms work. Developers work with larger building blocks and can thus take bigger steps and thus have applications in production faster. Platform teams within organizations especially like this way of working, according to Duffy.
Of course, Pulumi can only offer this open-source development environment if it also generates revenue somewhere. To that end, it has developed Pulumi Service. This is a SaaS solution that gives you, in Duffy’s words, “visibility and control” over what you have built on the IaC platform. Duffy compares the relationship between the open-source Pulumi Universal Infrastructure-as-Code and the commercial Pulumi Service to the relationship between Git and GitHub.
Pulumi Service is the management layer on top of the open-source platform. Among other things, you can track and see here exactly who made what changes to the code and at what point in time. It also adds the necessary things that are particularly indispensable within larger organizations. One example is IAM. You can use Active Directory, Okta or whatever tool you need for this. As long as it supports SAML 2.0.
Pulumi Service also offers an interesting addition in the area of Secrets Management. The moment you save anything that looks like a password, Pulumi notifies/warns you to encrypt this data. A final component of Pulumi Service that we want to highlight here is the Automation API. This allows developers to use IaC as a library, no longer just as a CLI. You still install Pulumi CLI when you use Automation API, but developers no longer have to interact with it. They deploy Automation API to do this for them, in the language of the underlying infrastructure. Especially in more complex rollouts, this should be of great benefit, according to Duffy.
We mentioned above that Pulumi Service is a SaaS solution. This does not mean that you cannot deploy it if you cannot, may not or do not want to use general cloud services. Pulumi also offers a so-called self-hosted version. You can install this in your own environment, including air-gapped environments. The promise is that you have and maintain full control over your data. In other words, databases and object stores are in your own network. There is also no communication from this version with the outside world, including Pulumi itself.
Can Pulumi provide depth in addition to breadth?
Basically, Pulumi allows organizations to get started with Infrastructure-as-Code at a higher level of abstraction. That should make IaC more accessible to more people, is the idea. Pulumi unlocks the full range of IaC offerings from all cloud service providers. According to Duffy, this feature makes it possible to set up organizations’ entire hybrid infrastructure for applications from a single solution. Other players in this segment often focus on a single component, such as serverless, containers and Kubernetes.
So Pulumi focusses on breadth, much more so than other players, with HashiCorp’s Terraform as the main competitor. It also offers migration services for other IaC platforms. This allows you to convert infrastructure built with Terraform, AWS CloudFormation, Azure Resource Manager and Kubernetes YAML to the language of your choice within Pulumi. According to Duffy, it doesn’t matter how this existing infrastructure is provisioned.
In the end, the breadth of Pulumi’s offering does raise the question to what extent this comes at the expense of the depth of the offering. Compromises have to be made somewhere. We saw this above with the templates, which offer only the most common patterns of the most common cloud architectures. In other words, you can do a lot with Pulumi, but not everything.
Duffy also readily acknowledges that Pulumi’s initial focus was primarily on breadth. It was mainly about offering the building blocks. As time went on, Pulumi also went into more and more depth. This was particularly possible because they developed almost everything together with customers. Then you can go very deep together. Continuously keeping up with everything that happens and changes in the world of IaC, however, is certainly a challenge, he recognizes. It’s something that Pulumi needs to be very aware of.
Things could go fast for Pulumi
We see in the rise of Pulumi’s IaC platform a parallel with low-code and no-code. That also once started as an idea to develop at a higher abstraction layer. For this, it also accepted some comprises to provide the larger building blocks. Meanwhile, low-code is becoming more widely accepted, not only for developing fringe applications, but also for developing core applications. In other words, it has become established.
A big difference between the rise of low-code and Pulumi’s ‘low-code IaC’ is that things move much faster now. Whereas low-code took about 10 years to really land, Pulumi’s offering (and others like it) will achieve this much more quickly. The more or less unexpected Series B investment round last year is a first proof of this. Pulumi is currently still a relatively modest player in the Infrastructure-as-Code market, but that could be about to change very quickly.
Read also: At the most recent edition of HashiConf, we heard that HashiCorp is also moving toward better visibility and more self-service with Terraform.