Platform engineering leads to so-called golden paths in the form of assured outcomes in software development projects. This software engineering construct and methodology is designed to provide developer infrastructure with self-service toolchains, services and processes encapsulated inside a so-called Internal Developer Platform (IDP). But how shiny is the golden path, are there mucky potholes along the way… and do we need special walking boots?
Martin Reynolds, field CTO at software delivery lifecycle automation company Harness, thinks the streets on this particular route to software development are – if not paved with gold – then at least tarmaced over with good intentions and potential efficiencies.
But why has platform engineering come about? It may be because software delivery is being redefined as the complexity of modern workflows outpaces the agility of traditional DevOps models. Platform engineering has emerged as a key factor in shaping the way organisations build, deploy and manage software at scale.
“This trend is moving at a breakneck pace – Gartner expects that by 2026, four out of five large software engineering organisations will have established platform engineering teams, up from just 45% in 2022. Such rapid adoption is being driven by a multitude of factors, including the overwhelming complexity of multicloud architectures and the toolchains required to build, deploy and run software across them, as well as growing regulatory pressure,” said Reynolds, during a technical briefing in London this week.
We might also suggest that because developers are problem solvers by nature, they have typically tried to solve this complexity by building their own internal tools, processes and workarounds. With every team taking its own approach, this leads to silos, duplicated effort and inconsistencies that ultimately hinder the innovation drive. Again, platform engineering can (potentially) act as an antidote.
A pathway to innovation
Reynolds suggests that platform engineering is the “ideal solution to the conundrum”, providing developers with access to infrastructure, tooling and frameworks as reusable products, supported by “golden paths” that make it easier to do the right thing and more difficult to do the wrong thing – thereby accelerating velocity.
“The key to success here is realising that the journey is as much about culture and process as it is about technology. With that in mind, there are two key areas that platform engineering teams should focus on to create these golden paths in their Internal Developer Platform (IDP),” said Reynolds, who lays this core pair of truths out as follows…
1. Modernise DevOps: Provide self-serve capabilities that allow teams to instantly provision environments, pipelines and infrastructure to meet the requirements of their build. Automation is key to making this possible, so infrastructure and pipeline configuration settings must be built into the IDP, eliminating the need for developers to apply these manually. As an added benefit, following these standard infrastructure configurations makes it easier for engineering teams to optimise cloud costs by eliminating wasted resource allocation.
2. Embed quality and resilience: Use policy-as-code practices to apply automated testing, security scanning and observability, removing the need for manual effort that can otherwise lead to blind spots. These capabilities maintain compliance by default, ensuring that new builds meet the minimum requirements for performance, reliability and security before being released to production. Modern techniques such as feature flagging, automated rollbacks and chaos engineering make these practices even more effective when made available in an IDP.
Platform engineering 2.0
“Many organisations have focused their IDPs on supporting developers with the initial deployment phase, delivering capabilities such as infrastructure as code (IaC), automated pipeline creation and security scans. The aim has been to enable developers to self-serve everything they need to get their code into production quickly and safely. But that’s only half the story,” explained Reynolds.
But, he says, the value of platform engineering really shines in day 2 operations, offering the capabilities developers need to manage and maintain their applications in the wild. From experiences drawn within the Harness team itself, Reynolds says that this is where many existing IDPs are falling short. He thinks that in order to build a fully mature IDP, platform engineering teams should equip their developers with standardised observability frameworks that support performance monitoring and optimisation, patching and compliance maintenance, incident response and automated rollback capabilities.
Driving continuous improvement
“Building the platform is just the beginning. Any investment in technology must be justified, which is why it is crucial for platform engineering teams also to track metrics that enable them to demonstrate tangible business outcomes and identify areas for improvement,” said Reynolds. “Technical metrics, such as DORA, that track developer performance through aspects like frequency of new releases, failure rates and the time it takes to recover from a production incident, are, of course, important. But engineering leaders need to go further and consider user adoption rates. They should measure how often developers self-serve, which templates and pipelines they use and how policy compliance is trending.”
To take this whole discussion even further, Reynolds and team say that platform engineering teams should examine how they can utilise AI to correlate technical metrics and user adoption rates, thereby revealing a more comprehensive picture of the impact of their IDP.
These insights will be invaluable to identifying the biggest priorities for future investments, helping to scale the platform to new heights and keeping developers at the forefront of innovation.