The DevOps engineer's handbook The DevOps engineer's handbook

Platform Engineering

Platform Engineering is a team structure you can use to manage complexity. A platform team creates a self-service internal developer platform (IDP). The IDP helps developers manage deployment pipelines, infrastructure, and operations tasks.

There are some essential qualities of Platform Engineering done well:

  • Use of the IDP isn’t mandatory
  • The IDP runs like a product, with developers treated as potential customers
  • You measure the IDP’s impact often

Platform Engineering is an example of Team Topologies in action, with the platform team helping many stream-aligned teams deliver value.

Platform Engineering is part of DevOps

Platform Engineering doesn’t replace DevOps. Instead, it tackles common problems organizations experience as they grow. Platform Engineering helps you to get the benefits of DevOps at scale. An internal developer platform (IDP) helps stream-aligned teams focus on delivering valuable features alongside running their software in Production.

An IDP addresses one or more shared concerns, such as:

  • Builds
  • Test automation
  • Deployments
  • Infrastructure
  • Monitoring
  • Maintenance
  • Cloud and cost management
  • Tool cost containment and supplier management

This is a long list of ways to address developer burnout with an IDP. Selecting a small set of big-impact areas is essential, rather than trying to do it all. One of the anti-patterns of Platform Engineering is moving all the pain into the platform team.

Platform teams benefit from the DevOps capabilities for culture, leadership, lean product management, and technical practices. You often find DevOps and Platform Engineering together. The Puppet State of DevOps Report found that high-performing DevOps organizations are more likely to use Platform Engineering.

Category% with platform engineering
Low8%
Mid25%
High48%

DevOps applies to all organizations that depend on software delivery and operational performance. Platform Engineering solves particular problems that can constrain performance, especially at scale.

Why Platform Engineering exists

Several common problems emerge when DevOps organizations scale:

  • Developers face increasing workloads
  • More demand from other teams
  • Developers need to learn new skills, technologies, and techniques

This can cause a high burden on developers, and the increased complexity can lead to cognitive overload.

Where an organization still needs to adopt DevOps, scaling results in larger work batches and slower releases. Adding more developers fails to increase the flow of valuable features. This is worse than the problems faced in DevOps.

If you recognize the DevOps scaling problem, consider a Platform Engineering approach. You can use our Platform Engineering decision flow to help.

How Platform Engineering fits into DevOps history

There are 3 recognizable movements to DevOps adoption. Organizations evolving their software process move through these stages at varying speeds. An organization that tries to fast-track DevOps and not take an evolutionary approach may have troublesome outcomes.

The 3 stages are:

  • Divided: Pre-DevOps, there are separate silos for development and operations with conflicting goals
  • Aligned: Aligned Goals and objectives to remove conflict between teams
  • Combined: Removing metaphorical walls allows cross-functional teams to build and run software

Platform Engineering exists to solve problems that emerge in the combined phase of DevOps adoption.

Divided

You may have worked in an organization where a developer’s job was ‘done’ after software passed all its tests. Someone else packaged, deployed, and operated the software. This divide was deeper than role definitions. Management would reward developers for delivering more features in less time and operations teams for stability.

The misalignment of goals kept development and operations in constant conflict. As a developer, you felt blocked by the need to wait for maintenance windows. Some technical decisions (like a choice of database technology) blocked progress if operations teams refused to support it. Similarly, management held operations teams responsible for any instability developers introduced.

The first stage of DevOps is to solve the alignment problem. Giving both teams shared goals removes the barriers preventing team collaboration.

Aligned

Development and operations teams must share the goal of regular and stable feature releases. This removes conflict and encourages collaboration.

Collaboration can start small. A developer and system admin reviewing error logs together after a deployment can be an excellent first step.

When developers see the error logs, they realize there’s value in improving the information in error messages. The feedback loop is complete, benefiting their software’s observability.

Operations monitoring dashboards will expand to include application and business metrics alongside infrastructure metrics.

A positive side-effect is everyone involved increases their knowledge beyond their previous silos. As understanding and empathy improve, everyone makes different decisions that are mutually beneficial. This leads to better outcomes for the software and the organization.

Combined

With DevOps well-established, the concept of a single team building and running software emerges. In some organizations, this meant moving operations experts into the development teams. In others, it involved developers taking on full responsibility, often called ‘you build it, you run it.’

There are 2 ways to be successful at ‘you build it, you run it’:

  1. Constrain design to ensure it stays simple, even as it scales
  2. Keep teams staffed with specialists who can manage more complex setups

Many teams had a complex setup but lacked knowledge across all members to share the workload.

Platform Engineering is the solution for organizations at odds with the success criteria. Often, that happens as teams and organizations scale up. Complexity increases as the number of teams grows. Workflows and toolchains become fragmented, and operations knowledge drops as more developers join the organization.

Who should use Platform Engineering

You should now understand the background behind the problems Platform Engineering can solve:

  1. Development teams overwhelmed by infrastructure and operations tasks
  2. Fragmentation due to a lack of alignment between teams

If you want to know if Platform Engineering would work for your organization, see our decision guide.

Getting the benefits of Platform Engineering

You must start your Platform Engineering journey by understanding problems experienced by your developers. To create a successful IDP, you must choose the tech stacks and workflows the platform should support based on the positive impact it could have.

You build a successful IDP with a thoughtful selection of hand-picked tools, not by buying 1 general-purpose platform. Platform Engineers design ‘golden pathway’ workflows and simplify deployments and operations.

A golden pathway is the choices the platform team decides to support. The IDP simplifies each pathway to reduce cognitive burden on developers. It also encourages others to adopt the IDP and the good decisions it represents.

For example, rather than development teams maintaining hundreds of infrastructure configuration lines in differently formatted files (JSON, YAML, XML), you create 1 internal format with a few key fields. This change makes it easier for developers to manage and aligns teams with sharing decisions about items not part of the internal format.

The key attributes of successful Platform Engineering efforts are:

  1. Having a platform as a product approach
  2. Treating developers as the customer
  3. Avoiding mandated adoption (the IDP should win market share)
  4. Making the platform self-service, with support
  5. Solving real developer problems, not inventing new tech stacks

As Platform Engineering teams form with highly skilled team members, a danger is they might think they know better than developers. When they approach the task with this mindset, they create something developers avoid using. Managers can compound this problem if they mandate the use of the platform. That signals managers also believe Platform Engineers are better than the developers.

Your IDP should compete for a share of the internal developer market. You can use the MONK metrics to guide your Platform Engineering efforts and DevEx metrics to track developer productivity.

Platforms aren’t a shared service

Traditional operations teams are a shared service. Developers raise tickets and wait for someone to do the work. For example, you’d submit a ticket to refresh the staging database. This request would sit in a queue until an operations team member picked it up and ran the steps to copy and anonymize data from a Production backup.

This is different from how a Platform Engineering team works. Instead of doing the work for you, the internal platform enables you to do the work. Refreshing the staging database would be available as an automated push-button action. Most self-service operations would trigger through an API call, allowing you to add tasks to automated workflows.

The only time you’d raise a ticket would be to get support using the platform. The platform team can improve documentation and simplify the product to reduce support requests. They can also network teams together to help each other use the platform.

Internal developer platforms are a product

The Puppet State of Platform Engineering Report highlights that Platform Engineering fails when the team doesn’t treat the platform like a product. Without a product focus, platforms don’t solve real problems. Having direct lines of communication between Platform Engineers and the users of the internal developer platform is a crucial sign the organization is taking the right approach.

Merely shifting cognitive load onto a Platform Engineering team doesn’t resolve a problem and may worsen things. High churn in your platform team can negatively impact stream-aligned teams using the IDP.

The Platform Engineering team should focus support on fewer scenarios than many teams have. One of Platform Engineering’s goals is encouraging new development work to align with tighter choices around technology and workflows. Less variety, more focus.

Platform Engineering shouldn’t run as a ticketed queue of work. The platform should allow developers to self-service their typical needs. The platform team can provide documentation, training, and support to help teams deploy and run software themselves.

Platform Engineering brings some design to a critical area of your organization. You don’t have to stop with one well-designed team. Translating your organization’s purpose into clear team structures can improve the flow of value to customers. The concept of a ‘platform team’ in Team Topologies doesn’t mean a Platform Engineering team creating an IDP. Your organization will have other chances to create platforms that improve performance in other areas.

You can design your organization using the Team Topologies team and interaction types. You’ll find them in the Team Topologies book in our DevOps reading list.

Internal developer platforms should be open-source

One of the benefits of creating a product aimed at developers is that they can contribute to it. You can make your platform internally open-source or inner-sourced, so developers can submit improvements. It’s a great way to ensure the platform solves real customer problems.

To get the most out of inner sourcing, make sure you:

  • Publish clear contribution guidelines
  • Keep documentation up to date
  • Guide contributors to solve problems with appropriate abstraction
  • Help contributors to use existing features to solve their problems

You need to decide on the right stage to introduce inner sourcing, usually after reaching broad adoption across several teams. InnerSource Commons has more information about inner-sourcing governance and patterns for successful adoption.

It’s not just about developers

Developer experience (DX) is crucial for solid software delivery performance, but you must balance it with the needs of others. There has been a rise in dysfunctional hyperfocus on developer experience to the detriment of the software they produce.

You can create great developer experiences that are bad for customers and damage the organization. Platform Engineering provides a clear way to improve developer experience while maintaining good customer and business outcomes. Golden pathways allow the right choice to be the easy choice for developers.

Happy and engaged developers create better software if you treat their experience as a part of a broader system.

The platform team can also perform valuable bridging between technical teams and other business areas, like governance, risk, and compliance.

Understanding business needs and helping development teams meet them without red tape adds value beyond technical concerns. The platform team must be exceptional at communication to keep DevOps collaboration alive.

Modern organizations strive to create the right environment for people to do their best work. This is a triple-win, benefiting employees, customers, and the business. Learn more about the benefits of employee engagement in the Gallup Q12 summary.

Summary

Platform Engineering is an example of Team Topologies in action. The team’s IDP should reduce complexity and cognitive overload by focusing on solving developer pain points.

A successful IDP is more likely to combine best-in-class tools rather than all-in-one DevOps platforms. Best-in-class tools include:

Platform Engineering aims to achieve high developer satisfaction without harming the experience for customers or the organization.

Next, you can learn when to adopt Platform Engineering and the patterns and anti-patterns of Platform Engineering.

Further reading

  • Team Topologies by Matthew Skelton and Manuel Pais (2019)
  • Adopting InnerSource by Danese Cooper and Klaas-Jan Stol (2018)

Help us continuously improve

Please let us know if you have any feedback about this page.

Send feedback

Categories:

Next article
Suitability