What are DevOps metrics?
You can use DevOps metrics to measure how well your development and operations practices are working. These measurements help you understand your team’s collaboration, delivery speed, and software quality.
When you track the right metrics, you’ll spot where work gets stuck, improve your processes, and align your technical work with your business needs.
Good metrics help your team deliver software faster with fewer bugs. But you need to be careful with DevOps metrics. Problems happen when teams misunderstand or misuse these measurements. For example, if you focus only on how often you deploy code without looking at quality, you might push unstable releases. And if you use metrics to compare teams or individuals, you can create unhealthy competition where people try to improve their numbers instead of making real progress.
Let’s look at the most valuable metrics and measurement approaches. I’ll share practical tips to help you measure DevOps work in ways that benefit your organization, the teams and individuals doing the work, and the people using the software.
This is part of a series of articles about DevOps.
Key categories of DevOps metrics
Process metrics
These metrics show how effectively your team delivers value to customers. They identify inefficient process elements like queues, waits, and failure demand. Here are the key ones you’ll want to track:
- Lead time for changes: Measures how long it takes for your code to go from commit to production. Shorter lead times indicate a more efficient delivery pipeline.
- Change failure rate: Tells you what percentage of your changes cause problems in production. This helps you understand how stable your processes are.
- Deployment frequency: Measures how often code is deployed to production. More frequent deployments reduce risk and bring earlier feedback, which is the reason to work in a more agile, efficient way.
Performance metrics
Performance metrics focus on the operational performance of systems and applications. Key metrics in this category include:
- Application latency: Measures how quickly your application responds to requests. The faster your response times, the better experience your users have.
- Throughput: Tracks the number of transactions or requests an application can handle within a specific timeframe. This helps you plan for growth and understand your system’s limits.
- System uptime: Tells you what percentage of time your system is available. The higher this number, the more reliable your service is for users.
Quality metrics
Quality metrics evaluate how well your software meets its requirements:
- Defect density: Shows you how many bugs exist per unit of code. This helps you track your code quality over time and highlights components with high defect rates.
- Customer-reported incidents: Tell you how many problems your users actually experience. This gives you a real-world user’s perspective of your software’s quality.
- Automated test coverage: Measures the percentage of code that is tested automatically. Higher coverage often leads to more reliable software.
Operational metrics
Operational metrics measure the efficiency and reliability of IT operations. Key examples include:
- Time to recover: Measures how quickly you can fix problems when they happen. A lower recovery time indicates better incident response practices and an effective pipeline to deploy fixes.
- Time between failures: Shows you how long your system typically runs before having problems. This helps you understand your system’s stability and may be more important to users than system uptime.
- Infrastructure utilization: Monitors the usage of servers, networks, and other resources. This lets you scale efficiently and avoid waste.
Cultural and collaboration metrics
Cultural and collaboration metrics assess the human aspects of DevOps, focusing on culture, teamwork, and organizational alignment. Metrics in this category include:
- Employee satisfaction: Surveys to gauge team morale and wellbeing, reflecting the health of the work environment. Happy teams deliver better results.
- Collaboration frequency: Shows how often your teams work together. This helps you track whether you’re breaking down silos effectively.
- Training and skill development: Measures the time or resources spent on upskilling team members, ensuring continuous learning and growth.
Key DevOps measurement frameworks
1. DORA metrics
The DORA metrics are widely regarded as key indicators of DevOps performance. These metrics focus on software delivery and operational efficiency, helping organizations benchmark their capabilities. The four primary DORA metrics are:
- Deployment frequency: Measures how often code is deployed to production. Frequent deployments are signs of working in small low-risk batches and indicate an efficient delivery pipeline with high automation and lightweight change approvals..
- Lead time for changes: Tracks how long it takes for you code to get from commit to production. Short lead times signify efficient pipelines and rapid iteration.
- Change failure rate: Measures the percentage of changes that cause production incidents. A lower rate suggests better testing practices and more stable delivery process.
- Failed deployment recovery time: Tracks how long it takes to get back into a good state after a bad deployment. Faster recovery indicate a robust incident response process and deployment pipeline for fixes.
Learn more about DORA metrics.
2. The SPACE framework
The SPACE framework, developed by researchers at GitHub, the University of Victoria, and Microsoft SPACE groups metrics into a customizable framework of five dimensions to evaluate developer productivity and wellbeing holistically:
- Satisfaction and wellbeing: Assesses factors like developer happiness, stress levels, and work-life balance to ensure teams are motivated and productive.
- Performance: Measures the quality and impact of a system or process, such the number of defects, system health, and feature usage, focusing on tangible results.
- Activity: Tracks actions like code commits, pull requests, and deployments, providing a view of work volume.
- Collaboration and communication: Evaluates team interactions, such as code review participation or documentation quality, reflecting the health of team dynamics.
- Efficiency and flow: Measures how smoothly work progresses through the pipeline, using metrics like the number of handoffs in a process or how often developers are interrupted.
Learn more about the SPACE framework.
3. MONK metrics
The MONK metrics provide a way for platform teams to measure the success of a Platform Engineering initiative. It combines 3 industry-standard metrics for internal developer platforms and adds outcome-focused measures that reflect the organization’s goals:
- Market share: Measures the size of the internal market, the number of internal users who could use the platform, and how many choose to do so.
- Onboarding time: Measures how long it takes for a new developer to get their first change to production and how long it takes a team to adopt the platform.
- Net Promoter Score: Assesses user satisfaction with the platform in terms of whether they would recommend that other teams adopt it.
- Key customer metrics: Contextual measures to track whether the platform is meeting its adoption goals. For example, an organization that adopts Platform Engineering to increase software delivery performance would measure the DORA metrics.
MONK metrics balance a benchmarkable set of standard metrics with goal-based measures that make sure your organization is getting the intended benefits of Platform Engineering.
4. Developer experience metrics
Developer experience (DevEx) metrics provide a three-dimensional framework to assess and enhance the productivity and satisfaction of development teams. These dimensions are:
- Feedback loops: Detects delays in processes that provide information to developers, like code reviews, build times, and test execution times. Shorter feedback loops help developers identify and address issues promptly, maintaining development momentum.
- Cognitive load: Measures the mental effort required to perform tasks, considering factors such as tool complexity, system architecture, and documentation clarity. Reducing cognitive load allows developers to focus more on coding and problem-solving.
- Flow state: Assesses the ability of developers to work without interruptions, achieving deep focus. Maintaining flow state leads to higher productivity and job satisfaction.
Learn more about DevEx metrics
5. Continuous Delivery statements
Continuous Delivery (CD) statements provide a qualitative assessment that helps teams evaluate their software delivery performance. Unlike specific technical practices, these statements focus on the capabilities that such practices should bring to a team. There are four key CD statements:
- Software is always deployable: Ensuring software can be deployed at all times needs early detection of functional bugs and continuous maintenance of quality attributes like security and performance. Techniques such as feature toggles or keystoning allow the software to be deployed without exposing unfinished features to users.
- Prioritize work to keep software deployable: Issues that prevent deployment should take precedence over new feature development. Addressing blockers like deployment-preventing bugs, unexpected component dependencies, or pipeline problems—ensures that critical problems can be fixed promptly.
- Fast, automated feedback for every change: Providing rapid, automated feedback to the entire team for each software change helps them quickly respond to issues that hinder deployments. Where a bug escapes, tests should be added to detect them automatically and prevent them re-occurring.
- On-demand automated deployments for any version and environment: On-demand deployments let any software version be deployed to any environment at the push of a button. This is achieved with fast, automated, repeatable, and reliable deployment processes. High-performing organizations often apply the same level of automation to infrastructure as to application deployments.
Learn more about Continuous Delivery statements
What are the dangers of DevOps metrics misuse?
Unintended side effects can undermine the value of your measurement system. Some organizations treat software delivery performance as the goal, but high performance is only relevant if it helps the organization attain its broader goals in delivering value.
An inappropriate measurement system has many downsides. The most significant damage caused by misusing metrics occurs when teams or individuals mistake a measurement as more important than the organization’s mission.
Here are the three most common mistakes in DevOps performance measurement:
- Elevation: Occurs when a metric gets reported too far up in an organization. Rather than use metrics to inform decisions, teams and individuals over-emphasize tasks that move the number. This results in sub-optimization, where outcomes worsen despite the elevated metric improvement.
- Aggregation: Happens when you measure against the wrong scale. A useful metric for measuring team and application performance may be misleading or damaging when applied with a different level of aggregation, like an individual or a whole department.
- Misuse: You misuse a metric when you change the purpose of the measurement. The classic misuse example is taking measures meant for continuous improvement and using them to compare teams. When you misuse metrics, you create the conditions where teams can game measurement systems. This destroys the value of the metrics and damages performance at the organizational level.
Tips for achieving a healthy DevOps measurement system
1. Use metrics for navigation
You might find it helpful to think about metrics as navigation aids. Imagine taking a long journey. You need a mix of different signals to arrive at your destination. At your first waypoint, like a train station, you need information to tell you which platform to use and if trains are running on time.
As you continue your journey, you use short-term signals and discard them. Long-term signals are more stable, like your eventual destination and stations where you change trains.
This metaphor allows you to assign one of 4 categories to each metric you use.
Destination metrics
Destination metrics show whether you’re achieving your organization’s goal and purpose. This is often financial, but not always.
While destination metrics are stable over the long term, they aren’t necessarily permanent. They tend to reflect the organization’s mission, so they may change if the mission changes.
Waypoint metrics
Waypoint metrics tell you if you’re heading in the right direction. Each time you pass a waypoint, you reduce uncertainty about the outcome. The metrics you choose as waypoints will respond earlier than destination metrics. Yet, the relationship you established gives you confidence they’ll result in the outcomes you want.
For example, you might use opportunities in the pipeline to predict future revenue. You must be cautious with goals for waypoint metrics. Having more opportunities is only helpful if you maintain some probability they’ll become revenue. Adding many low-probability opportunities will damage your destination metrics, as you’ll waste time on them rather than likelier sales.
Signpost metrics
Signpost metrics provide concrete signals for smaller groups within the organization. They provide local goals that connect the group’s work to the organization’s efforts. These metrics will differ for each team and will have a recognizable local flavor.
A contact center team will have very different measurements from a sales or finance team. When a team improves performance against a signpost metric, it should connect to organizational performance. That doesn’t mean you should create goals for these metrics, as this can have counter-productive effects on outcomes.
Tracking metrics
A tracking metric is local and often temporary. You would use a tracking metric to narrow the focus on part of your role and improve it. For example, if your build times were causing a bottleneck, you’d measure them while you made improvements. Once the constraint moves, you’ll stop measuring build times and focus on the next one.
It’s important to keep trimming tracking metrics to avoid overloading yourself with information. Removing them from your dashboard makes room for new metrics, though you could leave an automated alarm to tell you if things slip.
You should stop measuring at the tracking level when their value no longer justifies the effort to collect them.
2. Ensure metrics are healthy, not harmful
The individuals and interactions involved in building software create a complex and dynamic system, so it’s challenging to measure. You need your best efforts to operate metrics that are healthy, not harmful.
You need to:
- Collect and use the data for its intended purpose
- Balance the need for fast feedback and early indicators with the importance of tracking real outcomes
- Constantly adjust to keep things healthy
Wherever possible, let teams collect measurements for their own use. By limiting the extent of publication for the metrics, you’ll have more control over how they get used. The metrics should help you to identify improvement opportunities. You should not use metrics to compare teams or rate individual performance.
As a team makes progress, stakeholders and customers will notice the improvement without needing the numbers.
Our white paper on measuring Continuous Delivery and DevOps provides general guidance on metric design and explains types and levels of measurement. You can also learn about the common mistakes in DevOps metrics on our blog.
3. Go beyond readily available numeric metrics
In Out of the Crisis, William Edwards Deming warned of the dangers of depending only on numbers to manage work.
Management by use only of visible figures, with little or no consideration of figures that are unknown or unknowable.
You shouldn’t rule out metrics that are hard to get or include metrics just because your existing tools make them easy to find. Select measurements that give you a real sign of progress and work out how to report on them.
It takes more than numbers to be successful in DevOps. Alongside a healthy set of metrics, you must have a strong sense of purpose for your organization and its products and services. You’ll also need to look beyond short-term outcomes in your search for success over the longer term.
More reading
These resources will help you find out more about DevOps metrics:
- Measuring Continuous Delivery and DevOps (white paper)
- Common mistakes in DevOps metrics
- The new DevOps performance clusters
- Out of the Crisis by W. Edwards Deming (1982)
Help us continuously improve
Please let us know if you have any feedback about this page.
