The DevOps engineer's handbook The DevOps engineer's handbook

Measuring DevOps with the SPACE framework

The SPACE framework focuses on the deep relationship between satisfaction, well-being, and productivity. It uses a multi-dimensional measurement system to help you design metrics for the reality of individuals working in teams. You can apply the template to a focused area you want to improve or the whole organization.

If you use a measurement system that prioritizes individual productivity, it discourages people from helping others. Equally, if someone becomes subverted entirely to the team’s needs, they can no longer complete their own work.

The 5 dimensions of SPACE

The SPACE framework includes 5 dimensions:

  • Satisfaction and wellbeing
  • Performance
  • Activity
  • Communication and collaboration
  • Efficiency and flow

You should use a mix of measures from the 5 dimensions, including instrumented and perceptual measurements. Perceptual data is the only way to determine how people feel, which is a crucial driver of productivity.

We explain the dimensions below with some ideas for data you could capture. The goal isn’t to put all these metrics in place but to choose measurements based on your circumstances.

Satisfaction and wellbeing

Satisfaction captures how fulfilled people feel with their work, team, tools, and culture. Wellbeing captures how healthy and happy they are.

This dimension often predicts future performance. People rate their satisfaction and wellbeing lower before their productivity falls.

System data:

  • Retention - how many people stay on a team and with the organization

Survey questions:

  • How satisfied are you with other employees?
  • Would you recommend your team to other people?
  • Do you have the tools you need to get your job done?
  • How would you rate your energy and enthusiasm for work?
  • How satisfied are you with the software delivery system?

Performance

The performance dimension captures the outcome of a system or process.

System data:

  • Quality - how many defects you have and the ongoing system health and reliability
  • Impact - feature usage, cost reduction, and business won and kept

Survey questions:

  • Customer satisfaction with a feature, the product, and your organization

Activity

Activity metrics track the work done, like how many bugs you’ve fixed.

Never use activity metrics in isolation. Increased activity does not necessarily mean improved outcomes.

System data:

  • Number of work items, commits, pull requests, builds, and deployments
  • Number of incidents and issues and their severity

Communication and collaboration

Team communication and collaboration often come at a cost to individual productivity. Yet, collaboration drives higher team performance.

System data:

  • Speed of integration (for example, if the team uses pull requests, how long does it take to review and merge them)
  • Presence and quality of documentation
  • Onboarding speed (how long before a new developer has code in production)

Survey questions:

  • How would you rate the quality of team meetings?

Efficiency and flow

Individuals need solid focus time to get into a flow state and stay productive. This dimension refers to the flow of work and information.

System data:

  • Number of hand-offs in a process
  • Whole value-stream cycle time
  • Amount of value-adding time in the value stream
  • Amount of wait time in the value stream

Survey questions:

  • Can you regularly get uninterrupted flow time?
  • How often are you interrupted?
  • How much time do you spend on interruptions?

Assembling a set of metrics

The SPACE framework doesn’t need every metric measured all the time. You should select a mix of instrumented and perceptual measures across at least 3 dimensions. Over time, you can adjust by adding and removing metrics to nudge behavior and communicate what the organization values. You can use the metric selection principles to reduce unintended side effects.

Keep individual data private and only share aggregated values at the team and organization levels. Don’t ask survey questions that compromise anonymity, as it limits how honestly people can answer. Even asking for a job title or department can discourage openness, especially for people in unusual roles or small teams.

Watch for external forces that might influence your data. Activity metrics often drop when a team or organization invests in the future. Attending training and conferences or when people take holidays, for example.

You should also pay attention to bias in your system, which you should attempt to uncover in the data. Look out for skewed code reviews or longer waits for pull requests for specific team members, as these signal biases you need to tackle. You can use survey responses to check for this, too.

Personality differences can impact survey responses. Asking people to compare their feelings to a previous period can help reduce individual measurement bias. For example, here are 2 ways to ask for the same information. The second question asks the individual to compare to their previous experience:

  1. How many interruptions do you get? Not many, a few, or a lot?
  2. Compared to last week, how many interruptions did you get? Fewer, about the same, or more?

Make sure you don’t discourage vital but invisible work. Focusing too much on individual productivity or instrumented data will discourage teamwork, like teaching others how to make changes in a complex system. If you set individual focus on personal productivity, like the number of commits, essential teamwork will not happen.

The SPACE framework balances the natural tension between individual, team, and organization performance. Other productivity measurements that hide these tensions result in sub-optimal performance.

The table below shows the SPACE framework matrix, with an example metric in each category. The metrics you choose can differ, and it’s important to test the broader questions you want to answer about your system.

SPACEIndividualTeamSystem
Satisfaction and wellbeingDeveloper satisfactionRetentionSatisfaction with deployment pipeline
PerformanceCode review velocityCode review velocityCustomer satisfaction
ActivityTime spent coding (vs other activity)Features completedDeployment frequency
Communication and collaborationQuality of meetingsDiscoverability of informationQuality of documentation
Efficiency and flowAmount of flow timeNumber of handoffsFlow of work through the system

Remember, you only need a mix of instrumented and perceptual data across 3 or more dimensions. You don’t have to fill in the whole matrix!

Summary

The SPACE framework prompts you to design metric systems. By using a combination of metrics from several dimensions, you can balance the natural tensions in real workplaces.

Further reading

Help us continuously improve

Please let us know if you have any feedback about this page.

Send feedback

Categories:

Next article
CD statements