The DORA metrics were developed by a Google team after more than five years of research. Today, these metrics are used by development teams across the world to measure performance.
You’re going to find this article useful if you’re still not sure what these metrics are and how they can help you improve your team’s performance. Here’s what you’ll learn:
- What are DORA metrics?
- Importance of DORA metrics
- 5 Key DORA Metrics to improve performance
- Deployment Frequency
- Change Failure Rate
- Average Lead Time for Change
- Mean Time to Recovery
Let’s jump right in.
What are DORA Metrics?
The acronym DORA comes from the research program DevOps Research and Assessment (DORA), under Google’s Alphabet Inc. DORA metrics are the product of six years of research to measure how well a company ships code. More specifically, they measure the performance of your software development process and teams.
In the modern software development landscape, you must react fast enough to meet the evolving needs of your customers. And while your reaction speed matters, it’s even more critical you offer stable services.
Because of that, you have to improve your DevOps teams constantly. DORA metrics give you objective data points to improve your products. They help you evaluate your software delivery team’s performance.
.jpg){kind=link}
The four performance metrics used to measure DevOps success are:
- Deployment frequency
- Change failure rate
- Average lead time for changes
- Time to restore service
These metrics give you valuable insights into how your engineering team is performing. They allow you continuously improve your DevOps team performance to get better outcomes. Depending on the results, you can then classify your company on a scale. The scale ranges from Low and Medium to High and Elite performers.
For example, Elite DevOps have a higher chance of achieving and exceeding performance goals.
At the end of the day, DORA metrics help you answer the question. Are we, as a team, doing better than we did last year?
DORA metrics have helped software development teams take a step into the future. Thanks to these metrics, you can create visibility and get useful factual data. That means you can use them as a benchmark to make better decisions. Using these data-driven insights will help you improve the effectiveness and efficiency of your operations.
DORA metrics have now become a standard in measuring DevOps performance. As a DevOps leader, these metrics will give you insights into key areas of growth. They’ll help you modernize, beat your competition and improve your operations.
Importance of DORA Metrics
One of the main reasons why DORA metrics came into being was to create a universally-accepted standard. That’s because every organization had its own performance metrics a few years ago.
As a result, you could not compare your organization’s performance to others in your industry. And it was difficult to identify trends, and you couldn’t create performance benchmarks. But, thanks to DORA metrics, you now have a standardized framework for measuring performance.
With these metrics in place, you can now measure how fast you ship code and how reliable the code is. Doing so gives you an idea of your DevOps team’s current performance. It also helps you identify areas you need to work on to improve the quality and speed of the software you deliver.
For DevOps or engineering leaders, DORA metrics will give you insights into your team’s performance. You can then make recommendations to engineering project management based on real data.
DORA metrics will show whether the team is meeting customer needs and expectations. That is, better values mean happier customers.
4 Key DORA Metrics to Improve Performance
Earlier, we outlined the four key metrics that can help you measure the performance of your DevOps teams. In this section, we’ll dig a little deeper into those metrics and what they represent.
- Deployment Frequency
Deployment Frequency (DF) refers to how often you deploy changes. It shows your speed and the team’s ability to meet continuous delivery goals.
While analyzing this metric, your team may fall into one of four categories. Check this out.
The team at DORA outlines the following baselines for DF:
- Low performers: Less than once semi-annually
- Medium performers: Once every month to six months
- High performers: Once every week to a month
- Elite performers: Several times every 24 hours
DF can show various things depending on your organization’s definition of team success. It could be anything from how often your release code to end users to how often you deploy code to production.
Regardless of what you measure, for your team to qualify as elite performers, they need to aim for regular deployments per day.
One of the best hacks to improve DF is for you to ship several small changes. This offers a couple of benefits, actually.
The first is that a high DF will reveal bottlenecks in your DevOps process or the complexity of a project. That can help your team improve as they get to learn and engage with the process more frequently. It also helps you see how to adjust the processes for faster deployment.
The second benefit of deploying smaller changes is it makes it easier to find and resolve bugs.
- Change Failure Rate
The change failure rate (CFR) is a percentage of how often the changes your team makes lead to a failure in production. Failure here includes rollbacks, degraded service, downtime, failed deployments, service incidents, unplanned outage, etc.
All failures, even those with quick fixes, contribute to the change failure rate.
From this, it’s clear that your change failure rate reflects code quality and stability. It shows you how effective your DevOps team is at applying changes. What’s more, your change failure rate helps leaders determine where to provide support.
Your CFR also helps answer one critical question. How much time is spent fixing problems instead of working on new code or projects?
To calculate your CFR, get the sum of all deployment failures and divide it by the sum of all successful deployments.
The range for CFR is featured in the image above. It includes Elite, High, Medium, and Low Performers.
CFR is calculated as a percentage, which is great because it gives you a more accurate picture of the failure rate of your team. That’s important because failures can increase as the team makes more changes.
For instance, if you’ve adopted CI/CD practices, you’ll probably deploy more changes than most other teams. And as you make more changes, the chances of seeing increased failures also go high.
Now, if you compare that number of failures to another team that deploys a few changes every week, you may get scared that your team is seeing too many failures. The CFR, therefore, levels the playing field, giving you an accurate rate depending on your velocity. It also makes it easier to compare your performance with other teams.
- Average Lead Time for Changes
The Lead Time for Change, or LTC, is the average time your team takes to go from a commit to production. It’s a great indicator of how agile your team is. It not only tells you the time it takes to apply changes but also how responsive your team is in meeting the evolving needs and demands of end users.
Moreover, it helps your DevOps and engineering leadership understand your team’s cycle time. They can then determine whether your team can handle a flood of requests on short notice.
Like DF, it helps determine or set the tempo or engineering velocity of shipping code in a company.
The Accelerate State of DevOps 2021 set the following as LTC performance benchmarks.
- Low Performers: Over six months
- Medium Performers: One to six months
- High Performers: One day to a week
- Elite Performers: Under one hour
The biggest benefit of using LTC is that it helps reveal poor DevOps processes. If it takes longer for commits to reach production, e.g., over a month, you’re dealing with process inefficiencies.
You’ll need to identify and streamline your processes to lower the LTC. You can introduce continuous integration and continuous deployment practices, for example.
We’ll advise you to monitor your LTC over time. Of course, you want the metric to improve over time. That would mean your team and processes are getting better.
- Mean Time to Recovery
The mean time to recovery (MTTR) or restore service is the average time it takes your team to resolve a disruption of service. This metric helps you evaluate two things. The first is how stable your code is. Two, how agile your team is when facing an unexpected challenge.
According to the State of DevOps report, these are the benchmarks for the mean time to recovery:
- Low Performers: More than six months
- Medium Performers: One day to a week
- High Performers: Under 24 hours
- Elite Performers: Under an hour
To lessen the blow on your value stream, there must be minimal to no downtimes. You can work on improving this by shipping smaller changes, as discussed earlier. Identifying and resolving issues is much easier when the change made is small. Using feature flags can also help you improve MTTR by quickly disabling the changes causing downtimes.
In Closing
DORA metrics provide the best way of measuring your team’s performance. However, you should always consider the context of all four metrics before making any changes.
For example, we’d discourage you from comparing something like the average lead time for change with other companies. What you want to do instead is compare your team’s performance over time.
Also, don’t obsess over improving these figures at the expense of the user experience. For example, it wouldn’t matter if you reduced the mean time to recovery by deploying quick fixes that are going to break down in no time.
Hopefully, this article has helped you understand the DORA metrics and how you can use them to monitor and improve your team’s performance.