How to measure the ROI of DevOps and what metrics to follow?

What DevOps is?

I am going to start with a quick reminder about DevOps, but to go further, here is an article to know everything about DevOps.

The DevOps methodology starts with an evolution of your internal organisation. Indeed the first thing about DevOps is the collaboration between developers and Ops. Thanks to better communication within your team and common objectives between Ops and Devs, you will:

• save time on incident resolution
• launch new features faster
• reduce risks through process automation
• significantly increase the satisfaction of your customers, and your developers at the same time.

Today, according to IDC, 20% of software development projects benefit from a DevOps approach. By 2021, this figure will increase to between 35% and 40%. It is no coincidence that so many IT teams are adopting the DevOps methodology for the project. Let’s see why and how to measure ROI to save human and material resources.

Why Should you measure DevOps ROI?

You probably already know the ROI formula:

But why should you measure it? ROI is the measure of the effectiveness of an investment in terms of profitability. Calculating ROI is an essential step before starting a new project. It allows you to plan your project, anticipate the cost and set objectives for the success of the project.

Calculating ROI will help you to see if your project is possible and start figuring out how to do it. But it is also a requirement if you want to get the budget from your management board. Would you imagine yourself going to the board to present your idea without any numbers to back up your project? You have to be able to prove to your company that the project you want to start will bring more value than the initial investment, otherwise, it will never be approved.

Even though some projects are easier to calculate and some others as DevOps are more complicated, it's an exercise you are going to have to go through. Let’s see what metrics you should measure and objectives that you should set.

What metrics to follow to measure DevOps ROI?

Downtime

According to a study conducted by IDC and AppDynamics, infrastructure dysfunction can represent a loss per hour of $8,580 to $686,250 depending on the size of the company. And on average, companies facing downtimes lose $163,674 per year.

In addition to this loss of money due to the lack of sales, human costs must also be taken into account. According to the same study, 35% of respondents said they had to spend 1 to 12 hours of working time to repair production incidents.

Well, it seems obvious that the first metric is the production incidents (server down, the path to payment down, ...). You should, therefore, calculate the average loss of revenue for your company for X minutes of downtime, then look at last year how long in total your software was down. And then add the total human time cost of incidents repair. It will be part of the Current value of Investment since it is going to increase the turnover thanks to a stable infrastructure. Be aware that having 0 downtime is difficult to reach. Even Google as an objective of 99,99% availability of the service. Set an objective of availability and leave this error margin in the calculation.

Productivity

The second metric is maybe harder to calculate. It is the productivity of the developers. The first point to highlight when a company switches to a DevOps organisation is the speed of production. According to a study, the DevOps leads to 30 times more deployments made for 60 times fewer incidents (downtime reduction), 440 times less time to deliver (more fluid development experience). It is therefore undeniable that DevOps will have an impact on your business. And speed is not at the expense of quality, quite the contrary! For one of our customers, the DevOps organisation has made it possible to have a log platform 99% available and to reduce the number of incidents per month by 42%.

This metric will give you an objective. For instance, have a continuous integration and deployment process must reduce the time between the merge and the production by 2 hours. If you have 3 developers that push feature in production every day. It is 6 hours per day that can be reallocated to produce even more new features. In fact, 43% of companies that are moving to DevOps do so to strengthen their time to market (according to the IDC report). This allows them to arrive at the right time in a new market, or to launch an innovative product at the right time before market shares are taken by competitors.

You have to ask yourself what you can do with all the time saved from faster deployment process and incidents time repair. Will this save you money on recruiting a developer? Will it allow you to create an innovative product and then gain market shares? Will it improve the work conditions for your developers and so reduce the employee turnover? This metric is specific to your company.

On the Cost of Investment side, you have to take into consideration the price of the DevOps agency that will help you or the recruitment of an Ops team for instance. As well as the price of new tools such as Gitlab, Gitlab CI, Docker, Kubernetes, Prometheus, etc.

Let’s summarise the calculation :

How to maximize your DevOps ROI?

Clear objectives

Before starting a project, you must define success. The objective to achieve must absolutely be measurable daily. This is necessary to clearly observe the impact of your actions on the objective. It must also be achievable within a given time and not be too ambitious with the time frame.

For example, one of our clients wanted to reduce the number of manual actions to only one click from the merge to the staging. We measured this KPI every week for 4 weeks to finally reach one click.

A very effective tool you can use to track your path to success is from the lean methodology: the Kaizen. To know more about how to set up a Kaizen to reduce the number of incidents, read this article.

Automation

Do you ever hear of the DevOps DishWasher by Brian Kelly? Well, Brian compares the DevOps to washing dishes. Some people directly wash the dishes when they are dirty and some other stock until they don’t have any clean dishes and then clean them all. None of the two techniques is truly efficient because it is not the optimal batch size. So he has imagined a DevOps Dishwasher utopia where:

• Dishes never pile up
• Dishes are cleaned as soon as they are dirtied
• We never worry about the effort required to clean dishes
• We don’t need to optimize the number of dishes to be cleaned at a time
• Dishwashing becomes an invisible and automatic process that frees us to focus on other things

Which corresponds to:

• A team that delivers small chunks of work very frequently
• A reliable, scalable Continuous Integration (CI) system to test every new code automatically
• Automated tests that are reliable and have sufficient coverage
• A Continuous Delivery (CD) system that automatically gets new code running in production
• A scalable infrastructure
• A team of Developers and Ops who collaborate effectively

An automated process allows your developers to focus on the production of new features and reduce the risk of human errors. The automation process is like a dishwasher, while it is running you can do something else and trust that it will be clean at the end.

Energy saving

We often hear about energy saving in our personal life but it can also be applied in your professional life. Indeed if you have a little traffic at night you can turn off all the environment except the production and reduce the number of clusters on the production environment. Doing that you will save energy but also money. You will see your invoice decrease since you are using less power. It is the advantage of the cloud and scalability. It is then going to reduce your Cost of Investment.

You now have all the keys to succeed in a DevOps project with a positive ROI. You know what metrics to follow to calculate it, and you know how to maximize your chance to increase your ROI. Do not hesitate to contact us if you have any questions.