Peter Drucker, a famous management consultant, once said, “If you can’t measure it, you can’t improve it.” This is true, because without knowing where you are, you cannot move forward. When you hire a DevOps team you want to see results. So how do you measure—and improve—the work of your DevOps team? In this article, I will share which key performance indicators you should check, and which will work against you.
I recently had a conversation with a friend who said his manager wanted to measure the team’s workload to make sure it was working at full capacity. In fact, the manager felt that if the teams didn’t keep up with their commitments, they were probably not working hard enough. I often hear this attitude from many companies. But let’s look at this in terms that most engineering managers should understand.
Imagine this scenario: you want to make sure DevOps team is “meeting their commitments”. They promised they would deliver something, and you have to hold them to it, right? But they seem to be failing to accomplish a good amount of work. In the last five sprints, the team committed 50 story points out of 100. That doesn’t look very productive, does it? Therefore, you will use this ratio to improve their performance. It appears to work. In this new sprint, the ‘lazy’ team decided to fulfill their commitments. They doubled their speed!
You are satisfied that you have managed the situation successfully. Now everyone is happy…except your customer, who is now dealing with a buggy application. What went wrong here? Let me show you.
Key performance indicators you could measure… and why they will not work.
There are many Key Performance Indicators (KPI), but I will focus on the main ones that seem like they would be helpful…but instead can be misleading.
Lines of Code
You might think that if a DevOps team is typing more lines of code per day, they are more productive. Therefore, it seems as if you would want to measure lines of code, or the number of ‘commits’, per day for each developer. But in reality, elite developers spend more time on thinking than typing. Read also: How to reduce cloud-operating costs
The code is only the documentation of your DevOps’ planning converted to a format that a computer can understand. It is better to have a smaller number of high-quality lines, because such small enhancements can result in the deletion of over 5,000 lines of code. Impressively efficient, right? In other words, if you measure your DevOps team’s productivity by lines of code, they could just add more code in order to look more productive. Copying and pasting older code is much easier than spending the time on developing a smarter business solution.
Number of Defects Fixed
It may seem that if we measure the number of defects fixed, there will be fewer defects as a result. But in reality, high-quality work results in a smaller number of defects in the final product. So, if you are focusing on the number of defect tickets the team has closed, you are prioritizing quantity over quality. Moreover, if you pay a bonus for defects fixed you create an opportunity for the team to create more defects, or waste time finding more minor defects instead of addressing larger critical issues. Read also: DevOps as a service
Committed vs. Completed:
If we measure DevOps team by how much they complete versus the schedule to which they commit, they will work hard to meet those deadlines and deliver above and beyond! Or at least, that is how this would work in a perfect world. In reality, your DevOps team may prioritize sticking to a schedule over producing quality code. Defects and underlying issues could be ignored. Measuring your DevOps team by ‘committed versus completed’ means that you care more about the plan than the value.
As Daniel Ariely, one of the founders of behavioral economics, has noted: “Human beings adjust behavior based on the metrics they’re held against. Anything you measure will impel a person to optimize his score on that metric. What you measure is what you’ll get. Period.”
Are you still thinking that higher velocity/throughput means a DevOps team is more productive? Velocity or throughput give us an average of how many things the team have completed during a fixed period of time. But if you measure DevOps in this way, they could create more tasks or increase the number of story points per task just to satisfy your requirement. It may look good on paper, but in fact this leads to a higher number of mistakes and bugs in the long run.
What really matters?So, how do you decide which KPIs are the most critical? President Dwight Eisenhower–the decorated military general–had a famous and simple way to quickly decide which tasks were most important. This simple quote will help you understand which tasks are critical and which tasks can wait–or be eliminated entirely.
“What is important is seldom urgent, and what is urgent is seldom important” – Dwight EisenhowerYou need to measure the things the customer cares about so your DevOps can make them happy. If the customer is unhappy, it really doesn’t matter what your metrics say. What does the customer care about? They want to receive quality work when they need it. Productivity is not about how many features are delivered or how fast the job is done. I would suggest that productivity is measured by customer value delivery, not team output. Read also: What should I know before migrating to clouds Monitor metrics that demonstrate how happy your customer with your service. Among them, check the number of downloads and uninstalls, customer satisfaction score, number of active users, user’s feedback, and churn rate. Consider this: there shouldn’t be too many metrics, otherwise you will not be able to track them properly. Your metrics should reflect what is important to you. If you believe that customer satisfaction is your priority, check what the end-user says about the results your DevOps team is producing.
I want to get started. Where do I begin?Does bringing DevOps into your business sound right for you? Do you want to get started, but do not know how to begin? Put your details into the form right now and get your project estimation absolutely for free! The future of your business starts right now!
DevOps folks understand the cloud and obsess about automation. They work with SCM engineers, Db Admins, IT, QA, etc., with the intent of breaking down silos between teams and facilitating trust between teams. This leads to an overall increase in IT performance, reduced lead time to new features, and fast recovery from failures.
A DevOps engineer has to make sure that the CI/CD pipeline is intact, which requires spending time on troubleshooting, analyzing, and providing fixes to issues. They are also responsible for maintaining and managing the infrastructure required for the CI/CD pipeline and making sure that it is running smoothly and being used optimally.
Deployment frequency: This indicator gives a picture of how frequently deployments are rolling out in an organization. Small but stable deployment frequency metrics are ideal for any organization. Deployment time: This key performance indicator lets you know the required time for a specific deployment to roll out after approval. Average recovery time: This metric allows you to evaluate how long it takes for your team to solve a failed deployment or a unique unspecified issue.
DevOps can play a significant role, but only if the whole company is working together to make the best use of the team. You need the business users to identify what is acceptable. You need operations staff to troubleshoot backend performance issues. Then you need the developers to point out where issues might be from what the code is telling them. Good communication between teams is the only way to get past performance issues.
The main aim of your DevOps project will be customer satisfaction by an effective and efficient project. Project management is also related to customer satisfaction. If you want to satisfy your customer with an effective and efficient project, you have to plan it in the best possible way and apply a strategy to it. Therefore, the main relationship between DevOps and project management is the management system. Both are focused on an effective project with customer satisfaction.
Dmitry has 5 years of professional IT experience developing numerous consumer & enterprise applications. Dmitry has also implemented infrastructure and process improvement projects for businesses of various sizes. Due to his broad experience, Dmitry quickly understands business needs and improves processes by using established DevOps tools supported by Agile practices.