From my experience, successful KPIs follow the following six rules:
Rule 1 – A KPI should be measurable. Don’t settle for boolean ‘yes or no’, or something amorphous that you can’t measure. Let’s say that you want to get into fitness. Let me ask you, what does ‘getting into fitness’ mean to you? Is it running a marathon? Is it cycling for three hours without stopping? Is it the ability to lift 50 kilograms? If you can’t measure it over time, or it only has two states of ‘yes or no’, you will not be able to track your progress. This means you cannot ensure that what you are doing is actually pointing you in the right direction.
When thinking about goals and KPIs, the first thing most people do is think about the goal they want to achieve. Really, you need to do the opposite. Think, instead, about what you want to change and how you can measure that. Great! You’ve found your KPI. Now you can attach a goal to it (which we will explore later in the article).
Let’s go back to the ‘get into fitness’ goal. Without thinking about what ‘getting into fitness’ means to you and how you can measure it, your goal is almost like a wish. There’s nothing wrong with this, but to be proactive in reaching your goals, think about your KPI first. Set a goal second.
Rule 2 – The meaning of the metric’s direction on your KPI should be obvious. There should be a clear understanding that your KPI’s direction is either good or bad. It shouldn’t be vague.
Rule 3 – Your KPI should reflect the value you are trying to bring and not the value’s outcome. Earnings are the outcome of a great product. Fewer pager duty alerts are the outcome of improved quality in design, code, and testing. Team velocity is the outcome of fewer dependencies and a smoother development experience. It is easy to set the KPI on the value’s outcome, but try to avoid this because the outcome can be achieved in different ways, and it can hide the thing you really want to change.
Rule 4 – The metric on your KPI should frequently change, at best daily, but no more than weekly. This is so you can track what you’re doing, and react fast to adjust your plans when things aren’t going according to plan.
Rule 5 – Your KPI’s metric should be something that you and your team members have the power to move. If your KPI depends on someone else, it will hurt your team’s ownership and accountability. Replace it with a KPI that has a metric you have direct influence over.
Rule 6 – The KPI should lead to action and not staleness. For example, the best way to reduce the number of bad deployments is to not deploy at all. If you choose a defensive KPI, it will lead to fear and staleness, and will not drive your team into action.
Source: Eric Rabinovich, How to measure and improve success in your engineering team, https://leaddev.com/productivity-eng-velocity/how-measure-and-improve-success-your-engineering-team