The folks leading complex companies are quite capable of understanding the underlying work, but they can only afford to play this card sparingly. There are so many demands on their attention, and metrics become the great filter to prioritize which areas to observe from afar, and which to drill into.
It takes some time for an organization to gain the new competencies to fulfill this insatiable need for metrics. Good metrics require a deep understanding of the data underneath, and building that familiarity is real work. The beginning of the metrics-era is characterized by churn in metric selection. Each learning inspires a refinement across your catalog of metrics, and most new metrics requires new instrumentation, pipelines, and sanitation.
This takes time, and is difficult work, but it’s predictable work. You put in more time, learn more, and your metric selection becomes a durable, useful materialization of the reality underneath.
Well, that’s how it usually works, anyway. As I’ve gotten more experienced in defining and using metrics, more and more of my time is spent dealing with areas that defy easy measurement. Areas that I’ve at times been tempted to describe as unmeasurable.
When I meet new infrastructure leaders, one of my first questions is usually one of:
- “How do you measure security at your company?”
- “How do you measure productivity on your team?”
Answers vary, but generally the state of measurement for both leaves a lot to be desired.
Source: Metrics for the unmeasurable, Will Larson, https://lethain.com/metrics-for-the-unmeasurable/