Or “What are the problems with implementing metrics in an organization?”
OKRs are another form of the use of metrics in an organization with the aim of improving results delivered to the business. Metrics fall under the general category of “bureaucracy” that an organization uses in order to ensure the expected outcomes of an organization are routinely achieved.
Bureaucracy is, in and of itself, not necessarily evil, although many treat it that way. In reality bureaucracy is required in order to coordinate, align, and replicate the activities of organizations; the larger and more diverse the organization, the more bureaucracy. What is evil is bureaucracy that appears useless, arbitrary, and capricious. Raise your hand if you have been the victim of processes that seem overly complicated, or metrics that don’t make sense, or spent your time working around something that gets in the way of doing what is right. These are examples of bureaucracy that has outlived its purpose.
Properly done metrics in the bureaucracy produces results for an organization. OKRs are similar. OKRs can produce results for an organization. In order to ensure we understand how to successfully apply this form of bureaucracy, we need to understand potential problems so we can ensure our implementation addresses them. Before implementing, or as you run pilots, you need to ask yourself:
“Is the juice worth the squeeze?”
Potential problems associated with metrics include:
Metrics can be used for “evil”: This is where metrics are used to judge or compare people or organizations or worse. Using metrics this way stifles psychological safety in the organization reducing effectiveness. We often set up metrics to “hold people accountable” because we fear (expect?) that without them people will do the wrong thing.
Metrics can contribute to organization inefficiency: How much effort does it take to create, manage, maintain, report on, and discuss your metrics. Does the benefit of having this effort clearly outweigh the improvement in results you are seeing?
Metrics can result in inattention to overriding goals of the organization: Since people will naturally focus on the metrics, the key results, the overall objective might be lost. The effect worsens where OKRs are cascading; people lose understanding of the overall goal, leading to blind spots in the execution, and failing to adapt to the situation.
Metrics can stifle innovation: OKRs, in particular, are supposed to leave people free to do what they think is right to achieve the objective. But if there is uncertainty and an expectation based on the set of key results, people will tend to focus on “hitting the number” rather than taking risk, innovating, and where there is a potential miss the metric. In the worst case, people can hide behind the metrics, declaring “success” while the organization suffers.
Metrics can slow down an organization’s ability to adapt: How long does it take to change a metric now that one is established in response to the changing environment?
Metrics can be poorly formed: Particular examples of poorly formed metrics are:
Those that are captured because they are easy to capture, rather than what we should be interested in which are often more difficult to measure.
Metrics that have been set up as targets based on thinking such as “the number was X before, and we want X+10%” whether or not this makes a significant impact on the business results: the metric is not really enough to understand whether we are doing well except in the most superficial sense.
Metrics can be dehumanizing and overly simplistic: Focussing on metrics can result in oversimplification of complex variables and interrelationships. For example, we might see an upswing in the number of sign-ups to our product and assume customers are delighted. But perhaps the users have been mandated to use the product, and are actually very unhappy.
None of this says that OKRs are “bad.” OKRs were designed to address a number of issues associated with the traditional application of metrics (KPI’s, scorecards, etc.) For this reason you are seeing a lot of discussions around OKRs.
But since OKRs have numbers associated with them (the key result) they potentially suffer from the same kind of problems the use of any metrics in an organization has. As we implement OKRs, we need to ensure we address potential shortcomings.