The Balanced Scorecard: How to Use KPIs Effectively

Key performance indicators (KPIs) are intended to be measures of an organization’s health. They are useful in that they offer objective, typically quantifiable, metrics. These data enable leadership to make informed decisions as well as aid in prioritization and ensure accountability. Managers who forego KPIs may be at risk of running their team blindly; intuition certainly has its place, but objective weights must counter it.

However, leaders must use KPIs in a broader context to maximize their utility and avoid negative consequences. If not used wisely—that is, as one feature in the larger landscape of a lab’s status—KPIs quickly morph from objective health measures to proxies for what cannot be quantified. This disconnect allows staff to game KPIs to meet departmental or personal goals without furthering strategic objectives.

Get training in Making Data-driven Decisions and earn CEUs.One of over 25 IACET-accredited courses in the Academy.

Making Data-driven Decisions Course

View Course

The pitfalls of KPIs

During the Vietnam War, US secretary of defense Robert McNamara took an approach of radical quantification to assess how the US was performing in the war. Rather than relying on traditional qualitative measures of wartime success, such as territory occupied, public mood, and progress toward strategic objectives, McNamara focused on one quantitative metric: the number of enemy combatants killed. He believed that the most straightforward vector to success was simply keeping the number of Vietnamese casualties above the number of US casualties.

Up until that point, McNamara’s approach had a successful track record. A Harvard Business School alum, McNamara had previously worked at Ford Motor Company, where he established his reputation by ruthlessly optimizing for wholly quantifiable metrics on the automobile production line. This systemic analysis, he reasoned, could be applied to the war effort and yield comparable results. “Things you can count, you ought to count; loss of life is one,” he said.

. . . Intuition certainly has its place, but objective weights must counter it.

But McNamara was wrong: “By late 1967, the US was no nearer to concluding the war; McNamara and President Johnson did not agree on strategy, and public opposition to the war had grown . . . McNamara’s name became inextricably linked with American failure in Vietnam . . . [evaluating American progress in the conflict] did not submit itself to numerical analysis.”

After the war, in 1972, sociologist Daniel Yankelovich termed the phrase “The McNamara Fallacy” to refer to the mistake of ignoring valuable, but not necessarily measurable, information to instead focus only on that which is quantifiable. This fallacy results in a skewed, incomplete view of operations, adversely affecting decisions and prioritization. Scientists are just as susceptible to this fallacy as anyone else. KPIs do not lend themselves to qualitative data, leading some labs to ignore insightful information because it is difficult to track. Furthermore, scientists are biased toward measurable data, but lab managers must take data that is not easily tracked (e.g., customer satisfaction or employee morale) into account when making decisions.

Goodhart’s Law

When it is appropriate to use KPIs, it is essential that they do not become more than just measures. In the words of British economist Charles Goodhart, “When a measure becomes a target, it ceases to be a good measure.”

Many labs make this mistake. KPIs should be measures and nothing more. Once leadership passes down an explicit goal to change a KPI’s measure by x value in y timeframe, employees will intuit some basic game theory: it is in their favor to take shortcuts and meet their KPI goal even at the expense of other business processes. Often, staff are motivated to game the KPI in the name of job security. This effect is amplified if management incentivizes the KPI target, such as making it the basis of performance bonuses or merit raises.

Cross-departmental alignment

Finally, KPIs are often set without consideration for the objectives of other departments or the organization at large. Consequently, the KPIs will not align across the organization; this leads to one team’s attempts to optimize for one of their KPIs inadvertently sabotaging the KPI of another department. An example:

A sales representative at a contract research lab is given a revenue KPI target. He must acquire more customers to meet this target, which will translate into more work for the lab. Simultaneously, the lab staff are hoping to increase quality KPIs. Additional projects would negatively impact product quality, lowering those KPIs and employee morale.

These three pitfalls of KPIs—the McNamara fallacy, Goodhart's law, and conflicting targets—can be remedied with a balanced scorecard approach.

Therein lies the challenge many organizations face: in the abstract, the lab and the sales rep have the same goal of ensuring the organization’s success (i.e., profitability). But individually, they have conflicting goals: the lab is incentivized to refuse more work to maintain their quality while the sales rep is incentivized to acquire more customers to drive revenue. These three pitfalls of KPIs—the McNamara fallacy, Goodhart’s law, and conflicting targets—can be remedied with a balanced scorecard approach.

What is a balanced scorecard?

In the January/February 1992 issue of Harvard Business Review, Harvard Business School professor Robert S. Kaplan, along with business theorist David Norton, PhD, published an article detailing a strategic planning and measurement system borne out of a year-long research project they had conducted with 12 companies pushing the boundaries of performance measurement. “[The balanced scorecard is] a set of measures that gives top managers a fast but comprehensive view of the business,” they wrote. The balanced scorecard offers a holistic view of an organization by examining it from four fundamental angles:

1. The financial perspective (“How do we look to shareholders?”)
2. Internal business perspective (“What do we excel at?”)
3. Innovation and learning perspective (“Can we continue to improve and create value?”)
4. Customer perspective (“How do our customers see us?”)

Each perspective is informed by strategic goals coupled with measures of progress toward those goals—effectively, KPIs.

Creating a balanced scorecard for a lab

What might a lab’s balanced scorecard look like? Here is an example scorecard for a lab embedded in a contract research organization:

How can a balanced scorecard curb KPI gaming?

1. Holistic view: A balanced scorecard allows one to see the downstream effects of certain actions. Causal relationships between processes are mapped, and leading indicators are coupled with relevant lagging indicators. As such, manipulating a metric becomes harder because the effects of potential manipulation are articulated.
2. Strategic alignment: Balanced scorecards align all aspects of a business with its broad strategy. This discourages KPI gaming because employees must deviate from that strategy to game a metric, effectively sabotaging the lab.
3. Qualitative data included: Accounting for qualitative data should be baked into balanced scorecards, which would ensure that leadership is taking it into account when making decisions.
4. A living document: Balanced scorecards are updated as the lab’s vision and strategy are adjusted. Siloed KPIs tend to be static, not moving in step with the organization to ensure that whatever is being measured (and thus optimized for) is what should be prioritized.

It’s important to internalize that crafting effective KPIs and implementing a balanced scorecard is an iterative process. It is a cycle of setting objectives, measuring outcomes, learning from results, adjusting targets, and deciding which metrics are worth tracking. No organization will perfect this process on the first try, nor should they expect to. Rather, success lies in recognizing the evergreen nature of the balanced scorecard and embracing continuous improvement. By doing so, labs can sustain success and navigate the pitfalls of KPIs effectively.

References:

1. https://www.rcpe.ac.uk/college/journal/medicine-and-mcnamara-fallacy#:~:text=The%20'McNamara%20fallacy'%20(also,%2C%20it%20is%20not%20important'

2. https://hbr.org/1992/01/the-balanced-scorecard-measuresthat-drive-performance-2