Innovation Metrics: Figure Out Which Projects to Pursue
Paul Puopolo, who became executive vice president of innovation at Dallas Fort Worth International Airport (DFW) in August 2018, arrived at his new job with a clear mandate to standardize the innovation function.
“DFW had always been innovative, but it was more of an ad hoc innovation model,” says Puopolo. Innovation was largely comprised of isolated early-stage initiatives with variable business cases and few formal tools or processes. Puopolo’s goal was to build an optimized innovation model – in his words, “repeatable, scalable, and pervasive” – capable of having a clear impact on the organization’s results.
Plenty has been written about the importance of creating a culture of innovation – and doing so has been essential to the evolution that’s taking place at DFW. Less discussed is the need for what might seem anathema to innovation: process. For companies that are betting their future growth on their ability to innovate, creating a process for measuring it is critical.
Veterans like Puopolo (who previously held innovation posts at MetLife, Highmark, and Humana) know that while innovation requires new thinking, it also demands rigor – and clear metrics for success – to deliver business value.
Three years into Puopolo’s tenure, DFW is no longer flying blind when it comes to its innovation journey. The airport’s innovation business function (emphasis on business and function) has a dashboard that tracks innovation programs and key performance indicators (KPIs). Performance on goals, rather than instincts, drives decision-making.
Among the innovation group’s projects are two designed to improve travelers’ experiences while meeting security requirements. In 2020, DFW began exploring Mastercard’s digital identity services and SITA’s Smart Path passenger processing application so that employees could access facilities using mobile face-scan technology (that ultimately could provide touchless passenger checkpoints). In December 2021, the group launched a self-service check-in and bag-drop process with Spirit Airlines that uses biometrics and meets regulatory requirements for identifying travelers.
DFW also tested seven new technologies to support location-based services in the terminal and is moving forward on a proof-of-concept to manage curbside passenger traffic, congestion, and transportation usage.
Corporate leaders across industries recognize the importance of innovation. According to a recent Boston Consulting Group survey, three-quarters of CEOs (a 10% jump over the previous year) indicated that innovation was a top-three priority for their organizations. Among these large companies, the biggest innovation spenders devote tens of billions of dollars to their efforts each year. However, there is little connection between the amount of money a company spends on its innovation efforts and what it achieves. PWC found that there is no long-term correlation between innovation spend and a company’s overall financial performance. What matters most is how companies manage their innovation resources and measure their efforts throughout the innovation lifecycle.
“Generating new ideas creates a lot of excitement, and so there’s a lot of focus there in companies,” says Andrew Binns, director with Change Logic, a consultancy that helps clients ideate, incubate, and scale new businesses inside existing organizations. “The difficulty is in making them happen.”
What separates companies that innovate successfully in the long term is not the number of ideas they generate. It’s not simply setting up a shiny new innovation lab (though that can certainly help). It’s not necessarily disrupting a given industry or product category. It’s assessing whether the company is investing in the right kinds of activities to achieve its goals, weeding out underperforming initiatives, and pivoting to new opportunities. And it’s applying this kind of discipline in uncertain circumstances.
Companies that derive value from their efforts have mastered the less sexy aspect of innovation: metrics. They expand beyond the traditional measures of success for R&D (innovation spend as a percentage of total sales, number of new ideas, and percentage of sales from new products). And they build an arsenal of benchmarks, such as:
- Conditions conducive to innovation (the percentage of employees who have received innovation training and gained new competencies, adoption of collaboration and innovation management tools)
- Qualitative impact (number of innovations that significantly advance existing businesses, number of new-to-company opportunities, active customers or users)
- Financial impact (ROI of new products, services, or business models)
- Innovation sourcing (best performing business units, team, or categories; third-party innovation)
- Idea pipeline tracking (ideas-to-project ratios, number of projects stopped, innovation pipeline performance)
If that sounds like a lot of work, that’s because it is. “This is the challenge,” says Binns, author of the forthcoming book Corporate Explorer: How Corporations Beat Startups at the Innovation Game. “Ideation is so exciting. Incubation is boring, hard work that you’ve got to sustain.”
However, it’s doable. In fact, it’s being done. Companies that are reaping the benefits of innovation put in place specific metrics, mindsets, and processes to better manage their innovation resources and investments. For organizations that are serious about transformation and growth, metrics are mandatory.
Scott Anthony, partner at global strategy and innovation consulting firm Innosight, says that innovation leaders have to make resource choices. “Which efforts should get more funding, and which shouldn’t? Which should be accelerated, and which should be killed? Is the overall spending on innovation too much or too little or just right? Without performance metrics, it’s hard to make these kinds of decisions,” says Anthony.
KPIs of the unknown (why measuring innovation is hard)
Managing by measuring is nothing new and is typically straightforward. Traditional KPIs monitor things we already know something about; innovation is different by design. It’s the opposite of business as usual. Yet, to yield results, it also requires the rigorous application of unique metrics for success. Without metrics and processes specific to innovation, companies may pull the plug too early on ideas and efforts that fail to deliver early results in traditional corporate performance metrics. Conversely, they may invest too much ahead of real testing and learning, betting big on what they hope is the next big thing. Or they may generate a lot of new ideas but have no way to assess, manage, incubate, or scale them.
So how do you measure innovation? Unfortunately, there are no one-size-fits-all metrics to deploy and few best practices to apply across initiatives or industries. The metrics will depend on what kinds of innovation an organization is pursuing, what the innovation goals are, where the efforts are in their innovation lifecycles, and how mature the organization’s innovation program or capabilities are – for a start.
“Every organization is different, so you need to innovate in context,” says Puopolo of DFW. That means you need to speak, define, and measure innovation in a way that works for your organization.
For example, in a culture that values process, innovation metrics will emphasize the efficiency and effectiveness of the innovation pipeline. The metrics might include not just the number of ideas entering the pipeline but also the diversity of ideas (in terms of sourcing or business area application), the rate at which ideas move through each stage of the innovation process, or an innovation pipeline index that incorporates a number of salient metrics.
At DFW, Puopolo began with some basic action-based process metrics to establish an innovation portfolio and pipeline. That meant tracking items like “the number of ideas in discovery” and the number of pilot tests and test programs transitioned to the business for implementation. It wasn’t until year two that his team began including culture-related metrics, such as the numbers of innovation training sessions, the training participants, and the executive sponsors of innovation. In year three, he put in place metrics to track the longer-term impact of the group’s projects in areas such as customer satisfaction, employee engagement, project and portfolio revenue generation or cost savings, number of new business models, and brand awareness.
Building a game-changing portfolio of innovation metrics and KPIs takes time
In 2020, Central European insurer Uniqa Insurance (a Change Logic client) launched an innovation-focused business unit called SanusX with its own budget, staff, governance, and metrics to measure progress. The structure protects the unit from the short-term pressures of the core business. The unit’s two levels of innovation metrics help leaders determine what innovations to pursue and demonstrate their value to the business. At one level, there are lead indicators of progress toward a business goal, which will vary based on the goal (such as rate of customer adoption of a new app or increased customer retention).
“These help them understand whether a proposed new venture is on track,” says Binns of Change Logic. There are also the more tactical objectives and key results measures that focus on execution in each sprint cycle.
In the first year, SanusX ran more than 30 experiments related to seven new business ideas and terminated two unnecessary projects. The insurer now has a rich pipeline of opportunities that are being accelerated and scaled with its partners and potential acquisitions. In its first year, the SanusX team introduced a digital COVID-19 application (developed in less than a month) to help clients coordinate testing, sample collection, results delivery, and follow-up communication, which enabled thousands of tests per day at a rate ten times faster than standard testing protocols at the time.
The outcomes don’t always happen overnight. They can take years. The innovations that win in the marketplace aren’t always a shiny new app or disruptive business model that rises from nowhere. Companies that take a performance management approach to innovation metrics aren’t wed to ideas; they’re focused on results that align with their business strategies. Sometimes the results look more like a slow march toward new products, business models, and ultimately growth, which involve partnerships, acquisitions, and new applications of existing technologies and resources — requiring consistent attention and potentially new roles from leaders including CEO, CIO, COO, and more.
Analog Devices, for example, had long been known for its signal processing circuits and semiconductors empowering mobile phones, cars, manufacturing machinery, hospital scanners, and satellites. In 2013, new CEO Vince Roche sought to offer customers not just components but systems that incorporated hardware, software, and analytics.
Working with Change Logic, Analog Devices reset its metrics for innovation. Managers who would otherwise be incentivized by short-term revenue gains needed new performance metrics to encourage innovation. All innovation projects were given a set of early success factors that, when tracked over time, would indicate if an initiative was heading in a positive direction. “They tell them whether the business is getting market traction,” says Binns. “A dashboard of lead indicators helps them manage uncertainty because they know whether the innovation is tracking to expectations or whether they need to pivot their approach.”
Eight years ago, Analog Device’s director of technology for the wireless communications business unit began exploring a software-defined radio solution that could accelerate 5G adoption using smaller, lower-cost, more energy-efficient technology capable of supporting eight times the bandwidth. Aside from engineering advances, key indicators were whether customers were committing to the end products that were signaling a shift toward 5G (such as new systems for base stations). That gave the company confidence that they were onto something. The system was an early win for the new innovation approach and today generates more than US$1.5 billion in revenue.
“Innovation is about managing uncertainty. Metrics tell you whether or not you are on track toward your goal and so increase confidence that the new venture will pay off,” says Binns. “But they do not eliminate the need for judgment and leadership.”
Where to begin: Best practices for innovation performance management
Managing innovation performance with the same rigor as traditional business factors (but using different metrics) is the key to long-term success. The specifics of metrics and governance will vary by an organization’s goals and needs. And achieving maturity in determining how best to measure and manage innovation will take time. There are several existing tools that leaders can use to determine where their organizations are on the innovation maturity scale (from ad hoc to optimized) and what gaps they need to address. Beyond that initial assessment, there are three categories of best practices that organizations can adopt to develop more effective innovation metrics over time:
1. Identify and address cultural obstacles
- Insulate innovation from short-term pressure. Many organizations address this by creating an innovation unit that is insulated from core business demands. Those without an innovation group can provide education on innovation-specific metrics, mindsets, and processes best suited to managing innovation resources and investments.
- Build behaviors. Measuring behaviors – such as participation in innovation boot camps – better assesses a cultural shift than standard KPIs. “Training employees on how to be everyday innovators, running boot camps, or hosting tech talks can impact engagement, recruiting, and retention,” says Puopolo of DFW. “These activities build the innovation culture but are often overlooked.”
- Celebrate innovation “not invented here.” The best use of innovation funding is not always building a new product or approach in-house. P&G, for example, tracks the number of ideas that come from thousands of outside partnerships.
- Communicate consistently and clearly. Commit to a regular cadence to tell the innovation story and educate the organization. At DFW, this approach helped build transparency and consistency that was critical in the early stages of the new innovation programs, Puopolo says.
2. Build measurement and decision-making processes
- Create some accountability metrics. In the formative days, it’s important to show that innovation performance management is on the right track. This means holding the team accountable for practicing what it preaches and for the pipeline’s progress. “Impact metrics are required to keep the naysayers at bay and to provide cover while your program matures,” says Puopolo of DFW.
- Get clear about the destination. It’s important to identify end goals before considering how to evaluate performance. Once you determine strategic areas to focus on, you can better determine what new initiatives to pursue. And thinking ahead to what might indicate whether each innovation is on track makes it easier to decide on success metrics for that initiative.
- Combine short- and long-term metrics. Short-term metrics will focus on activities and long term on business impact. Puopolo says he tracked things such as number of business cases, pilot projects, killed projects, innovation training, and participants in the early days to build credibility and show early wins. Later, he added metrics such as revenue generation, cost savings, employee engagement, and customer satisfaction.
- Track the rejected ideas. A good innovation program will nix a great number of ideas. This seemingly negative metric is a positive indicator: It suggests that the organization is investigating a broad swath of opportunities. When the number is low, the organization may not be trying hard enough. Providing an accounting of this activity is also useful to an innovation group’s development, according to Puopolo, who says, “While it seems negative, it shows the methodology and the value of the process.”
3. Manage innovation metrics at the portfolio level
- Consider a balanced scorecard approach. Innosight’s Anthony advises tracking a selection of metrics related to inputs and outputs to measure performance of innovation. Using the approach of a balanced scorecard (the performance management report that companies use to track strategy implementation activities and outcomes), innovation leaders can identify strategic goals and indicators in categories such as financial impact, customer impact, internal processes, and employee learning. Anthony says the purpose of an innovation scorecard is to track whether the organization has the right raw materials for innovation, how effective it has been at generating innovations, and what value those innovations have created.
- Measure portfolio returns. Like a financial portfolio, organizations can measure their overall returns on innovation investment. A basic calculation takes the profits or cash flows produced (or expected) by innovation and divides that figure by the cumulative investment required to create those returns. Anthony offers a more nuanced approach to calculating innovation magnitude (financial contribution divided by successful ideas), innovation success rate (successful ideas divided by total ideas explored), and investment efficiency (ideas explored divided by total capital and operational investment).
- Leave room for learning. There is the danger of being too beholden to metrics. The process of measurement should serve the goal of innovation – not the other way around. Some of the greatest value of experimentation is the learning that emerges. Thus, it’s important to build some flexibility into the management and measurement of innovation.
Innovation metrics will change over time. A focus on innovation pipeline metrics in the beginning should evolve to include other metrics as the innovation program grows. A mature innovation program can report real business impacts such as increased customer loyalty or satisfaction, revenue growth, cost savings, and efficiency gains.
Puopolo’s DFW innovation group has seen its metrics evolve from scratch. Today, it has a one-page dashboard that highlights innovation processes and performance, including the number of ideas in different innovation stages, with a mix of initiative types (incremental, differentiating, transformational). The dashboard tracks which departments are innovation adopters and key strategic or cultural efforts that are underway. “As your innovation program matures, your metrics should adapt as well,” Puopolo says. “What numbers best measure the program’s progress will be based on where you are in your innovation journey.”