Measuring the productivity of product development is a complex and multi-faceted task, significantly different from manufacturing operations. Output from manufacturing is increasingly automated and can be summarized as emanating from “equipment assisted by people.” Output from engineering and the cross-functions that work with engineering to bring about a product that can be repeatedly made is usefully summarized as emanating from "people assisted by equipment.” The people in R&D and product development turn out the designs, not the equipment. Therefore, measuring the productivity of people is highly important.
Departments. People, for the most part, are organized into departments according to the roles they perform for the company. Those functional departments supply expertise to company activities and projects. Since the 1930s, organization science has shown that people are affected by the dynamics of the departments they work in—both positively and negatively. Note that we’re not calling them teams. But, in truth, departments are also a type of team.
Measuring the output and productivity of departments is risky because managers then try to optimize their department’s output. This, in turn, lowers projects’ outputs (whose resultant IP and products are the actual generators of revenues and profits). And, therefore, also reduces the effectiveness of the organization as a whole.
Measuring departments is important, but beware of measuring output. The focus should be on the competencies, hiring/firing/turnover, capacity, innovativeness, and other areas relating to the department’s capabilities and service levels to meet its demand.
Projects. The second key measurement area is projects. Why? Because this is where management authorizes investments in order to earn revenues and profits. Each investment is a wager that risks money. Some wagers have high risk and some have low risk, but each is a bet. Those bets are expected to be the future of the company (if not individually, then certainly collectively). It is absolutely critical to measure the ability to consistently and systematically execute projects. It’s also critical to have company-wide metrics that show averages, medians, modes, totals, and other cross-project measures across the types and sizes of projects undertaken.
Measuring projects should not be confused with measuring the product specs and values that are the purpose of those projects. Every project has to have product measures. But the values of product measures within each project are unique to that product and cannot be used on products in other projects. Generically, they can (and are), but that type of measurement—such average reliability and average defects per unit—resides in other departments. Those types of metrics are handled elsewhere and fed back. In summary, common “standardized” measures should reside across all projects, and should not be confused with unique product measures in each project.
Improvement efforts. The third key measurement area is improvement efforts. Most improvement activities are limited in duration. They come and go and rarely return. They last a few months to a few years, such as putting in new software or training in a new skill set. Each effort needs its own unique metrics, analogous to unique product metrics within a project described above. However, the result of improvements is best measured by monitoring aggregate increases in output, productivity, or quality.
CXO-level performance. The fourth and final area is to measure the collective results of everything. These are the measures that interest the CEO and investor community. Given all the money invested (the input), what is the value of the revenues/profits and intellectual property generated (the output)? Measuring product development productivity is defined as its output divided by its input.
In short, measuring productivity is essential and must be done to some extent at all levels in the organization. If the measured optimization of a system is targeted to the lower levels of the system, then the system as a whole is generally not optimized. Measuring productivity and output in aggregate at the top is the place to do it.
