A product’s design determines its features, materials, performance, time to market and costs. Its quality, however, depends on the quality of the processes used to develop and make it. This doesn’t mean good designs only come from healthy design processes, but the odds of good results are surely higher if the processes are healthy.
So, how healthy are your product development processes?
Check-up List
One way to assess a company’s design processes is to have those involved in those processes answer the following list of questions. I developed this list based on 30 years of teaching design in academia and industry and on what others have found. (Responses to the questions are either always, mostly, sometimes, occasionally, never or not applicable).
All the team members should answer these questions independently, and the entire team should then go then through the answers one at a time and discuss them. If there is a lack of trust in the organization, assessments can be done anonymously and then combined to protect identities. (Of course, a lack of trust is a sign of a process in trouble.)
By itemizing measures that average less than “mostly,” you create a list of areas to improve. It is then up to the team to use this itemized assessment to develop tasks aimed at improving the design process.
The Questions
1. Do design teams follow a design process?
Most engineers get a poor education in “design process.” It should be taught much like statics and heat transfer, as a fundamental building block of engineering. But it isn’t at most universities. Many new engineers have natural instincts for good design processes, but without training and practice, they will never be truly competitive.
When new graduates take that first job, they usually get on-the-job-training by working with mentors, but many don’t even learn the basics. Thus, there is often a lack of knowledge on the best design practices, and everyone is flying by the seat of their pants using improvised design processes.
There are also sometimes disconnects between how design is done and how the company says it should be done. When I visit customers, I always ask to see their design process documentation (often called “new product introduction” process). I then ask to see proof that the company is following it. Sometimes it exists, sometimes it is developed after the fact, but most of the time, there a disconnect between what “should” be done and what is done. This is not to imply that what is written is the gospel, but there should be a company protocol to close the gap between “what should be done” to “what is done.”
2. Is that design process successful more often than not?
Successful design projects meet their cost and performance goals. Projects that are challenged miss some of the goals, and failed ones drag on and are sometimes canceled. Ideally, all projects are successful, but this is a lot to ask.
The data in the above table is from the software design industry. Every year the Standish Group, an organization developing resources and tools to help develop software projects, publishes the Chaos Report. This is a self-reported survey of thousands of software developers. Results from five years are shown in the table.
What is important is that fully half of all projects are challenged with an average of 20% failing. These values may actually be worse than that as they are self-reported. Further, there is no reason to believe that the results for hardware and systems are significantly different.
3. Does the process include customers or their representatives in the design team?
Many companies have a disconnect between marketing/sales and engineering. Marketing/sales should link customers into the design process. Companies should borrow a practice from the Scrum method: assign a team member to be the product owner (PO). The PO provides the customers’ voices and can explain what needs to be done and why. They are accountable for product value and often are technical marketing people and play an active role in the team.
At some companies, however, the customers’ need for the product have been conjured up by engineers. This is often risky. Of course, there are the Steve Jobs and Elon Musks who push the market, but most products are pulled by a market need. Each company determines how much emphasis to place on the customer’s voice. This lets the design team knows how much to listen to the customers. An organization’s design processes should be consistent with the market pull.
4. Does the process accommodate developing product requirements and managing changes in them?
At one extreme are projects with no written requirements and, at the other end, are projects where requirements are fully defined up front, given to product development, and assumed complete and immutable.
The middle ground, however, is likely the best approach, and it was used by Saab in developing its Gripen E jet fighter. There were about 300 requirements for the fighter, and each covered one or more of Saab customers’ demands (see graphic below). These were reviewed every three months and updated as needed.
Of these, about 15 defined the need for the oxygen system providing life-sustaining air in flight and after ejection. This set of macro-requirements was all that was provided to 40 engineers on the seven teams that did the design work. They developed the majority of the requirements to be met by the oxygen system.
Pushing all the detailed requirements down to the design teams made the process agile and capable of quickly responding to changes.
5. Does the process support an even pace of development?
Many design processes built around the stage-gate model have the work done in one stage reviewed periodically at a gate. Often these gates are months apart and feedback and needed changes only get communicated at these points. Further, for many products, a flurry of changes appears just before the design is released to manufacturing as the product becomes finalized. These situations make for uneven development and generally raises the costs.
A healthy design process avoids these spikes by building in feedback throughout it. This lets teams respond to changes throughout the process instead of in costly clumps. Smoothing the process improves team morale and is generally less costly, as needed changes are caught sooner.
6. Does the process include best design practices?
A company wanting to streamline and improve its design process hired a consultant who taught it Failure Modes and Effects Analysis (FMEA), a widely used best practice for design. Unfortunately, the consultant did not follow up and, as a result, the company was not only doing FMEA poorly; it was doing the analysis after the design was completed. The company was not using it to develop quality products.
Companies should not use seagull training for best practices—where consultants fly in, leave a little memory of their visit and fly off. It takes time and effort to get new tools and methods to stick. One way to help them stick is to establish metrics so that the before and after results can be measured and compared.
There are two sources of best practices: external and internal. External sources include books, articles, consultants and conferences. Of equal importance is developing new best practices within the company. Many Lean methods evolved internally in Toyota and became best practices, and Scrum methods began life in software development teams.
7. Does the process lead to great product architectures?
“Perfection” in physical products requires three elements:
- Design from function to form.
- Design from stable fixed interfaces.
- Design for simplicity.
These three are the building blocks of great product architectures. Product architecture has been defined as “the way functions are allocated or grouped in physical chunks (components or assemblies) and the arrangement of interaction of those chunks.” Paying attention in going from function to form and the functional and physical interactions through stable fixed interfaces leads to simplicity and “perfection.”
One of the highest compliments I have received as a product designer was at a trade show where an admiring customer spent 10 minutes staring at my BikeE AT recumbent bicycle design and finally said, “That is too simple.”
The BikeE architecture was based on an extruded aluminum frame that provided a simple, stable fixed interface with the other components. The development of the configuration was driven by the need to comfortably support the rider, easily adjust for different-sized riders, support the suspension and have a good steering geometry. During the 1990s, more than 3,000 BikeEs like this were sold.
8. Does the process support the team’s reflection on the process and foster incremental improvements in it?
One major features of Scrum are built-in reflection periods scheduled during the design process. These meetings let team members share honest feedback on what’s going well and what could be improved. The goal is to discuss and document what should change next time around.
A company doesn’t have to be doing Scrum to borrow the idea of scheduling regular review meetings.
9. Does the process encourage set-based design, developing parallel solutions to the furthest reasonable point?
Set-based design is a Lean technique that keeps sets of design options open until the last possible moment. Designers have been doing this long before Lean came around. The term “set-based” stems from Toyota and Deming in the 1950s.
The alternative to set-based design is point-based design, the linear process of developing a single concept with greater and greater detail. At first, this seems efficient because you only work on one concept from start to finish. In an ideal world with no uncertainty, this would be the perfect design process.
However, design is all about making decisions with uncertain and evolving information. Thus, each set of alternatives has its own uncertainties that can only be reduced through development. As work to reduce each uncertain feature progresses and time runs out, the best possible product evolves.
10. Is incremental testing integrated into the process?
Many sequential design processes carry out testing late in development. Testing should be done continuously throughout product development. Testing can be physical or virtual but begins as early as conceptual design. Concepts should be tested before they are adapted for the product. In the ideal world, tests should evolve along with the product and parts, subsystems and assemblies tested as they are refined. (This is more feasible if teams pay attention to design goals and product architecture, as discussed in questions 4 and 8).
11. Does the process create a motivating environment?
Unmotivated engineers develop uninteresting products and have little commitment to the company. The design process can improve the motivation to develop good products. In a survey of product engineers considering using Agile methods, only 28% cited improved team morale as a possible benefit of adopting Agile. After they had, however, 61% said morale improved and that was one of the most common benefit reported. Good team morale is part of motivating individuals to do their best work.
The point here is not that you must immediately start using use Agile methods to motivate. But it is clear the process affects the team morale and, therefore, the quality of the products generated.
12. Does the process enable interactions among team members?
Good design requires good communication amongst team members. Scrum and other Agile methods emphasize daily stand-up meetings and team co-location. Both of these improve team interaction and communication. However, the reality of 2020 makes it hard to do either whether using Scrum, Stage-Gate or other processes.
The figure above shows the over-the-wall design method, with silos and one-way communication from customers to marketing to engineering design and to manufacturing. There can also be siloing in design teams.
One company I am familiar with has a small team; one person develops PC boards, one does firmware, one does packaging, and so on. Prior to March 2020, they all sat in one room and spent their time on one project. They found that spending 15 minutes each morning in a stand-up meeting answering three questions was beneficial: What did I do yesterday? What will I do today? And What obstacles are blocking the team’s progress? Now that they all work from home, these daily check-ins are even more essential.
This article gives you a template for evaluating the health of your team’s design process using 12 measures. For details on solving any design process issues uncovered, visit www.mechdesignprocess.com/design-process-checklist.
David G. Ullman is a product designer who has taught, researched and written about design for over 35 years. He is an emeritus professor of mechanical design at Oregon State University. More information on this topic can be found in his books The Mechanical Design Process 6th ed. and Scrum for Hardware and Systems Design, both available at www.mechdesignprocess.com.