Test-Suite Rebalancing Reduces Time To Market

Has your company been around for a long time? Did your products start off as purely mechanical, then evolve to electromechanical, then to electronic, and are now software-driven hardware? Are the historical underpinnings of your testing and quality organizations rooted in manufacturing operations and validating physical products? If so, then rebalancing the test suite may shorten your time to market.

Rebalancing a test suite is not easy. It requires much thought and patience, and it is not inexpensive. But lengthy time to market is also not inexpensive if the revenue streams of all products could be brought forward another one or two months. Because ROI justifications are unique to each company, rebalancing will be worth it for some firms and not others.

When was the last time your company systematically analyzed all the tests in the company's product testing suite? Of course, like inventory management, there will be A and B and C items, and obsolete inventory as well. But the question remains: When was the test suite last analyzed systematically?

The earlier bugs are discovered, the cheaper it is to fix them in both actual and elapsed time. Bugs occurring prior to beginning formal engineering change control are not the greatest concern as the entire cross-functional product team is generally not affected. However, the cost of bugs fixed under change control grows geometrically as time marches forward.

These change orders typically consume a couple of months or more out of each new product-development schedule. And they increase development costs. If unit sales volumes are high, development cost is not generally a concern, as it gets amortized over many units. If unit sales volumes are not large, development costs may be as important as time to market.

Finally, late changes and fixes often also increase product cost. Unless the sales price of the product can be bumped up to hold the planned unit gross margin constant, unit profit takes a hit. Historical test suites that have not been looked at in a while and/or company organizations rooted in physical testing may be reducing product ROI across all product lines.

There are many types of tests and testing: proof of design, proof of process, white box, black box, system, regression, life cycle, HALT, HASS, and a dozen others. But, with an open mind and competent staff, just about all categories of tests can be pulled forward to some degree. This is especially true in older companies.

Careful examination of line-item tests in company test suites will yield a number of findings. Some tests will proof design, but aren’t applied until well after the product is in the pilot or ramp-up phase. These tests are all candidates for being moved earlier to the prototype stage, or even earlier than that, by converting them to algorithms that run against solid models or EDA schematics and layouts.

Some tests are done in the field just before launch, or by agreement with an initial group of customers. Must all these tests still wait until this late stage? Or, has the product's evolution to electronics-based and software-driven hardware now created the opportunity to conduct some of these tests earlier in development?

The test suite of a company is a major asset worth millions. It evolved as the company evolved. For older companies, most of the test-suite asset was put in place during the mechanical and electromechanical eras. Somewhat understandably, the organizations chartered with building and maintaining the test suite did not foresee how quickly software would overlay the product line. As such, even more recently developed line-item tests were placed where they had been historically. Historical placement may still be occurring today.