Standardizing Computational Models: Verification, Validation and Uncertainty Quantification

Computational models and simulations have had an important role in engineering analysis since as far back as the 1960s. It is widely recognized that the use of modeling and simulation tools can make processes safer, more innovative and less costly than physical testing and can improve time to market for new products and designs. Increasing reliance on simulations for a vast array of complex engineering projects has made it necessary to use verification, validation and uncertainty quantification (VVUQ) to ensure that the results from simulations are credible. 

Recognizing this challenge, the American Society of Mechanical Engineers (ASME) officially formed a committee to address the issue of verification and validation in 2001. The ASME V&V 10 Subcommittee in Computational Solid Mechanics undertook the task of writing proposed guidelines. Its membership consisted of solid mechanics analysts, experimenters, code developers and managers from industry, government and academia.

Represented industries included aerospace/defense, commercial aviation, automotive, bioengineering and software development; represented government agencies include the U.S. Department of Defense, the Department of Energy, and the Federal Aviation Administration. By 2006, the first edition of ASME V&V 10, Guide for Verification and Validation in Computational Solid Mechanics, was published. 

As the initial standard was developed, so too did the scope of the V&V (now VVUQ Committee). The standards committee’s name was changed in 2021 from V&V to VVUQ in order to highlight the importance of uncertainty quantification when analyzing the results of modeling and simulation activities. VVUQ is cross-cutting and used by a wide array of engineering disciplines and industries where computer simulation is accessible for purposes such as design, testing and analysis.

The need for VVUQ standards in many areas is shown by the different standards development subcommittees that were added over the decades. These groups now address not only solid mechanics, but also include standards on fluid dynamics and heat transfer and medical devices.

Additional standards are being developed in the areas of nuclear thermal fluids behavior, advanced manufacturing, energy systems, artificial intelligence and machine learning, pharmaceutical products and airframe structures. All these subcommittees operate under the ASME VVUQ Standards Committee. 

Ensuring Consistency 

The ASME Standards Committee on VVUQ in Computational Modeling and Simulation standards provide procedures for assessing and quantifying the accuracy and credibility of computational models and simulations. The goal of the standards is to not only describe the semantics of VVUQ, but to define the process of performing VVUQ in a manner that facilitates communication and understanding among the various performers and stakeholders. 

In order to apply the procedures for establishing simulation credibility, one must first understand the general VVUQ concepts:  

• Verification—the process of comparing the computational (e.g., computer) model to the underlying mathematical model/equations. Verification answers the question, “Did we solve the problem/equations correctly?” 
• Validation—the process of comparing the computational model to the empirical data (e.g., How well does the computational model predict reality?). Validation answers the question, “Did we solve the correct problem/equations?” 
• Uncertainty Quantification—the process of mathematically assessing the uncertainties in modeling & simulation and experiments to determine the impact on final results. 

Consistent use of these concepts and terminology has been an issue across different industry groups and has been a continual issue in the development of consistent standards. In 2022, the first VVUQ 1 Verification, Validation, and Uncertainty Quantification Terminology in Computational Modeling and Simulation standard was published to try to standardize terminology for effective communication and understanding among the various performers and stakeholders. That standard is available for download on the ASME website.

Achieving Simulation Credibility in Analysis of an Aircraft Structure   

The following example from the aviation industry shows how the concepts from the currently published ASME standards are applied and involve input from different users or stakeholders such as engineers, scientists, analysts, experimenters, programmers, managers, decision-makers and regulators.  The example begins with the Validation Case diagram from the V&V 10 Standard on Computational Solid Mechanics and walks through the process steps to be followed for analyzing an aircraft’s structure. 

An aircraft’s structure is to be analyzed under conditions such as static loadings to determine optimum design and risks (e.g., fatigue). A hierarchical approach may be used to first investigate a component of the aircraft such as the wing before the higher-level subsystems are analyzed so as not to miss detailed processes occurring at these lower-level components. A model of the wing is developed in three parts (conceptual, mathematical, computational):

The conceptual model is defined to represent the basic structure of the wing. For this case, a wing can be represented by a tapered cantilever beam.  

The mathematical model is defined to represent the equations that govern the behavior of the beam representing the wing. For this case, equations including force, moment, stress and deflection with boundary conditions at the fixed and free ends comprise the mathematical model.  

The computational model is defined as the discretized form of the mathematical model so that it can be analyzed using a computer (e.g., Finite Element Analysis). 

After model development, a user is to perform code verification to determine if the computational model is performing as expected and that there are no “bugs” in the code. Code verification involves testing the code output against exact analytical solutions that are already known for the cantilever beam model under specific loading conditions. Following code verification, calculation verification (solution verification) is performed and involves estimating the error in putting the equations for the cantilever beam in approximate or discretized form. 

After obtaining simulation results, the user is to perform validation to determine how well the simulation results compare to empirical data, which can be obtained from sources including experiments. Performing uncertainty quantification is important for assessing how sources of uncertainty in modeling and simulation, experimentation or the real-world system affect the final output.

For instance, should any adjustments be made to the cantilever beam model to better represent the physical system and reduce uncertainty? If needed, the analysis is to be repeated with potential modifications to the model or experiment before analyzing the next (higher) level in the hierarchy.  

Potential Benefits in Increased Safety, Innovation, and Cost Savings 

Following the processes introduced in each of the ASME VVUQ standards can go a long way to help engineers explain the input parameters included in computational models and simulations and the inherent uncertainty levels of the results. This establishes the credibility of the results and can help users make predictions based on different engineering conditions. The reference shown below contains a quick guide for understanding VVUQ.  

Performing VVUQ enables stakeholders to invest in processes that are safer, more innovative and less costly than physical testing; improve time to market for new products and designs; obtain business objectives more efficiently; and advance digital engineering. Standards development activities in VVUQ are important to standardize practices in building credibility in computational modeling and simulation and make sound engineering decisions.

ASME standards in this space are already applied by many disciplines and industries including, but not limited to, computational solid mechanics, fluid dynamics and heat transfer and medical devices. The VVUQ subcommittees listed below are developing standards and supporting publications for wider adoption in specific application areas.    

Michelle Pagano is a standards & certification project engineer at the American Society of Mechanical Engineers. She has more than five years of experience in managing standards development projects and engaging with stakeholders to address industry and technological needs for various technical areas. Kate Hyam is the director of nuclear codes and standards at ASME, responsible for ASME’s portfolio of standards applicable to nuclear facilities and technology.