Standard for Verification and Validation in Computational Solid Mechanics

AN INTERNA TIONAL S T AND ARD

ASME V&V 10-2019

[Revision of ASME V&V 10-2006 (R2016)]

ASME V&V 10-2019

[Revision of ASME V&V 10-2006 (R2016)]

Standard for

Verification and Validation in Computational Solid Mechanics

AN INTERNATIONAL STANDARD

Date of Issuance: March 6, 2020

The next edition of this Standard is scheduled for publication in 2024.

ASME issues written replies to inquiries concerning interpretations of technical aspects of this Standard. Interpretations are published on the Committee web page and under https://go.asme.org/InterpsDatabase. Periodically certain actions of the ASME V&V Committee may be published as Cases. Cases are published on the ASME website under the V&V Committee Page at https://go.asme.org/VnVcommittee as they are issued.

Errata to codes and standards may be posted on the ASME website under the Committee Pages to provide corrections to incorrectly published items, or to correct typographical or grammatical errors in codes and standards. Such errata shall be used on the date posted.

The V&V Committee Page can be found at https://go.asme.org/VnVcommittee. There is an option available to automatically receive an e-mail notification when errata are posted to a particular code or standard. This option can be found on the appropriate Committee Page after selecting “Errata” in the “Publication Information” section.

ASME is the registered trademark of The American Society of Mechanical Engineers.

This international code or standard was developed under procedures accredited as meeting the criteria for American National Standards and it is an American National Standard. The Standards Committee that approved the code or standard was balanced to assure that individuals from competentand concerned interestshave had an opportunity to participate. The proposed code or standard was made available for public review and comment that provides an opportunity for additional public input from industry, academia, regulatory agencies, and the public-at- large.

ASME does not “approve,” “rate,” or “endorse” any item, construction, proprietary device, or activity.

ASME does not take any position with respect to the validity of any patent rights asserted in connection with any items mentioned in this document, and does not undertake to insure anyone utilizing a standard against liability for infringement of any applicable letters patent, nor assume any such liability. Users of a code or standard are expressly advised that determination of the validity of any such patent rights, and the risk of infringement of such rights, is entirely their own responsibility.

Participation by federal agency representative(s) or person(s) affiliated with industry is not to be interpreted as government or industry endorsement of this code or standard.

ASME accepts responsibility for only those interpretations of this document issued in accordance with the established ASME procedures and policies, which precludes the issuance of interpretations by individuals.

No part of this document may be reproduced in any form, in an electronic retrieval system or otherwise,

without the prior written permission of the publisher.

The American Society of Mechanical Engineers Two Park Avenue, New York, NY 10016-5990

Copyright © 2020 by

THE AMERICAN SOCIETY OF MECHANICAL ENGINEERS

All rights reserved Printed in U.S.A.

CONTENTS

Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv

Committee Roster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v

Correspondence With the V&V Committee . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii

1 Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

3 Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

4 Model Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

5 Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

6 Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

7 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Mandatory Appendices

iii

FOREWORD

Since the mid-1960s, computer simulations have come to dominate engineering mechanics analysis for all but the simplest problems. This reliance on complicated simulations makes a systematic program of verification and validation (V&V) necessary to ensure the accuracy of these simulations. This Standard describes such a program.

The concept of systematic V&V is not new. The software development community has long recognized the need for a software quality assurance (SQA) plan for scientific and engineering products. The Institute of Electrical and Electronic Engineers (IEEE) was the first to publish and adopt guidelines and standards for engineering SQA appropriate for developers. SQA guidelines, while necessary, are not sufficient to cover the issues of computational physics and engi- neering or the vast array of problems to which end users apply the codes. To fill this gap, the concept of application- specific V&V was developed.

Scientific and engineering communities have been exploring application-specific V&V since the mid-1990s. The Department of Defense’s Defense Modeling and Simulation Coordination Office (DMSCO) produced recommended prac- tices suitable for large-scale modeling and simulation in 1996. However, these DMSCO guidelines do not directly focus on the details of computational physics and engineering. The American Institute of Aeronautics and Astronautics produced the first V&V guidelines tailored for detailed analyses in the area of computational fluid dynamics (CFD) in 1998.

Recognizing the need for a similar set of guidelines for computational solid mechanics (CSM), members of the CSM community formed a committee under the auspices of the United States Association for Computational Mechanics in 1999. The American Society of Mechanical Engineers (ASME) Board on Performance Test Codes (PTC) granted the committee official status in 2001 and designated it the PTC 60 Committee on Verification and Validation in Computational Solid Mechanics. In 2008, an overarching committee for multiple V&V application areas was established by ASME as the V&V Standards Committee on Verification and Validation in Computational Modeling and Simulation. ASME reorganized the committees under the V&V Standards Committee, and the PTC 60 Committee was renamed the V&V 10 Subcommittee on Verification and Validation in Computational Solid Mechanics.

The V&V 10 Subcommittee (previously PTC 60 Committee) undertook the task of writing the proposed guidelines. Its membership has consisted of solid mechanics analysts, experimenters, code developers, and managers from industry, government, and academia. Represented industries include aerospace/defense, commercial aviation, automotive, bioen- gineering, and software development; represented government agencies include the Department of Defense, the Depart- ment of Energy, and the Federal Aviation Administration.

Early discussions within the V&V 10 Subcommittee revealed an immediate need for a common language and process definition for V&V appropriate for CSM analysts and their managers and customers. The first edition of ASME V&V 10, Guide for Verification and Validation in Computational Solid Mechanics, described the semantics of V&V and defined the process of performing V&V in a manner that facilitates communication and understanding among the various performers and stakeholders.

The Guide was approved by the V&V 10 Subcommittee and was approved and adopted by the American National Standards Institute in 2006. Since that original edition was released, the issues and problems of V&V in CSM have been studied through discussion and the generation of supporting documentation, including an example problem standard, ASME V&V 10.1. That work contributed to the maturation of the discipline and influenced this revised edition, which is now titled Standard for Verification and Validation in Computational Solid Mechanics.

This Standard is available for public review on a continuing basis. This provides an opportunity for additional public- review input from industry, academia, regulatory agencies, and the public-at-large.

ASME V&V 10 was approved by the V&V Standards Committee on March 28, 2019 and was approved and adopted by the American National Standards Institute on July 23, 2019.

iv

ASME V&V COMMITTEE

Verification and Validation in Computational Modeling and Simulation

(The following is the roster of the Committee at the time of approval of this Standard.)

STANDARDS COMMITTEE OFFICERS

T. M. Morrison, Chair

B. H. Thacker, Vice Chair

K. M. Hyam, Secretary

STANDARDS COMMITTEE PERSONNEL

M. D. Benedict, Air Force Research laboratory

J. Bischoff, Zimmer Biomet

S. W. Doebling, Los Alamos National Laboratory

K. Dowding, Sandia National Laboratories

C. J. Freitas, Southwest Research Institute

Y. Hassan, Texas A&M University

M. Horner, ANSYS, Inc.

K. M. Hyam, The American Society of Mechanical Engineers

J. S. Kaizer, U.S. Nuclear Regulatory Commission

D. M. Moorcroft, Federal Aviation Administration

T. M. Morrison, U.S. Food and Drug Administration

S. Rachuri, U.S. Department of Energy

R. R. Schultz, Consultant

V. Sharma, Exponent, Inc.

B. H. Thacker, Southwest Research Institute

A. F. Emery, Contributing Member, University of Washington

V&V 10 SUBCOMMITTEE — VERIFICATION AND VALIDATION IN COMPUTATIONAL SOLID MECHANICS

D. M. Moorcroft, Chair, Federal Aviation Agency

J. L. O’Daniel, Vice Chair, U.S. Army Corps of Engineers, Engineer Research and Development Center

M. Pagano, Secretary, The American Society of Mechanical Engineers

M. C. Anderson, National Nuclear Security Administration

S. Atamturktur, The Pennsylvania State University

M. D. Benedict, Air Force Research Laboratory

J. Budzien, Los Alamos National Laboratory

R. M. Ferencz, Lawrence Livermore National Laboratory

L. Gutkin, Kinectrics

S. L. Kieweg, Sandia National Laboratories

D. Lancaster, Pratt & Whitney

I. Lopez, Lawrence Livermore National Laboratory

W. L. Oberkampf, William L. Oberkampf Consulting

G. Orient, Sandia National Laboratories

C. F. Popelar, Southwest Research Institute

C. Robeck, Thornton Tomasetti - Weidlinger Applied Science

B. H. Thacker, Southwest Research Institute

J. V. Cox, Alternate, Sandia National Laboratories

1. R. Witkowski, Alternate, Sandia National Laboratories

   B. Benowitz, Contributing Member, Thornton Tomasetti

   1. W. Doebling, Contributing Member, Los Alamos National Laboratory

   2. K. Hasselman, Contributing Member, Consultant

   S. R. Hsieh, Contributing Member, Lawrence Livermore National Laboratory

2. Jiang, Contributing Member, General Electric Co.

R. W. Logan, Contributing Member, Consultant

H. U. Mair, Contributing Member, Johns Hopkins University Applied Physics Laboratory

T. L. Paez, Contributing Member, Thomas Paez Consulting

R. Rebba, Contributing Member, General Motors R&D Center

C. P. Rogers, Contributing Member, Crea Consultants, Ltd.

1. F. Schultze, Contributing Member, Los Alamos National Laboratory

2. Teferra, Contributing Member, Consultant

   
   

   v

   
   

   CORRESPONDENCE WITH THE V&V COMMITTEE

   
   

   General. ASME Standards are developed and maintained with the intent to represent the consensus of concerned interests. As such, users of this Standard may interact with the Committee by requesting interpretations, proposing revisions or a case, and attending Committee meetings. Correspondence should be addressed to:

   
   

   Secretary, V&V Standards Committee

   The American Society of Mechanical Engineers Two Park Avenue

   New York, NY 10016-5990

   https://go.asme.org/Inquiry

   
   

   Proposing Revisions. Revisions are made periodically to the Standard to incorporate changes that appear necessary or desirable, as demonstrated by the experience gained from the application of the Standard. Approved revisions will be published periodically.

   The Committee welcomes proposals for revisions to this Standard. Such proposals should be as specific as possible, citing the paragraph number(s), the proposed wording, and a detailed description of the reasons for the proposal, including any pertinent documentation.

   
   

   Proposing a Case. Cases may be issued to provide alternative rules when justified, to permit early implementation of an approved revision when the need is urgent, or to provide rules not covered by existing provisions. Cases are effective immediately upon ASME approval and shall be posted on the ASME Committee web page.

   Requests for Cases shall provide a Statement of Need and Background Information. The request should identify the Standard and the paragraph, figure, or table number(s), and be written as a Question and Reply in the same format as existing Cases. Requests for Cases should also indicate the applicable edition(s) of the Standard to which the proposed Case applies.

   
   

   Interpretations. Upon request, the V&V Standards Committee will render an interpretation of any requirement of the Standard. Interpretations can only be rendered in response to a written request sent to the Secretary of the V&V Standards Committee.

   Requests for interpretation should preferably be submitted through the online Interpretation Submittal Form. The form is accessible at https://go.asme.org/InterpretationRequest. Upon submittal of the form, the Inquirer will receive an automatic e-mail confirming receipt.

   If the Inquirer is unable to use the online form, he/she may mail the request to the Secretary of the V&V Standards Committee at the above address. The request for an interpretation should be clear and unambiguous. It is further rec- ommended that the Inquirer submit his/her request in the following format:

   
   

   Subject: Cite the applicable paragraph number(s) and the topic of the inquiry in one or two words. Edition: Cite the applicable edition of the Standard for which the interpretation is being requested.

   Question: Phrase the question as a request for an interpretation of a specific requirement suitable for general understanding and use, not as a request for an approval of a proprietary design or situation. Please provide a condensed and precise question, composed in such a way that a “yes” or “no” reply is acceptable.

   Proposed Reply(ies): Provide a proposed reply(ies) in the form of “Yes” or “No,” with explanation as needed. If entering replies to more than one question, please number the questions and replies.

   Background Information: Provide the Committee with any background information that will assist the Committee in understanding the inquiry. The Inquirer may also include any plans or drawings that are necessary to explain the question; however, they should not contain proprietary names or information.

   
   

   vi

   Requests that are not in the format described above may be rewritten in the appropriate format by the Committee prior to being answered, which may inadvertently change the intent of the original request.

   Moreover, ASME does not act as a consultant for specific engineering problems or for the general application or understanding of the Standard requirements. If, based on the inquiry information submitted, it is the opinion of the Committee that the Inquirer should seek assistance, the inquiry will be returned with the recommendation that such assistance be obtained.

   ASME procedures provide for reconsideration of any interpretation when or if additional information that might affect an interpretation is available. Further, persons aggrieved by an interpretation may appeal to the cognizant ASME Committee or Subcommittee. ASME does not “approve,” “certify,” “rate,” or “endorse” any item, construction, proprietary device, or activity.

   
   

   Attending Committee Meetings. The V&V Standards Committee regularly holds meetings and/or telephone confer- ences that are open to the public. Persons wishing to attend any meeting and/or telephone conference should contact the Secretary of the V&V Standards Committee. Future Committee meeting dates and locations can be found on the Committee Page at https://go.asme.org/VnVcommittee.

   
   

   vii

   
   

   PREFACE

   
   

   The ASME V&V 10 Subcommittee on Verification and Validation (V&V) in Computational Solid Mechanics is creating a family of standards that together present a comprehensive picture of the standards and practices governing this process.

   1. ASME V&V 10-2006, Guide for Verification and Validation in Computational Solid Mechanics, was the first edition of ASME V&V 10. Intended as an overview of V&V, it also included background material and definitions necessary to understand the other standards in the series. It contains definitions of key terms associated with V&V, and it provides context for the role of V&V in engineering as well as an overview of key aspects of application. ASME V&V 10-2019 is the first revision of that Guide. Since publication of the first edition, the field of V&V has matured to the point that ASME V&V 10’s title has been changed from “Guide” to “Standard.”

   2. ASME V&V 10.1-2012, An Illustration of the Concepts of Verification and Validation in Computational Solid Mechanics, is a follow-on publication that illustrates the steps in the V&V process described in ASME V&V 10-2019 through a worked example. This Standard is intended to provide a more concrete look at how to translate the process of V&V 10-2019 into the reality of an engineering project.

   3. ASME V&V 10.2 is currently under development with the working title The Role of Uncertainty Quantification in Verification and Validation of Computational Solid Mechanics. This Standard is intended to take a deeper look at the importance of uncertainty quantification (UQ), types and characterization of uncertainties, introduction to UQ meth- odologies, and how UQ is applied during each phase of the V&V process.

   4. ASME V&V 10.3 is currently under development with the working title The Role of Validation Metrics in Verification and Validation of Computational Solid Mechanics. This Standard is intended to provide a primer of mathematical metrics to measure the difference between calculated results and either analytical or semianalytical solutions (in the case of code verification) or experimental measurements (in the case of validation).

Readers are encouraged to begin with ASME V&V 10-2019 as it lays the groundwork, but may find a concurrent reading of ASME V&V 10.1-2012 beneficial, as it closely follows the V&V process described through an example. ASME V&V 10.2 and ASME V&V 10.3 should be read subsequent to ASME V&V 10-2019, as the foundation of ASME V&V 10-2019 is necessary to understand the significance of the deeper treatments in ASME V&V 10.2 and ASME V&V 10.3.

viii

ASME V&V 10-2019

STANDARD FOR VERIFICATION AND VALIDATION IN COMPUTATIONAL SOLID MECHANICS

1 EXECUTIVE SUMMARY

Program managers need assurance that computational models of engineered systems are sufficiently accurate to support programmatic decisions. This Standard provides the technical community — engineers, scientists, and program managers — with guidelines for assessing the credibility of computational solid mechanics (CSM) models. Verification and validation (V&V) are the processes by which evidence is gathered to determine the accuracy of the computer model for specified conditions. These accuracy results, along with uncertainty quantification (UQ), contribute

to the determination of the credibility of the model for the conditions of its intended use.

Professional organizations differ in their definitions of V&V. The American Society of Mechanical Engineers (ASME) V&V 10 Subcommittee on Verification and Validation in Computational Solid Mechanics has chosen definitions consistent with those published by the Department of Defense (DoD) (ref. [1]) and by the American Institute of Aeronautics and Astronautics (AIAA) (ref. [2]). Verification assesses the numerical accuracy of a computational model regardless of the physics being modeled. Both code verification (addressing errors in the software and numerical algorithms) and calcula- tion verification (estimating the numerical errors due to under-resolved discrete representations of the mathematical model) are addressed. Validation assesses the degree to which the computational model is an accurate representation of the physics being modeled. It is based on comparisons between numerical simulations and relevant experimental results. Validation is essential in assessing the predictive capability of the model in the physical realm of interest, and it must address uncertainties that arise from both experimental and computational procedures.

As shown in Figure 2.3-1, the general V&V process begins with a statement of the intended use of the model and pertinent information about the system being modeled so that the relevant physics are included in both the model and the experiments performed to validate the model. Modeling and experimental activities are guided by the response quantities of interest and the accuracy requirements for the intended use. Experimental outputs intended for validation for compo- nent-level to system-level tests should, whenever possible, be provided to modelers only after verification and the numerical simulations for those outputs have been performed.

Ideally, the V&V process for a particular application ends with acceptable agreement between model predictions and experimental outputs, after the uncertainties in both have been taken into account. If the agreement between model and experiment is not acceptable, an assessment should be performed to determine why agreement was not met and, poten- tially, the processes of V&V repeated by updating the model and performing additional experiments. Successful comple- tion of the validation process, demonstrated by satisfactory agreement between simulation and experiment, means that the model adequately reproduces the experimental measurements that have been obtained.

Once the validation process is successfully completed, the model should be assessed to determine if its predictive capability, including relevant uncertainties, is adequate for conditions where no experimental data are available. Since most models are developed for use where experimental data are not available, predictive capability must address a much wider range of uncertainties than validation. This Standard introduces the concept of predictive capability but does not go into detail because of the early stage of development of this field.

Finally, it is important to document all V&V activities. In addition to preserving the compiled evidence of V&V, docu- mentation records the justifications for important decisions such as selecting primary response quantities and setting accuracy requirements. Documentation thereby supports the primary objective of V&V: to build confidence in the predic- tive capability of computational models.

The guidance provided herein will enable managers and practitioners of V&V to better assess and enhance the cred- ibility of CSM models. Upon reading about the process described in this Standard and illustrated in ASME V&V 10.1-2012, engineers may be left with the sense that the real-world constraints of the engineering environment (i.e., schedule and budget) do not allow for sufficient project scope to complete the V&V process to a satisfactory level of rigor. Users of this Standard are nonetheless encouraged to provide a V&V foundation for their engineering calculations and identify any associated uncertainties and risks.

1