Guide for Verification and Validation in Computational Solid Mechanics

A N A M E R I C A N N A T I O N A L S T A N D A R D

Intentionally left blank

ASME V&V 10-2006

Guide for Verification and Validation in Computational Solid Mechanics

AN AMERICAN NA TIONAL S T AND ARD

Three Park Avenue • New York, NY 10016

Date of Issuance: December 29, 2006

This Guide will be revised when the Society approves the issuance of a new edition. There will be no addenda issued to ASME V&V 10-2006.

ASME issues written replies to inquiries concerning interpretations of technical aspects of this document. Periodically certain actions of the ASME V&V Committees will be published as cases. Cases and interpretations are published on the ASME Web site under the Committee Pages at https://cstools.asme.org as they are issued.

ASME is the registered trademark of The American Society of Mechanical Engineers.

This Guide was developed under procedures accredited as meeting the criteria for American National Standards. The Standards Committee that approved the Guide was balanced to assure that individuals from competent and concerned interests have had an opportunity to participate. The proposed code or standard was made available for public review and comment that provides an opportunity for additional public input from industry, academia, regulatory agencies, and the public-at-large.

ASME does not “approve,” “rate,” or “endorse” any item, construction, proprietary device, or activity.

ASME does not take any position with respect to the validity of any patent rights asserted in connection with any items mentioned in this document, and does not undertake to insure anyone utilizing a standard against liability for infringement of any applicable letters patent, nor assumes any such liability. Users are expressly advised that determina- tion of the validity of any such patent rights, and the risk of infringement of such rights, is entirely their own responsibility.

Participation by federal agency representative(s) or person(s) affiliated with industry is not to be interpreted as government or industry endorsement of this code or standard.

ASME accepts responsibility for only those interpretations of this document issued in accordance with the established ASME procedures and policies, which precludes the issuance of interpretations by individuals.

No part of this document may be reproduced in any form, in an electronic retrieval system or otherwise,

without the prior written permission of the publisher.

The American Society of Mechanical Engineers Three Park Avenue, New York, NY 10016-5990

Copyright © 2006 by

THE AMERICAN SOCIETY OF MECHANICAL ENGINEERS

All rights reserved Printed in U.S.A.

CONTENTS

Foreword iv

Committee Roster v

Correspondence With the PTC 60 Committee vii

Preface viii

1. Executive Summary 1

2. Introduction 1

3. Model Development 8

4. Verification 12

5. Validation 16

6. Concluding Remarks 19

Figures

1. Elements of V&V 2

2. Hierarchical Structure of Physical Systems 4

3. Example of Bottom-Up Approach to V&V 4

4. V&V Activities and Products 6

5. Path From Conceptual Model to Computational Model 9

Table

1 PIRT Example 9

Mandatory Appendices

iii

FOREWORD

Since the mid-1960s, computer simulations have come to dominate engineering mechanics analysis for all but the simplest problems. With today’s increasing reliance on complicated simula- tions using computers, it is necessary to use a systematic program of verification and validation (V&V) to ensure the accuracy of these simulations. This document is intended to describe such a program.

The concept of systematic V&V is not a new one. The software development community has long recognized the need for a quality assurance program for scientific and engineering software. The Institute of Electrical and Electronic Engineers, along with other organizations, has adopted guidelines and standards for software quality assurance (SQA) appropriate for developers. SQA guidelines, while necessary, are not sufficient to cover the nuances of computational physics and engineering or the vast array of problems to which end-users apply the codes. To fill this gap, the concept of application-specific V&V was developed.

Application-specific V&V has been the focus of attention for several groups in scientific and engineering communities since the mid-1990s. The Department of Defense’s Defense Modeling and Simulation Office (DMSO) produced recommended practices suitable for large-scale simulations. However, the DMSO guidelines generally do not focus on the details of first-principles–based computational physics and engineering directly. For the area of computational fluid dynamics (CFD), the American Institute of Aeronautics and Astronautics (AIAA) produced the first V&V guidelines for detailed, first-principle analyses.

Recognizing the need for a similar set of guidelines for computational solid mechanics (CSM), members of the CSM community formed a committee under the auspices of the United States Association for Computational Mechanics in 1999. The American Society of Mechanical Engineers (ASME) Board on Performance Test Codes (PTC) granted the committee official status in 2001 and designated it as the PTC 60 Committee on Verification and Validation in Computational Solid Mechanics. The PTC 60 committee undertook the task of writing these guidelines. Its membership consists of solid mechanics analysts, experimenters, code developers, and managers from industry, government, and academia. Industrial representation includes the aerospace/ defense, commercial aviation, automotive, bioengineering, and software development industries. The Department of Defense, the Department of Energy, and the Federal Aviation Administration represent the government.

Early discussions within PTC 60 revealed an immediate need for a common language and process definition for V&V appropriate for CSM analysts, as well as their managers and customers. This document describes the semantics of V&V and defines the process of performing V&V in a manner that facilitates communication and understanding among the various performers and stakeholders. Because the terms and concepts of V&V are numerous and complex, it was decided to publish this overview document first, to be followed in the future by detailed treatments of how to perform V&V for specific applications.

Several experts in the field of CSM who were not part of PTC 60 reviewed a draft of this document and offered many helpful suggestions. The final version of this document was approved by PTC 60 on May 11, 2006 and was approved and adopted by the American National Standards Institute on November 3, 2006.

iv

PERFORMANCE TEST CODE COMMITTEE 60

Verification and Validation in Computation Solid Mechanics

(The following is the roster of the Committee at the time of approval of this Guide.)

OFFICERS

1. E. Schwer, Chair

   H. U. Mair, Vice Chair

   R. L. Crane, Secretary

   
   

   
   

2. C. Anderson, Los Alamos National Laboratory

J. M. Burns, Liaison Member, Burns Engineering

J. A. Cafeo, General Motors Corporation

COMMITTEE PERSONNEL

D. K. Pace, Consultant

T. Paez, Sandia National Laboratories

A. B. Pifko, Consultant

1. L. Crane, The American Society of Mechanical Engineers

2. W. Doebling, Los Alamos National Laboratory

J. H. Fortna, ANSYS

M. E. Giltrud, Defense Threat Reduction Agency

J. K. Gran, SRI International

T. K. Hasselman, Acta, Inc.

H. M. Kim, Boeing

R. W. Logan, Lawrence Livermore National Laboratory

H. U. Mair, Institute for Defense Analyses

A. K. Noor, Old Dominion University

W. L. Oberkampf, Sandia National Laboratories

J. T. Oden, University of Texas

L. Proctor, MSC Software

J. N. Reddy, Texas A & M University

P. J. Roache, Consultant

L. E. Schwer, Schwer Engineering

P. E. Senseny, Consultant

M. S. Shephard, Rensselaer Polytechnic Institute

D. A. Simons, Northrop Grumman

W. G. Steele, Jr., Liaison Member, Mississippi State University

B. H. Thacker, Southwest Research Institute

T. G. Trucano, Sandia National Laboratories

R. J. Yang, Ford Motor Co.

Y. Zhao, St. Jude Medical

The Committee acknowledges the contributions of the following individuals associated with the Committee:

G. Gray, Former Member, Los Alamos National Laboratory

F. Hemez, Alternate, Los Alamos National Laboratory

R. Lust, Alternate, General Motors Corp.

1. Nitta, Alternate, Lawrence Livermore National Laboratory

   V. Romero, Alternate, Sandia National Laboratories

   
   

   v

   
   

   PERFORMANCE TEST CODES STANDARDS COMMITTEE

   OFFICERS

   J. G. Yost, Chair

   J. R. Friedman, Vice Chair

   J. Karian, Secretary

   
   

   
   

   P. G. Albert

   R. P. Allen

   J. M. Burns

   W. C. Campbell

   M. J. Dooley

   A. J. Egli

   J. R. Friedman

   G. J. Gerber

   P. M. Gerhart

   T. C. Heil

   R. A. Johnson

   D. R. Keyser

   S. J. Korellis

   COMMITTEE PERSONNEL

   
   

   M. P. McHale

   P. M. McHale

   J. W. Milton

   S. P. Nuspl

   A. L. Plumley

   R. R. Priestley

   J. A. Rabensteine

   J. W. Siegmund

   J. A. Silvaggio

   W. G. Steele

   J. C. Westcott

   W. C. Wood

   J. G. Yost

   
   

   HONORARY MEMBERS

   W. O. Hays

   F. H. Light

   
   

   
   

   R. L. Bannister

   R. Jorgensen

   MEMBERS EMERITI

   
   

   G. H. Mittendorf

   R. E. Sommerlad

   
   

   
   

   vi

   
   

   CORRESPONDENCE WITH THE PTC 60 COMMITTEE

   
   

   General. ASME Codes are developed and maintained with the intent to represent the consensus of concerned interests. As such, users of this Guide may interact with the Committee by requesting interpretations, proposing revisions, and attending Committee meetings. Correspondence should be addressed to:

   Secretary, PTC 60 Committee

   The American Society of Mechanical Engineers Three Park Avenue

   New York, NY 10016-5990

   Proposing Revisions. Revisions are made periodically to the Guide to incorporate changes that appear necessary or desirable, as demonstrated by the experience gained from the application of the Guide. Approved revisions will be published periodically.

   The Committee welcomes proposals for revisions to this Guide. Such proposals should be as specific as possible, citing the paragraph number(s), the proposed wording, and a detailed descrip- tion of the reasons for the proposal, including any pertinent documentation.

   Interpretations. Upon request, the PTC 60 Committee will render an interpretation of any requirement of the Guide. Interpretations can only be rendered in response to a written request sent to the Secretary of the PTC 60 Committee.

   The request for interpretation should be clear and unambiguous. It is further recommended that the inquirer submit his/her request in the following format:

   Subject: Cite the applicable paragraph number(s) and the topic of the inquiry.

   Edition: Cite the applicable edition of the Guide for which the interpretation is being requested.

   Question: Phrase the question as a request for an interpretation of a specific requirement suitable for general understanding and use, not as a request for an approval of a proprietary design or situation. The inquirer may also include any plans or drawings which are necessary to explain the question; however, they should not contain proprietary names or information.

   Requests that are not in this format will be rewritten in this format by the Committee prior to being answered, which may inadvertently change the intent of the original request.

   ASME procedures provide for reconsideration of any interpretation when or if additional information that might affect an interpretation is available. Further, persons aggrieved by an interpretation may appeal to the cognizant ASME Committee or Subcommittee. ASME does not “approve,” “certify,” “rate,” or “endorse” any item, construction, proprietary device, or activity. Attending Committee Meetings. The PTC 60 Committee regularly holds meetings, which are open to the public. Persons wishing to attend any meeting should contact the Secretary of the

   PTC 60 Committee.

   
   

   vii

   
   

   PREFACE

   
   

   This document provides general guidance for implementing verification and validation (V&V) of computational models for complex systems in solid mechanics. The guidance is based on the following key principles:

   1. Verification (addressing programming errors and estimating numerical errors) must precede validation (assessing a model’s predictive capability by comparing calculations with experiments).

   2. The need for validation experiments and the associated accuracy requirements for computa- tional model predictions are based on the intended use of the model and should be established as part of V&V activities.

   3. Validation of a complex system should be pursued in a hierarchical fashion from the component level to the system level.

   4. Validation is specific to a particular computational model for a particular intended use.

   5. Simulation results and experimental data must have an assessment of uncertainty to be meaningful.

Although the state of the art of V&V does not yet lend itself to writing a step-by-step perform- ance code/standard, this guide provides the computational solid mechanics (CSM) community with a common language and conceptual framework to enable managers and practitioners of V&V to better assess and enhance the credibility of CSM models. Implementation of a range of V&V activities is discussed, including model development for complex systems, verification of numerical solutions to governing equations, attributes of validation experiments, accuracy requirements, and quantification of uncertainties. Remaining issues for further development of a V&V protocol are identified.

viii

ASME V&V 10-2006

GUIDE FOR VERIFICATION AND VALIDATION IN COMPUTATIONAL SOLID MECHANICS

1. EXECUTIVE SUMMARY

   Program managers need assurance that computa- tional models of engineered systems are sufficiently accurate to support programmatic decisions. This docu- ment provides the technical community — engineers, scientists, and program managers — with guidelines for assessing the credibility of computational solid mechan- ics (CSM) models.

   Verification and validation (V&V) are the processes by which evidence is generated, and credibility is thereby established, that computer models have adequate accu- racy and level of detail for their intended use. Definitions of V&V differ among segments of the practicing commu- nity. The PTC 60 committee has chosen definitions con- sistent with those published by the Defense Modeling and Simulation Office of the Department of Defense (DoD) [1] and by the American Institute of Aeronautics and Astronautics (AIAA) in their 1998 Guide for the Verification and Validation of Computational Fluid Dynamics [2], which the present American Society of Mechanical Engineers (ASME) document builds upon. Verification assesses the numerical accuracy of a compu- tational model, irrespective of the physics being mod- eled. Both code verification (addressing errors in the software) and calculation verification (estimating numerical errors due to under-resolved discrete repre- sentations of the mathematical model) are addressed. Validation assesses the degree to which the computa- tional model is an accurate representation of the physics being modeled. It is based on comparisons between numerical simulations and relevant experimental data. Validation must assess the predictive capability of the model in the physical realm of interest, and it must address uncertainties that arise from both experimental and computational procedures.

   Although the state of the art of V&V does not yet lend itself to writing a step-by-step performance code/ standard, the guidance provided here will enable man- agers and practitioners of V&V to better assess and enhance the credibility of CSM models. The PTC 60 Committee recognizes that program needs and resources vary and that the application of the guidance provided in this document to specific cases must accommodate specific budget and risk considerations. The scope of this document is to explain the principles of V&V so that practitioners can better appreciate and understand

   how decisions made during V&V can impact their ability to assess and enhance the credibility of CSM models.

   As suggested by Fig. 1, the V&V processes begin with a statement of the intended use of the model so that the relevant physics are included in both the model and the experiments performed to validate the model. Modeling activities and experimental activities are guided by the response features of interest and the accuracy require- ments for the intended use. Experimental outcomes for component-level, subsystem-level, or system-level tests should, whenever possible, be provided to modelers only after the numerical simulations for them have been performed with a verified model. For a particular appli- cation, the V&V processes end with acceptable agreement between model predictions and experimental outcomes after accounting for uncertainties in both, allowing application of the model for the intended use. If the agreement between model and experiment is not acceptable, the processes of V&V are repeated by updat- ing the model and performing additional experiments. Finally, the importance of documentation in all of the V&V activities should be emphasized. In addition to preserving the compiled evidence of V&V, documenta- tion records the justifications for important decisions, such as selecting primary response features and setting accuracy requirements. Documentation thereby sup- ports the primary objective of V&V: to build confidence in the predictive capability of computational models. Documentation also provides a historical record of the V&V processes, provides traceability during an engi- neering audit, and captures experience useful in men-

   toring others.

   
   

2. INTRODUCTION

CSM is playing an increasingly important role in the design and performance assessment of engineered sys- tems. Automobiles, aircraft, and weapon systems are examples of engineered systems that have become more and more reliant on computational models and simula- tion results to predict their performance, safety, and reli- ability. Although important decisions are made based on CSM, the credibility (or trustworthiness) of these models and simulation results is oftentimes not ques- tioned by the general public, the technologists who design and build the systems, or the decision makers

1