Page Last Updated: Friday, 06 July 2018 11:34 EDT, 2002, 2008, 2011, 2012, 2016, 2018

Complex Operational and Organizational Problems

Topic: Verification, Validation & Accreditation (VV&A)

Dr. Dean S. Hartley III

Verification is the process of discovering the degree of success in implementing a conceptual model into computer code.  Validation is the process of discovering the nature and extent of the correspondence between the computer results and the real world.  Accreditation is the determination that a computer program is good enough for the purpose at hand.  Generally, data verification and validation are also included in the entire VV&A process.  Independent verification and validation (IV&V) is normally a component of VV&A and consists of an outside agent performing some verification and validation.  However, the bulk of the verification and validation procedures consists of the internal testing performed by the model builders and the user evaluations that take place each time the model is used.

Because most modern computer models are too complex for complete verification (certainly in one "sitting"), verification is a process that continues throughout the lifetime of the model.  On the other hand, the real world is so complex that computer models of the subtler aspects often address areas of the real world that are poorly understood.  Accordingly, validation must be an incremental and imprecise process.  These two facts make the need for official accreditation more than a formality:  at any given time, a model will be imperfectly understood, although its services may be needed.  Even for models of well-understood processes, such as projectile trajectories, accreditation may be more than a formality.  For example, if computational speed requires an imperfect approximation of the process, it will be required to determine if the known errors are sufficiently small to yield useful results.  In all cases, the purpose of the model use is important in the accreditation decision:  a model that is acceptible for one use may be totally unacceptible for another use.

Select one of the links below.

Year Client Project
1973 SAGA Convert combat models from IBM to Honeywell.
1973 SAGA Define documentation standards for combat models.
1975 SAGA Perform IV&V of the Tac Contender model.
1986 J8 Support the Modern Aids to Planning Program (MAPP).
1988 JAD Validate attrition algorithms using historical data.
1989 TRAC Perform IV&V of the Simulator Network for Training (SIMNET-T).
1990 Future Battle Lab Evaluate the use of SIMNET in WAREX 3-90.
1991 CFCK Analysis Build the Korean Battle Simulation Center (BSC).
1993 J8 Perform IV&V of the Future Theater Level Model (FTLM) conceptual model.
2001 Defense Modeling and Simulation Office (DMSO) Specify the architecture and create an OOTW Toolbox.
2002 Oak Ridge National Laboratory Study the problems of commonality in the Future Combat System (FCS).
2003 Hartley Consulting Create the Interim Semi-static Stability Model (ISSM).
2004 MORS Support running a workshop on GWOT.
2007 Defense Advanced Research Projects Office (DARPA) Create a VV&A methodology for the COMPOEX system.
2007 Naval Research Lab (NRL) Provide advice on DIME/PMESII tools.
2008 Hartley Consulting Create the DIME/PMESII VV&A Tool.
2008 MORS Support running a workshop on analyzing the impact of emerging societies.
2008 MORS Support running a workshop on understanding the consequences of catastrophic events.
2008 National Defense University (NDU) Support running an HSCB workshop.
2009 CERDEC Perform V&V on the HSCB Testbed.
2009 Hartley Consulting Create a community of interest on DIME/PMESII related discussions.
2010 TRAC Create an IW metric ontology.
2011 Hartley Consulting Improve the IW metric ontology to a total ontology.
2011 US SOUTHCOM Training Perform validation on the DEXES-II conceptual model.
2012 Air Force Research Laboratory Support the GLENS project development.
2012 TRAC Create a the IW metric ontology 2 from the total ontology.
2013 Hartley Consulting Create a model metadata ontology.
2014 Hartley Consulting Create an IW ontology GUI for the IW metric ontology 2.
2017 Hartley Consulting Extend the IW ontology to Unconventional Conflict and then to Modern Conflict.

Return to Website Entrance.

Solving Complex Operational and Organizational Problems
Dr. Dean S. Hartley III, Principal