T-VEC Tool Qualification
T-VEC has an up-to-date qualification suite that was applied in all prior FAA and FDA certifications. The T-VEC tools are categorized as “Verification Tools” that cannot introduce errors but may fail to detect them. The qualification suite is compliant with FAA Software Approval Guidelines, 8110.49, Chapter 9, Qualification of Software Tools Using RTCA/DO-178B.
The T-VEC qualification package includes a Qualification Plan that defines the plan for performing verification of formalized specifications that refine the Operational Requirements. The set of operational requirements trace to formal specifications and associated tests to verify each component of the T-VEC system, including the compiler, test generator, coverage analysis, and test driver components. In addition, tests are performed to demonstrate that the various operations and parameters that control these components operate properly. Analysis has been performed to ensure that the supported data types, language constructs, and a comprehensive set of subsystem integration tests have been verified. The outputs are verified manually, and all source specifications, tests, and test results are controlled within a CM system. The results of test execution are formally documented in the Verification Results. The Tool Qualification Accomplishment Summary summarizes the results of the tool qualification process, identifies the tool versions and associated operating environment, and describes and references the relevant tool qualification data.
Qualification of TTM
The key capabilities required for normal use of the TTM modeling system are the primary focus of the TTM qualification. Functionality of the tools that contribute to the TTM modeling and its related integration to support test vector generation, requirement-based coverage analysis, and test driver generation are in scope of the qualification. All other user functionality that is not directly related to the production of test vectors and test drivers is not in the scope of the TTM qualification (e.g., model searching features).
The primary focus for the TTM qualification is to ensure that the model translator provides a well-formed representation of each type of modeling construct. The TTM model translator converts a model, which is composed of tables of functions, conditions, events, and modes, into a test specification model. The TTM modeling language and constructs rely on a subset of the capabilities of the T-VEC system because the TTM language is currently not as expressive as the T-VEC specification language.
In a manner similar to the T-VEC qualification, the specification and associated tests cover the modeling language and data types that are mapped to the constructs of T-VEC specification language. The verification of these constructs has been verified in the T-VEC qualification. The purpose of the tests associated with these requirements is to demonstrate that each construct for each data type is properly translated into a valid T-VEC representation. Requirement traceability has been verified to ensure that each specified TTM requirement is translated into a T-VEC specification that will result in test vectors to cover that requirement. The basic approach to the specification supports automated testing, with automated results comparison. The results have been validated manually, and all artifacts are configuration controlled.
The tool qualification for the Simulink and Stateflow translator is continually evolving. The philosophy of this translator is the same as that of the TTM translator. The objective is to ensure that all constructs that can be represented in the Simulink and Stateflow are completely and consistently translated to a T-VEC specification. In addition, there are a number of translation features that can be specified in the translation interface to enable various types of structural and decision coverage over various modeling constructs of the Simulink and Stateflow language. The following summarizes some verification categories for the tests of the qualification suite:
- Primitive blocks (modeling constructs) tests exercised with coverage options
- Stateflow tests
- Subsystem integration coverage tests
- Signal range propagation tests
- Inline tests
- Test driver generation tests
- Test sequence tests