T-VEC Vector Generation System
The T-VEC Vector Generation System (VGS) is the engine to supports:
- Model analysis
- Test vector generation
- Test driver
- Test results analysis
- Project status and measurement
The VGS was first used in the FAA certification of the Traffic and Collision Avoidance (TCAS) system in 1990. T-VEC was applied to the entire MD90 (McDonnell Douglas) Electrical Power System Variable Speed Constant Frequency (VSCF) system that was FAA-certified in January of 1995.
In 1998 VGS was integrated with the original SCR tool. Since that time it has continued to evolve and support other integrations with various modeling tools such as MATRIXx and more recently Simulink/Stateflow.
VGS has a wide array of test vector generation options, but the default vector generation functions are accessible to user through a single menu option called Build that performs the build process based on the dependencies, much like make. The Build process perform a number of steps to support Test Vector Generation.
This function is also provide through a command or console function to permit batch-oriented test generation.
Test Vector Generation
The test vector generation produces a set of test vectors that include the inputs, expected outputs, and requirement traceability link. These vectors are generic in that the inputs and expected outputs are based on the names from the model. Users often visualize the vectors in a html table matrix to examine and assess the values produced in the process.
The resulting vectors can be applied to unit, integration, or system testing. The applicability of the vectors to theses various layers of testing is based on the model.
Test Driver Generation
Test drivers generation transforms generic test vectors into environment and programming language-specific test drivers.
The test driver generator and associated support tools use programmable schemas and user object mappings that relate the model variable to the implementation (or simulation) interfaces that are required to automatically execute the test driver in some target test environment.
Test drivers have been produced in many languages, such as C, C++, Java, SQL/ODBC/JDBC, XML, SOAP WinRunner, JCL, Perl, Python, Ada, Basis and VB, Custom (graphics),Assembler, shell, command languages, emulators, proprietary, more . . .
A Test Harness is often constructed manually to support a standard way to inject test inputs, control the execution, retrieve test output (or outcomes), and potentially to support other reporting. The Test Driver Generator produces the test harness as a byproduct of the generation process. It is based on a test driver schema.
Test Driver Schema
The test driver generator processes two user-defined schemas. One specifies the format of the expected outputs file and the other specifies the format of the target-specific test driver. These are referred to as the expected outputs and test driver schemas, respectively. Once at test driver schema is created, it can be reused to produce test drivers (and the associated harness or harness interfaces) for all testing associated with a specific test environment.
This is the process step where the user must cause the execution of the generated test driver against the test environment. The test driver causes the inject of test inputs and captures the actual output produced by the execution. These results are then compared with the expected outputs. VGS provides a test results comparison function to automate this activity.
In the Simulink Tester for T-VEC (SL2TVEC), test execution is done automatically as part of the Generate and Test function executed in the SL2TVEC GUI. Test execution is performed against the manually generated code, auto-generated code, or Matlab Simulation.
For tests derived from TTM models, these test might be executed in a variety of ways (see Examples). One handy mechanism is to use the VGS Toolbox to link to a mechanism for quickly performing the test execution.
Reports, Status and Measures
There are various reports produced by the VGS that are also hyperlinked to reports produced by TTM. Such reports provide status information, model defect summaries, with links to the detailed reports that hyperlink back to the TTM model or Simulink model. The reports include (all reports are in HTML and/or XML format):
- Model report - representation of TTM model
- Requirement report - representation of TTM requirements
- Project status report - Hyperlink to all other reports, with project measures
- Test vectors - representation of vectors
- Coverage analysis report - model defect status, with hyperlink to model defects
- Test results report - summary of test execution pass/fail for each vector
- Detailed defect report - provide model defect details