about Modified Condition/Decision Coverage

A place for discussing topics that do not fit into the other VGS categories

about Modified Condition/Decision Coverage

Postby jack » Thu Aug 07, 2008 7:47 am

Hi,

As i know MC/DC is a code coverage metric,so what is the meaning of that TVEC can support MC/DC and Extended?
Is it means that model also have MC/DC metric?
jack
 
Posts: 16
Joined: Sat Mar 15, 2008 9:59 am

Re: about Modified Condition/Decision Coverage

Postby busser » Thu Aug 07, 2008 4:58 pm

HI Jack

MC/DC is actually a logical decision coverage metric that is most typically applied to decisions in executable code. In DO-178B it is applied at both the 3rd generation language level (C/C++, Ada, etc.) and also at the object code level. However, it is a general concept that can be applied to any language with an execution semantics. Simulink is also such a language. It is no different than C/C++ or Ada or any other form of coding language. It is simply graphically code rather than ascii text based code. Simulink model are compiled (using the RTW's Target Language Compiler (TLC) into a 3rd generation language such as C/C++ which is then compiled further into assembly/object level code for linking into a target executable.

MC/DC coverage in Simulink Tester is the goal of testing all logical decisions in the Simulink model, to an MC/DC coverage level, and this coverage is often measured in the code that gets generated by the RTW from the model by tools like LDRA.

In T-VEC VGS there is an option for Extended MC/DC which is a term that is sometimes applied to distinguish simple MC/DC which only requires 2 tests for

if (x >= y) then
else

Whereas Extended MC/DC requires 3 tests.

This VGS option is actually not very important in SImulink Tester because the same concepts are covered by ForcePaths and ForceCondition translation options. However, it does affect TTM translations of requirements models where >= and <= are actually decisions rather than active blocks that compute boolean signal values like in Simulink.
busser
Site Admin
 
Posts: 52
Joined: Thu Mar 13, 2008 7:42 pm

Re: about Modified Condition/Decision Coverage

Postby jyoung » Wed Mar 18, 2009 3:02 pm

My first of what will probably be many questions.

I have a simular setup in simulink that I'm using to understand what T-vec is doing. I have an if block that makes its decision based on u1<u2 (both are model inputs). Also I have the inpts constrained to 0-100

I do the translation and build. The result is 4 test vectors. [1.0e-8, 0.0] [1.0e2, 9.999999999e1], [0.0, 0.0], and [1.0e2,1.0e2] these are [u2, u1].

My question is why do I need 4 vectors ? Can I not show complete coverage with 2 vectors? One to go down the If path and the other to go down the else path. I think this a fundamental question about what t-vec is doing when it generates a set of vectors.

On a side note if I run thsi through DV I get a set of two vectors. WIthout any constraints they are [0,0], [0,-1]. I'm noty saying either is better I'm just trying to figure out the difference.

Thanks
jyoung
 
Posts: 33
Joined: Wed Mar 18, 2009 2:25 pm

Re: about Modified Condition/Decision Coverage

Postby busser » Wed Mar 18, 2009 4:10 pm

My question is why do I need 4 vectors ? Can I not show complete coverage with 2 vectors? One to go down the If path and the other to go down the else path. I think this a fundamental question about what t-vec is doing when it generates a set of vectors.


Coverage is not really the only desirable result. Analysis and detection of errors is also one of the goals of the test vector produced by T-VEC.

By default, T-VEC generates 2 test vectors for each DCP (Domain Convergence Path - i.e. logical constraint path). One vector is produced using LOW BOUND favoring rules and the other is produced using HIGH BOUND favoring rules. Beyond simple MCDC coverage is the examination of constraint solution sets near the boundaries of the solution space. It is a well known fact (and part of the foundation of Domain Testing Theory) that coders/specifiers/modelers, etc. make most mistakes near the domain boundaries of their problem space. For example, use of > where >= was really the required relationship. T-VEC's LOW/HIGH bound mode vector generation process attempts to provide default test vectors that hit more of the domain boundary points than just MCDC alone.

T-VEC's general algorithm first solves the constraint equations that govern the outputs being produced (the constraints of the DCP path) simultaneously in terms of variable domains. Once a stable input space is identified, it begins collapsing this space to a final single point, by collapsing the variables that make up the input space to a single point, one at a time, then allowing the convergence process to collapse the input space further around that variables chosen single point value. It continues to follow this approach until all variables are fixed to a single value, which identifies the point in the input space that defines the whole set of input values for that test vector. It then plugs these values into the output equation to come up with expected output values. The LOW/HIGH bound vector generation concept is what is used to decided how to collapse each individual un-fixed input variable to a single value. In LOW bound mode, the minimum possible value for that variable, at the current stage of the convergence process, is chosen as the fix-point value. In HIGH bound mode, it is the maximum value of the current domain.

Note, the domain of values is not necessarily the default input domain for the variable, it is the domain that results from solving the constraint equations of the DCP. In your example, even though both u1 and u2 had initial 0..100 domains, after applying the u1<u2 relations, these domains were converged so that any point in u1 will have a corresponding point in u2 that satisfies the u1<u2 relation.

There are other reasons for having a default minimum set of test vectors that include both LOW and HIGH bound vector generation passes, but those reasons are beyond the scope of your question.

On a side note if I run thsi through DV I get a set of two vectors. WIthout any constraints they are [0,0], [0,-1]. I'm noty saying either is better I'm just trying to figure out the difference.


Your point about "without any constraints" is precisely the issue - what good are points like [0,0], [0,-1]. Any one, any tool, can blindly pick points in the open unconstrained domain, then plug them in and see what happens and make a decision as to what to do with it. T-VEC is designed to eliminate this manual and tedious trial and error steps in testing by eliminating the "execute it (or simulate it) and see what happens" processes. The translation of Simulink models into T-VEC form identifies all of the logical paths in the model to be the focus of the test vector generation process and then identifies a set of input values that specifically cover that logical path. This is the reason it can then compute the expected results, because it knows exactly what output relationships are governed by that logic path - so it can plug in the inputs and compute the results - and do so independently from Simulink ! It is never a good idea to allow the tool used to do simulation and code generation to also be its own oracle to the expected results. Such a testing process is no better than software testers looking at the code being tested in order to come up with the tests. A successful test in such situations proves nothing more than the compiler did what it said that it would do. In the case of Simulink/Stateflow, using it as the oracle presumes that the RTW code generator is perfect and can't make mistakes. This is not the case. We have found many situations where the RTW code produced different results than was produced by the simulation engine and/or different results than T-VEC or a human would produced if calculating by hand.

I hope this answered your basic questions. Please feel free to continue with more questions on this topic, or any other.
busser
Site Admin
 
Posts: 52
Joined: Thu Mar 13, 2008 7:42 pm

Re: about Modified Condition/Decision Coverage

Postby jyoung » Wed Mar 18, 2009 4:59 pm

Thanks Bob, as always a very detailed response.

So what I take away is that T-VEC generates additionl test vectors beyond MCDC to provide addiution confidence in the models. The Upper and Lower bound are choosesn because that is the most likley location within teh design space where an error might occure. This is a good additional feacture and explains the difference from DV. I beleive they generate the minimal number of vectors to get MCDC or whatever coverage level you select.

Thanks for the help
jyoung
 
Posts: 33
Joined: Wed Mar 18, 2009 2:25 pm

Re: about Modified Condition/Decision Coverage

Postby busser » Thu Mar 19, 2009 8:54 am

So what I take away is that T-VEC generates additionl test vectors beyond MCDC to provide addiution confidence in the models.


Basically, yes. While T-VEC is great for generating both test vectors and test drivers, independently from either design models (Simulink/Stateflow) and/or requirements models (TTM - a modern industrial, extended version of SCRTool) - its primary goals is model analysis - i.e. the detection of errors in the models. Test vectors (and subsequent test drivers) are side effects of the model analysis process.

I beleive they generate the minimal number of vectors to get MCDC or whatever coverage level you select.


It has never been T-VEC's goal to just get a credit check mark in some standards/guidelines checklist. The goal has always been improving quality and cost reduction through early phase detection of errors and mis-understandings of requirements.
busser
Site Admin
 
Posts: 52
Joined: Thu Mar 13, 2008 7:42 pm


Return to General Topics

Who is online

Users browsing this forum: No registered users and 0 guests

cron