DO-178B and DO-178C
There are many questions related to DO-178B support from T-VEC as well as the implications from the proposed DO-178C. This page provides some information related to these topics, including the important topic of tool qualification, which is required to support a certification effort.
Software tools used to automate aspects of software development or verification that will be applied for certification credit, without formal review, must be qualified. The tool qualification task can be nontrivial and may be seen as a hindrance when choosing to use some types of tools on programs governed by this constraint. However, a major survey conducted as part of the FAA’s independently commissioned Streamlining Software Aspects of Certification (SSAC) program within the civil avionics development community included a section on tool qualification. The results of this survey indicated that 60% of the respondents considered the cost attributed to tool qualification to be small or negligible, 36% considered the cost to be substantial, and only 4% considered the cost to be prohibitive (Hayhurst et al. 1999). Fortunately for most users of T-VEC tools, the tool qualification cost should be minimal because the tool qualification packages have been developed for several tool components of the T-VEC tool set.
DO-178 and Tool Qualification History
The use of software tools to assist in developing application software has been common practice since the first assembler program was written to assist in the creation of machine code programs (Salmon 1993). While the correct functionality of a software tool has always been of a concern to the user of the tool, it is not typically a concern of the customer of the application. The responsibility for the functionality of the application is typically left to the application’s developer; the customer is not typically interested in how the developer actually produced the application software.
However, in the context of applications that have an impact on health or safety, or other high assurance-related areas such as security, the concerns for correct functionality are much greater. In such domains, verification of correct functionality under possibly all operating conditions often is not only a concern of the customer but also of regulatory agencies such as the FAA and FDA. These concerns are formalized in officially published guidelines such as the FAA’s DO-178B (U.S. DOT 1999). These guidelines form the basis for FAA approval of software-based avionics applications by establishing software development process completion and functionality verification criteria that must be met by the application developer in order to be approved by the FAA for commercial aviation use. These guidelines, DO-178B specifically, include sections on the use of software tools in the application development process and also introduce the concept of tool qualification. Consequently, to fully understand the meaning tool qualification and its impact on the use of T-VEC, a brief history of its derivation is useful.
History of DO-178B (Paraphrased From Johnson )
In the avionics industry, software was initially viewed as an inexpensive and more flexible way to extend the functionality of mechanical and analog-electrical systems. However, it was quickly realized that the usual statistical approaches to assess the safety and reliability would not work for flight-critical software. An alternative means of assessment, one that addressed design errors rather than component failure rates, was required. From this need, the first version of DO-178 was created.
DO-178 was created to provide a basis for software certification by identifying and documenting software development “best practices,” but it was written primarily at a conceptual level. To be compliant, applicants for certification were required to meet the “intent” of DO-178, but there were few details about how to actually do this. Rules of use were developed by trial and error over time. DO-178 was the first to introduce the concept that software verification requirements were dependent on the safety criticality of the software. It divided software applications into three categories: critical, essential, and nonessential.
DO-178 also established the relationship between the software certification process and the other relevant Federal Aviation Regulations (FARs), such as the Type Certification Approval, the Technical Standard Order (TSO) Authorization, and the Supplemental Type Certification.
While the first version of DO-178 introduced the concept of software certification, the lessons learned from attempting to apply it quickly pointed out the need for revision. RTCA Special Committee 152 was formed to produce this revision, DO-178A, which was published in 1985. DO-178A turned out to be very different from DO-178.
DO-178A described, in systematic and structured detail, software development and verification processes, something that was deemed missing from DO-178. It maintained the safety categories of software applications, critical, essential, and nonessential, but also introduced the associated concept of software certification Levels 1, 2 and 3 corresponding to these criticality categories. This was done to allow for variance between application criticality and software certification level. The idea was that while an entire application would be assigned a criticality level, the level of certification effort for the software making up the application could be reduced according to the system design and implementation techniques. For example, if adequate partitioning of the design could be shown, some of the software of a critical application may only need a Level 2 or 3 certification effort, rather than Level 1, because of its lesser role in the overall functioning of the critical application. As with DO-178, attempting to apply DO-178A led to many new problems. Interpretation of certification requirements differed from one FAA region to another. Certification deliverables were frequently contended between applicants and the certification authorities and misinterpretation of DO-178A’s intent sometimes led to entire software development life cycles being disallowed because they did not follow traditional waterfall-like processes. Lastly, there was an overall lack of understanding and appreciation for the purpose of the certification requirements by the industry in general.
During the time of DO-178A, the avionics industry was undergoing a major shift from analog to digital systems, and software was being applied in ever larger and more complex applications. Many new vendors of avionics applications came into existence and were now subject to software certification constraints for the first time. The lack of available DO-178A documentation, training material, development standards, and experienced people became apparent, and the industry again came to the conclusion that a new version of DO-178 was needed to address these issues and incorporate the new lessons learned from application of DO-178A.
To address these concerns, DO-178B was designed to satisfy the following three primary goals.
- Develop objectives for the life-cycle processes
- Provide a description of the activities and design considerations for achieving those objectives
- Provide a description of the evidence indicating the objectives have been satisfied
The intention was to provide enough information and detail as possible in order to lessen the burden and increasing demand on the few experienced software certification people that were available.
DO-178B made two more changes: nomenclature for describing safety criticality was changed from critical, essential, and nonessential to catastrophic, hazardous/severe-major, major, minor, and no effect; software Levels 1, 2, and 3 were changed to A through E.
One of the more significant aspects of DO-178B was the change made in software verification requirements. Much greater emphasis was placed on requirements-based testing than in earlier versions. This was done to reduce or eliminate the practice of focusing software testing primarily on structural (i.e., white box) coverage without fully testing the requirements, which would then lead to the need for additional tests. Requirements-based testing, with emphasis on the structure coverage of these functional tests, was seen as a more cost-effective and meaningful way to conduct verification testing.
The emphasis on requirements-based testing resulted in a new and significant focus area in DO-178B to ensure the traceability between requirements, code, and the tests that verify that the code satisfy the requirements. Traceability evidence is required to be thorough and bidirectional, demonstrating that requirements trace forward to code and tests and satisfy requirements, and trace backward from tests to code and to the requirements for which the tests were created. At the higher levels of safety and criticality, DO-178B requires that this traceability evidence show 100% structural code coverage.
DO-178B requires that verification and traceability evidence be made available for audit and analysis by the FAA or their DER to support an assessment of completeness and correctness. While this was also true for prior versions, the enhanced rigor and completeness of verification evidence required by DO-178B has proven to be a significant burden to the avionics industry. Some industry representatives have complained that the certification process requires an inordinate amount of time and expense, with some aspects of that process contributing little or no value. In particular, the FAA has received numerous complaints about software aspects of the certification process. (Hayhurst et al. 1998)
The human effort involved in producing the degree of systematically rigorous and complete verification and traceability evidence required by DO-178B can be overwhelming. In addition, as the complexity grows, the cost to produce the associated verification evidence grows exponentially with the size of the application. While tools that provide assistance with development, verification, and traceability have been utilized and undergone qualification throughout the time frame of DO-178 and its three versions, the tools designed specifically to meet the demands of DO-178B are being counted on for automated assistance far beyond those of their predecessors. Consequently, the reliance upon such tools without independent review of their results and artifacts also continues to grow. Thus, the subject of tool qualification takes on additional weight in the context of DO-178B.
SC-205 is the official designation in the US of the RTCA subcommittee dedicated to updating DO-178B - SOFTWARE CONSIDERATIONS IN AIRBORNE SYSTEMS AND EQUIPMENT CERTIFICATION. This is the primary document on which the FAA certification of all commercial software-based aerospace systems is based. DO-178B was released in 1992 and is mirrored in Europe by ED-12B, the same document but with a European designation. SC-205 is responsible for revising DO-178B/ED-12B to bring it up do date with respect to current software development and verification technologies. The new document will be called DO-178C/ED-12C and is due to be finalized during 2010.
The committee work is divided into 7 Subgroups:
- SG1: SCWG Document Integration
- SG2: Issues and Rationale
- SG3: Tool Qualification
- SG4: Model Based Design and Verification
- SG5: Object-Oriented Technology
- SG6: Formal Methods
- SG7: Safety Related Considerations
The Model Based Design and Verification subgroup (SG4), isthe largest of the working groups with about 45 members (Jan 2007). All work is collected and coordinated via a web-site that is a collaborative work management mechanism, hosted at Embry Riddle University and maintained by Dr. Matte Jaffe.
This site is open to the public, so anyone can view the working artifacts, but it does require registration and setting up an account in order to access it. There are currently about 700 users of this web-site, while only about 130-140 current participants in the committee itself.
The work is focused on bringing DO-178B/ED-12B, finalized in 1991, up to date with respect to current software development practices, tools, and technologies. (For a very brief explanation of the history of this document, including the justification for the current work, see http://www.aviationtoday.com/cgi/av/show_mag.cgi?pub=av&mon=0705&file=perspectives.htm).
There are many topics being discussed that directly affect SSCI members and their application of technologies such as TAF, as well as their more traditional software development practices. Among these are strong calls for clarification/refinement of the definitions and boundaries between the key DO-178B concepts of High Level Requirements, Low Level Requirements, and Derived Requirements and a better definition of the exit/entry criteria between systems requirements and system design (ARP4754 -http://www.answers.com/topic/arp4754) and that of software requirements and software design (which is the domain of DO-178B - http://www.answers.com/DO-178B). Other topics such as what does verification mean in a model-based development paradigm and can model simulation or formal methods replace some or all software testing activities.
Model Based Design and Verification
Each areas is centered around a number of the issue papers that were submitted for discussion and resolution by Subgroup 4. These issue papers were written to address concerns and make recommendations related to Model Based Development and its impact on the DO-178B guidelines and guidance material as related to the certification of software based airborne systems. They cover topics such as:
- What is source code in the context of automatic code generators?
- Can coverage of a model’s logical paths during simulation of a model in the modeling environment be granted credit for MCDC structural coverage in the presence of a qualified (read “perfect”) auto-code generator?
- What are high level vs. low level vs. derived requirements and should design models such as Simulink/MatrixX/SCADE be considered low level requirements
RTOS in Certification
Here's an example of a question that may be asked by some users have developed a RTOS that used in airborne system, and they want their RTOS to be certified under DO-178B.
DO-178B is much more than just a certification, it is a process that starts the same day as the development project itself. It is nearly impossible to take an existing application that was not developed in cooperation with the FAA or European equivalent and get it certified for any DO-178B levels, and definitely impossible for Level A applications. At the very beginning of a DO-178B development project, a document called Plans for Software Aspects of Certification is supposed to be created and supplied to the certification authority for approval. This plan describes the process and tools and procedures that the developer will utilitize during the development and verification of the avionics software application. This plan also describes the application and its use in terms of safety impact on the aircraft. The developer and the certification authority will decide what level of safety criticality the application should be developed to and certified to. This determined the objectives and deliverables that must be met and produced during the development and verification phases of the project.
To take an existing application like a customer's RTOS and get it certified under DO-178B, and entire development effort would be necessary - not to produce the actual code itself - but to go back and develop all of the additional material necessary to support the certification. For example, DO-178B Level A requires that verification testing include sufficient tests for 100% MCDC level object code coverage, and all of the tests are required to be traceable back to specific software requirements that the code was designed to satisfy. There are a lot of objects to be met and deliverables required to be produced and sent or made available to the certification authority.