DOE Software Quality Assurance Criteria Review and Approach Documents

Safety Software Quality Assurance

Draft 3/31/2006

Functional Area Goal

An effective software quality management system is implemented that integrates and drives processes applicable to DOE's mission and work which complies with requirements of 10 CFR 830 Subpart A, Quality Assurance, and DOE Order 414.1C, Quality Assurance.

This document is intended to provide specific guidance for conducting oversight of safety software quality assurance functions at the various DOE/NNSA Offices and Sites. Performance Objectives and Lines of Inquiry were developed in conjunction with the DOE Guide 414.1-4, Safety Software Guide for Use with 10 CFR 830 Subpart A, Quality Assurance Requirements, and DOE O 414.1C, Quality Assurance, June 17, 2005 . Safety software quality assurance functions must also comply with the general guidance for conducting oversight of the quality assurance program functions. This general guidance is contained in the DOE Guide 414.1-1A, DOE Guide 414.1-1A, Quality Assurance Management System Guide, May 31, 2001 and its associated criteria review and approach document (CRAD).

Requirements

The specific safety software quality assurance functions must meet the quality assurance requirements in the following DOE directives. Where program specific requirements are identified below, additional requirements must be met. The criteria review and approach may need to be enhanced to address the applicable requirements.

The Carlsbad Field Office (CBFO) and the DOE Office of Civilian Radioactive Waste Management quality assurance programs were developed to comply with regulatory requirements that are in effect at the time an NRC license is issued for their respective sites. The EPA Administrator is the only authority that can invoke a change to the federal regulations applicable to WIPP which are different from those cited in the CRAD. Therefore, in conducting a review of the WIPP or the Yucca Mountain Project quality assurance programs, regulatory requirements applicable to those sites should be invoked where ever specific requirements are referenced in this CRAD Performance Objective and Lines of Inquiry.

• Quality Assurance Rule 10 CFR 830 - Nuclear Safety Management.
• Quality Assurance Order, O 414.1C - Quality Assurance, June 17, 2005.
• DOE-STD-1172-2003 - Safety Software Quality Assurance Functional Area Qualification Standard, December 2003.
• Program specific requirements: NNSA Quality Management Policy, QC-1; DOE/CBFO-94-1012, DOE Carlsbad Field Office, Quality Assurance Program Description; DOE/RW-0333P, DOE Office of Civilian Radioactive Waste Management, Quality Assurance Requirements and Description.

Guidance:

• DOE G 414.1-4, Safety Software Guide for Use with 10 CFR 830 Subpart A, Quality Assurance Requirements, and DOE O 414.1C, Quality Assurance, June 17, 2005.
• American Society of Mechanical Engineers (ASME) NQA-1-2000, Quality Assurance Requirements for Nuclear Facility Applications, 2000.

Performance Objective 1: Contractor Program Documentation:

While this performance objective is focused on the Contractor, there are instances where DOE may be the performer of the work. Where DOE is the performer the contractor requirements as stated herein apply to the DOE organization doing the work.

1.1 Safety Software Inventory:  An inventory of safety software is identified, documented, and maintained.

Criteria:

1. Safety software, as identified in DOE O 414.1C, is identified for each DOE nuclear facility (as defined in 10 CFR 830 and DOE O 414.1C).
    
2. An inventory list of the safety software is documented.
    
3. The software inventory includes all versions of specific applications in use at the publication date of the inventory list, and includes software that has been removed and designated not to be used.

Suggested lines of inquiry and review approach

1.2 Safety Software Graded Approach: Establish grading levels for safety software. Document those grading levels in the QAP. [Suggested rewrite] Grading levels for safety software are established and documented in the QAP.

Criteria:

1. Grading levels for safety software are defined in accordance with the graded approach (as defined in 10 CFR 830 and DOE O 414.1C).
    
2. The safety software graded approach is documented in the appropriate Contractor or DOE QAP or other DOE approved quality assurance program document.
    
3. The contractor or DOE quality assurance program is approved by the appropriate DOE authority.

Suggested lines of inquiry and review approach

1.3 Safety Software Quality Assurance Consensus Standard: ASME NQA 1-2000, Quality Assurance Requirements for Nuclear Facility Applications, or other national or international consensus standards that provide an equivalent level of quality assurance requirements as ASME NQA 1 2000, are specified by the user and approved by DOE.

Criteria:

1. If another national or international consensus standard(s) is used, an equivalency to ASME NQA-1-2000, Quality Assurance Requirements for Nuclear Facility Applications is performed and documented.
    
2. If an equivalency is performed, it is complete and ensures that the alternate standard(s) provides the equivalent level of quality assurance requirements for all 10 work activities as ASME NQA-1-2000. This equivalency addresses all requirements in ASME NQA-1-2000 Part I and Part II applicable to software.
    
3. The consensus standard(s) used for safety software is documented in the appropriate contractor or DOE QAP or other DOE approved quality assurance program document.
    
4. The contractor or DOE QAP or DOE approved quality assurance program document is approved by the appropriate DOE authority.

Suggested lines of inquiry and review approach

Performance Objective 2: Contractor Program Implementation:

Work processes involving safety software must be developed and implemented using the DOE approved national or international consensus standards (as per the criteria in Section 1.3 above) and must include the following elements.

2.1 Facility Design Authority Involvement: Facility design authority is involved in the identification of software requirements specification, acquisition, design, development, verification and validation (including inspection and testing), configuration management, maintenance, and retirement.

Criteria:

1. The facility design authority is an integral part of the safety software development and/or acquisition by providing input to, reviewing, and/or approving safety software development and acquisition documents including: safety software requirements, procurement or acquisition contracts and other documents, software development plans and documents, verification and validation documents, software configuration management documents, software acceptance criteria, software maintenance documents, and software retirement plans and documents.

Suggested lines of inquiry and review approach

2.2  Select and Implement Work Activities:  Using DOE approved grading levels (as per the criteria in Section 1.2 above) and ASME NQA-1-2000, Quality Assurance Requirements for Nuclear Facility Applications, or other DOE approved national or international consensus standards that provide an equivalent level of quality assurance requirements as ASME NQA 1 2000, applicable SQA work activities are selected and implemented from the following list to ensure that safety software performs its intended functions.

1. Software project management and quality planning.
2. Software risk management.
3. Software configuration management.
4. Procurement and supplier management.
5. Software requirements identification and management.
6. Software design and implementation.
7. Software safety.
8. Verification and validation.
9. Problem reporting and corrective action.
10. Training of personnel in the design, development, use, and evaluation of safety software.

Criteria:

1. The approved (as per the criteria in Section 1.2 above) graded approach is used to select and implement the 10 safety software work activities.
    
2. ASME NQA-1-2000, Quality Assurance Requirements for Nuclear Facility Applications, or other national or international consensus standards that provide an equivalent level of quality assurance requirements as NQA 1 2000, is used to implement these work activities.

Suggested lines of inquiry and review approach

2.3 Software Project Management and Quality Planning: Software project management and quality planning depicts the organizational structure that supports the software life-cycle stages and deliverables, and influences and controls the quality of the software.

Criteria:

1. Software project management and quality planning depicts the organizational structure, responsibilities, and authorities for those managing, performing, and assessing the software projects.
    
2. SQA activities, software practices, and documentation are periodically assessed.
    
3. Software quality activities are effectively implemented.

Suggested lines of inquiry and review approach

2.4 Software Risk Management: Software risk management utilizes a proactive and disciplined approach to assess and control software risks.

Criteria:

1. Potential software risks are identified as required by the grading level.
    
2. Likelihood and consequences of the safety software failure are determined.
    
3. Risks are prioritized.
    
4. Risk avoidance, mitigation, and/or transfer strategies are created.
    
5. Risks are monitored.

Suggested lines of inquiry and review approach

2.5 Software Configuration Management: Software configuration is defined, maintained, and controlled until the software is retired.

Criteria:

1. Software configuration items are identified, baselined and controlled.
    
2. A baseline labeling system is established and implemented.
    
3. Proposed software changes are documented, evaluated, and approved.
    
4. Only approved changes are implemented.

Suggested lines of inquiry and review approach

2.6 Procurement and Supplier Management: Acquired safety software, either commercial off-the-shelf (COTS) software or custom-developed for DOE, meets the appropriate level of QA based on risk, safety, facility life-cycle, complexity, and project quality requirements.

Criteria:

1. Procurement documents identify the technical and quality requirements.
    
2. Acquired software meets the technical and quality requirements.
    
3. Suppliers' QA programs meet or exceed the QA requirements specified in the procurement documents.
    
4. Procurement documents specify supplier reporting of software defects to the purchaser and the purchaser's reporting of defects to the supplier.

Suggested lines of inquiry and review approach

2.7 Software Requirements Identification and Management: Safety software functions, requirements, and their bases are defined, documented and managed throughout the safety software life-cycle.

Criteria:

1. The software requirements are documented and consistent with the system safety basis.
    
2. The functionality, performance, security including user access requirements, interface and safety requirements for the safety software are complete, correct, consistent, clear, testable, and feasible.
    
3. The documented software requirements are controlled and maintained. Changes to the software requirements are reflected in all appropriate documentation.
    
4. Each requirement is uniquely identified, defined, objectively verified and validated.

Suggested lines of inquiry and review approach

2.8 Software Design and Implementation: The safety software design depicting the logical structure, information flow, logical processing steps, data structures and interfaces are defined and documented. The design is properly implemented in the safety software.

Criteria:

1. The design, including interfaces and data structures, is correct, consistent, clearly presented, and feasible.
    
2. The design is completely and correctly implemented in the safety software.
    
3. The design is traceable to requirements and the relationship and traceability of design to requirements is controlled and maintained throughout the software life-cycle.
    
4. Impact of design activities on requirements and verification and validation is documented and appropriate life-cycle documents are revised to ensure traceability and adequate testing.

Suggested lines of inquiry and review approach

2.9 Software Safety: The design of the safety software components is developed to ensure the software modules perform their intended safety function in a manner consistent with the implementation of the safety system.

Criteria:

1. Software systems are analyzed at the design component level to ensure adequate safeguards are implemented to eliminate or mitigate the potential occurrence of a software defect that could cause a system failure impacting safety.
    
2. Safety software is designed with simplicity and isolation of safety functions.
    
3. Where appropriate, fault tolerance and self-diagnostics are implemented in the safety software design.

Suggested lines of inquiry and review approach

2.10 Verification and Validation (V&V): The V&V process and related documentation for software are defined and maintained to ensure, (1) the software correctly performs all its intended functions; (2) the software does not perform any adverse unintended function; and (3) the operation of the software addresses the original purpose, intent, and plan.

Criteria:

1. Safety software deliverables are verified, and validated for correct operation using reviews, inspections, assessments, observation, and testing techniques.
    
2. Relevant abnormal conditions are evaluated for mitigating unintended functions through testing, observation, or inspection techniques.
    
3. Traceability of safety software requirements to software design and acceptance testing is performed.
    
4. New versions of the safety software are verified and validated to ensure that the safety software meets the requirements and does not perform any unintended functions.
    
5. V&V activities are performed by competent staff other than those who developed the item being verified or validated. This may overlap with the training work activity.
    
6. New versions of the safety software are verified and validated to ensure that unchanged functionality has not been affected by software changes or additions.

Suggested lines of inquiry and review approach

2.11 Problem Reporting and Corrective Action: Methods for documenting software problem reporting, and corrective action for safety software errors and failures are established, maintained, and controlled.

Criteria:

1. Documented practices and procedures for reporting, tracking, and resolving problems or issues are defined and implemented.
    
2. An evaluation process exists for determining if the reported problem is a safety software defect, error, or something else.
    
3. Organizational responsibilities for reporting issues, approving changes, and implementing corrective actions are identified and found to be effective.
    
4. For safety software defects and errors, the defect or error is correlated with the appropriate software engineering elements, identified for potential impact, and all users are notified.
    
5. For acquired safety software, procurement or other acquisition agreement documents identify the requirements to both the supplier and purchaser/customer to report problems to each other.

Suggested lines of inquiry and review approach

2.12 Training of Personnel in the Design, Development, Use, and Evaluation of Safety Software: Personnel are trained/qualified and capable of performing assigned work. Continuing training to personnel to maintain job proficiency is provided.

Criteria:

1. A training or indoctrination program exists for each of the following applicable personnel assignments:
   1. safety software analysis;
   2. software development (concept to retirement);
   3. operations and use; and
   4. assessment or evaluation of safety software.
       
2. The training/indoctrination provides for continuing education and training to improve their performance and proficiency.
    
3. Training/indoctrination is commensurate with the scope, complexity, and importance of the tasks and the education, experience, and proficiency of the person.

Suggested lines of inquiry and review approach

Performance Objective 3: DOE Line Management Oversight Documentation:

Federal personnel have the technical competency to perform SQA responsibilities; Federal personnel with SQA responsibilities have technical competency to carry out their duties. Technical qualification requirements are specified in technical qualification standards. The process has been coordinated with the Federal Technical Capability Panel (FTCP) in accordance with the requirements of DOE M 426.1-1A, Federal Technical Capability Manual, dated May 18, 2004, and DOE STD-1172-2003, Safety Software Quality Assurance Functional Area Qualification Standard, dated December 2003.

Criteria:

1. DOE/NNSA has identified federal personnel to perform SQA responsibilities.
    
2. Federal personnel are qualified in accordance with the DOE M 426.1-1A, Federal Technical Capability Manual, dated May 18, 2004 and the DOE STD-1172-2003, Safety Software Quality Assurance Functional Area Qualification Standard, dated December 2003.
    
3. Where a DOE headquarter or line management organization develops, manages or is responsible for safety software, DOE O 414.1C safety software requirements apply to that organization. Performance Objective 1 and 2 should be applied to that DOE organization. Refer to DOE O 414.1C for applicability to DOE headquarters and line organizations.

Suggested lines of inquiry and review approach