Quality Assurance

Draft 3/1/2006

FUNCTIONAL AREA GOAL: A management system is implemented that integrates and drives processes applicable to DOE's mission and work which complies with requirements of 10 CFR 830 Subpart A, Quality Assurance Requirements, and DOE Order 414.1C, Quality Assurance.

REQUIREMENTS:

GUIDANCE:

EXCEPTION: This document is intended to provide general guidance for conducting oversight of Quality Assuring Management functions at the various DOE/NNSA Offices and Sites. Performance Objectives and Lines of Inquiry were developed in conjunction with the DOE Guide 414.1-1A, Quality Assurance Management System Guide, 5-31-01. The guide is generic in that it reflects the latest requirements and practices that are expected to be found in a quality assurance program. The Carlsbad Field Office (CBFO) and the DOE Office of Civilian Radioactive Waste Management are exceptions since their quality assurance programs were developed to comply with regulatory requirements that are in effect at the time an NRC license is issued for their respective sites. The EPA Administrator is the only authority that can invoke a change to the federal regulations applicable to WIPP which are different from those cited in the CRAD. Therefore, in conducting a review of the WIPP or the Yucca Mountain Project quality assurance programs, regulatory requirements applicable to those sites should be invoked where ever specific requirements are referenced in this CRAD Performance Objective and Lines of Inquiry.

NOTE: While Software Quality Assurance (SQA) and Suspect/Counterfeit Items (S/CI) are mentioned in this Quality Assurance CRAD, they are included in the context of DOE Order 414.1C, Quality Assurance, since the Order speaks to the entire spectrum of quality interests. Separate CRAD's have been developed for SQA and S/CI and should be referred to for specific and detailed guidance in those areas.

Performance Objective 1: Contractor Quality Assurance Program Documentation

Contractor line management has established a comprehensive and integrated program specific quality and safety management system applicable to DOE's mission and work which encompasses requirements of 10 CFR 830 Subpart A, Quality Assurance, and DOE Order 414.1C, Quality Assurance.

Criteria:

  1. The contractor has assigned and identified a senior management position responsible for the development, implementation, assessment, and improvement of a Quality Assurance Program.
     
  2. The contractor has implemented QA criteria as defined in Attachment 2 of the DOE O 414.1C, paragraph 3, Contractor Requirements Document (CRD); paragraph 4, Suspect/Counterfeit Items (S/CI) prevention requirements; and paragraph 5, safety software. The graded approach is described regarding how the QA criteria of the Order are applied by the contractor.
     
  3. The Contractor utilizes appropriate national or international consensus standard where practicable and consistent with contractual or regulatory requirements, and has identified the standard(s) approved for use by the site office. Suggested standards include the following:
  4. The contractor applies additional standards, where practicable and consistent with contractual or regulatory requirements and as necessary to address unique/specific work activities (e.g., development and use of safety software or establishing the competence of a testing and calibration laboratory). [Note: these standards may be referred to as "voluntary standards." However, once a practicable standard is adopted through regulation, code, contract, QAP, or procedure, compliance with the standard is required and is not voluntary.]
     
  5. The contractor has integrated, where practicable and consistent with contract or regulatory requirements, quality management system requirements as defined in the Contractor Requirements Document (DOE O 414.1C, Attachment 2), and other quality or management system requirements in DOE directives and external requirements. Examples include the following:
  6. Changes to the contractor's Quality Assurance Program are processed as follows:
  7. The contractor submits annually QAP changes made during the previous year to DOE for review and approval. In the submittal, changes are identified, the reason for the changes, and the basis for concluding that the revised QAP continues to satisfy the requirements of the existing CRD. [The contractor may make changes to an approved QAP at any time. Editorial changes made to correct spelling, punctuation, grammar, etc., do not require explanation.]
     
  8. The contractor's QAP has addressed the most recent revisions of the following guidance documents:

Suggested Lines of Inquiry and Review Approach

Performance Objective 2: Contractor Quality Assurance Program Implementation

2.1 Quality Assurance Program:  The Contractors Quality Assurance Program description documentation describes programs and processes that comprise the total scope of the quality assurance management system. The documentation includes a description of the organization structure, functional responsibilities, levels of authority, and interfaces for those managing, performing and assessing work.

Criteria:

  1. The QAP addresses and provides a description of the organization structure, functional responsibilities, levels of authority, and interfaces for those managing, performing and assessing work.
     
  2. The QAP emphasizes senior management commitment to quality and safety and that all work and every component and employee of the organization is included within the scope of the quality management system.
     
  3. The QAP is approved by senior management, updated annually if needed, and changes are reviewed and approved by DOE.
     
  4. The management processes include planning, scheduling, and describes how resources are selected and allocated for work.
     
  5. The contractor's quality assurance management system includes management assessments, and independent assessments as defined by laws, regulations, and DOE directives such as quality assurance program requirements and other structured operational awareness activities; incident/event reporting processes, including occupational injury and illness and operational accident investigations; quality improvement mechanisms such as worker feedback; quality issues management; lessons-learned programs; and performance indicators/measures.
     
  6. The contractor quality assurance management system includes methods for the flow down of quality requirements within the organization and to subcontractors and suppliers.
     
  7. Contractor quality assurance management system performance data are reviewed, analyzed, compiled, documented and made available to DOE line management as part of formal contract performance evaluation.

Suggested Lines of Inquiry and Review Approach

2.2: Personnel Training and Qualification:  Contractor worker qualification and training requirements are defined and implemented to ensure personnel achieve and maintain the capabilities to perform assigned work.

Criteria:

  1. Requirements and formal processes have been established and implemented that ensure personnel responsible for managing and performing quality assurance activities possess appropriate experience, knowledge, skills and abilities that are commensurate with their responsibilities.
     
  2. Worker qualification requirements are defined.
     
  3. Management plans for and commits resources to facilitate the training and qualification of personnel.
     
  4. Management ensures that persons hired or transferred into positions meet the appropriate requirements for performing assigned work.
     
  5. Job proficiency maintenance and training requirements are defined and worker information needed is provided.
     
  6. Project/task unique training requirements are identified and the required knowledge for performing work is effectively provided to workers.
     
  7. Management solicits and encourages worker input to the development of task training.
     
  8. Site/facility/specific training modules include consideration of safety, emergency plans, and security and operations information necessary for personnel to prepare for and perform their assigned duties.
     
  9. Institutional training conveys general information about the organization's mission, vision, goals, and the importance of all workers in the achievement of each element.
     
  10. Training includes personnel responsible for managing, planning, performing, controlling, and overseeing work.
     
  11. Training plan content is based on current site, facility, and organization procedures; technical and professional references; and past organization/industry experience.
     
  12. Training plans specify the type of training records to be maintained.

Suggested Lines of Inquiry and Review Approach

2.3: Quality Improvement: Management actions are initiated for improving the quality of products, processes, and services by establishing priorities, promulgating policy, allocating resources, communicating lessons learned, and resolving significant management issues and problems that hinder the organization from achieving its quality objectives.

Criteria:

  1. Quality problems are identified, evaluated, causes and significance determined, and management disposition action is documented.
     
  2. Primary management emphasizes for improving quality is placed on the prevention of quality problems.
     
  3. Management balances safety and mission priorities when considering improvement actions.
     
  4. Management encourages employee participation in planning, developing, exploring and implementing new ideas for improving quality in products, processes, and services.
     
  5. Quality problems and other quality related information (both positive and negative) from various internal and external sources are reviewed and analyzed to identify improvement opportunities in the quality management system, processes, items, products, or services.
     
  6. The effectiveness of implemented improvements is monitored and tracked to follow up on the adequacy of the actions to improve quality.
     
  7. Items, services, and processes that do not meet established requirements are identified and controlled.
     
  8. Corrective action processes require the identification of "root-causes" and result in corrective action plans that are developed to prevent their recurrence.
     
  9. Management has implemented a "Lessons Learned" program that utilizes feedback from within and other related organizations to identify potential quality improvement processes.
     
  10. Quality problems identified as Type "A" Accident event, or a Price-Anderson Enforcement, or DOE Inspector General Issues are tracked through resolution.

Suggested Lines of Inquiry and Review Approach

2.4: Documents and Records: The contractor's documents and records management system is effective in supplying documents for personnel to safely and correctly perform their assigned responsibilities.

Criteria:

  1. Documents that prescribe work control processes, specify requirements, or establish designs are prepared, reviewed, approved, issued, and revised in a controlled manner. Examples include the following:
  2. The document control system is effective in supporting management actions to ensure that safety and quality management requirements and functions are communicated to workers.
     
  3. Records are specified, prepared, reviewed, approved, and maintained.
     
  4. Records support management actions and decisions and provide evidence that work was correctly performed.
     
  5. Records management controls are adequate for the development and preservation of records for all record forms that are maintained (e.g., electronic, written, or printed; microfilm; photographs; radiographs; optical disks; etc.)
     
  6. Schedules for records retention and disposition are consistent with requirements of DOE O 200.1, Information Management Program, dated 9-3-96.
     
  7. Hardware and software tools used to create and store records are maintained to ensure that records are retrievable.

Suggested Lines of Inquiry and Review Approach

2.5.1: Work Processes: Work processes are carried out by qualified personnel using approved procedures, instructions, and equipment under administrative, technical, and environmental controls to achieve a planned end result.

Criteria:

  1. Work is performed consistent with technical standards, administrative controls, and hazard controls adopted to meet regulatory or contract requirements using approved instructions, procedures, etc.
     
  2. Line management and workers cooperate to identify processes that can be improved based on feedback prior to and following implementation of the work process.
     
  3. Management solicits input from workers in the development of work processes and communicates an expectation of worker accountability for quality and safety in the performance of work.
     
  4. Management ensures that the following are clearly identified and conveyed to workers before they begin work:
  5. Procedures, work instructions, or other means used to define work processes are documented and controlled.
     
  6. The scope and documentation detail is commensurate with the complexity and importance of the work, the skills required to perform the work, the hazards and risks or consequences of quality problems in the product, process, or service, and the need to meet regulatory and contract requirements.
     
  7. Control of processes, skills, hazards, and equipment are specified, understood, and documented.

Suggested Lines of Inquiry and Review Approach

2.5-2: Work Processes - Item Identification and Control: A process for the identification and control of items is in place and is aggressively being applied.

Criteria:

  1. Management processes for the identification and control of items are in place to:
  2. The identification and control process is applied from manufacture or receipt through delivery, installation, or use.
     
  3. Item configuration control requirements, based on a graded-approach, are defined. Requirements may include item unique part, lot, heat, model, version, serial number, installation, and record requirements that enable traceability to the item.
     
  4. Items and materials are maintained and stored to prevent their damage, loss, or deterioration.
     
  5. Equipment/tools used for process monitoring or data collection are calibrated, maintained, and properly secured.

Suggested Lines of Inquiry and Review Approach

2.6: Design: The contractor's design management process provides for the control of design functions and interfaces that enables producing quality design output products that effectively support facility design, construction, start-up, operation and maintenance functions.

Criteria:

  1. The design process provides controls for design inputs, outputs, verification, design changes and configuration control.
     
  2. Sound engineering/scientific principles and appropriate standards are applied to the design of items and processes.
     
  3. Applicable requirements are incorporated into design bases when developing design documents and changes thereto.
     
  4. Design programmatic, technical, and administrative interfaces are identified and controlled.
     
  5. Qualified individuals or groups other than those who perform the work perform design verification/validation of the adequacy of design products.
     
  6. Design products are verified and/or validated prior to their approval and implementation.
     
  7. The selection of applicable design control requirements are guided by safety analyses that establishes the following:
  8. Design documentation provides acceptance, inspection, testing, and maintenance criteria to ensure continuing reliability and safety of items.
     
  9. Design documentation includes approved computer codes utilized for preparing designs and documentation.
     
  10. Design records include documentation such as design inputs, calculations, and analyses; engineering reports; design outputs; design changes; design verification activities; and related documents that provide evidence that the design process was completed correctly.
     
  11. Design inputs include contractual requirements and customer expectations.
     
  12. Design inputs include information such as design bases, health and safety considerations, expected life cycle, performance parameters, codes and standards requirements, and reliability requirements.
     
  13. Design processes translate design inputs into design output documents that are technically correct and compliant with end user requirements.
     
  14. Design output documents support other processes such as dose and risk assessments, procurement, manufacturing, assembly, construction, testing, operation, inspection, maintenance, and decommissioning.
     
  15. As-built conditions for structures, systems and components are reflected in final design output documents.
     
  16. Design verification processes are defined and performed by technically knowledgeable persons separate from those who performed the design.
     
  17. Methods for conducting a graded approach to design verification are defined.
     
  18. Design verification methods include design reviews, alternative analyses, qualification testing, and peer reviews.
     
  19. Interim design verification activities are conducted under controlled conditions involving support of other work such as procurement, manufacture, construction, or testing.
     
  20. Design verification is completed before relying on structures, systems, and components to perform their function.
     
  21. Design changes that result from temporary modifications, or installed designs that are dispositioned "use-as-is," undergo the same level of design review as for original designs.
     
  22. Computer software used to originate or analyze design solutions that serve to mitigate potential accidents is controlled in accordance with DOE Guide G 414.1-4, Safety Software Guide, dated 6-17-05 (or latest revision). All other computer software used to analyze designs that might have safety, operational, or programmatic consequences is verified and controlled.

Suggested Lines of Inquiry and Review Approach

2.7: Procurement: The procurement process ensures that items and/or services provided by suppliers meet the requirements and expectations of end users.

Criteria:

  1. The procurement process is planned and controlled to ensure the following:
  2. Procurement processes have been implemented to prevent introduction of suspect/counterfeit items (S/CI's) and provide a method to detect them before they are released for use.
     
  3. Procurement requirements are commensurate with the importance of the end use of the purchased item or service.
     
  4. Management controls are in place and are being followed for DOE procurement and subcontracts through applicable DOE Orders, the Department of Energy Acquisition Regulation (the DEAR) in 48 CFR subchapters A through H, and the Federal Acquisition Regulation (FAR), in 48 CFR 970 et.seq.
     
  5. Existing procurement management systems adequately respond to end-user requirements and reflect requirements of the QA Rule 10 CFR 830, Nuclear Safety Management, and DOE Order 414.1C, Quality Assurance.
     
  6. The procurement process provides direction on how the contractor responsible for nuclear facilities ensures that subcontractors and suppliers satisfy criteria of 10 CFR 830.122.
     
  7. Procurement documents include or reference requirements and acceptance criteria for purchased items and services.
     
  8. Procurement documents include specifications, standards and other applicable documents as referenced in design documents.
     
  9. Critical parameters and requirements are specified such as document submittals, product related documentation, problem reporting, administrative documentation, personnel or materials qualifications, tests, inspections, performance expectations for services, and reviews.
     
  10. Procurement processes for safety-related structures, systems, and components are graded based on their importance to safety and the protection of workers, public and environment.
     
  11. Prospective suppliers of structures, systems, and components that are important to safety and/or the protection of workers, public and environment are required to have a quality assurance program and their quality programs have been verified to conform to contract requirements that includes their capability to meet performance and schedule requirements.
     
  12. Assessments of supplier's capability and facilities include the portions or all of the following:
  13. Qualified supplier's performance capabilities are periodically verified and monitored to confirm their continuing capabilities.
     
  14. Monitoring of supplier performance capability includes some are all of the following:
  15. The procurement process provides for the identification of inspections and tests to ensure conformance with purchase requirements.
     
  16. Procurement documents specify critical or important acceptance parameters for inspection.
     
  17. Inspections include verification that specified documentation has been provided by the supplier and that items were not damaged during shipment.
     
  18. Inspections include some are all of the following:
  19. Supplier-generated documents for structures, systems and components are accepted through the procurement system and controlled and processed by the end-user organization. (Documents include certificates of conformance, drawings, analyses, test reports, maintenance data, nonconformance documentation, corrective actions, approved changes, waivers, and deviations.
     
  20. Items procured for safety applications in nuclear or high-risk related activities are procured on either of the following basis:
  21. Commercial-grade items intended for use in nuclear safety or high-risk activities are procured in accordance with documented processes using recognized consensus standards.
     
  22. Critical design characteristics defined by the design organization are utilized in item selection and acceptance requirements. They may include the following:
  23. Multi-site procurements are performed only when interfaces and responsibility agreements have been clearly defined and established in writing with regard to quality requirements.

Suggested Lines of Inquiry and Review Approach

2.8: Inspection and Acceptance Testing: The contractor's inspection and test program requirements are effective in verifying that physical and functional aspects of items, services, and processes meet requirements and are fit for acceptance and use.

Criteria:

  1. Established acceptance and performance criteria are used for inspecting and testing specified items, services, and processes.
     
  2. Planning for inspections/tests includes provisions for the following:
  3. Technically qualified personnel are given the authority to access appropriate information and facilities to verify acceptance requirements perform inspections/tests.
     
  4. Inspection/test personnel are qualified and independent of the activities being inspected/tested and have the freedom to report the results of the inspections/tests.
     
  5. Inspection/test results are evaluated and verified by qualified personnel of the design organization to document that requirements are satisfied.
     
  6. Inspection and test records identify the following:
  7. Items or services that meet acceptance requirements are appropriately identified to enable correct application for their intended use.
     
  8. When requirement parameters for items are changed, accepted items are reviewed by persons knowledgeable of design requirements to determine if re-inspection and testing of item(s) is (are) necessary.
     
  9. Measuring and test equipment (M&TE) used for inspections, tests, monitoring, and data collection is calibrated, maintained, and controlled using a documented process.
     
  10. M&TE is checked before use to ensure it is of the proper type, range, accuracy, and precision and is uniquely identified and traceable to its calibration data.
     
  11. Procedures are developed that establish requirements for testing, retesting, adjusting, and recalibrating M&TE.
     
  12. M&TE is calibrated to standards traceable to the National Institute of Standards and Technology or other nationally recognized standards when appropriate, or if no standard exists, the basis for calibration is documented.
     
  13. The M&TE used is traceable to the item for which acceptance is established.

Suggested Lines of Inquiry and Review Approach

2.9: Management Assessment: Managers periodically assess their functions to determine how well their organization is meeting both customer and management performance expectations and mission objectives, to identify strengths or opportunities for improving performance, and to correct identified problems.

Criteria:

  1. Management assessments are planned, scheduled, conducted and results are documented.
     
  2. Problems that hinder the organization from achieving its objectives are identified, corrected, and personnel are trained on the changes.
     
  3. Managers view their participation in the assessment process as essential to ensure their gaining a first-hand understanding of how well the management system is functioning.

Suggested Lines of Inquiry and Review Approach

2.10: Independent Assessment: Contractor senior management has established a process to obtain an independent assessment of the organization's programs, projects, contractors, and suppliers.

Criteria:

  1. Senior management ensure that independent assessments are planned, scheduled and conducted to measure item and service quality and the adequacy of work performance to promote improvement.
     
  2. The management structure ensures there is sufficient authority and freedom from line management to enable independent teams to effectively perform assessments.
     
  3. Persons conducting independent assessments are technically qualified and knowledgeable in the areas they are assigned to assess and have had no previous responsibility for the work being assessed.
     
  4. Assessments include the following:
  5. Results of independent assessments are analyzed to determine if problems are local or global and appropriate remedial management action(s) is(are) initiated to correct problems.
     
  6. Findings from assessments are tracked and progress towards their closure are monitored and reported to senior management.
     
  7. Corrective actions that are implemented to address findings are verified for their adequacy and follow up reviews are performed to determine their effectiveness in correcting problems.
     
  8. The management system includes a "Lessons-Learned" reporting mechanism to enable other management groups to benefit from identified management issues. 

Suggested Lines of Inquiry and Review Approach

Performance Objective 3: DOE Line Management Oversight: DOE line management has established and implemented effective oversight processes that enable evaluating the adequacy and effectiveness of contractor quality assurance management systems and DOE oversight processes.

Criteria:

  1. The DOE Office has developed and implemented a quality assurance program or is working to an approved "umbrella" QAP, and employees have been trained on the application of the QAP. The QAP implements QA criteria as defined in paragraph 4b of DOE O 414.1C, Quality Assurance, using a graded approach where applicable.
     
  2. The DOE Office QAP uses national or international consensus standards where practicable and is consistent with regulatory requirements (e.g., 10 CFR 830) and identifies the standards used. Examples of appropriate standards include the following:
  3. The DOE Office applies additional standards, where practicable and consistent with contractual or regulatory requirements and as necessary to address unique/specific work activities (e.g., development and use of safety software or competence of a testing and calibration laboratory). [Note: These standards are sometimes referred to as "voluntary standards." However, once a practicable standard is adopted through regulation, code, contract, QAP, or procedure, compliance with the standard is required and is not voluntary.]
     
  4. The DOE Office integrates, where applicable, contract or regulatory requirements, quality management system requirements as defined in DOE O 414.1C, Quality Assurance, the S/CI prevention process, the Corrective Action Management Program (CAMP), and Safety Software Quality Requirements with other quality or management system requirements in DOE directives and external requirements, including as applicable the following:
  5. DOE line managers and supervisors communicate an attitude of responsibility and accountability for quality of work performed within their assigned scope of management responsibilities. Personnel at all levels understand their responsibilities and are committed to quality performance.
     
  6. The requirements for quality are clearly defined and understood by each DOE management unit and sufficient resources are provided for the unit to accomplish their mission.
     
  7. Quality measures are applied in a graded manner to non-safety related systems, equipment, and activities. Measures that are applied are based on the importance to safe and reliable facility operation.
     
  8. Quality audits and surveillances are scheduled based on the importance of the activity and past performance issues in selected areas.
     
  9. Individuals who have demonstrated capability for identifying performance issues perform quality audits and surveillances.
     
  10. Quality monitoring results are documented and corrective actions are tracked to ensure adequacy of management actions taken to correct problems.
     
  11. Results of surveillances and audits are reported to line management for identification of root causes and appropriate corrective action.
     
  12. Substantive quality problems are reported to senior management.
     
  13. The effectiveness of actions taken to correct quality problems are periodically reviewed to ensure the adequacy of corrective actions that have been implemented.

Suggested Lines of Inquiry and Review Approach



< BACK TO PREVIOUS PAGE