Feedback and Continuous Improvement

Draft

FUNCTIONAL AREA GOAL: An integrated process that makes use of all available performance feedback information to drive continuous improvement with the eventual goal of institutionalizing the attributes of a High Reliability Organization.

REQUIREMENTS:

DOE 5480.19, “Conduct of Operations Requirements for DOE Facilities.”
DOE Policy 450.4 "Integrated Safety Management"
DOE Order 440.1a "Federal Employee Occupational Safety and Health Program."
DOE Order 231.1a "Occurrence Reporting and Processing of Operations Information."
DOE Order 414.1c "Quality Assurance."
DOE Order 442.1a "DOE Employee Concerns Program"
DOE P 226.1, Department of Energy Oversight Policy
DOE O 226.1, Implementation of Department of Energy Oversight Policy
10 CFR 830, Nuclear Safety Management

GUIDANCE:

Draft DOE Order 210.x: Corporate Operating Experience Program
DOE G 450.4-1B, Integrated Safety Management System Guide

Performance Objective 1: Contractor Program Documentation

Contractor Line management has established a comprehensive and integrated operational assurance system which encompass all aspects of the processes and activities designed to identify deficiencies and opportunities for improvement, report deficiencies to the responsible managers, complete corrective actions, and share in lessons learned effectively across all aspects of operation.

Criteria:

A program description document that fully details the programs and processes that comprise the contractor assurance system has been developed, approved by contractor management, and forwarded to DOE for review and approval. The program description is reviewed and updated annually and forwarded to DOE for review and approval.
The contractor's assurance system includes assessment activities (self-assessments, management assessments, and internal independent assessments as defined by laws, regulations, and DOE directives such as quality assurance program requirements) and other structured operational awareness activities; incident/event reporting processes, including occupational injury and illness and operational accident investigations; worker feedback mechanisms; issues management; lessons-learned programs; and performance indicators/measures.
The contractor's assurance system monitors and evaluates all work performed under their contract, including the work of subcontractors.
Contractor assurance system data is formally documented and available to DOE line management. Results of assurance processes are periodically analyzed, complied, and reported to DOE line management as part of formal contract performance evaluation.
Contractors have established and implemented sufficient processes (e.g., self-assessments, corporate audits, third-party certifications or external reviews, performance indicators) for measuring the effectiveness of the contractor assurance program.
Requirements and formal processes have been established and implemented that ensure personnel responsible for managing and performing assurance activities possess appropriate experience, knowledge, skills and abilities commensurate with their responsibilities.

Performance Objective 2: Contractor Program Implementation

2.1 Assessments & Performance Indicators: Contractor Line management has established a rigorous and credible assessment program that evaluates the adequacy of programs, processes, and performance on a recurring basis. Formal mechanisms and processes have been established for collecting both qualitative and quantitative information on performance and this information is effectively used as the basis for informed management decisions to improve performance.

Criteria:

Line management has established and implemented a rigorous assessment program for performing comprehensive evaluations of all functional areas, programs, facilities, and organizational elements, including subcontractors, with a frequency, scope and rigor based on appropriate analysis of risks. The scope and frequency of assessments are defined in site plans and program documents, include assessments of processes and performance-based observation of activities and evaluation of cross-cutting issues and programs, and meet or exceed requirements of applicable DOE directives.
Rigorous self-assessments are identified, planned, and performed at all levels periodically to determine the effectiveness of policies, requirements, and standards and the implementation status.
Appropriate independent internal assessments are identified, planned and performed by contractor organizations or personnel having the authority and independence from line management, to support unbiased evaluations.
Line managers have established programs and processes to routinely identify, gather, verify, analyze, trend, disseminate, and make use of performance measures that provide contractor and DOE management with indicators of overall performance, the effectiveness of assurance system elements, and identification of specific positive or negative trends. Approved performance measures provide information that indicates how work is being performed and are clearly linked to performance objectives and expectation established by management.
Line managers effectively utilize performance measures to demonstrate performance improvement or deterioration relative to identified goals, in allocating resources and establishing performance goals, in development of timely compensatory measures and corrective actions for adverse trends, and in sharing good practices and lessons learned.

2.2 Operating Experience: The Contractor has developed and implemented an Operating Experience program that communicates Effective Practices and Lessons Learned during work activities, process reviews, and incident/event analyses to potential users and applied to future work activities.

Criteria:

Formal processes are in place to identify applicable lessons learned from external and internal sources and any necessary corrective and preventive actions, disseminate lessons learned to targeted audiences, and ensure that lessons learned are understood and applied.
Line managers effectively identify, apply, and exchange lessons learned with the rest of the DOE complex. Lessons learned identified by other DOE organizations and external sources are reviewed and applied by line management to prevent similar incidents/events.
Formal programs and processes have been established and implemented to solicit feedback or suggestions from workers and work activities on the effectiveness of work definition, hazard analyses and controls, and implementation for all types of work activities, and to apply lessons learned.
Employee concerns related to management of DOE and NNSA programs and facilities are promptly and thoroughly reported and investigated in accordance with applicable DOE directives.

2.3 Event Reporting: Contractor line management has established and implemented programs and processes to identify, investigate, report, and respond to operational events and incidents and occupational injuries and illnesses.

Criteria:

Formal programs and processes have been established to identify issues and report, analyze, and address operational events, accidents, and injuries. Events, accidents, and injuries are promptly and thoroughly reported and investigated, including the identification and resolution of root causes and management and programmatic weaknesses, and distribution of lessons learned.
Reporting of operational events, accidents, and injuries are conducted in accordance with applicable nuclear, security, environment, occupational safety and health, and quality assurance requirements, applicable DOE directives, and contract terms and conditions. Trending analysis of events, accidents, and injuries are performed in accordance with structured/formal processes and applicable DOE directives.

2.4 Issues Management: The Contractor has developed and implemented a formal process to evaluate the quality and usefulness of feedback, and track to resolution performance and safety issues and associated corrective actions.

Criteria:

Program and performance deficiencies, regardless of their source, are captured in a system or systems that provides for effective analysis, resolution, and tracking. Issues management system elements include structured processes for determination of risk, significance, and priority of deficiencies; evaluation of scope and extent of condition; determination of reportability under applicable requirements; identification of root causes; identification and documentation of corrective actions and recurrence controls to prevent recurrence; identification of individuals/organizations responsible for corrective action implementation; establishment of milestones based on significance and risk for completion of corrective actions; tracking progress; verification of corrective action completion; and validation of corrective action implementation and effectiveness.
Issues management processes include mechanisms to promptly identify the potential impact of a deficiency and take timely actions to address conditions of immediate concern, including stopping work, system shutdown, emergency response, reporting to management, and compensatory measures pending formal documentation and resolution of the issue.
Processes for analyzing deficiencies, individually and collectively, have been established that enable the identification of programmatic or systemic issues. Line management effectively monitors progress and optimizes the allocation of assessment resources in addressing known systemic issues.
Processes for communicating issues up the management chain to senior management have been established and based on a graded approach that considers hazards and risks. Line management receives periodic information on the status of identified deficiencies and corrective actions and holds organizations and individuals accountable for timely and effective completion of actions. Line management has executed graded mechanisms such as independent verification and performance-based evaluation to ensure that corrective action and recurrence controls are timely, complete, and effective. Closure of corrective actions and deficiencies are based on objective, technically sound, and verified evidence. The effectiveness of corrective actions is determined on a graded basis and additional actions are completed as necessary.
Results of various feedback systems are integrated and collectively analyzed to to identify repeat occurrences, generic issues, trends, and vulnerabilities at a lower level before significant problems result.
Individuals or teams responsible for corrective action development are trained in analysis techniques to evaluate significant problems using a structured methodology to identify root and contributing causes and corrective actions to prevent recurrence.

Performance Objective 3: DOE Line Management Oversight

DOE line management have established and implemented effective oversight processes that evaluate the adequacy and effectiveness of contractor assurance systems and DOE oversight processes.

Criteria:

DOE line management has established a baseline line management oversight program that ensures that DOE line management maintains sufficient knowledge of site and contractor activities to make informed decisions concerning hazards, risks and resource allocation, provide direction to contractors, and evaluate contractor performance.
DOE line oversight program includes assessments, operational awareness activities, performance monitoring and improvement, and assessment of contractor assurance systems. Documented program plans have been established that define oversight program activities and annual schedules of planned assessments and focus areas for operational awareness. Operational awareness activities must be documented either individually or in periodic (e.g., weekly or monthly) summaries. Deficiencies in programs or performance identified during operational awareness activities are communicated to the contractor for resolution through a structured issues management process.
DOE line management monitors contractor performance and assesses whether performance expectations are met; that contractors are assessing site activities adequately; self-identifying deficiencies; and, taking timely and effective corrective actions. Responsibilities for line oversight and self-assessment are assigned and managers, supervisors, and workers are held accountable for performance assurance activities. Deficiencies must be brought to the attention of contractor management and addressed in a timely manner.
DOE line management requires that findings must be tracked and resolved through structured and formal processes, including provisions for review of corrective action plans.
DOE line management regularly assess the effectiveness of contractor issues management and corrective action processes, lessons learned processes, and other feedback mechanisms (e.g., worker feedback). DOE line management must also evaluate contractor processes for communicating information, including dissenting opinions, up the management chain.
DOE line management must verify that corrective actions are complete and performed in accordance with requirements before findings identified by DOE assessments or reviews are closed, and requires that deficiencies are analyzed both individually and collectively to identify causes and prevent recurrences.
DOE line management has established appropriate criteria for determining the effectiveness of site programs, management systems, and contractor assurance systems, and includes consideration of previous assessment results, effectiveness of corrective actions and self-assessments, and evidence of sustained management support for site programs and management and assurance systems. Review criteria are based on requirements and performance objectives (e.g., laws, regulations, DOE directives), site-specific procedures/manuals, and other contractually mandated requirements and performance objectives.
DOE line management has established and maintained appropriate qualification standards for personnel with oversight responsibilities, and a clear, unambiguous line of authority and responsibility for oversight.
Line management periodically reviews established performance measures to ensure performance objectives and criteria are challenging and focused on improving performance in known areas of weakness.
DOE line management has established effective processes for communicating line oversight results and other issues up the DOE line management chain, using a graded approach based on the hazards and risks. Established processes include provisions for communicating and documenting dissenting opinions. Formal structured processes for resolving disputes for oversight findings and other significant issues have been implemented, and include provisions for independent technical reviews for significant findings.
An effective employee concerns program been established and implemented in accordance with DOE Directives that encourages the reporting of employee concerns and provides thorough investigations and effective corrective actions and recurrence controls.

< BACK TO PREVIOUS PAGE