Lessons Learned Programs

Content

• Introduction
• Notice Summary
• Applicability
• Lessons-Learned Program Elements
• Lessons Learned Process
• Lessons Learned Information Resources
• Recommendations
• References

Introduction

This notice is one in a series of publications issued by the Office of Nuclear and Facility Safety to share nuclear safety information throughout the Department of Energy complex. For more information, contact Dick Trevillian, Office of Operating Experience Analysis and Feedback, Office of Nuclear and Facility Safety, U.S. Department of Energy, Washington, DC 20585, telephone (301) 903-3074. No specific action or responses are required solely as a result of this notice.

Safety Notices are distributed to U.S. Department of Energy Program Offices, Field Offices, and contractors who have responsibility for the operation and maintenance of nuclear and related facilities, and to other organizations involved in nuclear safety. Written requests to be added to or deleted from the distribution of Safety Notices should be sent to: BR Richard L. Trevillian, EH-33, Room E-460 GTN, U.S. Department of Energy, Washington, DC 20585.

The ESH Office of Information Management maintains a file of Safety Notices and supporting information. Copies can be obtained by contacting the Office of Information Management at (301) 903-0449 or by writing to the Office of Information Management, U.S. Department of Energy, EH-72/Suite 100, CXXI/3, Washington, DC 20585.

Notice Summary

In March 1994, the Deputy Associate Secretary for Field Management established a DOE Lessons-Learned Process Improvement Team (LLPIT) to develop a complex-wide lessons-learned program that would standardize and facilitate identification, documentation, sharing, and use of lessons learned from operating experiences throughout the DOE complex. Recently, the LLPIT issued DOE-STD-7501-95, Development of DOE Lessons Learned Programs¹, and DOE-HDBK-7502-95, Implementing U.S. Department of Energy Lessons Learned Programs². These non-mandatory documents provide the standard for lessons-learned programs and guidance for implementation of the Standard.

Various DOE orders, rules, and requirements require that lessons learned be identified, evaluated, shared, and incorporated and many DOE sites have developed local lessons-learned programs. However, because a comprehensive DOE standard for developing and implementing lessons-learned programs did not exist, identification, sharing, and use of lessons learned have been inconsistent and sporadic within the DOE complex.

This Safety Notice describes the elements of an effective lessons-learned program and its processes as established in DOE-STD-7501-95 and DOE-HDBK-7502-95. It also provides information on lessons-learned resources such as the electronic DOE Lessons Learned Information System, and makes recommendations for managers at DOE facilities to develop, evaluate, or improve their lessons-learned programs.

Applicability

This notice applies to all organizations and operations, contractors, and subcontractors, who work under the direction of DOE and should be processed as a lessons-learned information source as described DOE-STD-7501-95.

Lessons-Learned Program Elements

The following elements are essential to an effective lessons-learned program.

• The following elements are essential to an effective lessons-learned program.
• Program description
• Promotion and communication plan
• Procedures
• Training
• Staffing and other resources
• Lessons-learned process activities
• Performance measures
• Effectiveness reviews

Management Support and Commitment

Section 3.2 of the Standard states: "Senior management has the responsibility to ensure that sufficient resources are provided to effectively implement a lessons-learned program." Senior managers are more likely to support a program that will improve organization efficiency and effectiveness, directly support management strategic goals, and be cost-effective.

Program Description

A locally designed program description provides the framework for a lessons-learned program and a basis for all related activities. Section 1.1 of the Handbook provides guidance on the purpose and content of the program description.

Promotion and Communication Plan

The more people know about the program, the more comfortable they are with it and the more likely they are to participate. Plans for promotion and communication of the program are necessary to gain the site-wide and DOE-wide involvement that is crucial to the success of a lessons-learned program.

Section 1.1 of the Handbook contains guidelines for promotion and communication plans. The Handbook also recommends customer feedback mechanisms, such as surveys and focus group meetings, as an important part of a communication plan.

Procedures

Procedures provide the steps necessary to implement and administer a lessons-learned program. They define organizational interfaces, establish roles and responsibilities, and promote consistency across multi-facility sites and the DOE complex, making it easier to understand and apply lessons-learned information from external sources.

Lessons-learned procedures should be formal and approved in the same manner as other operating and administrative procedures. They should be linked directly to the program plan and top-level management policies or directives to give lessons learned legitimacy within the organization.

Section 4.4 of the Standard includes a list of elements typically included in lessons-learned procedures. Appendix III of the Handbook contains examples of lessons-learned procedures.

Training

Section 4.3 of the Standard states: "Lessons learned personnel should be trained on implementing this technical standard and the organizational lessons learned program. Personnel training should include how to identify and research sources of lessons learned; how to develop, use, analyze, and disseminate lessons learned; how to identify trends based on lessons learned; and how to incorporate lessons learned into processes, operations, procedures, and training." Section 2.2 of the Handbook provides extensive guidance on training lessons-learned personnel.

Two types of training are important to the success of a lessons-learned program.

General Employee Training

ncluding lessons-learned information in the general employee training program is an excellent way to introduce the concept of lessons learned to new employees and to re-emphasize it to current employees. Section 2.2 of the Handbook describes how to include lessons learned in general employee training.

Training for Lessons-Learned Personnel

Specific knowledge of the lessons-learned process is useful for personnel directly involved with coordination, management, or administration of the program. Expertise with the lessons-learned process, combined with general technical knowledge of the facility and knowledge of organizational strategic goals, will aid in efficiently identifying and filtering this information and help select what is applicable and warrants further analysis.

Staffing and Other Resources

Section 4.2 of the Standard defines the roles and responsibilities of DOE and contractor managers and individuals who will implement, use, and contribute to the DOE lessons-learned program. The role of the lessons-learned coordinator is key to the success of the program. The coordinator typically provides day-to-day direction of the program and is the lessons-learned point-of-contact for the site.

Section 4.1 of the Standard states: "By taking advantage of existing resources, the impact to current programs should be negligible." The cost of implementing a lessons-learned program can be quickly outweighed by savings from implementing good practices and avoiding resource-consuming negative events, especially if existing resources are efficiently used.

Lessons-Learned Process Activities

There are essentially two parts of the lessons-learned process: (1) development and dissemination and (2) utilization and incorporation. These processes are described in detail later in this Notice and in the Standard and the Handbook.

Performance Measures

The Handbook recommends the use of performance measures to determine how well the process is being implemented and to identify areas needing improvement. Performance measures should be related to the goals and objectives of the program. Section 4.1 of the Handbook is a valuable source of information for developing performance measures for lessons-learned programs as well as other programs.

Effectiveness Reviews

Performance measures can sometimes focus only on measuring and improving processes while overlooking measuring the utility of the overall program. Effectiveness reviews are intended to evaluate the "big picture." They compare the costs and benefits and seek ways to improve the utility of the program.

Performance measures can be used as a tool for effectiveness reviews. When used, they should effectively demonstrate the utility of the lessons-learned program. They should show that the program not only improves safety, but reduces costs associated with accidents, inefficiencies, and redundancies. Performance measurement activities should be sensitive to the need for early success stories to demonstrate the utility of the lessons-learned program in the short term, as well as the need for performance measures that improve the process over the long term.

Lessons Learned Process

Development and Dissemination

This process involves capturing, analyzing, and distributing internal events or experiences for distribution within the organization and to the DOE complex. The key here is the capture and dissemination of internal events, in contrast to the second element which involves capture and use of external events. This is where lessons learned originate. Following is a description of the basic activities that make up this part of the lessons-learned process.

Identify Internal Sources of Lessons-Learned Information

Section 3.1 of the Handbook and section 5.1 of the Standard list internal lessons-learned information sources, most of which are part of existing programs.

Review Information for Applicability and Significance

The next step is to systematically review the internal sources to identify lessons that may be useful to other DOE organizations. The product from this step is a set of lessons learned for which documents will be developed and disseminated.

Develop Lessons-Learned Document

Section 3.1 of the Handbook and section 5.2 of the Standard describe development of lessons-learned documents. Documents may be developed for internal or DOE-wide distribution. Documents to be distributed DOE-wide must be in the format defined in the Standard, which ensures that all lessons learned distributed complex-wide are consistent, easy to access, and easy to understand.

Technical Review and Approval

Before being submitted for security review and distribution, lessons-learned documents should be technically reviewed to ensure accuracy, completeness, and applicability. This step should be performed before the security review to ensure that changes resulting from technical review do not alter the security classification of the document.

Security Classification and Control Review

Section 5.4 of the Standard states: "Prior to dissemination, lessons-learned documents shall be reviewed for compliance with organizational security requirements, reviewed by an Authorized Derivative Classifier for security classification, or reviewed by a Reviewing Official for Unclassified Controlled Nuclear Information." Documents distributed on the DOE Lessons Learned Information System must not contain classified information or Unclassified Controlled Nuclear Information.

Dissemination

Sections 5.5 of the Standard and section 3.2 of the Handbook address dissemination of lessons-learned documents. The Handbook contains many ideas and examples on how to disseminate these documents. The first step is to identify recipients who could benefit from the information in the document. Should the document be distributed locally, to other DOE sites or facilities, or DOE-wide?

Having established who should receive the document, the most effective method of dissemination can then be determined. For local distribution, E-mail, hard copy, or local-area-networks may be used. For distribution to other sites or facilities, possibilities include hard-copy distribution, E-mail, wide-area-networks, and the Lessons-Learned Information System (LLIS). For DOE-wide distribution, the LLIS, which is described later in this Notice, should be used.

Guidance contained in sections 5.2, 5.4, and 5.5 of the Standard should be observed when distributing lessons learned through the LLIS.

Utilization and Incorporation

This part of the lessons-learned process is related to capturing, analyzing, and applying internally and externally-developed lessons learned, events, or experiences. Through this process we learn through others' experiences and improve the safety and efficiency of operations.

Section 3.3 of the Handbook provides guidance on this part of the process. It states: "There is no reason to document lessons learned and make this information available to others if it is not going to be used (i.e., fed back into processes and activities). This element of the lessons-learned process is critical yet frequently does not receive adequate attention."

Following is a description of the basic activities that make up this part of the process.

Identify Potential Sources of Lessons Learned

Sources of lessons-learned information include internal organizations, other DOE facilities, other Federal agencies, and industry. Sections 5.1.2 and 5.1.3 of the Standard list information sources from the DOE complex and areas for which there is information from Federal and industry sources.

Appendix XI of the Handbook, "Index of Professional and Industry Sources of Lessons Learned," is a valuable reference that lists industry lessons-learned information sources.

Although much information can be obtained from hard copy distribution, electronic gathering is rapidly becoming the method of choice for obtaining lessons-learned information for the following reasons.

• Most information that can be obtained via hard-copy is also available electronically.
• Electronic information is typically searchable and quickly accessible.
• Frequently-accessed sources can be marked for quick reference.
• Many tools are now available to efficiently isolate and retrieve information electronically.

Determine Which Sources Will Be Screened

The Internet and LLIS contain many information sources that can be screened for lessons learned. However, availability of resources typically limit how much can be screened. Therefore, qualified personnel must determine which sources will be systematically screened for applicability to their facility. The selected sources should have a high probability for containing applicable lessons learned. For example, an environmental restoration site may wish to systematically screen the following lessons-learned information sources.

• Occurrence Reporting and Processing System
• Operating Experience Weekly Summaries
• Safety Notices
• Environment, Safety, and Health Bulletins
• Type A, B, and C Investigation Reports
• Lessons learned publications from other environmental restoration sites
• Information on the Environmental Protection Agency Internet gopher

Screen Lessons-Learned Information for Applicability and Significance

The next step is to screen lessons-learned sources for items that are applicable and may warrant corrective actions. Because of the wide diversity in subject matter and the large number of documents encountered, this task is best performed by an individual with a broad technical base and experience in investigative techniques who can quickly determine applicability.

Distribute to Appropriate Personnel for Review

Selected items are distributed commensurate with their significance and the potential for corrective actions. Some items may be included in general distribution while others may be assigned to specific people for detailed reviews of applicability and the need for further actions.

Determine If Actions Are Required

A lessons-learned program would be futile if it did not result in changes to implement good practices or avoid negative incidents. Those reviewing lessons learned should do so with thoughts toward identifying specific actions that will improve safety and efficiency at their facility.

To ensure technical adequacy, conclusions and recommended actions should be independently reviewed by knowledgeable individuals. Group or peer reviews are particularly beneficial because of the wider perspective and knowledge that can be applied to the analysis and recommendations.

Experience has shown that a recommended change is more likely to be accepted and implemented when input on the proposed recommendation is obtained from the group or organization that would implement it. Their input provides valuable information on resource limitations, technical abilities, cost/benefit, schedule, and other factors affecting the feasibility of a recommendation.

Implementing Corrective Actions

Once identified, corrective actions must be implemented if they are to have a positive impact. The Handbook advises that timeliness and assignment to a responsible party are important elements of corrective action implementation. The longer it takes to implement an action, the higher the probability that the event the recommendation was intended to prevent will occur.

The Handbook states: "When appropriate, the Lessons Learned Coordinator should track corrective actions to ensure timely completion and provide monthly status reports to the Facility Manager." Existing tracking systems can usually be used to track actions from lessons-learned reviews. Appendix IX of the Handbook contains examples of corrective action tracking.

Lessons Learned Information Resources

This section describes lessons-learned information resources that are particularly useful to DOE organizations in addition to those already discussed (i.e., the Standard and Handbook).

DOE Lessons Learned Information System

The Lessons Learned Information System (LLIS) is an electronic system that uses the Internet to provide access to information from local lessons-learned programs, as well as other federal agencies and industry. It is described in detail in Appendix D of the Standard and section 3.2 of the Handbook. The LLIS is comprised of the DOE Technical Information Service and the Lessons Learned List Server.

DOE Technical Information Service

The DOE Technical Information Service (TIS) is a collection of seamless applications that use a graphical interface and the Internet to provide quick and easy access to DOE lessons-learned information on the Internet. TIS also provides access to a variety of DOE reference material, technical information useful to DOE organizations, and gateways to other federal agency and industry information on the Internet.

TIS provides the following basic services to access lessons-learned information.

• Bulletin board services (Gopher)
• Electronic mail services (E-mail)
• News groups
• World Wide Web Services (Mosaic)
• DOE Lessons Learned Home Page (Site Servers)

Lessons Learned List Server

The List Server is a tool that uses an E-mail messaging service for rapid dissemination of urgent or important lessons-learned information to DOE site lessons-learned coordinators and other key persons. Lessons learned, formatted in accordance with the guidelines in the Standard, are sent to the List Server in an E-mail message. The List Server then sends the lesson to all subscribers. The List Server is restricted to DOE and contractor personnel.

Lessons-Learned Publications

There are a number of DOE-wide publications that contain useful lessons-learned information. Many are available electronically on the Internet via TIS.

DOE Occurrence Reporting and Processing System

The Occurrence Reporting and Processing System (ORPS) is the most extensive source of lessons-learned information in the DOE complex. It is a mainframe computer database that catalogs all reportable events at DOE facilities. The database, which can be searched, includes event descriptions, causes, corrective actions, and lessons learned from the events.

Nuclear Regulatory Commission Publications

The Nuclear Regulatory Commission issues documents with lessons-learned information from the commercial nuclear industry. Documents that may contain useful information for DOE facilities include Information Notices, Bulletins, and Generic Letters. These documents are also available on the Internet.

Recommendations

Developing effective lessons-learned programs on both local and DOE-wide levels is advantageous from two viewpoints. The first is personnel safety, which should be the highest priority for all DOE operations. An effective lessons-learned program reduces injuries and other threats to facility, environmental, and public safety by applying lessons learned from previous incidents. Second, effective lessons-learned programs are cost-beneficial. The cost of implementing such programs can be quickly outweighed by savings from implementing good practices and avoiding resource-consuming negative events, especially if existing resources are efficiently used.

The Office of Nuclear and Facility Safety encourages the development and use of lessons-learned programs at all DOE facilities and makes the following recommendations towards that end.

• Managers at facilities without a lessons-learned program should develop of one as soon as practical. Management at these facilities should provide commitment and support for development of such a program. The Standard and the Handbook should be used as primary references for developing the program.

• Managers at facilities with existing programs should review their programs against the guidelines in the Standard and Handbook. Areas that tend to be weak and require attention include (1) use of external lessons-learned information, (2) action identification and implementation, (3) performance measurement, and (4) effectiveness reviews.

As stated in the Standard: "A complex-wide program will help to prevent recurrences of negative experiences, highlight best practices, and spotlight innovative ways to solve problems or perform work more safely, efficiently, and cost effectively."

For more information on the LLPIT or to obtain a list of people to contact for lessons-learned information, contact Richard Trevillian, EH-33, (301) 903-3074 or Internet E-mail address: dick.trevillian@hq.doe.gov.

For more information on the LLIS and Lessons Learned List Server, contact Bill McQuiston, DOE-Idaho, (208) 526-7873 or Internet E-mail address: mcw@tis.inel.gov.

References

1. DOE-STD-7501-95, Development of DOE Lessons Learned Programs, DOE Office of Scientific and Technical Information, (615) 576-8401, May 1995.

2. DOE-HDBK-7502-95, An Easy-Reference Guide to Implementing the DOE Lessons Learned Standard, DOE Office of Scientific and Technical Information, (615) 576-8401, May 1995.