July 10, 2003, Board letter establishing a 120-day reporting requirement regarding hoisting and rigging operations at the Pantex Plant

[DNFSB LETTERHEAD]

July l0, 2003

The Honorable Everet H. Beckner

Deputy Administrator for Defense Programs

National Nuclear Security Administration

U.S. Department of Energy

1000 Independence Avenue, SW

Washington, DC 20585-0104

Dear Dr. Beckner:

The staff of the Defense Nuclear Facilities Safety Board (Board) recently conducted a review of hoisting and rigging operations at the Pantex Plant. A number of significant deficiencies were identified by the site contractor, BWXT Pantex, and the Pantex Site Office following the recent hoisting occurrence in the mass-properties facility at the site. The Board’s staff noted issues, including improper maintenance of safety-class hoists and inconsistent training, as discussed in the enclosed report.

A number of corrective actions are in various stages of planning and implementation to address the identified issues. Although it is too early to judge the effectiveness of the proposed corrective actions, the Board remains concerned and urges prompt attention to this important safety program.

Therefore, pursuant to 42 U.S.C. § 2286b(d), the Board requests a status briefing within 120 days of receipt of this letter or when a final set of corrective actions for the issues noted in the enclosed report has been clearly defined and funded-whichever occurs sooner.

Sincerely,

John T. Conway

Chairman

c: Mr. Mark B. Whitaker, Jr.

Enclosure

DEFENSE NUCLEAR FACILITIES SAFETY BOARD

Staff Issue Report

June 20, 2003

MEMORANDUM FOR: J. K. Fortenberry, Technical Director

COPIES: Board Members

FROM: J. Shackelford, C. Shuffler

SUBJECT: Hoisting and Rigging at the Pantex Plant

This report documents a review of the hoisting and rigging program at the Pantex Plant, conducted during the week of May 26, 2003. Members of the staff of the Defense Nuclear Facilities Safety Board (Board) J. Shackelford and C. Shuffler, together with outside expert R. West, evaluated the site’s hoisting and rigging program, focusing in depth on the root-cause analysis, corrective actions, and lessons learned from recent hoist-related occurrences. Other major areas of the review included the following:

The standards and codes required by contract and implemented by the contractor for the Pantex Plant, BWXT Pantex (BWXT), in its hoisting and rigging program.

The hazard and accident analyses performed for Zone 12 bays and cells and the safety classification of hoisting and rigging equipment.

Facility operations, including simulations of pre-engineered production lifts.

Training and qualification programs for personnel involved in hoisting and rigging operations.

The procurement, installation, inspection, testing, and maintenance of hoisting and rigging equipment, including cranes, hoists, and below-the-hook lifting devices.

BWXT’s review of the January 2003 hoist brake failure in Building 12-60 revealed several significant deficiencies in the hoisting and rigging program. In response to the review, BWXT and the Pantex Site Office (PXSO) established several corrective action programs to address needed improvements. The Board’s staff found these programs to be in various stages of development, approval, and implementation, but noted that there is no clear indication that resources and funding are available to complete them. The staff also identified some additional issues that had not been addressed.

Codes and Standards. The Board’s staff reviewed the codes and standards in the BWXT contract related to hoisting and rigging and found that the majority of the applicable industry guidance is appropriately referenced in the standards/requirements identification documents (SRIDS). However, one significant exception involved American Society of Mechanical Engineers (ASME) NOG-1, Rules for Construction of Overhead and Gantry Cranes. Department of Energy (DOE) Guide 420.1-1, Nonreactor Nuclear Safety Design Criteria and Explosive Safety Criteria, requires the use of ASME NOG-1 for the construction of safety-significant and safety-class cranes. The Board’s staff observed that this standard was not referenced in the SRIDS, and that BWXT had not evaluated the safety-class hoists and cranes at Pantex with respect to the guidance contained in the standard. BWXT personnel acknowledged the staffs concern and committed to evaluating the safety-class hoisting equipment using ASME NOG-1.

Safety Classification. BWXT’s current authorization basis classifies the 83 facility hoists used for nuclear explosive operations as safety-class equipment. The requirements for this equipment are set forth in Design Feature (DF) 3 of the site Technical Safety Requirements (TSRs). The DF requires the hoists and cranes to be designed and installed in accordance with applicable sections of 29 Code of Federal Regulations (CFR) 1910.179 and the ASME B30 series for load rating, brakes, and stops.

The current documented safety analyses for Zone 12 facilities, which have been approved by DOE but not yet implemented by the site, postulate swinging impacts involving weapons traveling at the maximum horizontal speed that BWXT calculated that a facility hoist could reach under normal operating conditions. However, the hoists at Pantex are pneumatic, with the speed of travel being determined by the inlet pressure to the hoists. Although the hoists are designed to operate at lower pressures (i.e., less than 90 psig), the facilities’ compressed air system can deliver significantly greater air pressures (i.e., in excess of 125 psig) under certain circumstances. To date, the contractor has been unable to determine the theoretical maximum hoist speeds at these higher pressures.

BWXT has proposed revisions to the approved, but unimplemented, authorization basis that would increase the maximum speed used for the accident analyses, but these revisions cannot be completed until new weapon response data are provided by the design agencies. The potential will still exist for the installed equipment to exceed this new, higher maximum speed. As a result, BWXT is considering engineering modifications to limit inlet air pressure, thus limiting the hoist travel speed. The staff observed that no implementation plan has been approved and funded to design and install any of these modifications.

The Board’s staff also noted that the travel stops installed on the equipment are not rated for the speeds that could be encountered in the currently postulated scenarios involving a runaway hoist. As a result, the staff questioned whether the current approved, but unimplemented, accident analysis is bounding with respect to the limiting impact scenarios.

Operations and Training. The Board’s staff reviewed facility hoisting and rigging procedures and observed operations in the training bays. Additionally, the staff conducted interviews and discussions with production technicians, system engineers, site management, and the training staff concerning appropriate actions on the part of production technicians during a postulated incident involving a runaway hoist. The staff noted significant inconsistencies among the responses expected of production technicians by management, training, and operations personnel during such an event.

During the January 2003 hoist brake failure in Building 12-60, the production technicians lost the ability to control the hoist with the pendant and resorted to guiding a nuclear explosive component into its fixture manually. BWXT management representatives indicated that they expected production technicians to depress the stop button on the hoist pendant while simultaneously dispatching a second technician to close the emergency shutoff valve supplying air to the hoist. They did not expect, or desire, production technicians to use other control functions on the pendant to redirect the hoist. The training staff, however, indicated that they trained production technicians to use all available means, including redirecting the hoist with other pendant control functions, to regain control of a malfunctioning hoist. The Board’s staff interviewed production technicians during two separate simulated hoisting operations, and received two different responses regarding the actions each production technician would take during such a scenario. Further, the Board’s staff observed that system engineering personnel had not evaluated the runaway hoist scenario with respect to expected equipment performance. The Board’s staff noted that BWXT has had more than 4 months since the actual brake failure incident to implement appropriate corrective actions and remedy any confusion about the desired response of production technicians during this type of failure. BWXT personnel acknowledged this training deficiency and are currently working to resolve it.

Review of the Building 12-60 Hoist Brake Failure, Root-Cause Analysis, and Corrective Actions. The Board’s staff reviewed the operational aspects, root-cause assessment, and corrective actions associated with the hoist brake failure in Building 12-60 that occurred on January 10, 2003. The details of this event are documented in Occurrence Report ALO-AO-BWXP-PANTEX-2003-0002. During the event, the facility hoist failed to respond to operator actions intended to control a nuclear explosive unit during lifting operations, and the unit slowly lowered to a resting position. During the event, operators took actions to secure the unit safely, and no adverse consequences or damage occurred. However, the event revealed a number of deficiencies in the hoisting and rigging program, as well as issues related to root-cause analysis and corrective actions.

Immediately following the incident, BWXT removed the hoist from service and performed a root-cause analysis to determine the cause of the failure. At the time, BWXT system engineers did not have detailed diagrams and manuals for this safety-class hoist. Therefore, they were unable to perform a rigorous failure modes and effects analysis (FMEA) to support the root-cause analysis. As a result, BWXT initially attributed the hoist failure to deficiencies in the control pendant, a conclusion later determined to be incorrect. After this initial (incorrect) root-cause analysis, the pendant was replaced, and the hoist was returned to service. Following additional questions and discussion with PXSO and the hoist manufacturer, however, BWXT again removed the hoist from service for further investigation. When the required drawings were received from the manufacturer, almost 1 week after the incident, an FMEA was performed. It showed that the root-cause initially identified by BWXT was not a credible failure mode.

During follow-up discussions with the manufacturer, BWXT noted that this type of hoist had been subject to similar failures in other applications. The manufacturer had determined that a design deficiency existed in the brake unit. This deficiency allowed a buildup of brake dust to bind the brake piston in its housing, preventing application of the brake. The manufacturer reported that a modification was available to limit the effect of this dust buildup. This modification was subsequently made to the applicable hoists. Brake inspections of the failed hoist revealed approximately 1 teaspoon of brake dust in the unit. BWXT personnel acknowledged that they had not been following the manufacturer’s recommendation to disassemble, clean, and inspect the brake periodically. The root-cause analysis was subsequently revised to reflect the fact that the primary cause of the incident was failure to perform the manufacturer’s recommended maintenance, with a secondary, contributing factor being related to the design deficiency.

BWXT determined that the initial root-cause analysis had been based on inadequate technical information and had incorrectly identified the cause of the failure. As a result, an inadequate corrective action plan had been implemented, and the hoist had been returned to service prematurely before the actual failure mode had been corrected. BWXT is currently working to upgrade the methodology supporting its corrective actions process. The incident also revealed significant weaknesses in BWXT’s program with respect to maintaining adequate technical information and documentation, as well as communicating with vendors to ensure that up-to-date information on operating experience is available for safety-class equipment. Finally, the event highlighted the need for significant improvements in BWXT’s efforts to ensure that all of the manufacturer’s recommended maintenance is either performed or appropriately evaluated for deferral or exclusion. The current corrective action plan for the hoist brake failure in Building 12-60 addresses this issue for the safety-class hoists at the Pantex Plant.

Maintenance and Procurement. As noted above, BWXT concluded that it had not adequately implemented the manufacturer’s recommended maintenance activities (or properly justified their omission) for safety-class hoists. Upon further review, BWXT determined that this problem existed with other vital safety systems at the site and indicated that corrective actions were under way to address this issue.

Following a request by the Board’s staff, it was found that BWXT did not have an adequate program for trending equipment failures and deficiencies. A report on crane and hoist deficiencies for 1998–2002, generated in response to the staffs request, showed maintenance problems and deficiencies in a variety of areas. Currently, no formal program exists to track and trend equipment failures for the safety-class hoists, but BWXT committed to establishing such a program in a corrective action plan. Additionally, the Board’s staff noted that the site had only recently incorporated predictive maintenance practices (e.g., lube oil analysis) into the maintenance programs for vital safety systems.

An incident that occurred in March 2003, involving the failure of draw cables for a lifting and rotating fixture, illustrated a deficiency in the site’s procurement process. A cable had failed in an unexpected manner, with the wire rope pulling out of the swaged end. Subsequent testing revealed that cables procured during the last 2 years had continued to fail in this unexpected manner and at loads significantly below the manufacturer’s rating. It was determined that the site had not implemented the appropriate administrative mechanisms to ensure that the procured components would meet the required quality control criteria. BWXT is currently working to resolve this procurement problem.

Conclusion. The Board’s staff noted a number of significant deficiencies in the Pantex Plant’s hoisting and rigging program. Most of these issues had been identified by BWXT and PXSO prior to the visit by the Board’s staff. A number of the deficiencies are generic in nature and applicable to other vital safety systems and program areas. Although corrective actions have been initiated or are being developed for these issues, there is no clear indication that resources and funding are available to complete all of the required actions. In view of the actions remaining to be accomplished, the Board’s staff will continue to follow resolution of these issues closely.