Office of Emergency Management
Oversight (HS-63)

 

 

 

 

 

Emergency Management Program Evaluation Inspector’s Guide

HS-63

March 2008

 


Preface

 

This Emergency Management Program Evaluation Inspector’s Guide provides detailed information and tools to assist inspectors assigned to evaluate the capabilities and performance of emergency management programs across the DOE Complex.  This inspector’s guide is designed specifically as an internal training tool for HS-63 inspectors.  HS-63 anticipates making periodic revisions to this guide in response to changes in DOE/NNSA program direction and guidance, insights gained from independent oversight activities, and feedback from DOE Headquarters, field offices, and sites, as well as external stakeholders.  Therefore, users of this process guide are invited to submit comments and recommendations to the Director, HS-63, at Steven.Simonson@oa.doe.gov.

 


Contents

 


Acronyms
AEGL        Acute Exposure Guideline Level
CAP          Corrective Action Plan
DOE          U.S. Department of Energy
DSA          Documented Safety Analysis
EAL          Emergency Action Level
EMG         Emergency Management Guide
EOC          Emergency Operations Center
EPA          Environmental Protection Agency
EPHA       Emergency Planning Hazard Assessment
EPI           Emergency Public Information
EPZ          Emergency Planning Zone
ERAP       Emergency Readiness Assurance Plan
ERO         Emergency Response Organization
ERPG       Emergency Response Planning Guidelines
FHA         Facility Hazards Analysis
HS-60 

Office of Independent Oversight
HS-63 Office of Emergency Management Oversight
JIC Joint Information Center
LEPC Local Emergency Planning Committee
MOA       Memoranda of Agreement
MOU Memoranda of Understanding
NARAC  National Atmospheric Release Advisory Center
NIMS      National Incident management System
NNSA     National Nuclear Security Administration
NRP        National Response Plan
OSHA     Occupational Safety and Health Administration
PRA        Probabilistic Risk Assessment
SAR        Safety Analysis Report
SERC      State Emergency Response Committee
TSR         Technical Safety Analysis
TEEL       Temporary Emergency Exposure Limit

 

 

 

 

Section 1 

INTRODUCTION

 

Purpose

 

As part of an effort to enhance the appraisal process, the Office of Independent Oversight (HS-60) and the Office of Emergency Management Oversight (HS-63) have prepared a series of documents that collectively provide comprehensive guidance and tools for evaluating emergency management programs across the U.S. Department of Energy/National Nuclear Security Administration (DOE/NNSA) complex.  The Independent Oversight Appraisal Process Protocols document describes the philosophy, scope, and general procedures applicable to all independent oversight appraisal activities.  The HS-63 Emergency Management Oversight Appraisal Process Guide describes specific procedures used by HS-63 in planning, conducting, and following up emergency management inspections.  The information in these appraisal process documents is not repeated here and therefore should be referred to when planning and conducting emergency management program inspections.  In particular, the HS-63 Emergency Management Oversight Appraisal Process Guide provides information regarding inspectors’ responsibilities and an overview of inspection activities.

 

This guide provides a set of detailed tools and references that inspectors can use to plan and conduct an inspection of the emergency management program.  These tools serve to promote consistency, ensure thoroughness, and enhance the quality of the inspection process.

 

Organization of Guide

 

This section of the guide provides an overview of inspection goals/areas of emphasis.

 

Section 2 (Program Elements) provides detailed guidance for inspecting emergency management program elements and includes descriptions of various types of data collection activities and the identification of common deficiencies identified in the past.

 

Section 3 (Analyzing Data and Interpreting Results) contains guidelines on how to analyze information gathered during data collection activities (including the impact that a deficiency in one program element may have on other program elements), interpret the significance of potential deficiencies, assign ratings to the supporting elements of a site’s emergency management program, and identify findings.

 

Appendix A provides references.

 

Overview

 

The tools contained in this guide are intended to be used at the discretion of the inspector.  Typically, inspectors select the tools that are applicable and most useful on a facility-specific and inspection-specific basis.  Although the guidelines presented here cover a variety of inspection activities, they do not and cannot address all emergency management program variations at all DOE/NNSA facilities.  The tools may have to be modified or adapted to meet inspection-specific needs, and in some instances, inspectors may have to design new activities and new tools to collect needed information.

 

The information in this guide does not repeat all of the detailed information in DOE directives.  Rather, it is intended to complement the directives by providing practical guidance for planning, collecting, and analyzing inspection data.  Inspectors should refer to this guide as well as DOE directives and other guidance documents at all stages of the inspection process.

 

One objective in developing HS-63 inspector guides is to provide a repository for the collective knowledge of HS-63’s most experienced inspectors that can be enhanced and updated as inspection methods improve and inspection experience accumulates.  Every attempt has been made to develop specific guidelines that are as useful as possible.  In addition to guidelines for collecting information, the inspection tools provide aids for prioritizing and selecting activities, then analyzing and interpreting results.  The specific guidelines should be viewed as suggestions rather than requirements, and they must be critically examined and interpreted on an inspection-specific basis, taking into account site-specific factors.

 

Inspection Goals

 

The primary inspection goal is to determine, with reasonable certainty, whether the emergency management program is both adequately meeting the appropriate standards established by DOE policy and is capable of providing appropriate protection to site personnel and the public in case of an accident at the site.  Accomplishing this goal requires that the inspection team determine whether the emergency management program is adequately managed, staffed, trained, equipped, and capable of performing all mission-related tasks and duties.

 

Compliance/Performance

 

While an emergency management program inspection includes both compliance and performance activities, a greater emphasis is placed on the performance aspect, as it is more useful in determining whether the emergency response organization (ERO) can perform its mission.  Many of the DOE emergency management requirements contained in DOE Order 151.1C are stated in performance terms: that is, they state a capability, duty, or integrated response that must be performed.  Therefore, compliance requires effective performance.  Even when dealing with policy requirements for which a compliance approach may seem appropriate (e.g., Does the training program contain the required elements?), the HS-63 approach for this topic is to go beyond compliance and determine the performance aspects of these requirements (e.g., Does the training program adequately prepare the emergency director to perform his/her mission?).  Therefore, whenever possible, data-collection activities for the emergency management program should be performance-oriented.

 

Planning Goals

 

The ultimate goal of planning is to anticipate and provide for every action necessary to conduct the highest quality inspection possible with the resources available.  That is an extremely broad goal, and it provides little structure for actual planning.  However, it is useful to focus the planning process on several narrower, yet major, goals.  Examples of such goals might include:

Planning Decisions

 

Based on analysis of the information gained from a scoping visit, site document review, discussion with other inspection team members, and discussion with the site points of contact, the inspection team leader must make a number of decisions, including:

Once these decisions have been made, individual inspection plans can be developed, and the detailed planning of data collection activities can proceed.

 

Relevant Site Documents

 

Many of the program elements are based on the site documents listed below.  The inspector should review the sections of these documents germane to his/her assigned review area during planning and/or data collection activities.  Other relevant documents that are specific to a program element are listed in the appropriate review area of Section 2.

Section 2A 

HAZARDS SURVEYS AND EMERGENCY PLANNING
HAZARDS ASSESSMENTS

 

General Information

 

Hazards surveys and emergency planning hazards assessments (EPHAs) form the basis for a site’s emergency management program.  The hazards survey serves to establish the “Base Operational Emergency Management Program.”  The hazards survey is intended to identify emergency management program needs that are different from those addressed by the EPHA.  Therefore, each facility/site should be included in a hazards survey, regardless of the need for an EPHA.  The hazards survey contains State, local, and Federal agency requirements related to facility and occupational safety, environmental and effluent controls, and hazardous materials management.  It also contains a listing of all hazardous materials and a qualitative evaluation, through a screening process, of whether the hazardous materials will need to be considered in an EPHA.  If hazardous materials meet or exceed specified screening threshold criteria, then a quantitative EPHA is performed to determine whether a hazardous material emergency management program is needed.  Such a program is needed if the EPHA indicates that release of hazardous materials may result in a classifiable emergency.  Other important information derived from the EPHA is the technical bases for determining the size of the emergency planning zone (EPZ), and identification of indications of barrier failures that should be used to develop emergency action levels (EALs).

 

Because the hazards survey and EPHA(s) form the basis for developing the emergency response program, deficiencies in these documents can lead to deficiencies in other emergency management program elements.  The Potential Impacts on Other Program Elements in section 3 of provides a description of these potential impacts.

 

Relevant Site Documents

 

In addition to the site documents listed in Section 1, the inspector may choose to review following site documents before or during the performance of onsite data collection.

Data Collection Activities.

Document Reviews

The following data collection activities have been found to be beneficial in evaluating this emergency management program element.  The inspector should choose which of the activities to perform based upon the focus of the evaluation and the site-specific features of the emergency management program.  Further guidance is found in the inspection plan.

 

Hazards Survey

 

A.  Inspectors should determine whether a site mechanism is in place that ensures required hazards survey elements are incorporated.  The mechanism could be an emergency planning document or procedure.  Review the mechanism used to develop the hazards survey and evaluate whether it specifies:

B.  Inspectors should review the hazards survey to determine whether it was prepared in accordance with the site mechanism and possesses all appropriate elements, in matrix or tabular format.  In addition, determine whether all facilities, including offsite hazardous facilities (e.g., water treatment plant using chlorine) and activities (e.g., transportation artery) that may impact the site are considered in the hazards survey.

 

C.  Inspectors should review the hazards survey to determine whether hazardous material screening, if performed as part of the survey process, is consistent with DOE Order 151.1C and DOE Guide 151.1-2, Technical Planning Basis.  The following activities should be performed to support this evaluation:

D.  Inspectors should cross-check emergency plan facility descriptions, the site map, and the listing of facilities in the hazards survey to determine whether the hazards survey includes all facilities.

 

E.  Inspectors should review the site’s hazardous material database(s) to determine whether all hazardous materials are listed in the hazards survey, or by reference, and have been identified.

 

F.   Inspectors should walk down a number of facilities, including the shipping and receiving warehouse, and observe transportation activities to check the accuracy of chemical, biological, and radiological hazardous material inventory database(s) used as the basis for determining amounts of hazardous materials that are used or stored in facilities.

 

G.  Inspectors should walk down the emergency response facilities to determine whether an up-to-date copy of the hazards survey is available in each facility.

 

Emergency Planning Hazards Assessments

 

H.  Inspectors should determine whether a site mechanism is in place that addresses EPHA development and maintenance.  The mechanism could be an emergency planning document or procedure.  Review the mechanism to determine whether it specifies:

I.    Inspectors should review the EPHA(s) to determine whether they were prepared in accordance with the mechanism.

 

J.   Inspectors should review the EPHA(s) to ensure an EPHA has been prepared for each facility containing hazardous materials that were not screened out.  Determine whether hazardous materials exceeding screening criteria are fully characterized (amount, location, condition of use, material properties, controls, etc.) to support development of scenarios and analysis of possible releases.

 

K.  Inspectors should review applicable EPHA(s), Design/Safety Basis Documents, and Vulnerability Analysis, if applicable, to determine whether the full spectrum of emergency events and conditions that could cause releases of hazardous material are analyzed.  For example, determine whether:

L.  Inspectors should review the EPHA(s) to determine whether barriers to release of hazardous materials are identified, together with the possible initiating events, accident mechanisms, and equipment failures.  In addition, determine whether indicators (alarms, instrument readings) of barrier failures for use in developing EALs are identified and whether areas where indications are lacking have been identified and action initiated (e.g., plant modification requests) to correct the condition.

 

M. Inspectors should review the EPHA(s) to determine whether facility and site boundaries have been correctly defined and whether meteorological assumptions are correct.

 

N.  Inspectors should perform an independent consequence assessment analysis on several scenarios to ensure that the EPHA(s) results are accurate and reproducible.

 

O.  Inspectors should review the EPHA(s) dispersion models for determining consequences to determine whether they are equivalent to emergency response models used by the ERO.

 

P.   Inspectors should review the EPHA(s) to determine whether correct protective action criteria are applied to potential release scenarios (e.g., Was the correct hierarchy of determining Acute Exposure Guideline Level (AEGL), Emergency Response Planning Guideline (ERPG), or Temporary Emergency Exposure Limit (TEEL) values used?  Was uranium considered as a toxicological hazard as well as a radiological hazard?).

 

Q.  Inspectors should collaborate with the Readiness Assurance inspector to confirm that the DOE/NNSA field office operations manager (designated representative) reviewed and approved the hazards survey and EPHA(s).  In addition, determine whether the DOE/NNSA field office operations manager reviewed, approved, and submitted the site EPZ to appropriate Headquarters staff.  Confirm that offsite jurisdictions concur with the EPZ determination.

R.  Inspectors should walk down the site’s emergency response centers to determine whether the consequences to onsite and offsite receptors of interest have been calculated and documented in readily available documentation.

 

S.   Inspectors should observe and evaluate the use and usability of EPHA(s) by consequence assessment personnel in the limited scope performance tests.

 

Common Deficiencies/Potential Concerns

 

This section identifies areas where concerns or deficiencies have been identified in previous inspections.  By reviewing this information before gathering data, inspectors can be aware of these deficiencies and concerns during interviews, walkdowns, and other data gathering activities.  However, the inspector should not simply focus his/her evaluation on whether these deficiencies exist at the site being evaluated, but rather should consider all aspects of this emergency management program element (including strengths and weaknesses).


 

Section 2B 

PLANS AND PROCEDURES

 

General Information

 

The emergency plan documents the emergency management program and describes both its administrative and response aspects.  The plan is executed through a series of implementing procedures, often called emergency plan implementing procedures, which contain the appropriate actions for both the Cognizant Field Office and contractor in response to an event.  At a minimum, the emergency plan will include the requirements of an Operational Emergency Base Program, which is based on an appropriate hazards survey and screening process and provides the framework for response to serious events as discussed in section 2A.  Biosafety facilities must have a designated Responsible Official and incident response plans that are integrated with the site’s emergency plan.  Sites with leased facilities must have documented arrangements that integrate the leased facility into the site program; including provisions for control and reporting of hazardous materials.

 

The administrative program established by the emergency plan and implementing procedures should include processes for the development, review, approval, and periodic update of plans and procedures; including supporting checklists and operator aids.  A document control system should ensure that procedures are available for use at all the response locations, and are the currently approved versions.  In addition, procedures and processes must be in place so that memoranda of understanding (MOUs) and agreement (MOAs) or mutual aid agreements with local, state, tribal, and Federal agencies are developed, approved and maintained

 

Relevant Site Documents

 

The inspector may choose to review any of the site documents listed in Section 1 before or during the performance of onsite data collection.  It is important to note that the following plans and procedures are typically evaluated by the specified functional area evaluator when assigned.

 

Inspection Element Evaluator
   
Hazards survey/EPHA process procedures Hazards survey/EPHA evaluator
Offsite Response Interface agreements EPI Evaluator
Training plans Training, Drill, and Exercise evaluator
EPI Plans and Procedures EPI evaluator
Corrective action/Issues management procedure Readiness Assurance evaluator

                                                                      

  

Data Collection Activities

 

The following data collection activities have been found to be beneficial in evaluating this emergency management program element.  The inspector should choose which of the activities to perform based upon the focus of the evaluation and the site-specific features of the emergency management program.  Further guidance is found in the inspection plan.

 

Document Reviews

 

A.  Inspectors should evaluate the emergency management Program Administration attributes by reviewing the emergency management plan(s) and implementing procedures to determine whether:

Note:  These program areas are generally inspected as separate, stand-alone areas and the inspector assigned to plans and procedures will not be directly involved in their inspection, but should verify that they are addressed in the emergency plan.

 

B.  Inspectors should evaluate if the emergency plan describes the ERO in sufficient detail to allow ERO personnel to understand and execute their integrated roles and responsibilities during an emergency, appropriately advise offsite authorities of the site’s emergency response, and enable responsible individuals to develop the emergency plan implementing procedures necessary for effective execution of the plan at all levels and locations.  To evaluate this review the emergency management plan and implementing procedures to determine whether:

C.  Inspectors should review the plan and procedures for initial actions and ERO activation to determine whether:

D.  Inspectors should review plans and procedures for ERO operations to verify that:

E.  Inspectors should review plans and procedures governing roles, responsibilities, and operations for some or all of the following: incident command post, emergency operations center, control center, technical support center, field monitoring teams, security, and fire and rescue.

 

F.    Inspectors should evaluate the categorization and classification functions by reviewing the emergency plan and associated implementing procedures to determine whether:

G.  Inspectors should evaluate the EALs and applicable classification implementing procedure(s), work instructions, and job aids to determine whether:

H.  Inspectors should evaluate the notifications and communications function by reviewing the emergency plan and implementing procedures to determine whether:

I.    Inspectors should evaluate the consequence assessment function by reviewing the emergency plan, implementing procedures, and other emergency management supporting documents to determine whether:

J.   Inspectors should evaluate the field monitoring component of consequence assessment by reviewing the emergency plan and implementing procedures to verify that the facility/site has processes in place to implement field monitoring and integrate the results into the consequence assessments, including:

K.  Inspectors should review quality assurance processes to verify that dispersion model software, including computational software and supporting databases, contain accurate information.

 

L.  Inspectors should evaluate the protective actions and reentry function by reviewing the sitewide emergency plan, building-specific emergency plans, and implementing procedures to ensure:

M. Inspectors should evaluate the emergency medical support function by reviewing the emergency plan and implementing procedures for emergency medical support to verify:

N.  Inspectors should evaluate the termination and recovery function by reviewing the emergency plan and implementing procedures to ensure:

Facility/Equipment Walkdowns

 

A.  Inspectors should perform a walkdown of site facilities to determine whether:

B.  Inspectors should walk down locations where consequence assessments and protective actions are formulated to determine whether:

C.  Inspectors should walk down the facilities where notifications are made to determine whether:

Interviews

 

A.  Inspectors should interview emergency management organization personnel responsible for maintaining the emergency plan and implementing procedures to evaluate:

B.    Inspectors should interview site personnel responsible for developing EALs to determine whether areas where the unescorted public are allowed can be evacuated within one hour of an emergency event and whether those areas are considered within the site boundary for emergency classification purposes.

 

C.  Inspectors should interview onsite emergency managers to determine whether they have a clear understanding of their responsibilities regarding determining and issuing protective actions for site personnel and protective action recommendations to state and local government organizations.  Determine emergency managers’ awareness of employees with any handicap, particularly deaf employees and those with mobility problems.  Interview the emergency response managers and initial decision makers responsible for classification to determine whether they are active participants in development, verification, or validation of EALs.  Inspectors should interview initial decision-makers to assess whether they can interpret initial assessment tools to refine pre-determined protective-action decision-making.

 

D.  Inspectors should interview emergency planners to determine whether:

E.  Inspectors should interview onsite personnel responsible for notifications and communications to determine whether the understanding of the individual(s) responsible for the notifications and communications is consistent with the plan and procedures.  (Interviews may reveal processes and procedures that have not been included in the emergency plan).

 

F.   Inspectors should interview the manager of the site’s emergency planning department to determine whether:

Common Deficiencies/Potential Concerns

 

This section identifies areas where concerns or deficiencies have been identified in previous inspections.  However, the inspector should not simply focus the evaluation on whether these deficiencies exist at the site being evaluated, but should consider all aspects of the emergency management program element (including strengths and weaknesses).  By reviewing this information before gathering data, inspectors can be aware of these deficiencies and concerns during interviews, walkdowns, and other data gathering activities.

 

ERO

EALs

Consequence Assessment

Protective Action and Reentry

 

Section 2C 

OFFSITE RESPONSE INTERFACES

 

General Information

 

Pre-planning is necessary to identify the interfaces for offsite organizations that are responsible for protecting the public or that will respond during an emergency.  This includes officials from local, state, tribal and Federal agencies who are responsible for assessing offsite protective action recommendations and implementing appropriate protective actions.  This element also includes the pre-planning necessary to obtain and provide mutual support during an event.  It includes arrangements with offsite agencies such as: local and state fire and law enforcement agencies, local hospitals and ambulance services, county or state public health officials, local and state emergency management officials, and Federal agencies (Federal Bureau of Investigation).  Plans should identify these interfaces and their expected role in an emergency, procedures should provide for their integration with the ERO, and agreements should be in place to ensure that roles and responsibilities are understood and accepted.

 
Interfaces with offsite response authorities and organizations are an integral part of the DOE/NNSA Comprehensive Emergency Management Program.  DOE Order 151.1C requires that effective interfaces be established and maintained to ensure that emergency response
activities are integrated and coordinated with the Federal, Tribal, State, and local agencies and organizations responsible for emergency response and protection of the workers, public, and environment and that they be in accordance with the requirements of the
National Response Plan (NRP) and National Incident Management System (NIMS).
 

This section provides guidance for inspectors reviewing this aspect of the emergency management program.  Specific areas of significance include the provision of:

The inspector assigned this area should coordinate and collaborate with the inspector responsible for Plans and Procedures and Emergency Public Information to effectively and efficiently execute the inspection plan.

 

Relevant Site Documents

 

In addition to the site documents listed in Section 1, the inspector may choose to review following site documents before or during the performance of onsite data collection.