oversight

Nuclear Waste: Preliminary Observations on the Quality Assurance Program at the Yucca Mountain Repository

Published by the Government Accountability Office on 2003-05-28.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                            United States General Accounting Office

GAO                         Testimony
                            Before the Subcommittee on Energy and
                            Water Development, Committee on
                            Appropriations, U.S. Senate

For Release on Delivery
Expected at 4:00 p.m. EDT
Wednesday, May 28, 2003     NUCLEAR WASTE
                            Preliminary Observations
                            on the Quality Assurance
                            Program at the Yucca
                            Mountain Repository
                            Statement of Robin M. Nazzaro, Director
                            Natural Resources and Environment




GAO-03-826T
                                                May 28, 2003


                                                NUCLEAR WASTE

                                                Preliminary Observations on the Quality
Highlights of GAO-03-826T, a testimony          Assurance Program at the Yucca
before the Subcommittee on Energy and
Water Development, Committee on                 Mountain Repository
Appropriations, U.S. Senate




A quality assurance program is                  DOE’s track record of correcting problems with its quality assurance
required by the Nuclear Regulatory              program is less than favorable. Recurring problems have persisted in the
Commission (NRC) to ensure that                 program despite DOE’s numerous attempts to correct them. DOE
the Department of Energy (DOE)
can safely construct and operate a
                                                evaluations and NRC oversight activities have concluded that the
high-level radioactive waste                    program still falls short of expectations.
repository. DOE is currently
preparing an application to NRC                 DOE’s 2002 quality assurance improvement plan represents the
for authorization to construct the              department’s most recent attempt to correct quality assurance problems,
repository. The quality assurance               including those involving the scientific models and software codes in the
program includes procedures to                  computer simulation that DOE will use to demonstrate the safety of the
assure NRC that the information
                                                repository. Because DOE is still in the process of implementing this plan,
DOE provides is verifiable and well
documented. DOE will use the                    it is too early to determine whether changes included in the plan will be
results of a computer simulation to             effective. However, notwithstanding these changes, DOE has recently
demonstrate that the repository                 identified further quality assurance problems, including recurring
can be safely operated over the                 problems with the data that will be used to support the NRC’s decision
10,000-year period required by the              on whether to authorize DOE to construct the repository.
Environmental Protection Agency’s
health and safety standards. Some               Based on previously identified weaknesses and recent indications of new
of the key elements of this
simulation are shown in the                     problems, we are concerned that DOE’s current efforts to improve its
illustration.                                   quality assurance program may not yield the results it hopes for. Our
                                                observation is further supported by NRC’s recent comment that DOE’s
This testimony is based on ongoing              quality assurance program has yet to produce outcomes necessary to
and published GAO work. The                     ensure that this program meets NRC requirements.
testimony provides the history of
DOE’s actions to correct quality                Illustration of Yucca Mountain Repository Waste Package Emplacement
assurance problems, the status of
DOE’s efforts to improve the
quality assurance program, and
preliminary observations on the
effect of quality assurance
problems on DOE’s ability to
successfully meet its 2004
milestone for submitting an
application to NRC requesting
authorization to construct the
repository.




www.gao.gov/cgi-bin/getrpt?GAO-03-826T.

To view the product, click on the link above.
For more information, contact Robin Nazzaro     Source: DOE.
at (202) 512-3841 or nazzaror@gao.gov.
    Senators Ensign and Reid:

    We are pleased to be here today to discuss the Department of Energy’s
    (DOE) quality assurance program for the Yucca Mountain repository
    project. As you know, Yucca Mountain is intended to serve as the nation’s
    permanent repository for high-level nuclear waste. DOE is currently in the
    process of preparing an application to the Nuclear Regulatory Commission
    (NRC) for authorization to construct the repository, which it expects to
    submit by December 2004. To ensure that DOE can safely construct and
    operate the repository, NRC requires DOE to have a quality assurance
    program. The quality assurance program is designed to include procedures
    to assure NRC that the information submitted to it is verifiable and well
    documented. Audits and management reviews are also built into the
    program to monitor whether workers follow these procedures. In cases
    where they are not followed, DOE must develop and implement corrective
    actions and monitor their effectiveness. An ineffective quality assurance
    program could potentially impede the application process and could
    precipitate potentially adverse health, safety, and environmental effects.

    In this context, you asked us to investigate the effectiveness of DOE’s
    efforts to improve its quality assurance program. Although we are still in
    the early stages of our investigation, we are prepared today to provide
    (1) the history of DOE’s actions to correct quality assurance problems,
    (2) the status of DOE’s efforts to improve the quality assurance program,
    and (3) preliminary observations on the effect of quality assurance
    problems on DOE’s ability to successfully meet its 2004 milestone for
    submitting an application to NRC requesting authorization to construct the
    repository.

    In summary:

•   DOE’s track record of correcting problems with its quality assurance
    program is less than favorable. Recurring problems have persisted in the
    program despite DOE’s numerous attempts to correct them. DOE
    evaluations and NRC oversight activities have concluded that the program
    still falls short of expectations.

•   DOE’s 2002 quality assurance improvement plan represents the
    department’s most recent attempt to correct quality assurance problems,
    including those involving scientific models and software codes that DOE
    will use to demonstrate the safety of the repository. Because DOE is still in
    the process of implementing this plan, it is too early to determine whether
    changes included in the plan will be effective. However, notwithstanding


    Page 1                                                          GAO-03-826T
                 these changes, DOE has recently identified further quality assurance
                 problems, including recurring problems with the data that will be used to
                 support the NRC’s decision on whether to authorize DOE to construct the
                 repository.

             •   Based on previously identified weaknesses and recent indications of new
                 problems, we are concerned that DOE’s current efforts to improve its
                 quality assurance program may not yield the results it hopes for. Our
                 observation is further supported by NRC’s recent comment that DOE’s
                 quality assurance program has yet to produce outcomes necessary to
                 ensure that this program meets NRC requirements.


                 The Nuclear Waste Policy Act of 1982 was enacted to establish a
Background       comprehensive policy and program for the safe, permanent disposal of
                 commercial spent fuel and other high-level radioactive wastes. DOE was
                 directed in the act to, among other things, investigate potential sites for
                 locating a repository. Amendments to the Act in 1987 directed DOE to
                 consider only Yucca Mountain, Nevada, as a potential site for a repository.
                 In 2002, the Congress approved the President’s recommendation of Yucca
                 Mountain as a suitable site for the development of a permanent high-level
                 waste repository. The next step in the process is for DOE to submit an
                 application to NRC for an authorization to construct the repository.

                 In order to ensure that the information submitted to NRC is verifiable and
                 well documented, NRC requires nuclear facilities to develop a quality
                 assurance program that includes a process to identify problems, develop
                 corrective actions, and monitor the effectiveness of these actions. Among
                 other things, such a quality assurance program is required to (1) train
                 personnel in quality assurance; (2) inspect activities that affect quality;
                 (3) establish controls over testing programs and test equipment, such as
                 ensuring that this equipment is properly calibrated; (4) establish and
                 maintain records, including records documenting the qualifications of
                 personnel performing repository work; and (5) verify compliance with the
                 rules and procedures of the quality assurance program to determine the
                 effectiveness of the program.

                 In carrying out its responsibility for the Yucca Mountain repository to
                 meet the Environmental Protection Agency’s (EPA) standards for
                 protecting public health and safety, as well as its standards, NRC provides
                 consultation and advice to DOE in the project’s pre-application period.
                 NRC officials are located onsite at the Yucca Mountain project office
                 where they conduct daily oversight of project activities, including


                 Page 2                                                          GAO-03-826T
                     observing and commenting on DOE’s quality assurance audits and
                     preparing bi-monthly reports on the overall status of the program.
                     Additionally, DOE and NRC hold quarterly quality assurance meetings and
                     conduct exchanges between staff on technical issues.


                     DOE’s quality assurance problems at the Yucca Mountain repository site
History of Actions   date back to the late 1980s. In a 1988 report, we identified significant
Taken to Correct     problems with the quality assurance program, noting that it failed to meet
                     NRC standards.1 We found that NRC had identified many specific concerns
Quality Assurance    from the oversight activities it had performed at Yucca Mountain. For
Problems             example, NRC noted that DOE’s heavy reliance on contractors and its
                     inadequate oversight of quality assurance activities would increase the
                     likelihood that DOE might encounter quality-related problems.
                     Furthermore, NRC said that the likelihood that the state of Nevada and
                     others would contest the licensing proceedings increased the probability
                     that DOE would have to defend its quality assurance program and the
                     quality of the work performed. NRC noted that DOE’s inability to properly
                     defend its work could result in additional expense and time-consuming
                     delays as program weaknesses are corrected. NRC also found that DOE
                     staff and contractors exhibited negative attitudes toward the function of
                     quality assurance, noting that participants appeared to lack a full
                     appreciation for what it took to get a facility licensed by NRC.

                     DOE was put on notice of these shortcomings, but the problems
                     continued. In its 1989 evaluation of DOE’s Site Characterization Plan, NRC
                     concluded that DOE and its key contractors had yet to develop and
                     implement an acceptable quality assurance program. In March 1992, based
                     on progress DOE had made in improving its quality assurance program,
                     NRC allowed DOE to proceed with its site characterization work, noting
                     that DOE had demonstrated its ability to evaluate and correct quality
                     assurance program deficiencies. A year and a half later, however, NRC
                     raised concerns with DOE about the acceptability of facility design
                     activities requiring quality assurance. NRC reported that it had no
                     confidence that DOE’s management plan for resolving quality assurance
                     issues related to the design activities would work because of DOE’s and




                     1
                      U.S. General Accounting Office, Nuclear Waste: Repository Work Should Not Proceed
                     Until Quality Assurance Is Adequate, GAO/RCED-88-159 (Washington, D.C.: Sept. 29,
                     1988).



                     Page 3                                                                   GAO-03-826T
                            the site contractors’ inability to effectively implement corrective actions in
                            the past.

                            DOE renewed its efforts to correct problems with its quality assurance
                            program starting in the late 1990s when its own audits at Yucca Mountain
                            identified quality assurance problems in three areas: data sources,
                            validation of scientific models, and software development. First, DOE
                            could not ensure that all the data needed to support the scientific models
                            could be tracked back to original sources or that the data had been
                            properly collected. Second, DOE had no standardized process to develop
                            the scientific models needed to simulate geological events. Finally, DOE
                            had no process for ensuring that the software being developed to support
                            the models would work. In response to the issues raised in the audits,
                            DOE issued a management plan in 1999 that prescribed remedies.
                            Following implementation of this plan, DOE considered the issues
                            resolved.

                            Model validation and software development problems, however,
                            resurfaced in 2001. New quality assurance audits found that project
                            personnel had not followed the required procedures for model
                            development and validation or established a timeline for completing the
                            models. In addition, these audits identified that project personnel had not
                            followed the software development process, prompting a prohibition on
                            further software development without prior management approval.
                            According to DOE, the significance of these new observations was
                            compounded by their similarity to those problems previously identified.


                            In July 2002, DOE provided NRC with a revised plan to correct its quality
Status of DOE Efforts       assurance problems at Yucca Mountain, including the problems with
to Improve Quality          scientific models and software codes. In constructing the plan, DOE
                            conducted an in-depth study of Yucca Mountain’s management and work
Assurance                   environment. The plan outlined five key areas needing improvement.
                            Specifically, it noted the need for

                        •   clarifying roles, responsibilities, accountability, and authority for DOE and
                            contractor personnel,

                        •   improving quality assurance processes and clarifying line management’s
                            quality responsibilities,

                        •   improving DOE and contractor written procedures,



                            Page 4                                                            GAO-03-826T
               •   implementing more effective and consistent corrective action plans to
                   preclude recurring quality problems, and

               •   improving the work environment where employees can raise program
                   concerns without fear of reprisal.

                   To fully address issues raised in the plan, DOE identified a total of 72
                   actions needed to correct the quality assurance program—35 to address
                   the five key areas, 12 to address model development issues, and 25 to
                   address software development issues. DOE recently reported that it had
                   completed 41 of the 72 actions. The management plan also included
                   performance measures to assess the effectiveness of the actions. DOE
                   recently reported, however, that the Yucca Mountain project still lacks
                   complete and useful performance measures and stated its intention to
                   have the appropriate performance measures in place by September 2003.

                   Since DOE began to implement its latest improvement plan, new quality
                   issues have emerged. In March 2003, DOE issued a “stop-work” order
                   preventing further use of a procedure intended to help improve DOE and
                   contractor quality assurance procedures. According to DOE, they
                   cancelled the use of the procedure and reverted back to the existing
                   procedure. In April 2003, DOE again found data-related problems similar
                   to the data verification problems identified in 1998. For example, DOE
                   found that, instead of verifying data back to appropriate sources, project
                   scientists had been directed to reclassify the unverified data as
                   “assumptions” which do not require verification.

                   At the April 2003 quality assurance meeting with NRC, DOE highlighted
                   several recent improvements to the quality assurance program. These
                   improvements included (1) management changes with DOE’s primary
                   contractor at the site, including a new president and a new director of
                   quality assurance, (2) increased line management involvement in quality
                   assurance, and (3) the integration of quality engineers with DOE line
                   employees. Despite this reported progress, an NRC official at the same
                   meeting commented that the quality assurance program had still not
                   produced the outcomes necessary to ensure the program is compliant with
                   NRC requirements.


                   Whether DOE can correct its quality assurance problems in time to meet
Preliminary        its milestone for submitting an application that is acceptable to NRC is not
Observations       clear. DOE’s unsuccessful efforts to address recurring quality assurance
                   problems, the identification of new problems since the issuance of its 2002


                   Page 5                                                          GAO-03-826T
           improvement plan, and NRC’s recent comment that DOE’s quality
           assurance program has yet to produce outcomes necessary to ensure that
           this program meets NRC requirements do not instill much confidence that
           the quality assurance problems will soon be resolved. An ineffective
           quality assurance program could impede the application process, leading
           to time-consuming and expensive delays as weaknesses are corrected, or
           ultimately prevent DOE from receiving authorization to construct a
           repository. Moreover, continued reliance on data that are unverifiable and
           thus could be inaccurate could lead to adverse effects in the course of the
           10,000-year period required by EPA’s health and safety standards. At the
           same time, now that the project has shifted from scientific investigation to
           preparing an application, DOE may now have the proper motivation and
           focus to correct recurring quality assurance problems given the integral
           role that quality assurance plays in the application process.


           As we continue our investigation, we will work to validate our
           observations and further assess the effectiveness of DOE’s efforts to
           improve its quality assurance program.

           Thank you, Senators Reid and Ensign. That concludes my testimony. I
           would be pleased to respond to any questions that you may have.



           Contacts and Acknowledgments
           For further information on this testimony, please contact Ms. Robin
           Nazzaro at (202) 512-3841. Individuals making key contributions to this
           testimony included Lee Carroll, Daniel Feehan, Thomas Kingham, Thomas
           Laetz, Chalane Lechuga, and Jonathan McMurray.




(360350)
           Page 6                                                          GAO-03-826T
                         The General Accounting Office, the audit, evaluation and investigative arm of
GAO’s Mission            Congress, exists to support Congress in meeting its constitutional responsibilities
                         and to help improve the performance and accountability of the federal
                         government for the American people. GAO examines the use of public funds;
                         evaluates federal programs and policies; and provides analyses,
                         recommendations, and other assistance to help Congress make informed
                         oversight, policy, and funding decisions. GAO’s commitment to good government
                         is reflected in its core values of accountability, integrity, and reliability.


                         The fastest and easiest way to obtain copies of GAO documents at no cost is
Obtaining Copies of      through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full-
GAO Reports and          text files of current reports and testimony and an expanding archive of older
                         products. The Web site features a search engine to help you locate documents
Testimony                using key words and phrases. You can print these documents in their entirety,
                         including charts and other graphics.
                         Each day, GAO issues a list of newly released reports, testimony, and
                         correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site
                         daily. The list contains links to the full-text document files. To have GAO e-mail
                         this list to you every afternoon, go to www.gao.gov and select “Subscribe to daily
                         E-mail alert for newly released products” under the GAO Reports heading.


Order by Mail or Phone   The first copy of each printed report is free. Additional copies are $2 each. A
                         check or money order should be made out to the Superintendent of Documents.
                         GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a
                         single address are discounted 25 percent. Orders should be sent to:
                         U.S. General Accounting Office
                         441 G Street NW, Room LM
                         Washington, D.C. 20548
                         To order by Phone:     Voice:    (202) 512-6000
                                                TDD:      (202) 512-2537
                                                Fax:      (202) 512-6061


                         Contact:
To Report Fraud,
                         Web site: www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in      E-mail: fraudnet@gao.gov
Federal Programs         Automated answering system: (800) 424-5454 or (202) 512-7470


                         Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800
Public Affairs           U.S. General Accounting Office, 441 G Street NW, Room 7149
                         Washington, D.C. 20548