IJnit,cvl States General Accounting Office RepGt to the Chairman, Comilittee on Interior and Insular Affairs, House of Representatives SURFACE MINING Interior Is Acting to Improve Its Applicant Violator System 141840 united states GAO General Accounting Office Washington, D.C. 20548 Accounting and Financial Management Division B-236866 June 22,199O The Honorable Morris K. Udall Chairman, Committee on Interior and Insular Affairs House of Representatives Dear Mr. Chairman: Your June 27,1989, letter requested that we evaluate what the Office of Surface Mining Reclamation and Enforcement (OGMRE)has done in response to our report entitled Surface Mining: Operation of the Appli- cant Violator System Can Be Improved (GAO/AFMIHXM, January 24, 1989). This letter provides a summary of actions taken by OSMREsince the completion of our last review and also provides information on the preliminary Department of the Interior-National Wildlife Federation agreement for improving the permit review process. OSMREinitiated a number of actions to improve the permit review pro Results in Brief cess, some of which have not yet been completed. While we see progress, the Applicant Violator System (AM) still is not reliable for determining whether applicants should be issued coal mining permits. Although the system’s accuracy rate increased from 64 percent at the time of our last review to 63 percent for the first 10 months of 1989, the present accu- racy rate still necessitates manual verification. To address the data accuracy problem, OSMREimplemented a redesigned system on May 1, 1990. While we did not assess the system, we were told that preliminary results of the first 2 weeks of its operations showed a 77-percent accu- racy rate. Subsequent to completion of our review work, a preliminary legal agree- ment was reached between the Department of the Interior and the National Wildlife Federation. (See appendix I.) The agreement called for various improvements to the permit review process, It has not yet been approved by the cognizant court. The Congress enacted the Surface Mining Control and Reclamation Act Background of 1977 to protect society and the environment from the adverse effects of surface coal mining operations. Section 610, a key provision of the act, requires denial of mining permits to applicants who have out- standing violations of mining regulations unless the applicant submits Page1 GAO/AFMD-9083SurfaceMining proof, to the satisfaction of the regulatory authority, that the violation has been or is in the process of being corrected. Interior has been criticized in GAOand congressional committee reports for not fulfilling this legislative mandate. A list of our prior reports is on the last page of this report. In addition, a 1986 court order provided for more effective implementation of the congressional mandate. In response, OSMREdeveloped the Applicant Violator System which is designed to link current applicants to outstanding violations, thus forming a basis for recommending permit issuance or denial. In a January 1989 report, we discussed fundamental problems we had identified in the system’s operations. The report said that subsequent manual verifications by OSMREhad reversed 46 percent of the AVS- generated recommendations. The high reversal rate was due primarily to inadequate and outdated information in the system’s data base. Our objectives were to (1) identify and assess actions taken to improve Objectives, Scope,and AVSoperations since completion of our previous review work in July Methodology 1988, (2) evaluate the system’s current ability to generate reliable permit issue and deny recommendations, and (3) provide a description of the legal agreement between Interior and National Wildlife on improving the permit review process. Cur work was conducted in the Washington, D.C., area. We interviewed OSMREofficials to identify actions taken to improve AVSoperations and reviewed the records supporting those actions. We reviewed the OSMRE Clearinghouse operations as well as the guidance it provided to the states to clarify their role in the permit process. We also reviewed oper- ating procedures, applications, violation records, and rules governing the issuance of permits. We reviewed agency statistics regarding the agreement rates between Avs-generated and manually verified recommendations, and an informal OGMREreport on the reasons for invalid AVSrecommendations. To deter- mine the system’s reliability, we also reviewed a selected judgmental sample of 30 of a total of 626 permit applications processed by the system in June and July 1989. Twenty of these were randomly selected. In order to more fully research the reasons for differing recommenda- tions, the remaining 10 were specifically selected because the AVSand Clearinghouse recommendations disagreed. We discussed the 30 cases Page2 GAO/AFMD-90-63 Surface Mining B-236866 with OGMREofficials and reviewed the documentation used to support its final issue or denial recommendations. In our earlier report, we recommended that OSMREcompare the AVSand National Wildlife permit review systems to identify improvement oppor- tunities, OSMREsuspended its comparison efforts pending settlement of a lawsuit filed by National Wildlife in April 1989 concerning Interior’s enforcement of the act and the operation of AVS.After a preliminary agreement was reached between Interior and National Wildlife on Jan- uary 24, 1990, OGMREadvised us it plans to study the National Wildlife system. Accordingly, we did not review this issue. As of May 21, 1990, the agreement had not been approved by the U.S. District Court for the District of Columbia, which is responsible for the case. Details are in appendix I. We also did not evaluate a recently implemented redesign of AVS,because system operations began after our review was completed. We discussed a draft of this report with OSMREofficials to confirm its accuracy and completeness and have incorporated any appropriate com- ments. We performed our review in accordance with generally accepted government auditing standards. OSMREhas taken several actions to improve its permit review process. Actions Taken to The major accomplishment was upgrading the data base primarily by Improve Application adding the Department of Labor’s Mine Safety and Health Administra- Processing tion mine identification numbers and, as recommended in our prior report, the Administration’s mine ownership and control data. OSMRE also solicited updated ownership and control information from the com- panies in its data base. Other actions include eliminating a backlog of unverified Avs-generated recommendations through productivity initiatives and providing more timely recommendations to states. In response to our prior report recom- mendation, rules defining ownership and control and requiring updating of the information on applications prior to permit issuance have been issued. In addition, OSMREimplemented a redesign of AVSon May 1, 1990. Details on each of these actions are in appendix II. Page2 B-236856 While OSMREhas acted to strengthen AVS,the system is still not pro- AVSDoes Not Yet ducing reliable permit issuance or denial recommendations. Therefore, Provide Reliable OGMREcontinues to manually verify the Ass-generated recommendations. Recommendationsfor During the first 10 months of 1989, the manual verifications agreed Issuing Permits with about 63 percent of the ASS-generated recommendations. This was a g-percent improvement over the agreement rate GAOfound during the first 6 months of AVSoperations in 1987 and 1988. To address the data accuracy problem, OSMREimplemented a redesigned system on May 1, 1990. While we did not assess the system, we were told that preliminary results of the first 2 weeks of its operations showed a 77-percent accu- racy rate. In addition to continuing data quality problems, we found that a major factor affecting the reliability of AVSrecommendations is the large number of incorrect links, i.e., the inability to correctly identify current applicants with outstanding violations. Although OGMREhas not deter- mined what is causing the problem, it stems at least in part from a potential weakness in the AVScomputer program. Appendix III provides a detailed discussion of our analysis of the relia- bility of AVSrecommendations. OSMREhas acted to improve AVSoperations. Nevertheless, data quality Conclusions problems, combined with a potential programming problem, continue to undermine its initiatives. While the accuracy of Avs-generated recom- mendations was 9 percent better than during the first 5 months of oper- ation, OSMRE'Smanual verifications reversed 37 percent of the AVS recommendations between January and October 1989. OSMREimplemented a redesigned AVSon May 1, 1990. Because system operations were initiated after our review work was completed, we were unable to assess its performance. However, a contractor study com- pleted during November 1989 endorsed the overall redesign concept and recommended several actions OSMREshould take to improve its develop- ment and implementation process. While all of OSMRE'Simprovement initiatives have merit, the only real measure of success is whether or not they substantially improve the accuracy of the Avs-generated recommendations. In the final analysis, making AVSa dependable performer will be contingent on successfully Page4 GAO/AFMD-90-63 SurfaceMining B-238856 upgrading data quality and ensuring that it can accurately link appli- cants and permittees to outstanding violations. OSMREbelieves that the revisions made to the system during its recent redesign project will improve its effectiveness and efficiency. Since (1) following the completion of our review, OSMREimplemented a redesigned AVSand (2) the legal agreement between National Wildlife and Interior has not yet been approved by the Court, we are not making any recommendations at this time. As agreed with your office, unless you publicly announce the contents of this report earlier, we will not distribute it until 30 days from its date. At that time, we will send copies of the report to the Secretary of the Interior, the Department of the Interior’s Director of the Office of Sur- face Mining Reclamation and Enforcement, and other interested parties. We will also make copies available to others upon request. Please contact me on 2759464 if you or your staff have any questions. Major contributors to this report are listed in appendix IV. Sincerely yours, Jeffrey C. Steinhoff Director, Financial Management Systems and Audit Oversight Page6 GAO/~9083 SurfaceMining Chdents 1 Appendix I 8 Preliminary Interior- National Wildlife Agreement on AVS Appendix II OSMREActions to Improve Application Processing Appendix III Analysis of the Reliability of AVS Appendix IV 17 Major Contributors to This Report Tables Table II. 1: Company Profile Project-Status as of 11 December 3 1,1989 Table III. 1: AVS and Clearinghouse Recommendation Agreement Rates, lo-Month Period Ending October 31,1989 Related GAO Products Abbreviations AVS Applicant Violator System GAO General Accounting Office OSMRE Office of Surface Mining Reclamation and Enforcement Page6 GAO/~9083 SurfaceMining Page7 GAO/AFMD-BM3SurfaceMining . Appendix I l?relmunary Interior-National Wildlife l Agreementon AVS On January 24,1990, a preliminary agreement was reached between Interior and National Wildlife. In exchange for National Wildlife drop- ping its legal action against Interior over enforcement of the surface mining law, Interior agreed to major changes to AVS. As of May 21,1990, the agreement had not been approved by the U.S. District Court for the District of Columbia. The preliminary agreement calls for OSMREto l complete, correct, and refine the AVS data base of potential ownership and control links between permit applicants and violators; l interface with data bases maintained by the Mine Safety and Health Administration, the Energy Information Administration, and the states; l augment and verify AVS data by reviewing federal inspection and legal files, company records, and audit information and by conducting field investigations; l formalize agreements with state surface mine regulators on screening permit applicants and verifying AVS information; . work with the Tennessee Valley Authority to block purchase of coal from violators of the surface mining law; l review permits already in force, and suspend or revoke any that should not have been issued because of uncorrected violations or ownership and control links with violators; l propose formal sanctions against people who intentionally give incom- plete or incorrect ownership and control information on coal mine permit applications; and l develop standards, through rule-making if necessary, for determining how AVS ownership and control links may be judged erroneous and for updating ownership and control information, Page8 GAO/AFMD-90-63 SurfaceMining . Ap&ndix II OSMREActions to improve Application Processing In our January 1989 report, we stated that OSMREhad not incorporated The System’s Data all of the relevant sources of mine ownership and control information Base Has Been into AVS. Instead, Mine Safety and Health Administration data was Upgraded obtained by the Clearinghouse during its manual verification process. Thus, we recommended that data sources, including the Administration, used during the manual verification process should be incorporated into AVS to improve the quality of the data available for the system-generated recommendations. OSMREhas acted to upgrade AVS, including entering Administration infor- mation into the AVS data base. The information added includes more Administration identification numbers that the Administration assigns to each mining activity and the related information regarding who owned or controlled the mine activities on specific dates. It also requested the 22,000 applicants and permittees in its data base to volun- tarily provide current data for updating the AVS data. Administration Numbers A March 1988 review of the AVS data base by OSMREdisclosed that only 5,000 of about 18,600 valid permits, or 27 percent, had Administration identification numbers. Because these numbers facilitate matching applicants to violators, OSMREtried to obtain additional numbers which would be incorporated in the system’s data base. At the conclusion of that effort, 16,228, or 88 percent, of the permits in the AVS data base had Administration numbers. Mine Ownership and Administration mine ownership and control information is now being Control Data entered in AVS, thereby improving the system’s ability to link applicants to violators. However, its usefulness is somewhat diminished because OSMREhas been slow in entering updated quarterly data. During our previous review, we found that the Administration owner- ship and control data was obtained and used by OSMREduring its manual verification process. As recommended in our prior report, OSMREhas incorporated Administration data in the AVS data base. This data, which the Labor Department requires to be updated quarterly, includes the dates that each owner, operator, and controller relationship existed. Such information enables AVS to determine who owned or controlled the violator at the time of the violation and thereby permits links of appli- cants to violators. At the conclusion of our prior review, ~SMRE informed us that it planned to enter the ownership and control data into AVS soon. In April 1989, OSMREentered the data as of October 1988 into AVS. While Page9 GAO/V Surf’aceMining Appendix II OSMRE Actions to Improve Application Processing OSMREhas continued to enter the quarterly updates it receives from the Administration, it has not done so promptly. For example, the Sep- tember 1989 quarterly data was not entered until January 6,199O. Company Profile Project In another effort to improve the quality of AVS data, OSMREinitiated a onetime “Company Profile” project in May 1988 to update the informa- tion it had on the 22,000 applicants and permittees in its data base. However, as of December 1989, only 3,181 of 8,462 responses received have been reviewed for updating the AVS data base. The 22,000 applicants and permittees were provided copies of the cur- rent ownership and control information in the AVS data base. The recipi- ents were requested to voluntarily update and correct the information and return it with supporting documentation by June 1,1988. As of December 31, 1989, OSMREhad received 8,452 responses. OGMREforwarded the 8,462 responses as they were received to the state regulatory authorities and instructed them to review and enter the changes in AVS. OGMREofficials advised us that, as of December 31, 1989, the states had reviewed 3,181, or 38 percent, of the 8,452 responses to update the AVS data base. However, as table II.1 shows, the degree of participation varies significantly. Some states processed all the responses, while others have processed few, if any. For example, while 4,253 responses were received from companies operating in Kentucky and Pennsylvania, only 10 have been used by the two states to update the AVS data base. OSMREofficials advised us that Kentucky updated AVS using its own ownership and control data base rather than the company- provided data. It attributed the slow progress in Pennsylvania to limited staffing but noted that a data entry staff member had been subse- quently added by the state to its AVS support staff. Page 10 GAO/AFMD-9083 Surface Mining Appendix II CBMRE Actlona to Improve Application --ml Table 11.1:Company Profile Project- Status as of Dqcember 31,1969 State or federal Responses Responses Percent field office received processed processed Alabama 190 170 89.5 Arizona 11 10 90.9 Colorado 30 20 66.7 Iowa 1 1 100.0 Illinois 146 127 87.0 Indiana 121 82 67.8 Kansas 6 5 83.3 Kentucky 2,728 0 0.0 Louisiana 1 0 0.0 Missouri 68 63 92.7 Maryland 84 77 91.7 Montana 35 0 0.0 North Dakota 34 34 100.0 New Mexico 7 0 0.0 Ohio 587 540 92.0 Oklahoma 75 66 88.0 OSMRE Western Field Office 55 21 38.2 Pennsylvania 1,525 10 0.7 Tennessee and OSMRE Eastern Field Office 208 4 1.9 Texas 3 3 100.0 Utah 20 20 100.0 Virginia 612 204 33.3 West Virginia 1,851 1,695 91.6 Wvomina 54 29 53.7 Total 8,452 3,181 37.6 In our prior report, we noted that many of the manually verified recom- Clearinghouse Backlog mendations had not been communicated to states in time to meet their Eliminated and planned permit issuance date. The Clearinghouse, a unit within OSMRE'S Verifications Division of Debt Management, is responsible for performing the manual verifications. Expedited Our current work showed that the manual verification process has been streamlined. In October 1988, the Clearinghouse revised its manual ver- ification process and was able to eliminate a backlog of 1400 unverified Avs-generated recommendations by December 1988. OSMREofficials told Page 11 GAO/Al?MD90-63 Surf’ace Mlnlng Appendix II OSMRE Actions to Improve Application Processing us that it would have taken 9 months to eliminate the verification backlog if OSMREhad not revised its procedures. While manual verifications of all Avs-generated recommendations are still being performed, the current process makes more efficient use of existing data, thus reducing redundant research. Since our January 1989 report, the Clearinghouse has changed its verification process by reviewing applications from the same or related companies as one sub- mission rather than as individual submissions. In addition, the Clearing- house began to use the information it had accumulated on those applicants who had submitted previous applications rather than to com- pletely redo all research from scratch. Reducing its redundant manual research has enabled the Clearinghouse to provide more prompt recommendations to the states. During our review, the Clearinghouse recommendations were being provided to the states within an average of 1 week after the Avs-generated recommenda- tion. The revised procedures have also enabled the Clearinghouse to phase out 16 contractor personnel. Status of Operating era1 critical data requirements or the roles and responsibilities of the Rules and Procedures system operators and users. In the report, we recommended that OSMRE expedite issuance of the Clearinghouse procedures and a rule for updating applicant information prior to permit issuance. We also called for monitoring state adherence to the recently issued ownership and control rule and the information update rule when it was published. OSMREhas published three new permit issuance rules, notified the states and federal field offices of the new rules, and requested and received the proposed amendments to their OSMRE-approvedprograms. No date has been established for issuance of the Clearinghouse procedures. The current draft procedures will have to be modified due to the recent rede- sign of AVS and the Interior-National Wildlife preliminary agreement. Redesignof AVS system was completed in September 1989, and testing began in January 1990. Major features of the system include: Y . States will have on-line system access, thereby allowing them to make permit issuance or denial decisions. Page 12 GAO/AFMD-3043 Surface Mining Appendix II OSMRE Actions to Improve Application Processing l The two basic files within the system’s data base will be an entity direc- tory and a violation file with related entities. . Changes in the automated operations will reduce the system processing time needed to establish business relationships between mining entities before matching to the violator file. . Quarterly updated Administration data will be accessible by users but will no longer be used to link applicants and violators. OSMREofficials believe that the revised system will permit quicker user access and reduce computer processing costs and time. Also, they believe that the redesigned system will eliminate the need for the Clearinghouse to manually verify the system-generated recommendations. Because OSMREhad retained a consultant to assess the redesign effort and system operations had not started prior to the completion of our work, we did not attempt to evaluate the redesign. Contractor Study of the OSMREawarded a contract in September 1989 to Data Computer Corpora- RedesignedAVS tion of America to perform an in-depth analysis of AVS and the National Wildlife permit review system. The analysis of the National Wildlife system was subsequently suspended until Interior-National Wildlife negotiations to resolve the issues in a lawsuit filed by National Wildlife in April 1989 were completed. National Wildlife and Interior reached a preliminary agreement on the lawsuit in late January 1990. OSMRE advised us that it plans to initiate a study of the National Wildlife system in June 1990. (See appendix I.) tract to Computech to assess the AVS Data Computer Corporation of America subsequently awarded a con- redesign. Computech’s report, which was issued on November 6, 1989, said that, although it had found some problems, the redesign effort presented a logical, cohesive concept for improving AVS and concluded that OSMREshould proceed. Page 13 GAO/AF’MD-fW63 SurPace Mining Appendix III Analysis of the Reliability of AVS In our January 1989 report, we stated that inaccurate and incomplete information prevented AVS from producing reliable permit issue and deny recommendations. At that time, we reported that, from the time of its implementation in October 1987 through March 3, 1988, about 46 percent of the AVS recommendations were reversed after they were man- ually verified by the Clearinghouse. To assess the current reliability of AVS, we compared the agreement rate of the automated system output and the manual verifications for the lo- month period ending October 31, 1989. As shown in table 111.1,the agreement rate, which fluctuated from a high of 88 percent to a low of 50 percent, averaged about 63 percent. Table 111.1:AVS and Clearinghouse Recommendation Agreement Rates, lo- Percentage of Month Period Ending October 31,1989 recommendations Month that agreed January 88.4 February 73.3 March 73.5 April and May (OSMRE combined data) 50.2 June 64.8 Julv 62.9 Auaust 65.1 September 69.6 October 62.8 Percentaae for lo-month period 63.4 Preliminary results of the first 2 weeks of operation of the redesigned AVS showed a 77 percent accuracy rate. As part of its efforts to improve AVS operations, OSMREreviews the Reasonsfor monthly AVS and Clearinghouse agreement rate to identify operational Differences trends and results. Based on these reviews, OGMREhas identified several overall reasons for the continuing differences between Avs-generated recommendations and the Clearinghouse manual verifications. The dif- ferences are primarily the result of renewal application submissions which contain outdated information; information obtained by the Clearinghouse after the applications have been entered into AVS; and Clearinghouse conditional approval recommendations. Finally, GAO and OSMREreviews disclosed that AVS may have a programming problem that Page 14 GAO/~90-63 Surface Mining Appendix M Analysis of the Reliability of AS% precludes proper processing of the crucial Administration ownership and control data. In addition, OSMREofficials told us that the primary reasons for the sharp drop in the agreement rates during April and May 1989 were the following: l Between February and June 1989, one state submitted a large number of renewal requests in addition to new permit applications. The renewals contained large amounts of inaccurate data. l In April 1989, Administration ownership and control information was incorporated in the AVS data base, which significantly expanded the information available for linking applicants to violators. In so doing, some of the data led to incorrect linkages. Finally, only the Clearinghouse staff is able to recommend that a permit be issued conditionally. It can do this based on its research indicating that an outstanding debt payment plan has been established or a settle- ment agreement is in place for the correction of an outstanding viola- tion. AVS does not have access to this information and accordingly would recommend denial based on an outstanding violation. Our review dis- closed that over the 10 months ended October 31,1989, the conditional issuance category reduced the agreement rate by about 6 percent. An OSMREanalysis and our test of AVS operations has identified auto- System Processing mated system programming problems. These problems have caused Problems incorrect linking of applicants to outstanding violations, thus reducing the accuracy of the Avs-generated recommendations. We selected a sample of 30 AVS recommendations to assess the reliability of the AVS operation. Our test identified 11 cases in which AVS made incorrect links between applicants and outstanding violations. For six cases, the Clearinghouse advised us that the incorrect linkages were caused by the system’s inability to accurately read the Administration identification numbers and ownership and control data. However, the Clearinghouse could not pinpoint the exact cause for the incorrect linkages. The Clearinghouse staff offered more specific causes for the other five cases. These incorrect linkages appeared to be due to problems in AVS identifying starting and ending dates for when specific individuals or companies owned or controlled specific mining operations. During our Page 15 GAO/AFMD-3083 Surface Mining Appendix III Aualysia of the Reliability of AVS detailed discussions with Clearinghouse staff on these five cases, they pointed out instances where the owner or operator of a mine had changed. As discussed earlier, Administration numbers identify the mine, not those operating or controlling it. Because the Administration ownership and control data showed that the current applicant was not involved at the time of the violation, the Clearinghouse staff concluded there was no basis for the system-generated recommendation to deny the permit and, accordingly, reversed it. Unless the system can precisely read and process information reflecting periods of mine ownership and control, any weaknesses in this ability to process Administration data increases the risk of arriving at the wrong decision. In June 1989, Clearinghouse officials conducted an informal review of applications received in April and May in which the AVS and Clearing- house recommendations disagreed. Of the 479 cases where the recom- mendations disagreed, the Clearinghouse determined that incorrect links of applicants to violators caused about 400 disagreements. Clearing- house personnel said that the incorrect links were due to AVS computer program logic problems, renewal applications with old data, and other reasons. They said that about half of the 400 disagreements were prob- ably attributable to weaknesses related to AVS'S use of ownership and control data, including the incorrect reading of ownership periods. Because the Clearinghouse could not identify the exact cause for the incorrect links, in June 1989, it initiated an effort to determine why AVS is making the incorrect links of applicants to violators and to make appropriate modifications. OSMREshortly thereafter deferred further analysis until receipt of renewal applications had decreased and current Administration ownership and control information had been entered into the system. As of the completion of our review, the analysis had not been resumed, even though AVS continued to experience the same problems. Page 16 GAO/AFMD90-63 Surface Mining ‘i Appendix IV Major Contributors to This Report John S. Reifsnyder, Assistant Director, (202) 634-5217 Accounting and Jack Pichney, Senior Accountant-in-Charge Financial Management Gary I? Chupka, Senior Accountant Dianne Langston, Staff Accountant Division, Christine E. Cirullo, Staff Accountant Washington, DC. Page 17 GAO/AFMD-9083 Surface Mining Page 18 GAO/AFMD-90.63 Surf&e Mining * . t Page 19 ,C. RelatedGAO Products Debt Collection: Interior Is Acting to Improve Its Collection of Civil Pen- alty Fees (GAo/AFww-i’3, August 16, 1989). Surface Mining: Operation of the Applicant Violator System Can Be Improved (GAO/AFND-~Q-31, January 24,1989). Debt Collection: Interior’s Efforts to Collect Delinquent Royalties, Fines, and Assessments (GAo/AFMD~~-~~BR, June 18,1987). (Bo6124) Page 20 GAO/AFlWD-It042 Surface Mining ‘l’ht~ first. five copitw of each report. art’ free. Atldit,ional copiw are !woo t*ach. ‘I’ht~rt- is a 2!5%discount. ou ordws for 100 or more copies mailtd to 8 sin&* address.
Surface Mining: Interior Is Acting to Improve Its Applicant Violator System
Published by the Government Accountability Office on 1990-06-22.
Below is a raw (and likely hideous) rendition of the original report. (PDF)