United States General Accounting Office GAO Report to the Chairman and Ranking Minority Member, Subcommittee on Readiness and Management Support, Committee on Armed Services, U.S. Senate March 2003 FINANCIAL MANAGEMENT DOD’s Metrics Program Provides Focus for Improving Performance GAO-03-457 a March 2003 FINANCIAL MANAGEMENT DOD's Metrics Program Provides Focus Highlights of GAO-03-457, a report to the Chairman and Ranking Minority Member, for Improving Performance Subcommittee on Readiness and Management Support, Committee on Armed Services, U.S. Senate The Department of Defense (DOD) The DOD Comptroller’s metrics showing significant reductions in payment has historically been unable to recording errors and in commercial and travel card payment delinquencies accurately account for and record were, in general, based on definitions and methodologies that were either its disbursements. In March 2002, consistent with or better than those used for prior reporting on these issues. the DOD Comptroller cited metrics Although the methodology used to calculate two of the cited measures that showed dramatic reductions in payment recording errors resulted in overstating the rates of improvement, our recalculation after (57 percent between October 2000 correcting for the methodology errors still showed positive—although less and October 2001), backlogs of dramatic—improvement trends. commercial payments (41 percent between April and October 2001), While we were able to verify the reductions in travel card delinquencies and travel card payment because the underlying data were available from an independent source, we delinquencies (34 percent for those could not verify the accuracy of the specific improvement percentages individually billed and 86 percent reported for payment recording errors and commercial payment for those centrally billed between delinquencies. DOD’s archaic and nonintegrated systems either do not January and December 2001). As a contain the transaction-level detail to support the completeness and result, the Congress asked us to accuracy of the metrics or they make it extremely onerous and time determine whether the cited reductions were (1) calculated consuming for the staff to gather and reconcile the needed detail. However, using consistent definitions and we were able to verify that DOD has made numerous policy, procedure, and methodologies, (2) properly systems changes that support an overall trend toward improved supported, and (3) effective performance in these areas. indicators of short-term financial management progress. If they could be verified, some of the cited metrics could be effective indicators of short-term financial management progress. However, if considered alone, delinquency rates are not necessarily good indicators for centrally billed travel cards or commercial payments. Placing too much GAO recommends the following: • Use definitions and criteria emphasis on paying bills promptly may tempt DOD staff to bypass important consistent with the Defense internal controls meant to ensure that the goods and services being paid for Finance and Accounting Service were properly authorized and actually received. when calculating and reporting metrics related to payment Despite shortcomings, the cited metrics have focused DOD’s attention on recording errors. highly visible financial management problems. As shown below, recent • Measure improvements in metrics issued by the DOD Comptroller indicate continuing improvements. individually billed travel card Reported Progress from Beginning Measurement Dates delinquencies by using same month to same month 100 Percent comparisons. 80 • Work with the military services 60 and other defense agencies to 40 develop performance measures 20 that complement the metrics 0 program for crosscutting issues. Oct. 01 Sept. 02 Commercial payment backlogs DOD concurred with our Payment recording errors recommendations. Centrally billed travel card delinquencies www.gao.gov/cgi-bin/getrpt?GAO-03-457. Source: GAO calculations using DOD data. To view the full report, including the scope and methodology, click on the link above. Note: Individually billed travel card delinquencies are not shown because of limitations in the data For more information, contact Gregory Kutz, on the measurement dates. (202) 512-9095 or email@example.com. Contents Letter 1 Results In Brief 4 Background 7 Cited Metrics Were Generally Based on Improved Definitions and Methodologies 11 Most Cited Metrics Are Not Verifiable 16 Cited Metrics Serve Important Purpose But Further Steps Are Needed to Sustain Improvements 25 Conclusion 28 Recommendations 28 Agency Comments and Our Evaluation 29 Appendixes Appendix I: Objectives, Scope, and Methodology 30 Appendix II: Comments from the Under Secretary of Defense 32 Appendix III: GAO Contact and Acknowledgments 34 Related Audit Reports 35 and Testimonies Figures Figure 1: Reductions in Payment Recording Errors Between October 2000 and September 2002 14 Figure 2: Reductions in the Number of Payment Recording Errors 17 Figure 3: Decrease in Vendor Pay Backlogs for the Period April 2001 through September 2002 21 Figure 4: Decrease in Centrally Billed Travel Card Delinquencies for the Period January 2001 through December 2002 22 Figure 5: Decrease in Individually Billed Travel Card Delinquencies for the Period January 2001 through December 2002 25 This is a work of the U.S. Government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. It may contain copyrighted graphics, images or other materials. Permission from the copyright holder may be necessary should you wish to reproduce copyrighted materials separately from GAO’s product. Page i GAO-03-457 DOD's Metrics Program A United States General Accounting Office Washington, D.C. 20548 March 28, 2003 Leter The Honorable John Ensign Chairman The Honorable Daniel K. Akaka Ranking Minority Member Subcommittee on Readiness and Management Support Committee on Armed Services United States Senate The Department of Defense’s (DOD) inability to accurately account for and record its disbursements has been a serious, long-standing, and much reported financial management problem. The department’s ability to improve its accounting has historically been hindered by its reliance on fundamentally flawed financial management systems and processes and a weak overall internal control environment. In fact, DOD’s complex and inefficient payment processes have generally inhibited the proper recording of transactions when they occur, including the prompt and proper matching of disbursements with obligations—a critical funds- control measure. Such payment recording errors mean that DOD does not know the true amount of funds that it has available to obligate and spend in each appropriation account. As a result, the department risks overspending or not effectively using all available funding for needed items. Page 1 GAO-03-457 DOD's Metrics Program Auditors have also reported on DOD’s history of delinquent payments to its commercial suppliers and for its government-issued individually billed travel cards. These weaknesses increased the risk of fraud and/or disbursement errors, including duplicate payments, payments in the wrong amount, or charges to the wrong accounts. In addition, travel card delinquencies and charge-offs have resulted in millions of dollars in lost rebates and increased fees. These payment problems contributed greatly to our decision to put DOD financial management on our high-risk list of areas that are vulnerable to waste, fraud, abuse, and mismanagement in 1995, a designation that continues today.1 To increase attention to these problems, the Defense Finance and Accounting Service (DFAS) accelerated and expanded the use of its existing financial management performance metrics several years ago, to include specific measures for payment recording errors and payment delinquencies. In his March 6, 2002, testimony before your Subcommittee, the DOD Comptroller reported significant improvements in these measures based on calculations using DFAS data. In particular, the Comptroller stated that • payment recording errors decreased by 57 percent between October 2000 and October 2001, • DOD’s backlog of commercial payments (i.e., delinquent unpaid invoices) had been reduced by 41 percent between April 2001 and October 2001, and 1 U.S. General Accounting Office, High Risk Series: An Overview, GAO/HR-95-1 (Washington, D.C.: February 1995); U.S. General Accounting Office, Major Management Challenges and Program Risks: A Governmentwide Perspective, GAO/OCG-99-1 (Washington, D.C.: January 1999); U.S. General Accounting Office, High-Risk Series: An Update, GAO-03-119 (Washington, D.C.: January 2003); U.S. General Accounting Office, Major Management Challenges and Program Risks: Department of Defense, GAO-03-98 (Washington, D.C.: January 2003). Page 2 GAO-03-457 DOD's Metrics Program • payment delinquencies decreased by 34 percent for personal (individually billed) travel cards and 86 percent for organizational (centrally billed) travel cards between January 2001 and December 2001.2 This report responds to your request that we review DOD’s reported improvements. As agreed with your office, our objectives were to determine whether (1) the cited performance measures were defined and calculated in a manner consistent with previous reporting on payment recording errors and delinquencies, (2) the cited improvement data were properly supported and represent real improvements in performance, and (3) the metrics are effective indicators of short-term financial management progress that can be sustained. In conducting this work, we visited various DFAS centers and gathered, analyzed, and compared information on how payment recording errors, commercial payment backlogs, and travel card delinquencies were defined, calculated, and reported both in the past and for the cited metrics. We spoke with center personnel about process and systems improvements and we gathered and analyzed relevant output that demonstrated the results of those changes. We also reviewed the various financial management metrics programs in place or being developed throughout DOD. However, as discussed later in this report, we were unable to independently verify the completeness and accuracy of the data that supported the cited metrics, with the exception of travel card information. We performed our work in accordance with U.S. generally accepted government auditing standards from June 2002 through February 2003. Details of our scope and methodology are in appendix I. We requested comments from the Secretary of Defense and the Under Secretary of Defense (Comptroller). DOD’s comments are reprinted in appendix II. We considered and incorporated DOD’s suggested technical comments as appropriate. 2 Individually billed accounts are held and paid by individual cardholders based on reimbursement of expenses incurred while on official government travel. Centrally billed accounts are used to purchase transportation or for the travel expenses of a unit and are paid directly by the government. Page 3 GAO-03-457 DOD's Metrics Program Results In Brief In general, the definitions and methodologies used to gather data for the DOD Comptroller reported metrics were either consistent with or better than those used for prior reporting related to payment recording errors, commercial payment backlogs, and travel card payment delinquencies. For example, in contrast to previous reporting, intransit transactions3 were appropriately included in the definition of payment recording errors for the cited metrics calculations. While the underlying data were more complete, however, DOD personnel agree that there may still be payment recording errors that have not been identified and properly categorized. In addition, the methodology used to calculate two of the cited measures resulted in overstating the rates of improvement. Our recalculation of the metrics after correcting for the methodology errors still showed positive—although less dramatic—improvement trends. We could not verify the accuracy of specific improvement percentages reported for payment recording errors and commercial payment delinquencies, in large part because of DOD’s archaic and nonintegrated systems. Either the transaction-level detail supporting the completeness and accuracy of the reported metrics was no longer available or it would have been extremely onerous and time consuming for DOD staff to gather and reconcile the transaction-level data. However, DOD was able to provide us with summary-level data that matched the amounts reported for these metrics and we did verify that DOD has made numerous policy, procedural, and systems changes that would support an overall trend toward improved performance in these areas. In contrast, we were able to obtain corroborating data from an independent source that properly supported the reductions in travel card payment delinquencies cited by the Comptroller. In addition, the reduction in delinquency rates for individual travel card payments is supported by our recent findings that the military services, in particular the Air Force,4 have begun to give delinquencies greater attention and have used travel card audits to identify problems and needed corrective actions. 3 Intransit transactions include payments that have not yet been received by the DFAS accounting office for recording and matching against the corresponding obligation. DOD began reporting intransits in response to a recommendation in a report titled Financial Management: Status of Defense Efforts to Correct Disbursement Problems, GAO/AIMD-95- 7 (Washington, D.C.: Oct. 5, 1994). 4 U.S. General Accounting Office, Travel Cards: Air Force Management Focus Has Reduced Delinquencies, but Improvements in Controls Are Needed, GAO-03-298 (Washington, D.C.: Dec. 20, 2002). Page 4 GAO-03-457 DOD's Metrics Program For metrics to be effective, they must be properly defined, correctly measured, and able to be verified. However, none of the DOD Comptroller’s cited metrics meet all of these criteria. In addition, some of the metrics may not be good indicators of financial management improvement if considered separately. For example, focusing only on delinquency rates for centrally billed travel cards and commercial payments may place too much emphasis on paying bills promptly. As a result, DOD staff may be tempted to shortcut important internal control mechanisms that are meant to ensure that the goods and services being paid for were properly authorized and actually received. We have previously reported on problems related to DOD individually billed travel card purchases and contract payments5 that indicate the need for increased attention to the propriety as well as promptness of such payments. Another limitation in the reported metrics for payment recording errors and commercial payment backlogs is that only DFAS performance is being measured and reported. Even though the military services and other defense organizations are key contributors to preventing and resolving these problems, these organizations do not have complementary measurement programs. Despite their shortcomings, the cited metrics have served an important purpose by focusing DOD’s attention on highly visible financial management problems, setting challenging goals, and encouraging staff to be diligent and innovative in their attempts to attain those goals. In general, the improvements cited by the Comptroller demonstrate what can be accomplished in the short term as a result of the focus and intensive effort day after day of DOD management and staff. According to recent information, DFAS is continuing to report improvements in the measured areas. • Payment recording errors and commercial pay backlogs for September 2002 show reductions of 26 percent and 35 percent, respectively, from the balances reported at October 2001. 5 GAO-03-298; U.S. General Accounting Office, Travel Cards: Control Weaknesses Leave Army Vulnerable to Potential Fraud and Abuse, GAO-02-863T (Washington, D.C.: July 17, 2002); U.S. General Accounting Office, Travel Cards: Control Weaknesses Leave Navy Vulnerable to Potential Fraud and Abuse, GAO-03-148T (Washington, D.C.: Oct. 8, 2002); U.S. General Accounting Office, Financial Management: Improvements Needed in Air Force Vendor Payment Systems and Controls, GAO/AIMD-98-274 (Washington, D.C.: Sept. 28, 1998). Page 5 GAO-03-457 DOD's Metrics Program • Centrally billed travel card delinquencies declined from 2 percent at December 2001 to 1.5 percent at December 2002—1.5 percent is equal to the delinquency rate for other federal agencies. • Individual travel card delinquencies were reduced from 12.2 percent at December 2001 to 8.3 percent at December 2002. However, DFAS has cautioned that, without modern integrated systems and the streamlined processes they engender, reported progress for payment recording errors and commercial payment delinquencies may not be sustainable if its workload is significantly increased or its staffing significantly decreased. As discussed in our January 2003 high-risk report, the keys to effective reform of DOD financial management include an integrated approach, sustained leadership, results-oriented performance measures, and appropriate incentives and consequences. In line with this, DOD is currently developing a departmentwide, balanced program of metrics that is intended to align with its strategic goals, focus on results, and achieve auditable reports. DFAS, the military services, and other defense agencies will all be supporting players in this program. From the individual performance measurement programs of the military services, defense agencies, and DFAS, certain metrics will be selected and reported to the top levels of DOD management for evaluation and comparison. In this scenario, it is important that DOD properly and consistently report the selected metrics and that the services, agencies, and DFAS develop complementary metrics programs to assist in identifying, measuring, and resolving crosscutting issues. We are making specific recommendations in this report to address weaknesses where (1) the DOD Comptroller’s office used methodologies that overstated improvements, (2) the military services were critical partners with DFAS in making improvements or resolving problems but were not being measured on their performance in these areas, and (3) changes in travel card metrics would improve performance evaluation. In comments on a draft of this report, DOD agreed with our recommendations and explained actions it is taking to implement them. Page 6 GAO-03-457 DOD's Metrics Program Background For years, auditors have reported long-standing weaknesses in DOD’s ability to promptly pay its bills and accurately account for and record its disbursements. Numerous of our and DOD Inspector General audit reports have cited deficiencies in management oversight, a weak internal control environment, flawed financial management systems, complex payment processes, delinquent and inaccurate commercial and vendor payments, and lax management of DOD’s travel card programs.6 Those deficiencies have resulted in billions of dollars in unrecorded or improperly recorded disbursements, over- and underpayments or late payments to contractors, and fraudulent or unpaid travel card transactions. Payment Recording Errors DOD’s disbursement processes are complex and error-prone. Although DFAS is responsible for providing accounting services for DOD, military service and other defense agency personnel play a key role in DOD’s disbursement process. In general, military service and defense agency personnel obligate funds for the procurement of goods and services, receive those goods and services, and forward obligation information and receiving reports to DFAS. Separate DFAS disbursing offices and accounting offices then pay the bills and match the payments to obligation information. Several military services and DOD agencies can be involved in a single disbursement and each has differing financial policies, processes, and stand-alone, nonstandard systems. As a result, millions of disbursement transactions must be keyed and rekeyed into the vast number of systems involved in any given DOD business process. Also, transactions must be recorded using an account coding structure that can exceed 75 digits and this coding structure often differs—in terms of the type, quantity, and format of data required—by military service. DFAS’s ability to match disbursements to obligation records is complicated by the fact that DOD’s numerous financial systems may contain inconsistent or missing information about the same transaction. Input errors by DFAS or service personnel and erroneous or missing obligation documents are two of the major causes of inconsistent information. 6 See the “Related Audit Reports and Testimonies” section for a list of GAO and DOD Inspector General reports related to payment recording errors, commercial payment problems, and travel card program deficiencies. Page 7 GAO-03-457 DOD's Metrics Program For calculating and reporting performance metrics related to payment recording errors, officials from the Comptroller’s office included the following categories. • Unmatched disbursements—Payments that were made by a DFAS disbursing office and received by a DFAS accounting office but have not yet been matched to the proper obligation. • Negative unliquidated obligations—Payments that have been matched to and recorded against the cited obligations but which exceed the amount of those obligations. • Intransits—Payments that have not yet been received by the DFAS accounting office for recording and matching against the corresponding obligation. • Suspense account transactions—Payments that cannot be properly recorded because of errors or missing information (e.g., transactions that fail system edit controls because they lack proper account coding) and are therefore temporarily put in a holding account until corrections can be made. For DOD to know how much it has spent and/or how much is still available for needed items, all transactions must be promptly and properly recorded. However, we reported as early as 1990 that DOD was unable to fully identify and resolve substantial amounts of payment recording errors. We also stated that DOD’s early reporting of these errors significantly understated the problems. For example, DFAS excluded $14.8 billion of intransits from its 1993 benchmark against which it measured and reported its progress in reducing recording problems in later years. In addition, DOD excluded suspense account transactions from its reporting of payment recording errors until as late as 1999. Finally, when negative unliquidated obligations, intransits, and suspense account transactions were reported, they were reported using net rather than absolute values.7 7 Collections, reimbursements, and adjustments are offset against disbursements when net amounts are reported. When absolute amounts are reported, collections, reimbursements, and adjustments are added to disbursements. Reporting net amounts can significantly understate the magnitude and impact of payment recording errors. Page 8 GAO-03-457 DOD's Metrics Program Commercial Payments DFAS has overall responsibility for the payment of invoices related to goods and services supplied by commercial vendors. As part of a reorganization effort in April 2001, DFAS separated its commercial payment services into two efforts—contract pay and vendor pay. Contract pay handles invoices for formal, long-term contract instruments that are typically administered by the Defense Contract Management Agency (DCMA). These contracts tend to cover complex, multiyear purchases with high dollar values, such as major weapon systems. Payments for contracts are made from a single DFAS system— Mechanization of Contract Administration Service (MOCAS). For fiscal year 2001, DFAS disbursed about $78 billion for over 300,000 contracts managed in MOCAS. The vendor pay product line handles invoices for contracts not administered by DCMA, plus miscellaneous noncontractual payments such as utilities, uniforms/clothing, fuels, and food. Vendor pay is handled by 15 different systems throughout DFAS and, annually, DFAS personnel pay nearly 10 million vendor invoices in excess of $70 billion. In general, DOD makes vendor payments only after matching (1) a signed contractual document, such as a purchase order, (2) an obligation, (3) an invoice, and (4) a receiving report. If any one of these components is missing, such as an obligation not being entered into the payment system, payment of the invoice will be delayed. According to DOD officials, approximately 80 percent of payment delinquencies are due to the delayed receipt of receiving reports by DFAS from the military service activities. Travel Cards DOD implemented the current travel card program in November 1998, through a DOD task order with Bank of America. This was in response to the Travel and Transportation Reform Act of 1998 (P.L. 105-264), which modified the existing DOD Travel Card Program by mandating that all government personnel must use the government travel card to pay official travel costs (for example, hotels, rental cars, and airfare) unless specifically exempted. The travel card can also be used for meals and incidental expenses or to obtain cash from an automatic teller machine. The intent of the travel card program was to provide increased convenience to the traveler and lower the government’s cost of travel by reducing the need for cash advances to the traveler and the administrative workload associated with processing/reconciling travel advances. Page 9 GAO-03-457 DOD's Metrics Program DOD’s travel card program, which is serviced through Bank of America, includes both individually billed accounts and centrally billed accounts. When the travel card is submitted to a merchant, the merchant will process the charge through its banking institution, which in turn charges Bank of America. At the end of each banking cycle (once each month), Bank of America prepares a billing statement that is mailed to the cardholder (or account holder) for the amounts charged to the card. The statement also reflects all payments and credits made to the account. For both individual and centrally billed accounts, Bank of America requires that the cardholder make payment on the account in full within 30 days of the statement closing date. If the cardholder—individual or agency—does not pay the monthly billing statement in full and does not dispute the charges within 60 days of the statement closing date, the account is considered delinquent. For individually billed accounts, within 5 business days of return from travel, the cardholder is required to submit a travel voucher claiming legitimate and allowable expenses, which must be reviewed and approved by a supervisor. DOD then has 30 days in which to make reimbursement. Although DOD, like other agencies, relies on its employees to promptly pay their individually billed accounts, DOD does have some tools to monitor travel card activity and related delinquencies, including Bank of America’s Web-based Electronic Account Government Ledger System (EAGLS). Using EAGLS, supervisors can obtain reports on their cardholders’ transaction activity and related payment histories. For the centrally billed accounts, the travel office at each military installation or defense agency must first reconcile the charges shown on the centrally billed travel charge card account with the office’s internal records of transportation requests. After reconciliation has been completed, the voucher is sent to DFAS for payment. Because the travel card program is fairly new, DOD does not have a long history of reporting statistics for delinquencies. However, in our previous reports and testimonies, we have reported that DOD’s individually billed delinquency rate is higher than that of other federal agencies. As of September 2002, DOD’s delinquency rate was approximately 7.3 percent, about 3 percent higher than other federal agencies. Among the military services, however, the Air Force had the lowest delinquency rate. As of September 2002, the Air Force delinquency rate was 4.8 percent, significantly lower than the rest of DOD. Even though the Air Force had lower numbers of delinquent accounts, we found that control environment weaknesses and breakdowns in key controls were departmentwide and Page 10 GAO-03-457 DOD's Metrics Program that these deficiencies led to instances of potential fraud and abuse with the use of travel cards in all the military services. Performance Metrics In 1998, DFAS developed its Performance Contract to focus on continued Programs achievement of its mission to provide responsive, professional finance and accounting services to DOD. As part of this contract with DOD, DFAS defined its performance objectives and identified specific performance measurement indicators. DFAS managers—and sometimes staff—are rated and rewarded based on their ability to reach annual reduction goals for each indicator. Performance metrics are now calculated monthly and the DFAS Director and the DOD Comptroller regularly review the results. Section 1008 of the National Defense Authorization Act for Fiscal Year 1998 (P.L. 105-85) directed the Secretary of Defense to submit a biennial strategic plan for the improvement of financial management to the Congress. In conjunction with the plan, the DOD Comptroller decided to develop a performance measurement system—a set of departmentwide metrics that will provide clear-cut goals for financial managers to monitor their progress in achieving reform. To begin this effort, the Comptroller adopted many of the DFAS performance measurement indicators because the DFAS metrics program had been underway for some time and was reporting successes. For payment recording errors and commercial payment backlogs in particular, the Comptroller’s metrics used information gathered and tracked by DFAS for its performance management contract. The metrics cited in the Comptroller’s testimony represent only a few of the financial management performance metrics developed to date. From a comprehensive set, the detailed metrics will be rolled up into “dashboard” metrics that will provide the Secretary of Defense and the Congress with a quick measure of DOD’s status in relation to critical financial management goals. This effort is part of an even larger effort by DOD to develop programmatic metrics for all of its operations. Cited Metrics Were In general, the definitions and methodologies for gathering the data used by DOD Comptroller officials to calculate the cited improvement percentages Generally Based on at the ending measurement date were either consistent with or better than Improved Definitions those used at the beginning measurement date or for prior reporting on payment recording errors, commercial payment backlogs, and travel card and Methodologies payment delinquencies. We did find that the reported metrics overstated Page 11 GAO-03-457 DOD's Metrics Program the rate of improvement in some areas because Comptroller officials included transactions that DFAS would not consider to be payment errors or because they chose an inappropriate comparison to measure travel card delinquencies. However, recalculation of the metrics after correcting for these factors still showed positive—although less dramatic—improvement trends. Payment Recording Errors DOD has gradually improved its reporting of payment recording errors over the years. DOD is now including all known categories of payment errors— unmatched disbursements, negative unliquidated obligations, intransits, and suspense account transactions—in its definition and, except in the case of intransits, is using absolute rather than net amounts in its calculations. However, the reporting of payment recording errors may not be complete. For example, work that we have performed on closed DOD accounts and on unliquidated obligations8 indicates that recording errors are not always identified or resolved appropriately. DFAS agrees that to properly manage and improve its payment processes, it must have a complete universe of payment recording errors. Therefore, DFAS personnel are currently working to determine whether the error categories identified to date contain all of the relevant transactions and whether other error categories exist. While the same basic methodologies were used for calculating the cited metrics at the beginning and ending measurement dates, Comptroller officials overstated DOD’s improvement percentages because the October 2000 calculation included transactions that did not meet the DFAS criteria for being considered payment errors while the October 2001 calculation did not include them. First, the October 2000 calculation for payment recording errors included all transactions that were being held in DFAS suspense accounts; however, DFAS uses certain suspense accounts to record collection transactions, such as accrued payroll taxes and receipts for the sale of military property, that are held temporarily before being distributed to the proper government agency or DOD entity. The transactions in these accounts, which DFAS labels as “exempt suspense accounts,” do not represent payment recording errors. In fiscal year 2001, 8 U.S. General Accounting Office, Defense Budget: Improved Reviews Needed to Ensure Better Management of Obligated Funds, GAO-03-275 (Washington, D.C.: Jan. 30, 2003); U.S. General Accounting Office, Canceled DOD Appropriations: $615 Million of Illegal or Otherwise Improper Adjustments, GAO-01-697 (Washington, D.C.: July 26, 2001). Page 12 GAO-03-457 DOD's Metrics Program DFAS Cleveland changed its practice of charging payroll taxes to suspense accounts and began appropriately accruing taxes in an accrued payroll tax account. As a result, payment recording errors as calculated by Comptroller officials at October 2001 were reduced by an estimated $7.5 billion—the amount of DFAS Cleveland’s accrued payroll taxes—even though payment processes were not improved at all. Second, in fiscal year 2001, DFAS Indianapolis corrected a reporting error by a defense agency that had been double-counting transactions in its suspense accounts. This resulted in an estimated $1.1 billion reduction from amounts reported in October 2000, even though no payment recording errors were corrected or resolved. In addition, Comptroller officials measured intransits using net rather than absolute values and did not adopt DFAS criteria for aging intransit and suspense account transactions. These practices affected the balances used to calculate the metrics at both the beginning and ending measurement dates. First, net rather than absolute values were used to calculate intransits at October 2000 and October 2001, which understated both balances by approximately $4 billion. When net amounts are reported, collections, reimbursements, and adjustments are offset against disbursements, thus reducing the balance of intransit transactions. Second, the reported metrics included all intransit and suspense account transactions at October 2000 and October 2001 regardless of their age. However, DOD allows 60 days to 180 days9 for the normal processing of various payment transactions because of systems limitations and the complexity of the department’s processes and, in line with these criteria, DFAS’s metrics related to payment errors only consider aged intransit and suspense account transactions. By not using DFAS’s criteria for aged intransit and suspense account transactions, the Comptroller officials overstated the balances of payment recording errors by approximately $6 billion at the beginning and $5 billion at the ending measurement dates. Figure 1 illustrates the effect on improvement rates of (1) eliminating exempt suspense accounts and double counting, (2) using DFAS’s criteria for aged intransits and suspense amounts, and (3) using absolute rather 9 According to DOD’s Financial Management Regulation (FMR), intraservice transactions should not remain in transit for more than 60 days and interservice and interfund transactions for more than 120 days. For suspense accounts, the DOD FMR allows 60 days for clearing most transactions and 180 days for interfund transfers. DOD has stated that the normal processing time for most transactions is 60 days. Page 13 GAO-03-457 DOD's Metrics Program than net amounts for intransits. Our recalculation shows an overall 46 percent reduction in payment recording errors between October 2000 and October 2001 rather than the 57 percent reduction reported by the Comptroller; however, the reductions are still significant and the trend is still overwhelmingly positive. Between October 2001 and September 2002, DOD continued to report that it had reduced payment recording errors. Comptroller officials calculated a 26 percent reduction during that period while our recalculation shows a 22 percent reduction. Figure 1: Reductions in Payment Recording Errors Between October 2000 and September 2002 Dollars in billions 30 27.20 25 20 15 11.78 8.74 8.66 10 4.68 3.63 5 0 Oct. 2000 Oct. 2001 Sept. 2002 Comptroller’s calculation GAO’s calculation Source: DOD Comptroller officials’ and GAO’s calculations using DFAS data. Page 14 GAO-03-457 DOD's Metrics Program Commercial Payment The metrics for commercial payment backlogs (delinquent unpaid Backlogs invoices) at April 2001 and October 2001 were calculated using consistent definitions and methodologies. An invoice was considered delinquent if payment was not made within the time frame established by the contract terms (e.g., by the 15th day after the invoice date) or, if no time frame was specified, on or before the 30th day after a proper invoice was received.10 DFAS reported information on delinquent invoices to Comptroller officials monthly using standardized input sheets. The total backlog percentages were then calculated by dividing the number of delinquent invoices outstanding by the total number of invoices on hand. According to the DOD Comptroller’s metrics, delinquent invoices for vendor pay decreased by 41 percent from April 2001 through October 2001 while delinquent invoices for contract pay decreased by 32 percent during that same period. Because DFAS officials stated that the decrease cited in the Comptroller’s metrics was primarily due to intensive focus placed on decreasing the backlog of delinquent vendor invoices, our review concentrated on vendor pay issues. Travel Card Delinquencies For the travel card metrics, consistent definitions and methodologies were used to gather the data and calculate the improvement percentages cited by the DOD Comptroller for January 2001 and December 2001. Travel card payments were considered delinquent if they were not paid within 60 days of the monthly statement closing date. Even though the terms of the travel cardholder’s agreement with Bank of America requires payment of the statement within 30 days of the statement closing date, it is industry practice to allow 60 days before the invoice is considered delinquent and interest is charged. Comptroller officials used a standard industry practice to calculate the travel card delinquency rates— the total dollar amount outstanding for 60 days or more was divided by the total balance outstanding. While the definitions and methodology were consistent with standard practices, the metrics comparison of delinquencies for individually billed 10 The principal guidance used for making payment to vendors is the Prompt Payment Act, as implemented by the Office of Management and Budget (OMB) in 5 Code of Federal Regulations (C.F.R.) Part 1315, “ Prompt Payment: Final Rule,” September 29, 1999. The Prompt Payment Act requires the government to pay interest if an invoice is not paid on time. Page 15 GAO-03-457 DOD's Metrics Program accounts in January to those in December could be misleading. As our recent work shows, individually billed travel card delinquencies have been cyclical, with the highest delinquencies occurring in January and February. Therefore, the most useful metrics would compare same month to same month, for example, January to January or December to December. If the Comptroller officials had compared individual travel card delinquencies at January 2001 to those at January 2002, the reported decrease would have been 16 percent as opposed to 34 percent. Most Cited Metrics Are DFAS only provided us with internally generated summary-level data that reconciled to the totals reported for payment recording errors and Not Verifiable commercial pay backlogs. DFAS did not provide us with detailed transaction-level data that supported those metrics. As a result, we were unable to test whether (1) all payment recording errors and delinquent commercial payments were properly included in the metrics and (2) the actions taken to resolve or correct payment recording errors were appropriate. For individual and centrally billed travel card delinquencies, we were able to obtain independent verification from a source outside DOD that supported the Comptroller’s metrics. Although we could not audit the reported metrics for all of the measured areas, we verified that DFAS and other DOD organizations have made numerous policy, procedure, and systems changes that would support an overall trend toward improved performance. For payment recording errors and commercial payment backlogs, perhaps the most significant change has been DOD’s inclusion of performance measures in its contracts with DFAS. The performance contract and an accompanying data dictionary provide specific, measurable reduction goals, which DFAS management— and in some cases staff—are held accountable for reaching. The resulting focus has fostered innovative process and systems improvements as well as better communication among the parties involved in preventing or resolving these problems. For example, DFAS holds monthly videoconferences with its centers and field sites to discuss progress and any impediments to reaching that period’s goals. Payment Recording Errors In general, DFAS centers did not maintain history files of all the transactions that were not promptly matched with obligations, created negative unliquidated obligations, were in transit longer than allowable, or were in suspense accounts during the period October 2000 through Page 16 GAO-03-457 DOD's Metrics Program October 2001—information that is necessary in order to verify the completeness and accuracy of the reported metrics. DFAS officials explained that the detailed data supporting the reported monthly totals are compiled by hundreds of DFAS field sites using numerous accounting systems and there is no specific requirement for the field sites to save the data. While some DFAS officials believe that it would be possible to recreate transaction-level detail to support month-end totals, the task would be extremely onerous and time consuming. Although we were unable to verify through audit procedures the accuracy of the reductions reported by the Comptroller, we did reconcile summary- level information provided by the DFAS centers to the metric amounts. We also verified that DFAS has made numerous policy and systems improvements that support a continuing trend of reductions in payment recording errors as illustrated by the metrics in figure 2. Figure 2: Reductions in the Number of Payment Recording Errors Dollars in billions 10 9 8 7 6 5 4 3 2 1 0 Oct. 2000 Oct. 2001 Sept. 2002 Suspense Intransits Negative unliquidated obligations Unmatched disbursements Source: GAO calculations using DFAS data. Page 17 GAO-03-457 DOD's Metrics Program DFAS has been working to reduce payment recording errors for more than a decade. In the late 1990s, DFAS consolidated most of its disbursing and accounting functions from 300 defense accounting offices into 5 centers, in large part to help streamline the payment recording process. DFAS has also been working with other DOD components to consolidate or replace about 250 outdated and nonintegrated financial and accounting systems. While the systems effort will take many years and must be accomplished within DOD’s overall plan for systems development and integration,11 DFAS has made, and continues to make, improvements in the policies and systems tools available to DFAS personnel for preventing and correcting payment recording errors. Since October 2000, DFAS has made several policy changes that have affected the payment recording process. In January 2001, DOD revised its official guidance12 to clarify and strengthen policies related to the prompt (1) recording of disbursements and obligations and (2) resolution of payment recording errors. If the military services or DOD components have not provided DFAS with accurate obligation information within specified time frames, the revision gave DFAS the authority to record obligations in order to resolve individual unmatched disbursements, negative unliquidated obligations, and certain suspense account transactions. DFAS also expanded its prevalidation policy, which it claims has been key to reducing payment errors associated with commercial contracts. Prevalidation requires that DFAS personnel ascertain that there is a valid obligation recorded in the accounting records before making a payment. Between November 2000 and October 2001, DFAS lowered the dollar threshold amount for transactions requiring prevalidation from $100,000 to $25,000. DFAS developed new systems tools13 for communicating accounting information among its centers and field locations that have reduced the amount of time DFAS personnel need to match disbursements to 11 Section 1004 of the National Defense Authorization Act for Fiscal Year 2003 (P.L.107-314, Dec. 2, 2002) requires the Secretary of Defense to develop a comprehensive financial management enterprise architecture plan no later than May 1, 2003. It also requires the preparation of a Transition Plan that will guide DOD in implementing its architecture plan, and must include a schedule for terminating “legacy systems.” 12 DOD 7000.14-R, Financial Management Regulation, Volume 3, Chapter 11. 13 Appendix I discusses the limited scope of our review of systems tools and systems improvements. Page 18 GAO-03-457 DOD's Metrics Program obligations. For example, since the late 1990s DFAS has implemented the following. • Electronic data access capability, which provides web access to contract, billing, and other documents pertinent to the payment recording process. Electronic access to these documents enables users to obtain information more quickly than in the past, when many documents were stored in hard-copy format. • Phase 1 of the Defense Cash Accountability System (DCAS), which provides a standardized, electronic means for DFAS centers to report expenditure data for transactions involving more than one military service (cross-disbursements). Prior to DCAS, the centers had different systems and formats for reporting this information to one another and to Treasury, a situation that increased the complexity of recording and matching cross-disbursements. According to DFAS officials, DCAS reduced the cross-disbursement cycle time from 60 days to 10 days. • The Standard Contract Reconciliation Tool (SCRT), which provides DFAS personnel a consolidated database for researching commercial contract records. Prior to SCRT, locating and accessing these records was difficult due to the variety of accounting, contracting, and entitlement systems involved. DFAS centers have also developed individual applications that have improved payment processes. For example, DFAS Indianapolis implemented an Access “Wizard” application to automate the process of matching intragovernmental expenditure transactions to obligation records. The program also enables center staff to identify transactions that have not been processed within 30 days so they can follow up with field accounting personnel. Commercial Pay Backlogs DFAS was unable to provide detailed transaction-level data that supported the metrics related to vendor payment backlogs—the most significant contributor to the reductions. DFAS only maintained summary-level data that were generated by the 23 DFAS field sites. Using standard definitions and standard summary spreadsheets, DFAS personnel collected the summary information monthly through data calls to the more than 15 different systems that track DOD vendor pay backlog information. As a result, we were only able to confirm that the summary information provided by DFAS reconciled to the amounts reported by the Comptroller. Page 19 GAO-03-457 DOD's Metrics Program We were unable to verify by audit the accuracy or completeness of that data. DFAS management has focused on reducing commercial payment backlogs since fiscal year 2000 and this focus is continuing through the present. According to its performance contracts, DFAS’s goal was to reduce the backlog by 15 percent per year beginning in fiscal year 2000 from a baseline of 48,000 delinquent invoices. In April 2001, DFAS centralized operational control of contract pay and vendor pay under one executive, who was given ultimate responsibility for meeting these performance goals. DFAS also made site-specific procedural changes to reduce the backlog of vendor payments. These included • hiring temporary contract and permanent staff in key sites, such as Limestone and Dayton; • forecasting when civilian employees in Europe would be taking vacation and then staggering vacation leave and/or hiring temporary help (e.g., in Germany, every civilian employee has 6 weeks of annual leave, which is usually taken during the summer); and • forming partnerships with the military services and defense agencies to improve their processing time for receiving reports, since DFAS must match the receiving report to the invoice before payment can be made. DFAS credits these and other changes for the continued reduction of the backlog of delinquent invoices. Figure 3 below illustrates the trend in the reduction of outstanding delinquent vendor invoices compared to the total number of invoices on-hand. Page 20 GAO-03-457 DOD's Metrics Program Figure 3: Decrease in Vendor Pay Backlogs for the Period April 2001 through September 2002 Percent 12 10 9.82 9.61 9.25 8 6.84 6.74 7.84 6.68 7.35 6.71 6 4.96 5.88 5.67 4.17 4 3.91 3.62 3.67 3.42 3.65 2 0 Jan. Feb. Mar. Apr. May June July Aug. Sept. Oct. Nov. Dec. Months 2001 2002 Source: DFAS data. Travel Card Delinquencies We were able to verify the reductions cited by the Comptroller in individual and centrally billed travel card delinquencies. We obtained travel card delinquency information from an independent source, the General Services Administration (GSA), that supported the Comptroller’s metrics. GSA receives information from individual travel card vendors, such as Bank of America, and prepares a monthly summary report for DOD that documents individual and centrally billed travel card delinquencies by military service or defense agency. We compared the GSA data to the cited metrics and verified that the reported reductions in travel card delinquencies were accurate. As with the other problem areas, DOD credits the decrease in travel card delinquency rates in both individual and centrally billed accounts primarily to increased management attention. For the centrally billed accounts, DOD has attributed the initial high delinquency rates to problems in transferring the travel card contract from American Express to Bank of America. When Bank of America was given the contract, its on-line travel information system, EAGLS, was not fully operational and therefore was unable to accurately process all of the travel Page 21 GAO-03-457 DOD's Metrics Program data being transferred by American Express. Because EAGLS contained incorrect account numbers, invoice information, and billing addresses, DOD agency program coordinators did not have the information necessary to determine which accounts were delinquent, in suspense, or canceled. While DOD and Bank of America officials were working jointly to identify and resolve the problems, centrally billed invoices became backlogged. Once the problems were resolved, DOD was able to reduce the backlog. As of December 31, 2002, DOD’s centrally billed delinquency rate was 1.5 percent, well below fiscal year 2002’s proposed goal of 3.0 percent and equal to the delinquency rate for other federal agencies. Figure 4 below shows the centrally billed delinquency rates from January 2001 through December 2002. Figure 4: Decrease in Centrally Billed Travel Card Delinquencies for the Period January 2001 through December 2002 Percent 16 14.3 14 12 9.9 10 8 6.2 6.2 6 5.5 4.8 4.7 4.3 4.1 4.2 4 2.6 3 2.3 1.9 2 2 1.7 1.3 1 0.4 0.8 1.5 0.2 0.4 1.5 0 Jan. Feb. Mar. Apr. May June July Aug. Sept. Oct. Nov. Dec. Months 2001 2002 Source: General Services Administration. Page 22 GAO-03-457 DOD's Metrics Program For individual travel cards, our recent work also supports the improved delinquency rates being reported by DOD. During the past year, we reported on the travel card programs for all three military services.14 In general, we found that the military services, in particular the Air Force, have given delinquencies greater attention and have used travel card audits to identify problems and needed corrective actions. We reported that all of the services are now holding commanders responsible for managing the delinquency rates of their subordinates. For example, Air Force management holds monthly command meetings where individual travel card delinquencies are monitored and briefed. The individual services have also implemented new programs to help reduce delinquencies, including the following. • In January 2003, the Army established two goals of not more than 4.5 percent of dollars delinquent and not more than 3 percent of accounts delinquent. The Navy has established a similar goal of no more than 4 percent delinquent accounts. • The Air Force is providing financial training to all inductees that includes developing a personal budget plan, balancing a checkbook, preparing a tax return, and understanding financial responsibility. The training also covers the disciplinary actions and other consequences of financial irresponsibility by service members. • The Navy has developed a three-pronged approach to address travel card issues: (1) provide clear procedural guidance to agency program coordinators (APCs) and travelers that is available on the Internet, (2) provide regular training to APCs, and (3) enforce the proper use and oversight of the travel card by using data mining to identify problem areas and abuses. • In January 2003, the Army issued two directives to its major commanders, which address a range of policy requirements, to include: (1) training for APCs and cardholders, (2) monthly review of cardholder transactions, (3) exempting and/or discouraging the use of the card for en route travel expenses associated with deployments, and (4) prohibiting use of the card for travel expenses associated with permanent change of station moves. 14 GAO-03-298; GAO-02-863T; GAO-03-148T. Page 23 GAO-03-457 DOD's Metrics Program In addition, DOD has implemented a number of departmentwide programs to improve the individually billed travel card program. Beginning in November 2001, DOD began a salary and military retiree pay offset program for delinquencies—similar to wage garnishment. In March 2002, the Comptroller created a Credit Card Task Force to address management issues related to the purchase and individually billed travel card programs. On July 19, 2002, the DOD Comptroller directed the cancellation of (1) inactive travel charge card accounts, (2) active travel card accounts not used in the previous 12 months, and (3) travel card accounts for which the bank cannot identify the cardholders’ organization. DOD is also encouraging individual cardholders to elect to have all or part of their travel reimbursement sent directly by DFAS to Bank of America—a payment method that is standard practice for many private sector employers. The Congress has recently addressed this issue in section 1008(a) and (b) of the National Defense Authorization Act for Fiscal Year 2003,15 which provides the Secretary of Defense the authority to require use of this payment method. According to DOD, about 32 percent of its individually billed cardholders elected this payment option for fiscal year 2002. As a result of these and other actions, DOD has been able to sustain reduced delinquency rates between October 2002 and December 2002, as illustrated in figure 5 below. However, DOD still needs to do more to address the underlying causes of the problems with its travel card program. In a recent testimony, we concluded that actions to implement additional “front-end” or preventative controls are critical if DOD is to effectively address the high delinquency rates and charge-offs, as well as potentially fraudulent and abusive activity. 16 As a result of our work on travel cards,17 the Congress included a provision in the Department of Defense Appropriations Act for Fiscal Year 200318 requiring the Secretary of Defense to evaluate whether an individual is creditworthy before authorizing the issuance of any government charge card. If this requirement is effectively 15 P.L. 107-314, Dec. 2, 2002. 16 GAO-02-863T. 17 See the “Related Audit Reports and Testimonies” section for a list of our reports and testimonies related to DOD travel cards. 18 P.L. 107-248, Oct. 23, 2002, Sec. 8149. Page 24 GAO-03-457 DOD's Metrics Program implemented, DOD should continue to improve delinquency rates and reduce potential fraud and abuse. Figure 5: Decrease in Individually Billed Travel Card Delinquencies for the Period January 2001 through December 2002 Percent 20 18.4 18 16 15.4 14 12.7 12.4 12.2 12.2 12 11.7 8.7 9.4 10 8.7 8.2 8.9 7.8 8 6.5 6.4 8.1 8.5 8.1 8.3 7.8 7.9 7.3 6 6.9 7 4 2 0 Jan. Feb. Mar. Apr. May June July Aug. Sept. Oct. Nov. Dec. Month 2001 2002 Source: General Services Administration. Cited Metrics Serve The metrics that the DOD Comptroller highlighted in the March 2002 hearing relate to areas that have received considerable congressional and Important Purpose But audit attention. As discussed earlier, the metrics program increased Further Steps Are management focus on these problem areas and led to improvements in policies, processes, and—in a limited way—systems. While some of the Needed to Sustain cited metrics could be effective indicators of short-term financial Improvements management progress, assuming they could be verified, others are not necessarily good indicators, particularly if taken alone. In addition, continued financial management progress will require additional actions. For example, the military services and other defense agencies are key contributors to preventing and resolving payment recording errors and commercial payment delinquencies but they do not have the same incentives to improve their performance in these areas. Also, because DFAS lacks modern, integrated financial management systems, preventing Page 25 GAO-03-457 DOD's Metrics Program and resolving payment delinquencies and errors require intensive effort day after day by DFAS and other DOD organizations, which could be difficult to sustain. Effectiveness of Measures The cited metrics for individual travel card delinquencies and payment recording errors could be effective indicators of financial management improvement. For payment recording errors, continuing reductions would indicate better controls over obligation, disbursement, and collection processes and that, as a result, DOD is less prone to fraud, waste, or abuse of appropriated funds. Monitoring the delinquency rates for individual travel card payments would provide DOD with an early indication that employees may be abusing their cards (i.e., using the cards for personal purchases) or having credit problems. However, improved delinquency rates do not necessarily indicate improved financial management of centrally billed travel cards or commercial payments. In fact, by placing too much emphasis on paying bills promptly, DOD staff may be tempted to shortcut important internal control mechanisms that are meant to ensure that the goods and services being paid for were properly authorized and actually received. We and DOD auditors have issued several reports on the improper use of individually billed travel cards at DOD and on over- and underpayments to DOD contractors but are just beginning work to identify and evaluate the adequacy of DOD policies, procedures, and controls related to purchases from vendors and centrally billed travel cards. As a result of these audits, we will likely recommend additional metrics related to program performance and internal controls for monitoring performance in these areas. Page 26 GAO-03-457 DOD's Metrics Program Sustainability of Measures such as the ones discussed in this report may be useful in the Improvements short term but may not be appropriate once DFAS has reengineered its business processes and modernized its systems. As DFAS and the military services develop integrated and/or interfaced financial management systems, many of the problems related to transaction recording errors should be eliminated. Based on the recent work we performed for your committee related to DOD’s enterprise architecture,19 however, these new systems are years away from implementation. Because DFAS lacks modern, integrated financial management systems, preventing and resolving payment delinquencies and errors require intensive effort day after day by DFAS and military service staff. As a result, DFAS has indicated that much of the reported progress to date is sustainable only if its workload is not significantly increased or its staffing significantly decreased. Until new systems and reengineered processes are in place, DOD can take a number of steps to help maintain improvements in these areas. First, continued leadership and focus by top management will be a major factor in the sustainability of progress made to date. Second, because DFAS alone cannot resolve DOD’s payment recording problems or payment delinquencies, integrated metrics programs across DOD will be important. As noted earlier in this report, while the military services and other defense agencies play key roles in obligating DOD funds, preparing obligation documents, receiving and preparing billing documents, preparing receiving reports, and recording transaction information into accounting systems, these organizations do not currently have complementary metrics programs. Thus the military services and defense agencies are not measured on the accuracy and timeliness of their payment processes even though their assistance is necessary for DFAS to make improvements and resolve problems. For example, commercial payment backlogs were largely due to failure by the military services in providing receiving reports to DFAS, yet service delays were not being measured. DOD is currently developing a departmentwide, balanced program of metrics that is intended to align with its strategic goals, focus on results, and achieve auditable reports. As contemplated, DFAS, the military 19 U.S. General Accounting Office, DOD Business Systems Modernization: Improvements to Enterprise Architecture Development and Implementation Efforts Needed, GAO-03-458 (Washington, D.C.: Feb. 28, 2003). Page 27 GAO-03-457 DOD's Metrics Program services, and other defense agencies will all be supporting players in this program. From the individual performance measurement programs of the military services, defense agencies, and DFAS, certain metrics will be selected and reported to the top levels of DOD management for evaluation and comparison. In this scenario, it is important that DOD properly and consistently calculate and report the selected metrics and that the military services, other agencies, and DFAS develop integrated metrics programs to assist in identifying, measuring, and resolving crosscutting issues. Conclusion As the cited metrics demonstrate, DOD can make meaningful, short-term progress toward better financial management while waiting for long-term solutions, such as integrated financial systems. Leadership, real incentives, and accountability—hallmarks of a good performance measurement program—have brought about improvements in DFAS policies and processes. The cited metrics are also serving as important building blocks for DOD’s current efforts to develop a departmentwide performance measurement system for financial management. However, before the payment recording error and commercial payment backlog metrics can be relied upon for decision-making purposes, they must be • verifiable, • properly defined and correctly measured, and • linked to the goals and performance measures of other relevant DOD organizations. In addition, because the reported improvements depend heavily on the day- to-day effort of DFAS staff, sustaining the progress may be difficult if DFAS has significant workload increases or staff decreases. Recommendations DOD systems do not provide the transaction-level support needed to verify the accuracy and completeness of many of its selected metrics. However, because DOD is currently working on developing an enterprisewide system architecture to guide its future systems development and implementation strategies, we are not making any recommendations in this report related to improving the underlying business systems. We did identify several steps that DOD could take now to improve the reported metrics. We are recommending that the DOD Comptroller Page 28 GAO-03-457 DOD's Metrics Program • use definitions and criteria that are consistent with DFAS definitions and criteria when calculating and reporting metrics related to payment recording errors, • measure improvements in individually billed travel card delinquencies by using same month to same month comparisons, and • work with the military service Assistant Secretaries for Financial Management to develop performance measures for the military services and other defense agencies in areas for which there is shared responsibility, in order to complement the DFAS metrics program. Agency Comments and In written comments on a draft of this report (see appendix II), the Under Secretary of Defense (Comptroller) stated that the department concurred Our Evaluation with our recommendations and described actions to address them. The department also provided several technical comments, which we have incorporated in the report as appropriate. We are sending copies of this report to other interested congressional committees; the Secretary of Defense; the Under Secretary of Defense (Comptroller); the Director, Defense Finance and Accounting Service; and the Assistant Secretaries for Financial Management (Comptroller) for the Army, the Navy, and the Air Force. Copies will be made available to others upon request. Please contact me at (202) 512-9505 or firstname.lastname@example.org if you or your staff have any questions about this report. Other GAO contacts and key contributors to this report are listed in appendix III. Sincerely yours, Gregory D. Kutz Director Financial Management and Assurance Page 29 GAO-03-457 DOD's Metrics Program Appendix I Objectives, Scope, and Methodology AA ppp ep ned n x id e x Iis As requested by the Chairman and Ranking Minority Member of the Subcommittee on Readiness and Management Support, Senate Committee on Armed Services, we undertook an assessment of the consistency, accuracy, and effectiveness of certain DOD-reported metrics related to payment recording errors, commercial payment backlogs, and delinquent travel card payments. Specifically, our objectives were to determine whether (1) the cited performance measures were applied and calculated in a manner consistent with previous reporting on payment delinquencies and recording errors, (2) the cited improvement data were properly supported and represent real improvements in performance, and (3) the metrics are effective indicators of short-term financial management progress. To complete this work, we visited DOD Comptroller offices and DFAS centers in Arlington, Cleveland, Columbus, Indianapolis, and Denver where we did the following. • Gathered, analyzed, and compared information on how payment recording errors, commercial payment backlogs, and travel card delinquencies were defined, calculated, and reported both in the past and for the cited metrics. • Reviewed GAO, DOD IG, and other service auditors’ reports for the past 10 years. • Reviewed DOD consolidated financial statement reporting of payment recording errors over the last 10 years. • Reviewed DOD policy for maintaining financial control over disbursement, collection, and adjustment transactions. This policy1 specifically describes the requirements for researching and correcting payment recording errors. • Obtained and analyzed the underlying summary spreadsheets from DFAS that were the information source for the Comptroller officials’ calculations for payment recording errors and commercial pay backlogs. DFAS gathers this information monthly through data calls from numerous systems used to process and account for payments. Although we requested the underlying detailed transaction-level data 1 DOD 7000.14-R, Financial Management Regulation, Volume 3, Chapter 11. Page 30 GAO-03-457 DOD's Metrics Program Appendix I Objectives, Scope, and Methodology supporting the spreadsheets so that we could perform audit tests, we were unable to obtain the detail-level data. • Obtained and analyzed the underlying summary spreadsheets from DFAS that were the information source for the Comptroller officials’ calculations for travel card delinquencies. Obtained independent summary data for travel card delinquencies from GSA and compared amounts to Comptroller-reported metrics. • Interviewed center personnel about process and system improvements and gathered and analyzed relevant output that demonstrated the results of those changes. Our review of new systems tools and purported systems improvements was limited: we did not validate whether systems changes followed appropriate requirements or whether they resulted in the production of reliable financial information. • Obtained explanations from officials from the Office of the Secretary of Defense regarding the metrics program and assessed whether the cited metrics are effective indicators of short-term financial management progress. The data in this report are based on DFAS records. With the exception of travel card delinquency rates, we were unable to independently verify or audit the accuracy of these data. We performed our work from June 2002 to February 2003 in accordance with U.S. generally accepted government auditing standards. We received written comments on a draft of this report from the Under Secretary of Defense (Comptroller). These comments are presented and evaluated in the “Agency Comments and Our Evaluation” section and are reprinted in appendix II. We considered technical comments from the department and incorporated them as appropriate but did not reprint them. Page 31 GAO-03-457 DOD's Metrics Program Appendix II Comments from the Under Secretary of Defense Appendx Ii Page 32 GAO-03-457 DOD's Metrics Program Appendix II Comments from the Under Secretary of Defense Page 33 GAO-03-457 DOD's Metrics Program Appendix III GAO Contact and Acknowledgments Appendx iI GAO Contact Molly B. Boyle (202) 512-9524 Acknowledgments Staff making key contributions to this report were Rathi Bose, Steve Donahue, Diane Handley, Fred Jimenez, and Carolyn Voltz. Page 34 GAO-03-457 DOD's Metrics Program Related Audit Reports and Testimonies Payment Recording Errors Defense Budget: Improved Reviews Needed to Ensure Better Management of Obligated Funds. GAO-03-275. Washington, D.C.: January 30, 2003. Canceled DOD Appropriations: Improvements Made but More Corrective Actions Are Needed. GAO-02-747. Washington, D.C.: July 31, 2002. Canceled DOD Appropriations: $615 Million of Illegal or Otherwise Improper Adjustments. GAO-01-697. Washington, D.C.: July 26, 2001. Department of Defense Office of the Inspector General. Trends and Progress in Reducing Problem Disbursements and In-Transit Distribution. Report No. 99-135. Arlington, Va.: April 16, 1999. Financial Management: Problems in Accounting for Navy Transactions Impair Funds Control and Financial Reporting. GAO/AIMD-99-19. Washington, D.C.: January 19, 1999. Financial Management: Improved Reporting Needed for DOD Problem Disbursements. GAO/AIMD-97-59. Washington, D.C.: May 1, 1997. DOD Problem Disbursements: Contract Modifications Not Properly Recorded in Payment System. GAO/AIMD-97-69R. Washington, D.C.: April 3, 1997. Financial Management: Improved Management Needed for DOD Disbursement Process Reforms. GAO/AIMD-97-45. Washington, D.C.: March 31, 1997. Financial Management: Status of Defense Efforts to Correct Disbursement Problems. GAO/AIMD-95-7. Washington, D.C.: October 5, 1994. Financial Management: Problems in Accounting for DOD Disbursements. GAO/AFMD-91-9. Washington, D.C.: November 9, 1990. Commercial Payment Department of Defense Office of the Inspector General. Controls Over the Backlogs Computerized Accounts Payable System at Defense Finance and Accounting Service Columbus. Report No. D-2002-113. Arlington, Va.: June 21, 2002. Page 35 GAO-03-457 DOD's Metrics Program Related Audit Reports and Testimonies Department of Defense Office of the Inspector General. Closing Overage Contracts Prior to Fielding a New DOD Contractor Payment System. Report No. D-2002-027. Arlington, Va.: December 19, 2001. Department of Defense Office of the Inspector General. Controls Over Vendor Payments Made for the Army and Defense Agencies Using the Computerized Accounts Payable System. Report No. D-2002-056. Arlington, Va.: March 6, 2002. Contract Management: Excess Payments and Underpayments Continue to Be a Problem at DOD. GAO-01-309. Washington, D.C.: February 22, 2001. Internal Controls: Reporting Air Force Vendor Payment System Weaknesses Under the Federal Managers’ Financial Integrity Act. GAO/AIMD-99-33R. Washington, D.C.: December 21, 1998. Financial Management: Improvements Needed in Air Force Vendor Payment Systems and Controls. GAO/AIMD-98-274. Washington, D.C.: September 28, 1998. Contract Management: Fixing DOD’s Payment Problems is Imperative. GAO/NSIAD-97-37. Washington, D.C.: April 10, 1997. Department of Defense Office of the Inspector General. Vendor Payments - Operation Mongoose Fort Belvoir Defense Accounting Office and Rome Operating Location. Report No. 97-052. Arlington, Va.: December 23, 1996. High Risk Series: Defense Contract Management. GAO/HR-95-3. Washington, D.C.: February 1995. Travel Card Delinquencies Travel Cards: Control Weaknesses Leave Navy Vulnerable to Fraud and Abuse. GAO-03-147. Washington, D.C.: December 23, 2002. Travel Cards: Air Force Management Focus Has Reduced Delinquencies, but Improvements in Controls Are Needed. GAO-03-298. Washington, D.C.: December 20, 2002. Travel Cards: Control Weaknesses Leave Army Vulnerable to Potential Fraud and Abuse. GAO-03-169. Washington, D.C.: October 11, 2002. Page 36 GAO-03-457 DOD's Metrics Program Related Audit Reports and Testimonies Travel Cards: Control Weaknesses Leave Navy Vulnerable to Fraud and Abuse. GAO-03-148T. Washington, D.C.: October 8, 2002. Travel Cards: Control Weaknesses Leave Army Vulnerable to Potential Fraud and Abuse. GAO-02-863T. Washington, D.C.: July 17, 2002. Department of Defense Office of the Inspector General. Acquisition: Summary of DOD Travel Card Program Audit Coverage. Report No. D- 2002-065. Arlington, Va.: March 18, 2002. (192060) Page 37 GAO-03-457 DOD's Metrics Program GAO’s Mission The General Accounting Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full- GAO Reports and text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents Testimony using key words and phrases. You can print these documents in their entirety, including charts and other graphics. Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to www.gao.gov and select “Subscribe to GAO Mailing Lists” under “Order GAO Products” heading. Order by Mail or Phone The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. General Accounting Office 441 G Street NW, Room LM Washington, D.C. 20548 To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202) 512-6061 To Report Fraud, Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm Waste, and Abuse in E-mail: email@example.com Federal Programs Automated answering system: (800) 424-5454 or (202) 512-7470 Public Affairs Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800 U.S. General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C. 20548 United States Presorted Standard General Accounting Office Postage & Fees Paid Washington, D.C. 20548-0001 GAO Permit No. GI00 Official Business Penalty for Private Use $300 Address Service Requested
Financial Management: DOD's Metrics Program Provides Focus for Improving Performance
Published by the Government Accountability Office on 2003-03-28.
Below is a raw (and likely hideous) rendition of the original report. (PDF)