Payment Processing: Statistical Sampling Plan for Voucher Prepayment Examination

Published by the Government Accountability Office on 2003-08-28.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

United States General Accounting Office
Washington, DC 20548

   August 28, 2003

   Ms. Martha Stearns
   Deputy Director
     Commercial Pay Services
   Defense Finance and Accounting Service
   P.O. Box 182317
   Columbus, Ohio 43218-2317

   Subject: Payment Processing: Statistical Sampling Plan for Voucher Prepayment

   Dear Ms. Stearns:

   This report responds to a request made by your office that we approve the use of a
   statistical sampling plan for voucher prepayment examination of contract payments.
   While Title VII, “Fiscal Guidance,” of the GAO Policy and Procedures Manual for
   Guidance of Federal Agencies requires prepayment examination of vouchers prior to
   their certification and payment, it permits the use of statistical sampling for vouchers up
   to $2,500. You requested a waiver of the $2,500 ceiling for statistical sampling and
   approval of an alternative voucher prepayment examination plan that would, among
   other things, use statistical sampling for certain vouchers up to a maximum of $500,000
   for contract payments.

   In order to respond to your request, we reviewed your letter and the information
   provided in the attachments. We also contacted your staff to discuss your request in
   more detail. We analyzed the sections of the law pertaining to this request and reviewed
   the legislative history of 31 U.S.C. § 3521(b), upon which Title VII is based. Section
   3521(b) of Title 31 provides for statistical sampling procedures. We did not test the
   Defense Finance and Accounting Service (DFAS) contract payment system; therefore,
   our response only addresses your request conceptually.

    Title VII of GAO’s Policy and Procedures Manual for Guidance of Federal Agencies. The manual is the official medium through
   which the Comptroller General promulgates (1) accounting principles, standards, and related requirements and material for the
   guidance of federal agencies in the development of their accounting systems and internal auditing programs, (2) uniform procedures
   for use by federal agencies, and (3) regulations governing GAO’s relationships with other federal agencies and with private concerns
   doing business with the government. 31 U.S.C. § 3521(b) states that heads of agencies may prescribe statistical sampling procedures
   to audit vouchers when they decide that economies will result. The term audit as used in 31 U.S.C. § 3521 (b) refers to the
   examination of disbursement vouchers prior to their certification for payment.

                                                  GAO-03-1067R DOD Voucher Prepayment Examination
At this time, we cannot approve your request for a waiver or your proposed alternative
sampling plan. Your submission, requesting exception to the $2,500 maximum for
statistical sampling in voucher prepayment examination, does not justify your projected
savings of approximately $4.2 million per year. Further, it is unclear how the proposed
plan will adequately protect the government’s interest. In addition, absent GAO approval,
DFAS certifying officers risk personal liability for improper payments should they rely
upon the voucher prepayment examination process under the proposed alternative
sampling plan.


As described in your letter and the information accompanying it and as explained by
your staff during subsequent discussions, DFAS is evaluating its existing contract
payment procedures with the intent of achieving cost-effective compliance with Title VII.
We support such initiatives to create a government that works better and costs less. At
the same time, agencies have the responsibility to protect the government’s interest.

The fiscal guidance in Title VII contains general accounting processing procedures that
are required by law. The requirements are designed to complement agencies’ existing
federal accounting, internal control, and systems standards. All basic vouchers, voucher
schedules, and invoices or bills used as vouchers must be certified for correctness and
legality by an authorized certifying officer. A certifying officer is accountable for and
required to personally reimburse the government for any illegal, improper, or incorrect
payment made by a disbursing officer because of his or her certification.2 In addition, a
disbursing official is accountable for ensuring that a voucher is in proper form, certified
and approved, and computed correctly based on the facts certified.3 In appropriate
circumstances, the Comptroller General may relieve a disbursing or certifying official
from liability.4

Electronic techniques and systems are being used to generate, process, transmit, and
store financial information in today’s federal financial environment, requiring
accountable officials to rely on the systems, controls, and personnel that process the
transactions. However, the accountable officials must have reason to believe that the
key processes and controls on which they rely are working. Further, even though all or
some portion of the payment process may be automated, the certifying officer’s basic
accountability and legal liability for the propriety of the payment remains unaltered.

As a viable alternative to the traditional practice of performing 100 percent prepayment
examination of vouchers, section 7.4E of Title VII authorizes agencies to establish
statistical sampling programs for the examination of vouchers in support of their
certification and payment, provided the vouchers do not exceed $2,500. Within this
  31 U.S.C. § 3528 (a)(4).
  31 U.S.C. § 3325 (a)(2).
  31 U.S.C. § 3528 (b).
  The most significant factors on which confidence in the system and controls should be based are (1) a well-defined organization
structure and flow of work, with appropriate separation of responsibilities and clearly written policies and procedures governing the
examination, approval, and certification of disbursement vouchers, (2) effective application of available technology and concepts to
achieve efficient and effective voucher processing, and (3) review of the voucher processing procedures and controls in sufficient
scope, depth, and frequency to provide reasonable assurance that key processing procedures and controls are working and reliable.

Page 2                                       GAO-03-1067R DOD Voucher Prepayment Examination
maximum, agencies are required to establish their own dollar limitations based on cost-
benefit analyses of their voucher examining operations.6 Any disbursing or certifying
officer relying in good faith on the statistical sampling procedure adopted by the agency
to disburse funds or certify a voucher for payment will not be liable for losses to the
government resulting from payment or certification of a voucher not audited specifically
because of the use of an authorized sampling procedure, provided that the agency has
diligently carried out collection actions prescribed by the Comptroller General.
Exceptions to the $2,500 maximum for vouchers that may be statistically sampled may
be authorized by the Comptroller General, on a case-by-case basis, when it is clear that
they are warranted economically and that the procedures and controls will adequately
protect the government’s interest.

Prepayment Examination Process at DFAS

DFAS’s voucher prepayment examination process is designed to ensure, among other
requirements, that (1) payment is permitted by law, (2) appropriation amounts are
available at the time and are being used for the intended purpose, (3) goods and services
have been received and conform to the requirements of the order or agreement, (4)
required administrative approvals have been obtained, and (5) quantities, prices, and
calculations are accurate.

According to the information submitted with your request, the DFAS Columbus Contract
Pay organization entitles8 about 78,000 vouchers monthly at an average total monthly
voucher value of $7.2 billion. Contractor invoices are processed and paid through a
contract administration and payment system—Mechanization of Contract Administrative
Services (MOCAS). Approximately 35,000 of the 78,000 monthly invoices are entitled
through an automated payment of invoices (API) process. In the API process, the
MOCAS system edit checks provide for the entitlement and automatic payment of these
invoices. The remaining 43,000 payment vouchers are manually entitled by
approximately 120 voucher examiners and supervisors. Vouchers are manually entitled
if, for example, the contract governing the voucher payment has been modified, the
contract has special payment instructions, or it fails the edits in the API process.

The DFAS Front End Analysis (FEA) organization was established to detect contract pay
voucher errors prior to disbursement. One primary function of the DFAS FEA
organization is to perform independent voucher prepayment reviews. FEA staff
judgmentally select vouchers at various points in the entitlement process and examine
them for various attributes, such as input errors, payment according to contract, and
availability of funds. According to your letter, FEA’s voucher examination procedures
are designed to assess MOCAS system errors or individual voucher examiner error rates
by nonstatistical sampling from each day’s manual entitlements.

  GAO’s Policy and Procedures Manual for Guidance of Federal Agencies, Title VII, “Fiscal Procedures,” ch. 7, sec. 7.4E.
  31 U.S.C. § 3521 (c).

  The entitlement process, according to DFAS, refers to the preparation of the payment package, including the matching of an invoice,
receiving report, and contract. Entitlement precedes prepayment review procedures.

Page 3                                       GAO-03-1067R DOD Voucher Prepayment Examination
Of the approximately 78,000 vouchers processed each month, 55,000 are greater than
$2,500 and therefore subject to 100 percent prepayment examination in accordance with
Title VII. Currently, FEA staff nonstatistically select and examine an average of 1,500
vouchers above $2,500 per month. In a sample of manually entitled invoices from
October 2000 through July 2001, FEA noted a primary error rate9 of 13.4 percent, with
some of those errors attributable to overpayments totaling approximately $4.8 million for
the 10-month period.

DFAS’s Proposed Alternative Sampling Plan for Prepayment Examination

Your alternative voucher prepayment examination plan for contract payments would
shift the role of the FEA independent voucher team, made up of 6.5 full-time equivalents,
from detecting input errors and detecting individual voucher examiner errors to playing
a key role in the prepayment certification process for accountable officers. Specifically,
FEA staff would perform a review of (1) 100 percent of all manually entitled payment
vouchers equaling or exceeding $500,000 and (2) a statistical sample of manually entitled
vouchers less than $500,000 combined with all API vouchers regardless of amount,
thereby excluding API vouchers from the 100 percent voucher examination threshold.
All vouchers selected as part of this sampling plan would be reviewed to ensure that they
were legal, proper, and correct for payment using standardized review criteria. Any
discrepancies noted would be reported, tracked, monitored, and resolved.

Your analysis noted that this more systematic examination of vouchers would ensure
that FEA staff review over 53 percent of total monthly contract payment dollars, as
compared to the 6.5 percent of the total dollars that are currently examined through the
judgmental sampling of manually entitled vouchers only. In turn, DFAS expects that this
heightened review will detect and correct more potential overpayments and result in
savings to the government. The analysis prepared by a contractor for DFAS concludes
that the 6.5 FEA full-time equivalents can perform 1,000 manually entitled voucher
examinations per month if the maximum threshold for statistically sampled vouchers
was raised from $2,500 to $500,000. Thus, it appears to us that the resources available
are driving the proposed threshold and sample size rather than the consideration of other
key factors, such as an appropriate risk level and acceptable error rate that would
provide reasonable assurance that the government’s interest is protected.

Our Assessment of the Proposed Alternative Sampling Plan

Your proposed alternative sampling plan, which includes waiving the $2,500 maximum
for statistical sampling of voucher prepayment examination for contract payments, does
not justify the proposed savings or provide assurance that the government’s interest will
be adequately protected. Further, DFAS certifying officers risk personal liability for
improper payments should they rely upon the prepayment examination process under
the proposed alternative sampling plan.

A primary error is defined in the business case analysis as “the first error on an invoice that has been reviewed.” An invoice may
have more than one error.

Page 4                                       GAO-03-1067R DOD Voucher Prepayment Examination
GAO’s Policy and Procedures Manual sets out a relatively straightforward formula for
agencies to use in deciding whether to adopt sampling in the certification of payment
vouchers under 31 U.S.C. § 3521(b). The cost of examining all vouchers should be
compared with the total costs of sampling (i.e., the cost of examining the sample plus the
amount of projected losses from undetected errors in vouchers not examined).10 Where
the costs of reviewing all vouchers exceed the costs of sampling plus estimated losses,
sampling offers savings. Savings are achieved when the combined costs of (1) examining
the sample and (2) projected losses due to undetected errors on invoices not examined
are less than the cost of examining all invoices. Through analysis, the sampling plan
must identify a tolerable error rate, the number of invoices to select for examination,
and the selection method.

Your request describes approximately $4.2 million in projected savings per year as a
result of the alternative sampling plan. Based on our review of your proposed sampling
plan, we question this projection. Specifically, adequate evaluation of sample results
involves defensible projections to the entire population using statistically valid samples.
The sample results in your plan cannot be projected to the population because it is not
clear how the monthly sample sizes were determined or how the samples were picked
from the monthly population. The DFAS analysis is centered on 10 monthly selected
samples of manually entitled vouchers, but does not document how the sample sizes,
ranging from 985 vouchers selected in July 2001 to 1,894 vouchers selected in April 2001,
were determined relative to the total monthly population of manually entitled vouchers.
In addition, the methodology used to record errors in the sample is unclear in the
analysis. The analysis indicates that the error rate of 13.4 percent is a result of 2,019
“primary” errors noted in the 15,021 vouchers reviewed. A primary error is defined in the
analysis as “the first error on a reviewed invoice.” However, an invoice may have more
than one error. Because total errors are not known or identified, the results cannot be
viewed as representative and used to project potential losses to the remaining vouchers
in the population. Hence the measured savings calculated as a result of FEA’s
prospective ability to detect and correct potential contract payment errors under the
proposed plan are unjustified.

We have additional concerns about whether this plan adequately protects the
government’s interest, based on the Department of Defense’s (DOD) long-standing
problems with accurately accounting for and reporting on disbursements. We have
previously reported on long-standing problems in MOCAS contract payments. For
example, recent DFAS data showed that almost $1 of every $4 in contract payment
transactions was for adjustments to previously recorded payments—$49 billion of
adjustments out of $198 billion in transactions. DOD had been developing a new system,
the Defense Payment Procurement System (DPPS), which was designed in part to
resolve DOD’s long-standing disbursements problems and streamline contract payment
processes. However, as we previously reported, DOD terminated DPPS in December

   GAO’s Policy and Procedures Manual for Guidance of Federal Agencies, Title VII, app. III.
   Tolerable error rate is the point at or below which savings occur.
   U.S. General Accounting Office, Canceled DOD Appropriations: $615 Million of Illegal or Otherwise Improper Adjustments, GAO-
01-697 (Washington, D.C.: July 26, 2001).
   U.S. General Accounting Office, DOD Business Systems Modernization: Continued Investment in Key Accounting Systems Needs
to Be Justified, GAO-03-465 (Washington, D.C.: Mar. 28, 2003).

Page 5                                    GAO-03-1067R DOD Voucher Prepayment Examination
2002 after 7 years of development due to poor program performance and increasing life
cycle costs. Consequently, DFAS will be relying on MOCAS, which was implemented in
1968, and other legacy systems to meet its requirements for the foreseeable future.14 In
addition, DOD contracts processed by DFAS contain numerous fund citations and
complex payment allocation terms. These factors, along with contract modifications,
changes in payment allocation terms, and nonstandardized contracts and payment terms
contribute to the fact that a large percentage of payment vouchers will need to be
manually entitled for the foreseeable future. Based on our prior findings, this area has
been very susceptible to errors.

One of the requirements imposed on the certifying officer is to ensure (or have in place a
system that can offer a reasonable degree of assurance of) the “legality of a proposed
payment under the appropriation or fund involved.”15 A certifying officer’s legal liability is
strict and arises automatically at the time of an illegal or improper payment.16 The
Comptroller General has authority to relieve a certifying officer from liability when we
find that the certification was based on official records and the official did not know, and
by reasonable diligence and inquiry could not have discovered, the correct information.

The Federal Managers’ Financial Integrity Act of 1982 (FMFIA) requires agency heads to
periodically evaluate their systems of internal control using the guidance issued by the
Office of Management and Budget (OMB).18 Title VII also provides that FMFIA reviews
should be designed, carried out, and reported in ways that show whether the voucher
processing and disbursing system and controls are reliable. To the extent practicable,
the needs of certifying and approving officials should be considered in advance of an
FMFIA review, and the review results should be made available or communicated to
them in reports tailored to their special interests. The FMFIA review is a good starting
place for assessing risk in the contract pay certification and disbursing process, which,
used in conjunction with an objective and scientifically developed approach
encompassing the criteria19 referred to above, can provide the guidance for alternative
plans that we could evaluate in the future.


Title VII allows for a deviation from the $2,500 limitation for using statistical sampling to
certify vouchers to the extent that there are resultant savings while adequately
protecting the government’s interest. Title VII does not distinguish between certification
of automated or manual entitlements, but provides for statistical sampling of vouchers
less than $2,500 as a technique in large-volume, highly automated voucher processing
operations. Any alternatives to 100 percent prepayment examination must be part of a
total system that includes well-defined responsibilities, organizational structures, and an
adequate set of internal controls. Until a sound statistical sampling plan with justifiable
   U.S. General Accounting Office, DOD Financial Management: Integrated Approach, Accountability, Transparency, and Incentives
Are Keys to Effective Reform, GAO-02-573T (Washington, D.C.: Mar. 20, 2002).
   31 U.S.C. § 3528(a)(3).

   54 Comp. Gen. 112, 114 (1974).
   31 U.S.C. § 3528 (b)(1).
   31 U.S.C. § 3512 (d).
   The criteria include (1) defining the universe of vouchers, (2) determining the size of the sample needed, and (2) analyzing the
results of the sample using the appropriate statistical procedure.

Page 6                                      GAO-03-1067R DOD Voucher Prepayment Examination
savings is presented, we cannot grant a waiver of the $2,500 maximum for statistical
sampling of contract payments or approve the alternative voucher examination plan
submitted. The proposed alternative sampling plan does not justify savings or
adequately protect the government’s interest. In addition, good faith reliance on this
particular plan would not relieve a disbursing or certifying official of liability for an
improper payment or certification of a voucher not audited.

                                          - - - - -

The contents of this report were discussed with Bill Felice of your staff. We hope our
comments are helpful. If you have any questions, or would like to discuss these matters
further, please contact me at (202) 512-6906 or Mary Arnold Mohiyuddin, Assistant
Director, at (202) 512-3087. This report will be available at no charge on GAO’s Web site
at http://www.gao.gov.

Sincerely yours,

McCoy Williams
Financial Management and Assurance


Page 7                         GAO-03-1067R DOD Voucher Prepayment Examination
This is a work of the U.S. government and is not subject to copyright protection in the
United States. It may be reproduced and distributed in its entirety without further
permission from GAO. However, because this work may contain copyrighted images or
other material, permission from the copyright holder may be necessary if you wish to
reproduce this material separately.