oversight

Foster Care Program: Improved Processes Needed to Estimate Improper Payments and Evaluate Related Corrective Actions

Published by the Government Accountability Office on 2012-03-07.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

             United States Government Accountability Office

GAO          Report to Congressional Requesters




March 2012
             FOSTER CARE
             PROGRAM
             Improved Processes
             Needed to Estimate
             Improper Payments
             and Evaluate Related
             Corrective Actions




GAO-12-312
                                             March 2012

                                             FOSTER CARE PROGRAM
                                             Improved Processes Needed to Estimate Improper
                                             Payments and Evaluate Related Corrective Actions
Highlights of GAO-12-312, a report to
congressional requesters




Why GAO Did This Study                       What GAO Found
Each year, hundreds of thousands of          Although ACF has established a process to calculate a national improper
the nation’s most vulnerable children        payment estimate for the Foster Care program, the estimate is not based on a
are removed from their homes and             statistically valid methodology and consequently does not reflect a reasonably
placed in foster care. While states are      accurate estimate of the extent of Foster Care improper payments. In addition,
primarily responsible for providing safe     the estimate deals with only about one-third of the federal expenditures for Foster
and stable out-of-home care for these        Care, and is therefore incomplete. ACF’s methodology for estimating Foster Care
children, Title IV-E of the Social           improper payments was approved by the Office of Management and Budget
Security Act provides federal financial      (OMB) in 2004 with the understanding that continuing efforts would be taken to
support. The Administration for
                                             improve the accuracy of ACF’s estimates of improper payments in the ensuing
Children and Families (ACF) in the
                                             years. ACF, however, continued to generally follow its initial methodology which,
Department of Health and Human
Services (HHS) is responsible for
                                             when compared to federal statistical guidance and internal control standards,
administering and overseeing federal         GAO found to be deficient in all three phases of ACF’s estimation methodology—
funding for Foster Care. Past work by        planning, selection, and evaluation. These deficiencies impaired the accuracy
the HHS Office of Inspector General          and completeness of the Foster Care program improper payments estimate of
(OIG), GAO, and others have identified       $73 million reported for fiscal year 2010.
numerous financial deficiencies
                                             Deficiencies in ACF’s Methodology to Estimate Foster Care Improper Payments
associated with the Title IV-E Foster         Estimation
Care program. GAO was asked to                methodology
determine the extent to which (1)             phase          Deficiencies by phase
ACF’s estimation methodology                  Planning       •    Methodology is limited to identifying improper payments for only one-third of
generated a reasonably accurate and                               the total federal share of foster care expenditures—maintenance payments.
complete estimate of improper                                •    The case-level population data ACF used to derive its foster care improper
                                                                  payment estimate does not contain the associated payment data needed for a
payments across the Foster Care
                                                                  direct estimate of the payment error rate and the total amount of dollars that
program and (2) ACF’s corrective                                  were improperly paid.
actions reduced Foster Care program           Selection      •    ACF has not established up-front data quality procedures over the case-level
improper payments. To complete this                               population data that states report and that ACF uses for improper payments
work, GAO reviewed HHS’s fiscal year                              estimation.
2010 improper payments estimation                            •    Sample selection process includes a high percentage of replacement cases
                                                                  due to inaccurate information contained in the case-level population data.
procedures, conducted site visits, and
                                              Evaluation     •    Methodology does not include procedures on how to identify payment errors
met with cognizant ACF officials.                                 related to underpayments and duplicate payments during the review of
                                                                  sampled cases across states.
What GAO Recommends                                          •    Methodology used to aggregate state-level improper payment data does not
                                                                  take into account each state’s margin of error, which is needed to calculate an
GAO is making seven                                               overall program improper payment estimate with a 90 percent confidence
recommendations to help improve                                   level generally required by OMB guidance.
ACF’s methodology for estimating
                                             Source: GAO analysis of ACF’s methodology to estimate Foster Care improper payments.
improper payments for the Foster Care
program and its corrective action            ACF has reported significantly reduced estimated improper payments for its
process. HHS agreed that its improper        Foster Care maintenance payments, from a baseline of 10.33 percent for fiscal
payment estimation efforts can and           year 2004 to a 4.9 percent error rate for fiscal year 2010. However, the validity of
should be improved, and generally            ACF’s reporting of reduced error rates is questionable. GAO found that ACF’s
concurred with four of the                   ability to reliably assess the extent to which its corrective actions reduced
recommendations and agreed to                improper payments was impaired by weaknesses in its requirements for state-
continue to study the remaining three        level corrective actions. For example, ACF used the number of cases found in
recommendations. GAO reaffirms the           error rather than the dollar amount of improper payments identified to determine
need for all seven recommendations.          whether or not a state was required to implement corrective actions. As such,
View GAO-12-312. For more information,
                                             some states with higher improper payment dollar error rates were not required to
contact Susan Ragland at (202) 512-8486 or   implement actions to reduce these rates. GAO also found deficiencies in ACF’s
raglands@gao.gov.                            Audit Resolution Tracking and Monitoring System that limited its ability to
                                             efficiently track and compare trends across states’ Single Audit findings.
                                                                                                          United States Government Accountability Office
Contents


Letter                                                                                      1
               Background                                                                   5
               ACF’s Methodology for Estimating Foster Care Program Improper
                 Payments Is Not Statistically Valid or Complete                          15
               Validity of Reported Foster Care Program Improper Payment
                 Reductions is Questionable                                               26
               Conclusions                                                                31
               Recommendations for Executive Action                                       32
               Agency Comments and Our Evaluation                                         33

Appendix I     Objectives, Scope, and Methodology                                         37



Appendix II    Title IV-E Expenditures by Type and by State for Fiscal Year 2010          41



Appendix III   Estimated Improper Payment Error Rates by State for Fiscal Year 2010
               Reporting                                                            43



Appendix IV    Comments from the Department of Health and Human Services                  45



Appendix V     GAO Contact and Staff Acknowledgments                                      53



Tables
               Table 1: Eligibility Criteria for the Title IV-E Foster Care Program         6
               Table 2: Deficiencies in ACF’s Methodology to Estimate Foster
                        Care Improper Payments                                            16


Figures
               Figure 1: Title IV-E Federal Share of Expenditures by Type for
                        Fiscal Year 2010                                                   9
               Figure 2: Title IV-E Eligibility Review 3-Year Cyclical Process            12




               Page i                                           GAO-12-312 Foster Care Program
Figure 3: Foster Care Program Outlays for Fiscal Year 2010
         Covered under IPIA Reporting                                                     17
Figure 4: ACF’s Calculation to Derive the National Improper
         Payment Estimate for the Foster Care Program                                     25




Abbreviations

ACF                        Administration for Children and Families
ACR                        Administrative Cost Review
AFCARS                     Adoption and Foster Care Analysis and Reporting
                            System
ARTMS                      Audit Resolution Tracking and Monitoring System
HHS                        Department of Health and Human Services
IPERA                      Improper Payments Elimination and Recovery Act
                            of 2010
IPIA                       Improper Payments Information Act of 2002
OIG                        Office of Inspector General
OMB                        Office of Management and Budget
PIP                        Program Improvement Plan
SACWIS                     Statewide Automated Child Welfare Information
                            System



This is a work of the U.S. government and is not subject to copyright protection in the
United States. The published product may be reproduced and distributed in its entirety
without further permission from GAO. However, because this work may contain
copyrighted images or other material, permission from the copyright holder may be
necessary if you wish to reproduce this material separately.




Page ii                                                   GAO-12-312 Foster Care Program
United States Government Accountability Office
Washington, DC 20548




                                   March 7, 2012

                                   The Honorable Thomas R. Carper
                                   Chairman
                                   The Honorable Scott Brown
                                   Ranking Member
                                   Subcommittee on Federal Financial Management, Government
                                    Information, Federal Services, and International Security
                                   Committee on Homeland Security and Governmental Affairs
                                   United States Senate

                                   The Honorable John McCain
                                   United States Senate

                                   Each year, hundreds of thousands of our nation’s most vulnerable
                                   children are removed from their homes and placed in foster care, often
                                   due to abuse or neglect. While states are primarily responsible for
                                   providing safe and stable out-of-home care for these children until they
                                   are returned safely home, placed with adoptive families, or placed in other
                                   arrangements, Title IV-E of the Social Security Act provides states some
                                   federal financial support in this area. 1 The Administration for Children and
                                   Families (ACF) under the Department of Health and Human Services
                                   (HHS) is responsible for administering this program and overseeing Title
                                   IV-E funds. HHS’s reported fiscal year 2010 outlays to states for their
                                   Foster Care programs under Title IV-E totaled more than $4.5 billion
                                   serving over 408,000 children as of September 30, 2010.

                                   Past work by the HHS Office of Inspector General (OIG), GAO, and
                                   others have identified numerous deficiencies in state claims associated
                                   with the Title IV-E Foster Care program. In particular, the HHS OIG found
                                   hundreds of millions of dollars in unallowable claims associated with Title




                                   1
                                    Codified, as amended, at 42 U.S.C. §§ 670-679c.




                                   Page 1                                             GAO-12-312 Foster Care Program
IV-E funding. 2 A 2006 GAO report also found variations in costs states
claimed under the Title IV-E program and recommended a number of
actions HHS should take to better safeguard federal resources. 3 In
addition, annual state-level audits have identified weaknesses in states’
use of federal funds, such as spending on unallowed activities or costs
and inadequate state monitoring of federal funding. 4

As required under the Improper Payments Information Act of 2002 (IPIA),
as amended, 5 HHS has identified its programs that are susceptible to
significant improper payments, including the Foster Care program. 6 HHS
has reported annually on estimated improper payment amounts for the




2
 Examples of HHS OIG reports include: HHS OIG, Audit of Allegheny County Title IV-E
Foster Care Claims From October 1997 Through September 2002, A-03-08-00554 (Jan.
4, 2011); Review of Title IV-E Foster Care Costs Claimed on Behalf of Delinquent
Children in Georgia, A-04-07-03519, (June 17, 2010); Review of California’s Title IV-E
Claims for Payments Made by Los Angeles County to Foster Homes of Relative
Caregivers, A-09-06-00023 (Oct. 2, 2009); and Philadelphia County’s Title IV-E Claims
Based on Contractual Per Diem Rates of $300 or Less for Foster Care Services from
October 1997 Through September 2002, A-03-07-00560 (May 19, 2008).
3
 GAO, Foster Care and Adoption Assistance: Federal Oversight Needed to Safeguard
Funds and Ensure Consistent Support for States’ Administrative Costs, GAO-06-649
(Washington, D.C.: June 15, 2006).
4
 Examples of state-level audit reports include: California State Auditor, State of California
Internal Control and State and Federal Compliance Audit Report for the Fiscal Year Ended
June 30, 2010 (Sacramento, CA: Mar. 29, 2011); KPMG, Government of the District of
Columbia Schedule of Expenditures of Federal Awards and Reports Required by
Government Auditing Standards and OMB Circular A-133, Year Ended September 30,
2010 (Washington, D.C.: Jan. 27, 2011); and State of Indiana, State Board of Accounts,
State of Indiana Single Audit Report July 1, 2009 to June 30, 2010 (Indianapolis, Ind.:
Feb. 25, 2011).
5
 Pub. L. No. 107-300, 116 Stat. 2350 (Nov. 26, 2002). IPIA was amended by the Improper
Payments Elimination and Recovery Act of 2010, Pub. L. No. 111-204, 124 Stat. 2224
(July 22, 2010). The changes made by this law to IPIA estimation and reporting were first
implemented for fiscal year 2011 reporting.
6
 The act defines an improper payment as any payment that should not have been made
or that was made in an incorrect amount (including overpayments and underpayments)
under statutory, contractual, administrative, or other legally applicable requirements. It
includes any payment to an ineligible recipient, any payment for an ineligible service, any
duplicate payment, payment for services not received, and any payment that does not
account for credit for applicable discounts. Office of Management and Budget (OMB)
guidance also instructs agencies to report payments for which insufficient or lack of
documentation was found as improper payments.




Page 2                                                     GAO-12-312 Foster Care Program
Foster Care program since 2005. 7 In its fiscal year 2010 agency financial
report, HHS reported estimated improper payments for Foster Care of
about $73 million, 8 which contributed to the total governmentwide
improper payment estimate of over $125 billion for that year. 9

Given continuing concerns about financial vulnerabilities in the Foster
Care program, you asked us to conduct a review of the program’s
improper payment estimate and reduction strategies. Specifically, our
objectives were to (1) determine the extent to which ACF’s estimation
methodology generated a reasonably accurate and complete estimate of
Foster Care improper payments and (2) determine the extent to which
ACF’s corrective actions reduced improper payments.

To address these objectives, we reviewed applicable improper payment
legislation, Title IV-E of the Social Security Act, as amended, and HHS
regulations on Title IV-E eligibility reviews, related Office of Management
and Budget (OMB) guidance and ACF’s internal guidance, results from
Title IV-E eligibility reviews, past GAO and OIG reports, and internal
control standards. We also reviewed improper payment information
reported in the Improper Payments Section of HHS’s fiscal year 2010
agency financial report.

To further address the first objective, we performed independent
assessments of ACF’s methodology, including statistical sampling
analysis using OMB’s statistical guidance, GAO guidance, and federal
internal control standards as criteria to determine the accuracy and
completeness of ACF’s reported fiscal year 2010 improper payment




7
  In its fiscal year 2005 performance and accountability report, HHS reported an improper
payment estimate for the Foster Care program for fiscal years 2004 and 2005. According
to HHS, the fiscal year 2004 error rate had not been finalized prior to the issuance of its
fiscal year 2004 performance and accountability report, and thus was not reported in that
publication.
8
 The reported estimate of improper payments in the Foster Care program for fiscal year
2011 is $72 million or 5.3 percent.
9
 GAO, Status of Fiscal Year 2010 Federal Improper Payments Reporting, GAO-11-443R
(Washington, D.C.: Mar. 25, 2011).




Page 3                                                     GAO-12-312 Foster Care Program
estimate for the Foster Care program. 10 As part of this assessment, we
conducted interviews with ACF officials and its contractor to clarify our
understanding of the methodology. We also obtained information about
ACF’s pilot to develop a methodology for estimating improper payments
related to administrative costs for the Title IV-E Foster Care program. The
scope of our review did not include an assessment of individual states’
processes or payment systems.

To further address the second objective, we reviewed agency policies
and procedures and states’ Program Improvement Plans (PIP) used to
address the root causes of improper payments identified from the Title IV-
E eligibility reviews, which ACF conducts to ensure that states are
claiming federal reimbursement only for eligible children; evaluated the
compliance thresholds ACF uses to require states to implement corrective
actions; and conducted interviews with officials from ACF’s central office.
We also obtained information about other ACF monitoring activities over
states, such as its tracking and monitoring of states’ efforts to address the
findings identified in the Single Audits. 11

In addition, we conducted site visits at 3 of ACF’s 10 regional offices
(Philadelphia, PA; Chicago, IL; and San Francisco, CA) where we
interviewed ACF staff on how Title IV-E eligibility reviews are conducted
and how they work with states to implement corrective actions and follow
up on Single Audit findings. We selected these three regional offices for
site visits because they provided oversight of states that collectively
claimed over half of the total federal share of Foster Care maintenance
payments made in fiscal year 2009, the most recent data available at the
time of our review for site visit selection. We also selected these regional




10
  OMB, Standards and Guidelines for Statistical Surveys (September 2006); OMB
Memorandum M-06-23, Issuance of Appendix C to OMB No. Circular A-123 (Aug. 10,
2006); GAO, Inflation Adjustments, (Washington, D.C.: Sept. 14, 2011); Assessing the
Reliability of Computer-Processed Data, GAO-09-680G (Washington, D.C.: July 2009);
Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1
(Washington, D.C.: November 1999); and Using Statistical Sampling, GAO/PEMD-10.1.6
(Washington, D.C.: May 1992).
11
  States, local governments, and non-profit organizations that expend $500,000 or more
in a year in federal awards are to have an audit conducted under the provisions of the
Single Audit Act, 31 U.S.C. §§ 7501-7507. Typically, this takes the form of a “Single
Audit,” which includes audit coverage of both the entity’s financial statements and the
federal awards it receives.




Page 4                                                   GAO-12-312 Foster Care Program
                      offices to achieve variation in the numbers of error cases and amount of
                      disallowed claims found during Title IV-E eligibility reviews.

                      We conducted this performance audit from February 2011 through March
                      2012 in accordance with generally accepted government auditing
                      standards. Those standards require that we plan and perform the audit to
                      obtain sufficient, appropriate evidence to provide a reasonable basis for
                      our findings and conclusions based on our audit objectives. We believe
                      that the evidence obtained provides a reasonable basis for our findings
                      and conclusions based on our audit objectives. See appendix I for
                      additional details on our objectives, scope, and methodology.



Background
Foster Care Program   Title IV-E of the Social Security Act provides for states to obtain federal
                      reimbursement for the costs of their Foster Care programs. 12 While states
                      may provide foster care services to a range of children outlined by state
                      laws and regulations, they may only claim Title IV-E Foster Care funds for
                      children meeting eligibility criteria outlined in the Social Security Act (see
                      table 1).




                      12
                        42 U.S.C. §§ 670-679c. Title IV-E also authorizes funds to states for Adoption
                      Assistance programs, which provide financial support to families who adopt eligible
                      children with special needs from the foster care system.




                      Page 5                                                   GAO-12-312 Foster Care Program
Table 1: Eligibility Criteria for the Title IV-E Foster Care Program

                                                                                                                                                     Social Security
Title IV-E Eligibility Criteria                                                                                                                        Act Sections
The state, Indian tribe, tribal organization, or tribal consortium has a foster care plan approved by the                                         471(a), (b); 479c(b)
Secretary of Health and Human Services that meets defined statutory criteria;
The child was placed in a state-licensed foster family home or child-care institution in accordance with                                        472(a)(1)-(2), (b), (c),
either a voluntary placement agreement between the state agency and the child’s legal guardian(s), or a                                                         and (f)
judicial determination that conditions in the home from which the child was removed were contrary to the
child’s welfare;
A judicial determination for removal of a child must include a determination that:                                                                  472(a)(1) and (2);
•   reasonable efforts have been made to preserve and reunify families by preventing or eliminating the                                                    471(a)(15)
    need for removing the child or making it possible for the child to safely return, or
•   other reasonable efforts have been made in accordance with a plan for the permanent placement of
    the child, if efforts to preserve the family are inappropriate (e.g., in instances of violent crime) or
    determined to be inconsistent with the permanency plan; that conditions in the home from which the
    child was removed were contrary to the child’s welfare and reasonable efforts were made to prevent
    removal;
The child’s placement and care are the responsibility of a state agency (or other public agency acting                                           472(a)(1) and (2)(B)
under an agreement with a state agency), Indian tribe, tribal organization, or tribal consortium operating
under a plan approved by the Secretary of Health and Human Services;
But for the removal of the child from the home, the child would have received aid under the Aid to                                                  472(a)(1) and (3)
Families with Dependent Children program as it was in effect on July 16, 1996, had it been applied for in
the month in which the voluntary agreement or judicial determination was made; and
If removal is the result of a voluntary placement agreement, eligibility terminates 180 days after removal                                         472(d), (e), and (f)
unless a state obtains a judicial determination that continued placement is in the child’s best interest.
                                              Source: Title IV-E of the Social Security Act., codified, as amended, at 42 U.S.C. §§ 670-679c.



                                              Title IV-E authorizes states to receive federal reimbursement for
                                              “maintenance payments” to support expenses for a foster care child, such
                                              as food, clothing, shelter, and school supplies. The federal government
                                              matches the amounts states pay for maintenance costs under the
                                              Medicaid rate. The Medicaid rate varies by state and by year and, for
                                              fiscal year 2010, ranged from 50 to 83 percent. In addition to
                                              maintenance costs, Title IV-E authorizes states to receive reimbursement
                                              for other costs incurred to manage the program. Those other costs and
                                              the allowable reimbursement rates 13 fall under the following three main
                                              categories:

                                              •     Child placement services and other administrative activities
                                                    (administrative costs), which generally cover expenses states incur in


                                              13
                                                  See 42 U.S.C. §§ 672(i), 674(a)(3).




                                              Page 6                                                                                 GAO-12-312 Foster Care Program
                                identifying eligible children, referring them to services, and planning
                                for permanent placement. These can also include administrative costs
                                used to serve foster care “candidate” children, who are at-risk for
                                foster care but still reside in the home. These costs are matched at 50
                                percent.
                           •    State and local training costs (training costs), which are matched at 75
                                percent.
                           •    Statewide Automated Child Welfare Information System (SACWIS)
                                development, installation, and operation costs (SACWIS costs).
                                SACWIS helps states manage their child welfare cases and report
                                related information to the federal government. These SACWIS costs
                                are matched at 50 percent.

                           Since 2002, HHS has also approved states to receive federal
                           reimbursement for demonstration project costs involving the waiver of
                           certain provisions of Title IV-E. 14 The waivers grant states flexibility in the
                           use of Title IV-E foster care funds for “demonstration projects” of
                           alternative services that promote safety, permanency, and well-being for
                           children in the foster care system, so long as the projects do not cost the
                           federal government more than the states would have received under Title
                           IV-E. As of June 2010, nine states have active Title IV-E waiver
                           agreements. 15


Program Trends and Costs   Data from HHS shows that the average number of children served by
                           Title IV-E Foster Care funds has declined, from over 197,000 in fiscal
                           year 2008 to 181,000 in fiscal year 2010.

                           HHS and child welfare experts have cited a number of reasons for this
                           decline. For example, they noted that a child is required to qualify for the
                           Aid to Families with Dependent Children program (a means-tested
                           program based on a federally defined poverty line) as it was in effect on


                           14
                             Section 208 of the Social Security Act Amendments of 1994, Pub. L. No. 103-432, §
                           208, 108 Stat. 4398, 4457 (Oct. 31, 1994), codified, as amended, at 42 U.S.C. § 1320a-9,
                           authorized state demonstration programs for which the Secretary was permitted to waive
                           some Title IV-E requirements. The Adoption and Safe Families Act of 1997, Pub. L. No.
                           105-89, § 301, 111 Stat. 2115, 2127 (Nov. 19, 1997) extended and expanded HHS’s
                           authority to use waivers for child welfare programs, allowing up to 10 new demonstration
                           projects each year.
                           15
                               The nine states that have active Title IV-E waiver agreements are California, Florida,
                           Illinois, Indiana, Iowa, Minnesota, Ohio, Oregon, and Wisconsin.




                           Page 7                                                      GAO-12-312 Foster Care Program
July 16, 1996, in order to be eligible for Title IV-E. Because income limits
for the program have remained static while inflation has raised nominal
incomes for some families, fewer children are eligible. For example, to be
considered as below the federal poverty line, a family comprised of 4
persons, including 2 children, had to have an annual income below
$15,911 in 1996, as compared to $22,113 in 2010. However, the $15,911
threshold continues to be used each year to determine eligibility for the
Title IV-E Foster Care program. In addition, states sometimes use other
federal programs for children who could otherwise have been claimed
under Title IV-E, because the other programs, such as Title XX’s Social
Services Block Grants, Medicaid, and Temporary Assistance to Needy
Families, provide federal reimbursement for a broader range of
services. 16

Of the $4.5 billion in total Title IV-E Foster Care funds paid to states in
fiscal year 2010, ACF reported that maintenance costs made up 34
percent of the total while administrative costs accounted for the largest
share of the costs at 44 percent. Figure 1 shows federal outlays, as
reported by HHS, by type of expenditure for fiscal year 2010. Title IV-E
expenditures by type and by state for fiscal year 2010 are presented in
appendix II.




16
 See GAO-06-649.




Page 8                                            GAO-12-312 Foster Care Program
                       Figure 1: Title IV-E Federal Share of Expenditures by Type for Fiscal Year 2010




Oversight Activities   ACF is responsible for the administration and oversight of Title IV-E
                       funding to states. ACF staff are located in ACF’s headquarters (Central
                       Office) and its 10 regional offices. 17 Collectively, these ACF offices
                       oversee states’ financial internal control processes for the Title IV-E
                       program and monitor their performance and compliance with federal child
                       welfare laws. One key oversight activity related to state Foster Care
                       programs is ACF’s Title IV-E eligibility reviews, as required under the
                       Social Security Act and HHS regulations. 18 ACF has conducted these
                       reviews since 2000. They are intended to help evaluate whether state
                       claims for federal reimbursement for Foster Care maintenance costs are
                       valid and accurate. Title IV-E eligibility reviews are to be conducted by
                       teams composed of both federal and state staff, and are to include (1)
                       desk reviews to ensure that the correct amount of maintenance costs was
                       claimed on behalf of foster care children during the review period and (2)




                       17
                         ACF regional offices are located in Boston, MA; New York, NY; Philadelphia, PA;
                       Atlanta, GA; Chicago, IL; Dallas, TX; Kansas City, MO; Denver, CO; San Francisco, CA;
                       and Seattle, WA.
                       18
                        42 U.S.C. § 1320a-2a; 45 C.F.R. § 1356.71.




                       Page 9                                                 GAO-12-312 Foster Care Program
site visits to states to ensure that maintenance costs were claimed only
for children who were eligible for the Title IV-E program.

As required by the Social Security Act and HHS regulations, there are two
stages of Title IV-E eligibility reviews, a primary and secondary review.
During a primary review, HHS regulations specify that the review team is
to examine a sample of 80 cases per state, selected from the Adoption
and Foster Care Analysis and Reporting System (AFCARS). 19 Each case
represents a child for whom a Title IV-E Foster Care maintenance
payment was made. If a primary review finds fewer than 5 cases with
errors in either the amounts paid on behalf of a child or in a child’s
eligibility for Title IV-E funds (5 percent of the cases reviewed or fewer),
ACF determines that the state is in substantial compliance with the
regulations. At that point, the state is scheduled to have another primary
review in 3 years. 20 On the other hand, if a primary review finds 5 or more
cases are in error (exceeding 5 percent of the number of cases
reviewed), ACF determines that the state is not in substantial compliance
with the regulations. In those instances, ACF requires states to develop a
Program Improvement Plan (PIP) designed to correct the areas of
noncompliance identified, such as payments to unlicensed providers or
incomplete criminal record checks. Any improper payments the review
teams identify during these reviews are classified as disallowed costs




19
  AFCARS is the federal information system that collects and processes data on children
in foster care and those who have been adopted under the auspices of state child welfare
agencies. AFCARS serves as central depository of various nationwide data on the foster
care program, as required by the Title IV-E legislation. ACF uses this system for, among
other purposes, determining and assessing outcomes for children and families, budget
planning and projections, and targeting areas for greater or potential technical assistance
efforts. The data in AFCARS are self-reported and maintained by the states, and are
subject to information system assessment reviews and federally mandated edit checks by
ACF.
20
  According to ACF, HHS employs a 10 percent error threshold as part of its sampling
methodology to determine the level of state compliance in meeting the federal
requirements in the Foster Care program. If during a primary review, in which 80 cases
are reviewed, 4 or fewer cases are found to be in error, HHS can be 91 percent certain
that no more than 10 percent of the entire population of Title IV-E Foster Care cases will
be in error.




Page 10                                                    GAO-12-312 Foster Care Program
that, in general, are to be returned to ACF or withheld from future
reimbursement claims. 21

States required to develop a PIP generally have 1 year to implement the
corrective actions specified in the PIP, after which a secondary review is
to be conducted. 22 During the secondary review, the review team is to
examine a sample of 150 cases, as outlined in HHS regulations. If 10
percent of cases or fewer are found to be in error and if the total dollar
amount found to be in error is less than 10 percent of the total dollar
amount reviewed, then ACF determines that the state is in substantial
compliance. Further, if the state exceeded only one of these secondary
review error thresholds, then ACF would also determine that the state is
in substantial compliance. Only in instances where the state exceeds both
the case percentage and dollar percentage error thresholds of 10 percent
would ACF determine that the state is not in substantial compliance. HHS
regulations require such a state to repay a disallowance percentage
applied to its Title IV-E claims during the review period. After conducting a
secondary review, ACF would then schedule another primary review in 3
years (see figure 2).




21
  States have the right to appeal disallowance decisions to HHS’s Departmental Appeals
Board. HHS’s Departmental Appeals Board is responsible for providing impartial,
independent review of disputed decisions in a wide range of HHS programs. It generally
issues final decisions on behalf of HHS, which may then be appealed to federal court.
22
  Given the time it takes to develop and implement a PIP, a secondary review is typically
conducted 2 to 3 years after the primary review.




Page 11                                                   GAO-12-312 Foster Care Program
Figure 2: Title IV-E Eligibility Review 3-Year Cyclical Process




                                           Another key ACF oversight activity related to state Foster Care programs
                                           is the monitoring of findings from state-level audits conducted under the
                                           Single Audit Act and OMB Circular No. A-133, known as Single Audits. 23
                                           The Single Audit Act requires an annual audit of states, local
                                           governments, and non-profit organizations that expend $500,000 or more
                                           of federal funds in a given year. ACF regional offices are to work with
                                           states to resolve Single Audit findings related to the Foster Care program
                                           to help ensure that states are using funds in accordance with program
                                           requirements and addressing financial management weaknesses. ACF
                                           started using the Audit Resolution Tracking and Monitoring System
                                           (ARTMS) in 2010 to provide online processing and real-time tracking of
                                           ACF’s audit follow-up process. The National External Audit Review
                                           Center is a specialized function of the HHS OIG that serves as a


                                           23
                                             OMB Circular No. A-133, Audits of States, Local Governments and Non-Profit
                                           Organizations (includes revisions published in the Federal Register on June 27, 2003, and
                                           June 26, 2007).




                                           Page 12                                                  GAO-12-312 Foster Care Program
                          clearinghouse to determine which state Single Audit report findings HHS
                          is responsible for resolving. When ACF receives Single Audit finding data
                          from the OIG’s National External Audit Review Center, HHS headquarters
                          staff upload the data into ARTMS and assign the audit finding data to the
                          appropriate ACF regional office staff for resolution. Consistent with OMB
                          Circular No. A-50, ACF considers an audit finding resolved when the
                          auditor and the state agree on action to be taken. 24 ARTMS provides
                          users with notifications of tasks to be performed, such as when an audit is
                          assigned to a financial management specialist for follow-up, and allows
                          them to submit and view all audit resolution information online.


Improper Payments         The Improper Payments Information Act (IPIA) was enacted in November
Information Act of 2002   2002 to enhance the accuracy and integrity of federal payments. 25 IPIA
and Implementing          requires agencies to:
Guidance                  •     review all programs and activities and identify those that are
                                susceptible to significant improper payments;
                          •     obtain a statistically valid estimate of the annual amount of improper
                                payments, including the gross total of over- and underpayments, in
                                those susceptible programs and activities;
                          •     report to the Congress estimates of the annual amount of improper
                                payments in their susceptible programs and activities and;
                          •     for estimates exceeding $10 million, implement a plan to reduce
                                improper payments.

                          OMB’s implementing guidance for IPIA, in effect for fiscal year 2010
                          reporting, required that for any programs and activities identified as
                          susceptible to significant improper payments, agencies must develop a
                          statistically valid methodology, or other methodology approved in
                          advance by OMB, to estimate the annual amount of improper payments,
                          including a gross total of both underpayments and overpayments. 26 The
                          Foster Care program was deemed a risk susceptible program and



                          24
                              OMB Circular No. A-50, Audit Followup (Sept. 29, 1982).
                          25
                              Pub. L. No. 107-300, 116 Stat. 2350 (Nov. 26, 2002).
                          26
                            OMB Memorandum M-06-23, Issuance of Appendix C to OMB Circular A-123 (Aug. 10,
                          2006). OMB has updated this guidance to reflect changes made by the Improper
                          Payments Elimination and Recovery Act of 2010, Pub. L. No. 111-204, 124 Stat. 2224
                          (July 22, 2010); these updates first took effect during the fiscal year 2011 reporting cycle.




                          Page 13                                                     GAO-12-312 Foster Care Program
therefore required to address the IPIA reporting requirements. 27 OMB
guidance also requires that, as part of their plan to reduce improper
payments for all programs and activities with improper payments
exceeding $10 million, agencies identify the reasons their programs and
activities are at risk of improper payments (also known as root causes),
set reduction targets for future improper payment levels and a timeline
within which the targets will be reached, and ensure that agency
managers and accountable officers are held accountable for reducing
improper payments.

ACF annually reports to HHS—for inclusion in its agency financial report
used to report to the Congress—an improper payment estimate for Foster
Care program maintenance payments based on results of Title IV-E
eligibility reviews required under the Social Security Act. For programs
administered at the state level such as Foster Care, OMB guidance
provides that statistically valid estimates of improper payments may be
provided at the state level either for all states or for all sampled states
annually. These state-level improper payment estimates should then be
used to generate a national dollar estimate and improper payment rate.
With prior OMB approval, ACF has taken its existing Title IV-E eligibility
review process, already in place under the Social Security Act, and
leveraged it for IPIA estimation. OMB granted this approval in December
2004 with the expectation that continuing efforts would be taken to
improve the accuracy of ACF’s estimates of improper payments in the
ensuing years. ACF provides a national estimated error rate based on a
rolling average of error rates identified in states examined on a 3-year
cycle. As a result, ACF’s IPIA reporting for each year is based on new
data for about one-third of the states and previous years’ data for the
remaining two-thirds of states. While each state sample represents a
distinct 6-month period under review, the national “composite sample”
reflects a composite period under review that encompasses a 3-year
period (the Title IV-E eligibility review 3-year improper payment cyclical
process). To calculate a national estimate of improper payments, ACF


27
   Prior to IPIA, OMB identified seven agency programs within HHS, including the Foster
Care program, as being susceptible to significant improper payments. The other six high
risk programs within HHS were Head Start, Medicare, Medicaid, Temporary Assistance for
Needy Families, State Children’s Insurance Program, and the Child Care and
Development Fund. Because OMB deemed these programs as high risk, HHS was
required to report estimated improper payments internally to OMB. After IPIA became
effective in fiscal year 2004, HHS began reporting improper payment information on
Foster Care and the other programs in its annual agency financial report.




Page 14                                                GAO-12-312 Foster Care Program
                          uses error rates that span a 3-year period of Title IV-E eligibility reviews in
                          the 50 states, the District of Columbia, and Puerto Rico. ACF applies the
                          percentage dollar error rate from the sample to the total payments for the
                          period under review for each state. Improper payment error rates by state
                          for fiscal year 2010, as calculated by ACF, can be found in appendix III.

                          On July 22, 2010, the Improper Payments Elimination and Recovery Act
                          of 2010 (IPERA) was enacted. IPERA amended IPIA, and established
                          additional requirements related to federal agency management
                          accountability, compliance and noncompliance determinations based on
                          an Inspector General’s assessment of an agency’s adherence to IPERA
                          requirements and reporting that determination, and an opinion on internal
                          controls over improper payments. Specifically, one new IPERA provision
                          calls for federal agencies’ Inspectors General to annually determine
                          whether their respective agencies are in compliance with key IPERA
                          requirements such as meeting annual reduction targets for each program
                          assessed to be at risk of and measured for improper payments, and to
                          report on their determinations to the agency head, the Congress, and the
                          Comptroller General.


                          ACF’s methodology, which resulted in a reported $73 million (or 4.9
ACF’s Methodology         percent) estimate of improper payments in the Foster Care program for
for Estimating Foster     fiscal year 2010, 28 had deficiencies in all three phases of its estimation
Care Program              methodology—planning, selection, and evaluation—when compared to
                          OMB’s statistical guidance, GAO guidance, and federal internal control
Improper Payments Is      standards, as summarized in table 2. Specifically, ACF’s estimation
Not Statistically Valid   methodology (1) did not consider nearly two-thirds of reported federal
                          Foster Care program payments for fiscal year 2010, (2) was not based on
or Complete               a probability sample of payments, (3) lacked specific procedures for
                          identifying underpayments and duplicate payment errors, and (4) used a
                          flawed process for aggregating state-level data into an overall national
                          error rate. As a result, ACF’s methodology is not statistically valid or
                          complete, and these deficiencies impair the accuracy and completeness
                          of its reported Foster Care program improper payment estimate.




                          28
                            The $73 million estimate of improper payments in the Foster Care program for fiscal
                          year 2010 consists of $9 million in underpayments and $64 million in overpayments.




                          Page 15                                                  GAO-12-312 Foster Care Program
Table 2: Deficiencies in ACF’s Methodology to Estimate Foster Care Improper Payments

Estimation
methodology phase               Deficiencies by phase
Planning                        •   Methodology is limited to identifying improper payments for only one-third of the total federal
                                    share of foster care expenditures—maintenance payments.
                                •   The case-level population data used to derive the foster care improper payment estimate
                                    does not contain the associated payment data needed for a direct estimate of the payment
                                    error rate and the total amount of dollars that were improperly paid.
Selection                       •   ACF has not established up-front data quality procedures over the case-level population
                                    data, self-reported by states, prior to sample selection.
                                •   Sample selection process includes a high percentage of replacement cases due to
                                    inaccurate information contained in the case-level population data.
Evaluation                      •   Methodology does not include procedures on how to identify payment errors related to
                                    underpayments and duplicate payments during the review of sampled cases across states.
                                •   Methodology used to aggregate state-level improper payment data does not take into
                                    account each state’s margin of error, which is needed to calculate an overall program
                                    improper payment estimate with a 90 percent confidence level generally required by OMB
                                    guidance.
                                       Source: GAO analysis of ACF’s methodology to estimate Foster Care improper payments.




Methodology Excludes                   ACF’s annual IPIA reporting for the Foster Care program is incomplete,
Most Foster Care Program               as it is limited to identifying improper payments for only one type of
Expenditures from                      program payment activity—maintenance payments. For fiscal year 2010,
                                       as shown in figure 1 of this report, maintenance payments represented 34
Estimate                               percent of the total federal share of expenditures for the Foster Care
                                       program. Administrative and other payments were not considered in
                                       ACF’s IPIA estimation process and thus, not included in the Foster Care
                                       program improper payment estimate of about $73 million for fiscal year
                                       2010. 29 Administrative costs accounted for 44 percent of the total federal
                                       share of expenditures for the Foster Care program, while other costs
                                       accounted for the remaining 22 percent. 30 These other costs include
                                       operational and development costs associated with SACWIS; training


                                       29
                                        According to ACF, the range for the national improper payment estimate for the Foster
                                       Care program is $68.0 to $77.3 million, at a confidence interval of 90 percent.
                                       30
                                          Administrative payments (which are based on a state’s average monthly Title IV-E
                                       administrative cost claims per case) make up a large portion of Title IV-E expenditures
                                       and can vary greatly across states. For example, for fiscal year 2010, administrative
                                       payments represent 71 percent of the total federal reimbursements for Alabama and North
                                       Carolina compared to 13 percent for West Virginia. Administrative costs are not subject to
                                       a maximum limit for reimbursement. The federal government generally reimburses the
                                       states for 50 percent of eligible administrative costs with no limit.




                                       Page 16                                                                         GAO-12-312 Foster Care Program
costs; and state demonstration projects to provide alternative services
and support for children in the Foster Care system. Figure 3 shows the
portion of Foster Care program outlays considered in ACF’s methodology
for estimating associated improper payments for IPIA reporting.

Figure 3: Foster Care Program Outlays for Fiscal Year 2010 Covered under IPIA
Reporting




Because ACF’s methodology does not include an estimate for improper
payments related to its administrative payment activity, the related
payment errors that meet the definition of improper payments were not
accounted for or included in the reported estimate for the Foster Care
program. OMB’s December 2004 approval of ACF’s proposed
methodology included an expectation that ACF would develop a plan and
timetable to test administrative expenses by April 2005.

Consistent with this expectation, in order to begin exploring the issues of
accounting for and including administrative costs, ACF established a
working group in 2006. Then, in 2007, ACF initiated an Administrative
Cost Review (ACR) pilot to examine how certain state agencies
accumulate costs that are included in their expenditure claims for federal
financial participation 31 and to identify improper administrative payments
within those pilot states. 32 Seven states volunteered for pilots held from


31
 Federal financial participation is the federal government’s share of expenditures made
by an entity in implementing a program administered by the agency.
32
  ACF does not currently have any initiatives under way to estimate improper payments
related to SACWIS, training, or demonstration projects.




Page 17                                                  GAO-12-312 Foster Care Program
                            fiscal years 2007 through 2011, and two more states are scheduled for
                            fiscal year 2012. 33 Pilot reports for two states have provided estimates for
                            a gross improper payment total of $11.3 million for the period October 1,
                            2008, through March 31, 2009. These amounts were not included in
                            ACF’s estimated amounts for improper payments. According to ACF, it
                            will use the results of the ACR pilots to determine the feasibility of
                            developing a methodology to estimate an administrative error rate as part
                            of the calculation of the national Foster Care improper payment error rate.
                            However, as of December 15, 2011, ACF had not yet made a decision
                            with respect to when these reviews would be implemented and,
                            ultimately, whether to establish a methodology to estimate improper
                            payments related to administrative costs.

                            Although ACF did not consider Foster Care administrative expenditures in
                            its fiscal year 2010 IPIA estimation process, its Title IV-E eligibility
                            reviews identified disallowed administrative costs (or improper payment
                            amounts) which were added to the amount of any claims disallowances.
                            For fiscal year 2010, disallowed administrative costs that ACF
                            documented from the Title IV-E eligibility reviews totaled $2.4 million;
                            however, this amount was not included in its Foster Care improper
                            payment estimate. According to ACF, administrative payments are not
                            currently included as part of the reported improper payment estimate
                            because this disallowed amount is based on a calculation and not directly
                            determined from a case file review. ACF calculates the administrative
                            cost disallowance by allocating an average administrative cost for any
                            ineligible time period identified during a Title IV-E eligibility review.


ACF’s Methodology for       The methodology ACF used to estimate improper maintenance payments
Selecting Cases to Review   was not based on a probability sample of payments, which is needed for
Is Also Deficient           a direct estimate of the payment error rate and total amount of dollars that
                            were improperly paid. 34 In 2004, ACF proposed three options to OMB as
                            approaches for estimating improper payments in the Foster Care


                            33
                             The seven states where ACF has conducted ACR pilots are Arizona, Texas, and New
                            York in fiscal year 2007; Iowa and Wisconsin in fiscal year 2009; and New Mexico and
                            North Carolina in fiscal year 2011; the two states scheduled for fiscal year 2012 are New
                            Hampshire and Mississippi.
                            34
                              A probability sample is defined as a sample from a population selected by some random
                            method such that each item in the population has a known, nonzero probability of being
                            drawn that can be calculated.




                            Page 18                                                   GAO-12-312 Foster Care Program
program. OMB approved ACF’s plan to derive the estimate using error
dollars per case from state review samples for its base error rate
calculation, with the expectation that continuing attention to the statistical
processes used would be needed to obtain the best estimate of
erroneous payment rates. OMB’s approval reflected the idea that the
methods initially used would incorporate annual improvements to the
accuracy of improper payment estimates. However, other than a change
in 2008 to derive the estimate using sample state dollar error rates, ACF
generally continues to use the same methodology outlined in 2004.

We found that ACF selected a sample from a universe of all cases
receiving Title IV-E Foster Care payments during the period under review.
This population of Foster Care cases is drawn from AFCARS. However,
AFCARS does not contain any Title IV-E financial data that links a Title
IV-E payment amount to each case file. Lacking such payment data, ACF
relies on states to provide payment histories for all cases selected for
review. According to ACF, it was already using AFCARS to select
samples for the Title IV-E eligibility reviews prior to IPIA implementation.
Consistent with OMB’s approval of its methodology, ACF opted to use
AFCARS data instead of creating or identifying a new data source for
meeting IPIA requirements. ACF officials stated that utilizing this existing
source of data reduced the burden on states by not requiring them to
draw their own samples and employed the AFCARS database in a
practical manner.

Notwithstanding ACF’s objective of leveraging existing data to estimate
foster care improper payments for addressing IPIA requirements, if ACF
is to make an estimate that accurately represents the target population of
Title IV-E payments, a more direct statistical method would be needed to
select a probability sample of Title IV-E payments. OMB’s Standards and
Guidelines for Statistical Surveys documents the professional principles
and practices that federal agencies are required to adhere to and the
level of quality and effort expected in all statistical activities. 35 According
to these standards and guidelines, probabilistic methods for survey
sampling are one of a variety of methods for sampling that give a known,
non-zero, probability of selection to each member of the target population.
The advantage of probabilistic sampling methods is that sampling error
can be calculated. For the purpose of making a valid estimate with a



35
 OMB, Standards and Guidelines for Statistical Surveys (Sept. 2006).




Page 19                                                GAO-12-312 Foster Care Program
measurable sampling error that represents a population, the sample must
be selected using probabilistic methods. The sample results can then be
used to make an inference about the target population, in this instance,
foster care cases that received a maintenance payment.

While it is possible for ACF to estimate a payment error rate and the total
amount of dollars improperly paid for maintenance payments using a
combination of AFCARS and supplemental payment data from the states,
this would require a more complex estimation methodology than ACF
currently uses. Based on our review of the sampling documentation
provided, ACF did not consider key factors such as variation in volume of
payments and dollars of payments across cases and states. In addition,
the population of data that ACF used to select its sample from is not
reliable because ACF’s sampling methodology did not provide for up-front
data quality control procedures to (1) ensure that the population of cases
was complete prior to its sample selection and (2) identify inaccuracies in
the data field used for sample selection. 36

During our review, we found that the population of Foster Care cases
from AFCARS contained inaccurate information on whether a case had
actually received a Title IV-E Foster Care maintenance payment during
the period under review, reflecting continuing concerns regarding the
accuracy and completeness of AFCARS data. Specifically, ACF had to
replace a high percentage of cases sampled from the database of Foster
Care cases for the fiscal year 2010 reporting period due to inaccurate
information in AFCARS. To ensure that a sufficient number of relevant
sample items are available for review, ACF routinely selects an “over-
sample” of cases—cases selected in addition to the required 80 or 150
cases initially selected for the primary and secondary reviews.

Of the original 4,570 sample cases ACF selected for testing in its primary
and secondary reviews for fiscal year 2010, 298 cases (almost 7 percent)
had to be replaced with substitutes taken from the “over-sampled” cases


36
  The data contained in AFCARS is self-reported by the states. ACF relies on existing
AFCARS Assessment Reviews which evaluate states’ automated information systems for
collecting, extracting, and submitting AFCARS data, and assess the accuracy of data
collected in accordance with the regulation requirements as well as ACF policy and
technical guidance. However, these reviews are not conducted annually for all states and
do not address verifying the accuracy or completeness of the data element—#59 Title IV-
E (Foster Care)—ACF uses to develop its population of foster care cases for estimating
improper payments.




Page 20                                                 GAO-12-312 Foster Care Program
because the selected cases had not received Title IV-E Foster Care
maintenance payments during the period under review. Of the 298 over-
sampled cases used to replace the cases initially selected, 63 cases
(more than 21 percent) then had to be replaced again because those
cases had also not received Title IV-E Foster Care maintenance
payments during the period under review. Although we were able to
determine how many sampled (or over-sampled) cases had to be
replaced because available records showed no Title IV-E payment was
received during the reporting period, neither we nor ACF were able to
determine the extent to which the opposite occurred—the extent to which
cases that had received a payment (and therefore should have been
included in the sample population) had not been coded as receiving Title
IV-E payments. As part of its sampling methodology, ACF has not
established procedures to identify any such occurrences. Therefore, ACF
could not determine whether its sampling universe was complete, i.e.,
whether all of the cases receiving a Foster Care payment were included
in the universe of cases from which it selected sample cases for review.

According to GAO’s Assessing the Reliability of Computer-Processed
Data, reliable data are defined as data that are reasonably complete and
accurate, meet intended purposes, and are not subject to inappropriate
alteration. 37 “Completeness” refers to the extent to which relevant records
are present and the fields in each record are populated appropriately.
“Accuracy” refers to the extent to which recorded data reflect the actual
underlying information. GAO’s Internal Control Management and
Evaluation Tool provides that reconciliations should be performed to
verify data completeness. 38 Also, data validation and editing should be
performed to identify erroneous data. Erroneous data should be captured,
reported, investigated, and promptly corrected.

ACF officials told us they are aware that AFCARS does not contain Title
IV-E payment data and acknowledged that they do not perform
procedures to identify incorrect or missing information in the population
prior to sample selection. However, ACF officials said they continue to
use the data to meet IPIA reporting requirements because it is the only
database that contains case-level information on all children in foster care


37
 GAO-09-680G.
38
 GAO, Internal Control Management and Evaluation Tool, GAO-01-1008G (Washington,
D.C.: August 2001).




Page 21                                            GAO-12-312 Foster Care Program
                              for whom the state child welfare agencies have responsibility for
                              placement, care, or supervision. However, without developing a
                              statistically valid sampling methodology that incorporates up-front data
                              quality controls to ensure complete and accurate information on the
                              population, including payment data, ACF cannot provide assurance that
                              its reported improper payment estimate accurately and completely
                              represents the extent of improper maintenance payments in the Foster
                              Care program.


ACF’s Methodology Does        In its fiscal year 2010 agency financial report, ACF reported that
Not Include Procedures to     underpayments and duplicate or excessive payments represented 25
Identify the Full Extent of   percent of the errors that caused improper payments. 39 While ACF’s
                              methodology for performing Title IV-E eligibility reviews included written
Errors Related to
                              guidance and a data collection instrument that focused on eligibility
Underpayments and             errors, it did not include procedures on how to search for and identify
Duplicate or Excessive        payment errors related to underpayments and duplicate or excessive
Payments                      payments during case reviews. Rather, ACF’s procedures only provided
                              that any observed underpayments and duplicate or excessive payments
                              are to be disclosed as findings in the state’s final eligibility review report.
                              Without detailed procedures to guide review teams in the identification of
                              underpayments and duplicate or excessive payments, ACF’s
                              methodology cannot effectively assure its review team identifies the full
                              extent to which any such underpayments or duplicate or excessive
                              payments exist in the Foster Care program.

                              As defined in IPIA, improper payments include both overpayments and
                              underpayments, and an agency’s estimate should reflect both types of
                              errors. IPIA also includes examples of improper payments, one of which
                              is duplicate payments. According to GAO’s Standards for Internal Control
                              in the Federal Government, operational information is needed to
                              determine whether the agency is achieving its compliance requirements
                              under various laws and regulations. 40 Information is required on a day-to-


                              39
                                 The other types of errors identified related to eligibility. These included providers not
                              licensed or approved, ineligible payments (e.g., therapy), a child not being eligible under
                              the Aid to Families with Dependent Children program at the time of removal, criminal
                              records check not completed, judicial determination regarding reasonable efforts to
                              finalize permanency plan not timely, and no judicial determination of reasonable efforts to
                              prevent removal.
                              40
                               GAO/AIMD-00-21.3.1.




                              Page 22                                                    GAO-12-312 Foster Care Program
day basis, to make operating decisions, monitor performance, and
allocate resources. Pertinent information should be identified, captured,
and distributed in a form and time frame that permits people to perform
their duties efficiently.

ACF compiles the results of all state eligibility reviews to determine the
most common types of payment errors. ACF officials told us that all
review team members receive the same training and the results of the
state reviews are analyzed to ensure consistency and reliability. For fiscal
year 2010, underpayments were the largest percentage of payment errors
(19 percent—as a percentage of all Title IV-E maintenance payment
errors identified in sampled cases). Duplicate or excessive payments
comprised 6 percent of the payment errors. However, the extent of
underpayments and duplicate or excessive payment errors identified
varied widely by state, and in some instances were not identified at all. 41
The lack of detailed procedures for identifying any such payment errors
may have contributed to the variation or whether the teams found any
errors. For example, our analysis of the Title IV-E eligibility reviews that
comprised the fiscal year 2010 foster care improper payment estimate
identified underpayments in 21 of 51 state reviews. Of the 21 states
where reviewers had identified underpayments, such payments ranged
from 1.3 percent to 12.0 percent of cases reviewed. Similarly, duplicate or
excessive payments were identified in only 16 of 51 states. Of the 16
states that had this type of error, these payments ranged from 1.3 percent
to 5.0 percent.

During our site visits, ACF regional officials told us that states have
differing claiming practices for certain expenses. Specifically, officials in
one regional office said that if a child became eligible during a specific
month, a state could have claimed through the first day of the month for
that child, but chose not to so as to not risk a potential error on a future
Title IV-E eligibility review. According to these regional officials, the
regional offices operate under the presumption that “if a state made the
decision not to claim certain expenses, then the failure to claim is not
considered an underpayment.” These types of decisions would be



41
  ACF did not identify underpayments in 31 of 51 state eligibility reviews and did not
identify duplicate or excessive payments in 36 of 51 state eligibility reviews. While we did
not assess the validity of the reported findings in these states, total absence of such errors
seems inconsistent with the general distribution of errors reported elsewhere. Also, Florida
was excluded from review due to a statewide waiver demonstration project.




Page 23                                                     GAO-12-312 Foster Care Program
                             discussed during the reviews but ACF guidance does not call for
                             decisions and their rationale to be formally documented. According to
                             ACF’s Title IV-E eligibility review guide, potential underpayments are to
                             be identified during a review of the case record and payment history.
                             ACF’s Title IV-E eligibility review guide provides that payment histories
                             should be submitted, but it does not specify what criteria reviewers are to
                             look for in order to determine instances of underpayments or duplicate or
                             excessive payments. In August 2011, ACF issued a new attachment to
                             the Title IV-E eligibility review guide to provide a tool for calculating and
                             reporting underpayments identified during eligibility reviews. This new
                             attachment provides a template for recording underpayments for the
                             period under review, but it does not provide guidance on how to identify
                             underpayments. Instances of duplicate or excessive payments are to be
                             reported on other existing attachments in the Title IV-E eligibility review
                             guide. However, none of these attachments offer additional guidance for
                             how to identify national underpayments or duplicate or excessive
                             payments.


ACF’s Aggregation            ACF’s fiscal year 2010 Foster Care program improper payment estimate
Methodology Does Not         did not appropriately aggregate state improper payment data to derive a
Generate a National Foster   national improper payment estimate (dollars and error rate). ACF
                             calculated the national estimate of improper payments each year using
Care Program Improper        data collected in the most recent eligibility review for each of 50 states,
Payment Estimate             the District of Columbia, and Puerto Rico. According to the information
                             ACF presented to OMB in December 2004, ACF’s methodology would
                             calculate the standard error of each state estimate, and of the national
                             estimate, to examine the extent to which the precision requirements as
                             specified in OMB’s implementing guidance for IPIA are met. However, the
                             methodology ACF actually used to aggregate this state-level improper
                             payment data does not take into account each state’s margin of error,
                             which is needed to calculate an overall program improper payment
                             estimate with a 90 percent confidence level generally required by OMB
                             guidance. Figure 4 depicts, at a high level, ACF’s calculation to derive the
                             national improper payment estimate for the Foster Care program.




                             Page 24                                          GAO-12-312 Foster Care Program
Figure 4: ACF’s Calculation to Derive the National Improper Payment Estimate for the Foster Care Program




                                        a
                                         A confidence interval is a statistical sample-based estimate expressed as a range of values, for
                                        example, $100 plus or minus $5. The sample is designed such that there is a specified confidence
                                        level for which the population value is being estimated is expected to be located within the interval.
                                        b
                                         OMB’s implementing guidance for IPIA requires agencies to consult a statistician to ensure the
                                        validity of their sample design, sample size, and measurement methodology. Estimates are to be
                                        based on a statistically random sample of sufficient size to yield an estimate with a 90 percent
                                        confidence interval of plus or minus 2.5 percentage points. Agencies may alternatively use a 95
                                        percent confidence interval of plus or minus 3 percentage points. However, the guidance also allows
                                        agencies to propose alternate sampling approaches for OMB approval prior to implementation.
                                        c
                                         GAO, Inflation Adjustments, (Washington, D.C.: Sept. 14, 2011).




                                        Page 25                                                            GAO-12-312 Foster Care Program
                         ACF has reported significantly reduced estimated improper maintenance
Validity of Reported     payments, from a baseline error rate of 10.33 percent for 2004 to a 4.9
Foster Care Program      percent error rate for 2010, but the validity of ACF’s reporting of reduced
                         improper payment error rates is questionable. Examples of corrective
Improper Payment         actions ACF has identified include reviews, the requirement for state
Reductions is            improvement plans, on-site training and technical assistance to states,
Questionable             and outreach to judicial organizations to educate them as to their role in
                         addressing Foster Care eligibility issues. However, the significant
                         weaknesses discussed previously concerning ACF’s estimation
                         methodology impaired the accuracy and completeness of ACF’s reported
                         improper payment estimate for the Foster Care program. Further, we
                         found that ACF’s ability to reliably assess the extent to which its
                         corrective actions reduced Foster Care program improper payments was
                         impaired by deficiencies in (1) its method for requiring when states
                         implement corrective actions and (2) information technology limitations
                         related to monitoring states’ Foster Care program-related Single Audit
                         findings.


Deficiencies in ACF’s    We identified three deficiencies in ACF’s process for implementing plans
Requirements for State   to reduce improper payments.
Foster Care Program
                         •   ACF did not use reported improper payment error rates—which are
Improper Payment             based on the dollar amount of improper payments identified in a
Corrective Actions           sample of state Foster Care cases—to determine whether or not a
                             state is required to implement corrective actions.
                         •   ACF’s measure for assessing corrective action effectiveness is
                             through performance on its secondary review of Title IV-E cases,
                             which has a more lenient passing standard than the primary review.
                         •   Not all types of payment errors are required to be addressed in the
                             PIP.

                         OMB’s implementing guidance for IPIA requires that agencies put in place
                         a corrective action plan to reduce improper payments. In addition, ACF’s
                         internal guidance requires states to implement corrective actions through
                         a PIP if, during the Title IV-E primary eligibility review, a state is found to
                         have 5 or more cases in error (exceeding 5 percent of the number of
                         cases reviewed). While ACF identifies state PIPs as a corrective action
                         strategy, it does not use the dollar-based estimated improper payments to
                         determine when a state is required to develop a PIP. Instead, ACF uses
                         the number of sample cases found in error to determine which states
                         should develop a PIP. Therefore, some states with improper payment
                         dollar error rates exceeding 5 percent were not required to implement



                         Page 26                                           GAO-12-312 Foster Care Program
corrective actions to reduce these rates. For fiscal year 2010 reporting,
ACF used the results of 44 primary eligibility reviews and 7 secondary
reviews. 42 Of the 44 state primary reviews, 13 had dollar-based estimated
improper payments greater than 5 percent; however, because ACF uses
case error rates as the determining factor for states’ compliance with their
primary reviews, not all states were required to complete a PIP. Of the 13
states, ACF determined 7 were noncompliant in their primary eligibility
reviews because the case error rate exceeded ACF’s threshold of 5
percent (more than 4 of the 80 cases were found in error) and thus, were
required to complete a PIP. The remaining 6 states were found
substantially compliant in their primary reviews as their case error rate
was below the established 5 percent threshold (4 or fewer cases were
found in error). The dollar-based improper payment rates for those 6
states ranged from 5.1 to 19.8 percent—based on the percentage of
improper payment dollars found in the sample. Because improper
payment rates are not used in applying the PIP corrective action strategy,
ACF’s method cannot effectively measure states’ progress over time in
reducing improper payments. It also cannot effectively help determine
whether further action is needed to minimize future improper payments.
This limits the extent to which states are held accountable for the
reduction of improper payments in the Foster Care program.

Upon a state’s implementation of its PIP, ACF conducts a secondary
review to determine whether errors found during the primary review have
been addressed. The secondary review is ACF’s principal tool to measure
a state’s success in implementing actions to reduce Foster Care program
improper payments. These reviews carry the potential financial penalty of
an extrapolated disallowance of the state’s federal share of Title IV-E
expenditures if the state is found to be noncompliant. However, because
ACF’s error threshold to be found noncompliant with a secondary review
is twice as high as that of the primary review (10 percent versus 5
percent), it limits ACF’s ability to provide an effective incentive for states
to focus continuing attention on causes of improper payments. Based on
our analysis of ACF’s Title IV-E eligibility reviews, 27 states have had at
least one secondary review between 2002 and 2010. Of the 27 states that
received a secondary review, 26 states passed this review (meaning that
the error rates were below the ten percent threshold) and only 1 state


42
  According to ACF, Title IV-E eligibility reviews for Florida were suspended pending
completion of the statewide Foster Care demonstration project, and therefore the results
were not included for fiscal year 2010 reporting.




Page 27                                                  GAO-12-312 Foster Care Program
failed. Of the 26 states that passed, 13 states (50 percent) would have
failed if the primary review error threshold of 5 percent was in effect. Of
those 13 states, we found at least 3 states that passed the secondary
review with a case error rate over 10 percent because the reported
improper payment dollar based error rate was below 10 percent. 43

The one state that failed its secondary review in 2003 received an
extrapolated disallowance in accordance with HHS regulations. Since the
eligibility reviews began in 2000, this is the only state found to be
noncompliant with its secondary review. While the extrapolated
disallowance is a financial penalty intended to encourage states to
address causes of improper payments, after incurring an extrapolated
disallowance, this state was again found to be noncompliant based on
ACF’s subsequent review in 2006. As such, the state was again required
to develop and implement a PIP to address the causes of errors identified
in this review. After implementing this PIP, the state was subject to
another secondary review and was found to be compliant with a case
error rate of 6.67 percent and dollar error rate of 2.84 percent. However,
this state would have failed if the primary review error threshold of 5
percent was in effect.

According to ACF officials, it established the 10 percent threshold for
compliance with secondary reviews in 2000 based on states’ error rates
at that time which were between 15 percent and 17 percent (in terms of
both cases and dollars). ACF officials told us the 10 percent appeared to
be a target that states could meet to demonstrate reductions in improper
payments over time. Also, the baseline estimated improper payment error
rate reported for the Foster Care program in 2004 was 10.33 percent.
Since establishing the 10 percent threshold in 2000, ACF has not
conducted a review to validate the continuing propriety of the
performance metric. GAO’s Internal Control Management and Evaluation
Tool provides that an agency should periodically review and validate the
propriety and integrity of both organizational and individual performance
measures and indicators. 44 According to this tool, performance



43
  Our analysis was limited to only identifying 3 states because ACF did not identify the
dollar error rate for 9 of the other 24 states. Therefore, there may be other states that
passed secondary reviews with an error rate over 10 percent because their dollar error
rate was below 10 percent.
44
 GAO-01-1008G.




Page 28                                                    GAO-12-312 Foster Care Program
                             measurement factors are to be evaluated to ensure they are linked to
                             mission, goals, and objectives, and that they are balanced and set
                             appropriate incentives for achieving goals while complying with law,
                             regulations, and ethical standards. For fiscal year 2010, ACF reported
                             that error rates for most of the states (33 of 51) were less than 5
                             percent. 45

                             In addition, ACF’s process for overseeing states’ implementation of
                             improper payment reduction actions has other weaknesses. ACF’s
                             guidance only requires that the PIP—required if a state has more than
                             four cases found in error in its primary review—address areas that the
                             eligibility review identified as needing improvement. Consequently, states’
                             corrective action plans may not address all types of previously identified
                             payment errors. There is nothing in the guidance to prevent states from
                             addressing other areas in the PIP. However, based on our review of the
                             guidance for developing PIPs and discussions with Central Office and
                             regional office staff, underpayments and other non-eligibility errors and
                             eligibility errors outside of the period under review might not be
                             addressed in the PIP if these types of errors were not a factor in a state’s
                             compliance. Not including all types of errors in states’ corrective action
                             plans reduces their effectiveness for addressing the causes of payment
                             errors as required under IPIA.


Limitations of ACF’s Audit   In 2010, ACF began using a departmentwide system, ARTMS, to track
Tracking System Impair       and monitor the resolution of audit findings for programs it administers,
Monitoring Corrective        including audit findings concerning Foster Care program payment
                             errors. 46 ACF utilizes ARTMS as its primary tool for monitoring states’
Actions
                             resolution of reported Single Audit findings. Single Audit findings for
                             states’ Foster Care programs have included, among other issues,
                             deficiencies in state oversight over subrecipients of federal funds, lack of
                             training of state personnel on program eligibility requirements, and
                             potential for unauthorized access to information systems to create and
                             approve cases. Single Audit reports generally include a summary of prior


                             45
                              Florida was excluded from review due to a statewide waiver demonstration project.
                             46
                               ARTMS, intended to be a HHS-wide system, is being rolled out in phases. Phase 1 was
                             rolled out in 2010 to include ACF. ARTMS was designed by ACF’s Office of Administration
                             (Office of Financial Services—Division of Financial Integrity; Office of Grants
                             Management; and Office of Information Services). With each phase of the roll out, the
                             agency will include upgrades to enhance the system.




                             Page 29                                                GAO-12-312 Foster Care Program
audit findings describing any recurring issues and any related corrective
actions undertaken by the state agency. Single Audits also generally
provide information about any deficiencies in state agencies’ systems and
processes that can be useful for ACF in monitoring federal expenditures
and identifying and reducing improper payments in the Foster Care
program.

According to ACF officials, ARTMS is designed to track and monitor the
resolution of Single Audit findings by audit report number, but the system
does not enable users to search for specific audit findings by type of
finding, grantee, state, region, or across years. As a result, regional
offices could not use ARTMS to examine trends in the types of findings in
their states in order to ensure that any systemic issues are addressed.
Limitations with ARTMS decrease ACF’s ability to leverage existing
agency data to identify reoccurring issues and other vulnerabilities such
as inadequate state monitoring of federal funding that might not be
identified during the 3-year eligibility review process and could lead to
improper payments. This lack of information could impair ACF’s and
regional offices’ ability to effectively monitor states’ efforts to reduce
improper payments and the effectiveness of corrective action strategies
implemented. According to GAO’s Standards for Internal Control in the
Federal Government, information should be recorded and communicated
to management and others within the entity who need it and in a form and
within a time frame that enables them to carry out their internal control
and other responsibilities. 47

ACF regional office officials acknowledged limitations with ARTMS related
to functionality in tracking findings. Specifically, a regional office official
told us that ARTMS was not designed to be able to generate reports of all
audit findings for an entire ACF region. Lacking such capability, a user
interested in aggregating the findings would have to obtain the audit
findings from each state and manually combine them outside of ARTMS.
This regional office used a separate internal spreadsheet to track
information related to the audit findings for all states in its purview.
Otherwise, staff would need to view each state’s Single Audit findings
individually within ARTMS, which could be time-consuming. Another
regional office we visited used a similar spreadsheet, which also included
information on the time it takes to close findings. A third regional office we



47
 GAO/AIMD-00-21.3.1.




Page 30                                           GAO-12-312 Foster Care Program
              visited also utilized an off-line spreadsheet as a means to track the
              clearance process for closing audit findings.


              A statistically valid approach for estimating improper payments would
Conclusions   help ensure that Foster Care program improper payment estimates are
              reasonably accurate and complete, reflecting all types of program
              payments including administrative costs, based on complete and accurate
              payment data, and aggregated using state-level margins of error.
              Developing and implementing a sound methodology is a critical program
              management tool for understanding and addressing financial
              vulnerabilities in the Foster Care program through approaches such as
              identifying underpayments and duplicate or excessive payment errors
              consistently across states. While ACF has reported an improper payment
              estimate and related reductions for the Foster Care program, the
              statistical validity of both is questionable. Further, ACF’s method for
              evaluating the effectiveness of states’ implementation of their corrective
              action plans has several significant weaknesses, including reliance on
              ineffective and dated metrics that do not consider states’ improper
              payment dollar error rates in conjunction with targets that have not been
              reassessed since 2000.

              Similarly, deficiencies in its system for monitoring Single Audit findings
              limit ACF’s ability to efficiently track and compare trends across states.
              This includes ACF’s ability to measure states’ progress in reducing their
              improper payment errors, as well as its ability to reliably and completely
              identify and correct vulnerabilities at the state level that could lead to
              improper payments. Although OMB’s approval reflected a stated plan for
              ACF to implement a process to annually improve the accuracy of its
              improper payment estimate, this has not resulted in substantial changes
              to the process ACF outlined in 2004. Given the financial accountability
              challenges reported for state-administered programs, the ongoing
              imbalance between revenues and outlays across the federal government,
              and increasing demands for accountability over taxpayer funds, improving
              ACF’s ability to identify, reduce, and recover improper payments is
              critical. It will be important for ACF to work closely with OMB in examining
              and updating its statistical procedures to help assure the validity of ACF’s
              estimates.




              Page 31                                         GAO-12-312 Foster Care Program
                      In order to more accurately and completely estimate improper payments
Recommendations for   for the Foster Care program and ensure that its methodology is
Executive Action      statistically valid, we recommend that the Secretary of Health and Human
                      Services direct the Assistant Secretary for the Administration for Children
                      and Families to take the following four actions:

                      •   augment procedures for estimating and reporting Foster Care
                          program improper payments, to include administrative costs;
                      •   develop and implement procedures to provide a statistically valid
                          methodology for estimating and reporting Foster Care program
                          improper payments based on complete and accurate payment data;
                      •   augment guidance to teams gathering state-level Foster Care
                          program improper payment estimate data to include specific
                          procedures to follow in identifying and reporting any underpayments
                          and duplicate or excessive payment errors; and
                      •   revise existing procedures for calculating a national improper payment
                          estimate for the Foster Care program to include a statistically valid
                          method for aggregating state-level margins of error to derive an
                          overall, inflation adjusted, program estimate.

                      To help ensure corrective action strategies effectively reduce Foster Care
                      program improper payments, we recommend that the Secretary of Health
                      and Human Services direct the Assistant Secretary for the Administration
                      for Children and Families to take the following three actions:

                      •   develop and implement procedures requiring states to implement and
                          report on corrective actions whenever a state’s estimated improper
                          payment dollar error rate exceeds a specified target level for the
                          program;
                      •   establish and implement procedures requiring periodic assessments
                          of state-level improper payment target levels, including targets
                          associated with Title IV-E secondary reviews, for the Foster Care
                          program for which states are to implement and report on corrective
                          actions; and
                      •   enhance ARTMS reporting capabilities to provide data on the status of
                          actions taken to address Single Audit findings concerning states’
                          Foster Care program payments, such as providing reporting
                          capabilities to allow ARTMS users to search for specific audit findings
                          by type of finding, grantee, state, region, or across years.




                      Page 32                                         GAO-12-312 Foster Care Program
                     We provided a draft of this report to the Secretary of Health and Human
Agency Comments      Services for comment. In its written comments, reprinted in appendix IV,
and Our Evaluation   HHS agreed that its improper payment estimation efforts can and should
                     be improved. HHS provided a summary of refinements that it had made to
                     its improper payment estimation methodology over the years and also
                     provided information on additional steps it planned to take. HHS stated
                     that our analysis would be a helpful resource as it continued to improve
                     its process. With regard to our seven recommendations to help improve
                     ACF’s methodology to estimate improper payments and its corrective
                     action process, HHS generally concurred with four of the
                     recommendations and agreed to continue to study the remaining three
                     recommendations. HHS also provided technical comments that we
                     incorporated, as appropriate.

                     HHS generally concurred with three recommendations we made related
                     to improving the improper payment estimation methodology for the Foster
                     Care program. Specifically, HHS generally agreed to (1) estimate and
                     report improper payments related to administrative costs, (2) provide
                     specific procedures to identify and report any underpayments and
                     duplicate or excessive payment errors, and (3) revise procedures for
                     calculating the aggregate state-level margins of error to derive an overall,
                     inflation adjusted, program estimate. HHS described several actions
                     currently under way to address these recommendations. Regarding the
                     first recommendation, HHS noted that it was continuing to pilot test the
                     Administrative Cost Reviews, described in this report, in fiscal year 2012;
                     however, HHS’s response did not indicate when it expects these reviews
                     will be fully implemented. In its response to the second recommendation,
                     HHS stated that additional guidance for identifying and reporting
                     underpayments and duplicate or excessive payment errors will be
                     included in the updated Eligibility Review Guide and review instrument
                     during the fiscal year 2012 review cycle. For the third recommendation,
                     HHS stated that it can and will adjust its calculation to incorporate
                     individual state margins of error in aggregating the state-level estimates
                     into the national program estimate. However, HHS also stated that it will
                     seek to determine whether making this revision would add sufficient value
                     given that the estimate spans 3 years and that inflation is relatively low.
                     We maintain that both aggregating state-level margins of error and
                     factoring for inflation are needed to implement a statistically valid method
                     for estimating improper payments.

                     For the other recommendation we made related to the improper payment
                     estimation methodology for the Foster Care program, HHS stated it would
                     continue to study our recommendation to develop and implement a


                     Page 33                                         GAO-12-312 Foster Care Program
statistically valid Foster Care improper payment methodology based on
complete and accurate payment data. HHS agreed that it should use the
best data available. In its comments on our draft report, HHS
acknowledged that it would be optimal to conduct a separate data
collection to obtain a universe of Title IV-E payments, but stated that it
needs to balance the goal of appropriate measurement with the cost and
burden placed on states. HHS described the quality controls in place over
the AFCARS data to help ensure the information is complete and
accurate prior to selecting case samples for its Title IV-E eligibility
reviews, which form the basis for its Foster Care improper payment
estimate. Examples of such controls include automated system edit
checks within AFCARS, AFCARS Assessment Reviews, and other
outreach efforts to improve state AFCARS reporting. Our report describes
some of the steps ACF has taken to address AFCARS data quality, but
we also point out limitations in these efforts. For example, the AFCARS
Assessment Reviews are not conducted annually for all states and do not
address verifying the accuracy or completeness of the specific data
element that ACF uses to develop its population of foster care cases for
estimating improper payments. HHS also stated that its use of
oversample cases demonstrated that its sampling and oversampling
process is working properly to exclude cases that do not meet the
selection criteria. While this process would identify some cases that did
not meet ACF’s selection criteria, our point in this report is that ACF’s
extensive reliance on the use of oversampling in its methodology is
indicative that the population of cases could contain additional
inaccuracies that may not be identified through its existing process.
Further, as we stated in our report, neither we nor ACF were able to
determine the completeness of the universe of cases used to estimate
Foster Care improper payments, that is, whether all cases that had
actually received a Title IV-E payment were properly coded as such.
Thus, the issues we identified with ACF’s sampling methodology,
provides limited assurance that the reported improper payment estimate
accurately and completely represents the extent of improper maintenance
payments in the Foster Care program.

With respect to our three recommendations to help ensure corrective
strategies effectively reduce Foster Care improper payments, HHS
concurred with one recommendation related to establishing and
implementing procedures for periodic assessments of state-level
improper payment target levels. HHS also agreed to consider another
recommendation to develop and implement corrective action procedures
whenever a state’s estimated improper payment dollar error rate exceeds
a specified target level for the program in conjunction with the


Page 34                                        GAO-12-312 Foster Care Program
recommendation it concurred with to implement periodic assessments.
HHS highlighted several actions it plans to take to enhance its efforts to
reduce improper payments, such as taking steps to reexamine and
explore the feasibility for lowering the error rate threshold and considering
ways to enhance existing Eligibility Review Guide instructions to address
any eligibility review findings that involve improper payments not
specifically requiring development of a corrective action plan. HHS stated
that it plans to further study our recommendation to enhance ARTMS
reporting capabilities to provide data on the status of actions taken to
address Single Audit findings concerning states’ Foster Care program
payments. HHS stated that the agency would study the value of potential
enhancements to ARTMS in light of the significant relevant data already
available (such as audit information by federal program, grantee name,
audit resolution status, and audit periods). Although these data elements
are currently available in ARTMS, search results are presented by
individual audit reports and include limited information. For example, the
search results provide only the number of findings associated with a
specific audit report but do not provide details of the individual audit
findings that would allow program managers to analyze, for example,
trends in the types of findings in states in order to ensure that any
systemic issues are addressed. We reaffirm our recommendation to
ensure that ACF is able to fully utilize ARTMS as a tool to analyze Single
Audit findings.


As agreed with your offices, unless you publicly announce the contents of
this report earlier, we plan no further distribution until 30 days from the
report date. At that time, we will send copies to the appropriate
congressional committees; the Secretary of Health and Human Services;
and other interested parties. In addition, the report will be available at no
charge on the GAO website at http://www.gao.gov.




Page 35                                          GAO-12-312 Foster Care Program
If you or your staffs have any questions about this report, please contact
me at (202) 512-8486 or raglands@gao.gov. Contact points for our
Offices of Public Affairs and Congressional Relations may be found on
the last page of this report. GAO staff who made key contributions to this
report are listed in appendix V.




Susan Ragland
Director
Financial Management and Assurance




Page 36                                         GAO-12-312 Foster Care Program
Appendix I: Objectives, Scope, and
              Appendix I: Objectives, Scope, and
              Methodology



Methodology

              The objectives of this report were to (1) determine the extent to which the
              Administration for Children and Families’ (ACF) estimation methodology
              generated a reasonably accurate and complete estimate of improper
              payments across the Foster Care program and (2) determine the extent
              to which ACF’s corrective actions reduced improper payments. To
              address these objectives, we reviewed the Improper Payments
              Information Act of 2002 (IPIA) requirements 1 and related Office of
              Management and Budget (OMB) guidance effective for fiscal year 2010, 2
              Department of Health and Human Services (HHS) regulations on Title IV-
              E eligibility reviews, 3 and ACF’s internal guidance including policies and
              procedures on conducting Title IV-E Foster Care eligibility reviews,
              computing improper payments, implementing corrective action plans for
              reducing improper payments, and monitoring and resolving audit findings
              in the Foster Care program. We also reviewed results from the Title IV-E
              eligibility reviews for periods 2000 through 2010, and prior GAO 4 and
              HHS Office of Inspector General (OIG) reports. 5 In addition, we reviewed
              improper payment information reported in HHS’s fiscal year 2010 agency
              financial report, Improper Payments Section. We reviewed these
              documents to understand ACF’s efforts to address IPIA requirements and
              to identify previously reported issues with ACF’s improper payment
              reporting.



              1
               Pub. L. No. 107-300, 116 Stat. 2350 (Nov. 26, 2002). IPIA was amended by the Improper
              Payments Elimination and Recovery Act of 2010, Pub. L. No. 111-204, 124 Stat. 2224
              (July 22, 2010). The changes made by this law to IPIA estimation and reporting were first
              implemented for fiscal year 2011 reporting.
              2
               OMB Memorandum M-06-23, Issuance of Appendix C to OMB Circular A-123 (Aug. 10,
              2006).
              3
               See 45 C.F.R. §§ 1355.35, 1355.36, 1356.71.
              4
               GAO, Foster Care and Adoption Assistance: Federal Oversight Needed to Safeguard
              Funds and Ensure Consistent Support for States’ Administrative Costs, GAO-06-649
              (Washington D.C.: June 15, 2006); and Foster Care: State Practices for Assessing Health
              Needs, Facilitating Service Delivery, and Monitoring Children’s Care, GAO-09-26
              (Washington, D.C.: Feb. 6, 2009).
              5
               Examples of HHS OIG reports include: HHS OIG, Audit of Allegheny County Title IV-E
              Foster Care Claims From October 1997 Through September 2002, A-03-08-00554 (Jan.
              4, 2011); Review of Title IV-E Foster Care Costs Claimed on Behalf of Delinquent
              Children in Georgia, A-04-07-03519 (June 17, 2010); Review of California’s Title IV-E
              Claims for Payments Made by Los Angeles County to Foster Homes of Relative
              Caregivers, A-09-06-00023 (Oct. 2, 2009); and Philadelphia County’s Title IV-E Claims
              Based on Contractual Per Diem Rates of $300 or Less for Foster Care Services from
              October 1997 Through September 2002, A-03-07-00560 (May 19, 2008).




              Page 37                                                  GAO-12-312 Foster Care Program
Appendix I: Objectives, Scope, and
Methodology




To further determine the extent to which ACF’s methodology generated a
reasonably accurate and complete estimate of improper payments across
the Foster Care program, we:

•   Performed an independent analysis of ACF’s sampling methodology,
    including a review of the sampling plan and other underlying
    documentation, as well as evaluated whether ACF’s sampling
    methodology complied with OMB statistical guidance, GAO guidance,
    and federal internal control standards 6 as criteria to determine the
    accuracy and completeness of ACF’s reported fiscal year 2010
    improper payment estimate for the Foster Care program. The scope
    of our review did not include an assessment of individual states’
    processes or payment systems that are the underlying data that ACF
    uses to support the national estimate of Foster Care improper
    payments.

•   Interviewed ACF officials such as the Acting Associate Commissioner
    for the Children’s Bureau, its contractor, and staff at selected regional
    offices such as program managers and financial specialists to gain an
    understanding of (1) the methodology that ACF uses to estimate
    improper payments in the Foster Care program in accordance with
    IPIA requirements 7 and (2) the administrative cost review pilot at five
    states to develop a methodology for estimating related administrative
    improper payments. We reviewed available reports for the five pilot
    reviews to identify what information ACF obtained from these reviews.

To further determine the extent to which ACF’s corrective action process
reduced improper payments, we:

•   Reviewed ACF policies and procedures to gain an understanding of
    reported corrective action strategies, including the Title IV-E eligibility
    review process and development of states’ Program Improvement
    Plans (PIP) used to address the root causes of improper payments,


6
 OMB Memorandum M-06-23, Issuance of Appendix C to OMB Circular A-123 (Aug. 10,
2006); OMB, Standards and Guidelines for Statistical Surveys (September 2006); and
GAO, Assessing the Reliability of Computer-Processed Data, GAO-09-680G (Washington,
D.C.: July 2009); Standards for Internal Control in the Federal Government,
GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999); and Using Statistical
Sampling, GAO/PEMD-10.1.6 (Washington, D.C.: May 1992).
7
 The scope of our review did not include an assessment of individual states’ processes or
payment systems.




Page 38                                                  GAO-12-312 Foster Care Program
Appendix I: Objectives, Scope, and
Methodology




    which are identified from the Title IV-E eligibility reviews. We reviewed
    applicable states’ PIPs for the period 2001 through 2010. We also
    inquired of ACF officials from the Program Implementation Division
    within the Children’s Bureau about other monitoring activities in place
    for states that did not have a PIP in place for IPIA reporting in fiscal
    year 2010.

•   Assessed compliance thresholds ACF uses to require states to
    implement corrective actions against actual performance data to
    assess the propriety of established performance measures. As part of
    this analysis, we reviewed our internal control standards as guidance
    to assess ACF’s evaluation of states’ efforts to implement corrective
    actions. 8

•   Conducted a walkthrough of ACF’s Audit Resolution Tracking and
    Monitoring System (ARTMS) to obtain an understanding of ACF’s
    monitoring activities to track and resolve state’s Single Audit findings
    for the Foster Care program. In addition, we interviewed officials in
    ACF’s Office of Information Services and the Division of Financial
    Integrity located in the Central Office and selected representative
    regional offices such as Regional Program Managers, and program
    and fiscal specialists to determine how ARTMS is used to identify and
    correct vulnerabilities that could lead to improper payments.

•   Examined states’ reported Single Audit 9 findings for fiscal years 2008
    through 2010 from ARTMS and a listing of HHS OIG reports on the
    Foster Care program to identify vulnerabilities or weaknesses in
    states’ operations that may not have been identified through ACF’s
    Title IV-E eligibility reviews.

•   Reviewed agency policies and procedures such as ACF’s Title IV-E
    Foster Care Eligibility Review Guide issued in March 2006 which
    includes the Title IV-E Foster Care Eligibility On-Site Review
    Instrument and Instructions; ACF’s user guide for ARTMS, version


8
 GAO, Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1
(Washington, D.C.: November 1999); and GAO, Internal Control Management and
Evaluation Tool, GAO-01-1008G (Washington, D.C.: August 2001).
9
 States, local governments, and non-profit organizations that expend $500,000 or more in
a year in federal awards are to have an audit conducted under the provisions of the Single
Audit Act, 31 U.S.C. §§ 7501-7507. Typically, this takes the form of a “Single Audit,” which
includes both the entity’s financial statements and the federal awards it receives.




Page 39                                                    GAO-12-312 Foster Care Program
Appendix I: Objectives, Scope, and
Methodology




    1.2; and ACF’s FY 2010 Corrective Action Plan to Reduce the
    Estimate Rate of Improper Payments in the Foster Care Program,
    dated November 12, 2010.

•   In addition, we conducted site visits to three of ACF’s ten regional
    offices (Philadelphia, PA; Chicago, IL; and San Francisco, CA). These
    three regional offices provided oversight of states who collectively
    claimed over half of the total federal share of Foster Care payments
    made in fiscal year 2009, the most recent data available at the time of
    our review for site visit selection. We also selected these regional
    offices to achieve variation in the numbers of error cases and amount
    of disallowed claims found during Title IV-E eligibility reviews, which
    ACF conducts to help ensure that states are claiming federal
    reimbursement only for eligible children. One region represented the
    highest number of error cases found in the Title IV-E eligibility reviews
    and the highest maintenance payment disallowance. Another region
    had the largest amount of foster care maintenance payments in fiscal
    year 2009 and the states within this region had high improper
    payment rates. Another region had a low number of error cases and
    improper payment issues relative to the high amount of maintenance
    payments it made to states in its purview. During these site visits, we
    interviewed agency personnel such as program managers, regional
    grants officers, and financial specialists to gain an understanding of
    how Title IV-E eligibility reviews are conducted and how the regional
    offices work with states on corrective actions and follow up on Single
    Audit findings. We also inquired about other ACF monitoring activities
    over states to address financial management weaknesses.

We conducted this performance audit from February 2011 through March
2012 in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit to
obtain sufficient, appropriate evidence to provide a reasonable basis for
our findings and conclusions, based on our audit objectives. We believe
that the evidence obtained provides a reasonable basis for our findings
and conclusions based on our audit objectives.




Page 40                                          GAO-12-312 Foster Care Program
Appendix II: Title IV-E Expenditures by Type
                                         Appendix II: Title IV-E Expenditures by Type
                                         and by State for Fiscal Year 2010



and by State for Fiscal Year 2010


                                                         Child
                                                    placement
                         Average                  services and      SACWIS
                         monthly            Net administration   (operations
                       number of    maintenance (administrative          and                              Demons-
State                    children     payments          costs) development)               Training         trations           Total
Alabama                    2,097     $ 6,151,654    $ 22,659,900       $ 2,234,570       $ 898,679               $-    $ 31,944,803
Alaska                       655       2,541,413        9,670,033        1,294,613        308,245                 -     13,814,304
Arizona                    4,403      34,920,745      25,433,283         1,625,049      10,213,887            4,056     72,197,020
Arkansas                   1,789      13,298,257      19,613,926                   -     7,629,291                -     40,541,474
California                33,188     189,058,341     434,338,308        41,470,987      74,006,011     464,542,684    1,203,416,331
Colorado                   2,041      21,048,905      37,310,491         2,833,886       1,432,169                -     62,625,451
Connecticut                1,670      33,837,323      26,140,861           494,901       1,008,418                -     61,481,503
Delaware                     201       1,415,097        2,336,241          251,308         45,551                 -      4,048,197
District of Columbia         902      17,088,155      13,730,035         2,639,434        216,026                 -     33,673,650
Florida                    6,127       (662,979)                  -      8,246,087               -     164,026,443     171,609,551
Georgia                    2,755      35,226,100      38,876,701         6,928,247        642,342                 -     81,673,390
Hawaii                       473       3,215,746      12,858,071                   -     2,188,051                -     18,261,868
Idaho                        963       3,148,911        4,823,821          322,585       1,213,690                -      9,509,007
Illinois                  13,292      67,605,231      99,941,388                   -     1,458,872       18,734,590    187,740,081
Indiana                    3,087      37,968,599      43,019,507           416,181       5,291,285        4,610,780     91,306,352
Iowa                       1,471      10,575,979      10,057,790           634,089       1,393,149        1,672,279     24,333,286
Kansas                     1,245      13,946,950      13,549,938                   -       98,523                 -     27,595,411
Kentucky                   2,921      27,357,551      12,950,173         1,823,554       4,863,147                -     46,994,425
Louisiana                  2,562      26,969,310      23,512,183        (2,529,684)      8,187,171                -     56,138,980
Maine                        957       5,963,933      11,220,480           502,557        385,164                 -     18,072,134
Maryland                   2,145      41,022,794      29,899,462         1,463,167       2,372,090                -     74,757,513
Massachusetts              2,191      26,328,104      33,629,458           526,268               -                -     60,483,830
Michigan                   4,165      27,686,604      65,368,746                   -      620,506                 -     93,675,856
Minnesota                  1,800      11,218,971      17,590,313         5,550,119       4,900,152        3,617,755     42,877,310
Mississippi                  999       8,190,833        6,493,427                  -             -                -     14,684,260
Missouri                   3,166      16,858,234      30,177,210         4,083,378       4,950,719                -     56,069,541
Montana                      627       4,679,312        5,282,883          943,474       1,358,777                -     12,264,446
Nebraska                   1,369       7,861,664        9,467,519          169,953       2,499,675                -     19,998,811
Nevada                     2,083      11,695,544      19,428,724         1,925,270       2,261,741                -     35,311,279
New Hampshire                436       3,424,675      11,753,640           470,028       1,298,133                -     16,946,476
New Jersey                 4,226      34,725,549      54,478,552         2,525,021       3,321,363                -     95,050,485
New Mexico                 1,092       5,367,458      14,420,493         1,562,400       3,368,379                -     24,718,730




                                         Page 41                                                     GAO-12-312 Foster Care Program
                                     Appendix II: Title IV-E Expenditures by Type
                                     and by State for Fiscal Year 2010




                                                    Child
                                               placement
                   Average                   services and      SACWIS
                   monthly             Net administration   (operations
                 number of     maintenance (administrative          and                                                          Demons-
State              children      payments          costs) development)                                    Training                trations               Total
New York            12,724      201,743,181           213,684,975                 5,193,320             9,882,339                              -   430,503,815
North Carolina       3,197       20,787,787             53,681,394                              -          874,011                             -    75,343,192
North Dakota           375        5,962,358               5,115,417                    17,411              385,436                             -    11,480,622
Ohio                 7,446      105,128,723             81,031,793                2,907,758             5,625,728                  300,500         194,994,502
Oklahoma             3,308       11,848,998             16,665,705                2,181,593             3,531,191                              -    34,227,487
Oregon               3,190       23,579,566             58,623,327                8,347,283             1,275,803               6,422,206           98,248,185
Pennsylvania        14,690      125,564,749           124,439,736                               -       5,255,437                              -   255,259,922
Puerto Rico               -                   -                        -                        -                     -                        -             -
Rhode Island           592        6,611,638               5,288,952               1,495,342                  75,712                            -    13,471,644
South Carolina       1,174       12,273,512             22,002,174                1,024,325             1,223,697                              -    36,523,708
South Dakota           590        3,749,543               2,628,588                    98,542              216,286                             -     6,692,959
Tennessee            2,981       20,262,663             12,901,854                9,376,444             3,107,492                (490,734)          45,157,719
Texas               11,971      142,903,432           104,371,896                               -       9,969,190                              -   257,244,518
Utah                   902        5,054,187             12,228,891                1,250,832             1,704,787                              -    20,238,697
Vermont                528        6,851,819               2,485,472                             -       1,203,723                              -    10,541,014
Virginia             2,870       27,490,610             27,827,516                              -          701,434                             -    56,019,560
Washington           4,159       20,774,711             59,527,687                5,207,194             5,952,490                              -    91,462,082
West Virginia        1,012       27,171,706               4,255,352               2,014,995                352,446                             -    33,794,499
Wisconsin            2,151       20,032,342             27,653,539                2,527,137             2,009,693                  792,644          53,015,355
Wyoming                120          500,589               1,176,256                  502,913               212,042                             -     2,391,800
Totals              181,078   $ 1,538,027,077 $ 1,995,624,081                $ 130,552,531 $ 201,990,143                   $ 664,233,203 $ 4,530,427,035
                                     Source: ACF’s Title IV-E Foster Care Fiscal Year 2010 Expenditures as Reported by States (May 4, 2011).




                                     Page 42                                                                              GAO-12-312 Foster Care Program
Appendix III: Estimated Improper Payment
                       Appendix III: Estimated Improper Payment
                       Error Rates by State for Fiscal Year 2010
                       Reporting


Error Rates by State for Fiscal Year 2010
Reporting

                                                   Sample          Sample overpayment
State                  Review date        underpayment rate                       rate      Sample error rate
Alabama                     08/2009                     0.00%                   0.96%                   0.96%
Alaska                      07/2009                     0.53%                   4.42%                   4.95%
Arizona                     03/2010                     0.00%                   1.16%                   1.16%
Arkansas                    08/2009                     0.38%                   7.19%                   7.57%
California                  09/2009                     1.77%                   6.20%                   7.97%
Colorado                    06/2009                     0.00%                   4.04%                   4.04%
Connecticut                 04/2009                     0.00%                   8.09%                   8.09%
Delaware                    04/2009                     2.58%                   7.69%                 10.27%
District of Columbia        09/2009                     0.70%                   3.06%                   3.76%
                                   a
Florida                                                                                                 0.00%
Georgia                     09/2009                     0.00%                   6.88%                   6.88%
Hawaii                      06/2010                     3.08%                   0.57%                   3.66%
Idaho                       04/2010                     0.92%                  18.88%                 19.80%
Illinois                    08/2007                     0.00%                  10.87%                 10.87%
Indiana                     01/2009                     2.33%                   8.84%                 11.17%
Iowa                        09/2007                     0.00%                   0.00%                   0.00%
Kansas                      08/2008                     0.40%                   1.37%                   1.77%
Kentucky                    11/2007                     0.48%                   1.66%                   2.14%
Louisiana                   09/2007                     0.61%                   0.72%                   1.33%
Maine                       06/2010                     0.00%                   0.81%                   0.81%
Maryland                    06/2008                     0.00%                  14.39%                 14.39%
Massachusetts               11/2009                     0.00%                  17.74%                 17.74%
Michigan                    06/2010                     2.56%                   1.78%                   4.34%
Minnesota                   03/2010                     1.64%                   3.49%                   5.13%
Mississippi                 07/2008                     0.00%                  16.80%                 16.80%
Missouri                    09/2008                     0.00%                   4.68%                   4.68%
Montana                     06/2009                     0.00%                   0.00%                   0.00%
Nebraska                    08/2009                     0.00%                   1.96%                   1.96%
Nevada                      06/2008                     0.15%                   3.84%                   3.99%
New Hampshire               12/2009                     0.00%                   0.18%                   0.18%
New Jersey                  08/2008                     1.97%                   3.11%                   5.08%
New Mexico                  07/2009                     0.00%                   3.62%                   3.62%
New York                    08/2009                     0.00%                   1.50%                   1.50%
North Carolina              06/2008                     0.00%                   1.46%                   1.46%
North Dakota                08/2008                     0.00%                   0.69%                   0.69%




                       Page 43                                                  GAO-12-312 Foster Care Program
                 Appendix III: Estimated Improper Payment
                 Error Rates by State for Fiscal Year 2010
                 Reporting




                                                     Sample                  Sample overpayment
State            Review date                underpayment rate                               rate         Sample error rate
Ohio                    07/2010                                 3.16%                        1.94%                   5.10%
Oklahoma                01/2010                                 0.00%                        0.00%                   0.00%
Oregon                  07/2008                                 0.69%                        2.48%                   3.17%
Pennsylvania            07/2007                                 0.15%                        3.04%                   3.19%
Puerto Rico             05/2007                                 0.00%                        10.76%                10.76%
Rhode Island            09/2007                                 0.00%                        14.31%                14.31%
South Carolina          05/2009                                 0.00%                        1.24%                   1.24%
South Dakota            05/2009                                 0.00%                        0.00%                   0.00%
Tennessee               06/2009                                 0.00%                        8.79%                   8.79%
Texas                   07/2009                                 0.15%                        0.00%                   0.15%
Utah                    09/2008                                 0.00%                        0.19%                   0.19%
Vermont                 05/2008                                 0.00%                        1.97%                   1.97%
Virginia                03/2007                                 1.48%                        4.09%                   5.57%
Washington              08/2007                                 0.82%                        0.75%                   1.58%
West Virginia           04/2008                                 0.00%                        4.44%                   4.44%
Wisconsin               08/2008                                 0.00%                        0.00%                   0.00%
Wyoming                 03/2010                                 0.00%                        0.00%                   0.00%
                 Source: ACF’s Title IV-E Foster Care Program Fall 2010 Error Rate Update.
                 a
                  According to ACF, Florida IV-E Reviews suspended pending completion of a statewide Foster Care
                 demonstration project and were therefore, not included in the national error rate.




                 Page 44                                                                     GAO-12-312 Foster Care Program
Appendix IV: Comments from the
             Appendix IV: Comments from the Department
             of Health and Human Services



Department of Health and Human Services




             Page 45                                     GAO-12-312 Foster Care Program
Appendix IV: Comments from the Department
of Health and Human Services




Page 46                                     GAO-12-312 Foster Care Program
Appendix IV: Comments from the Department
of Health and Human Services




Page 47                                     GAO-12-312 Foster Care Program
Appendix IV: Comments from the Department
of Health and Human Services




Page 48                                     GAO-12-312 Foster Care Program
Appendix IV: Comments from the Department
of Health and Human Services




Page 49                                     GAO-12-312 Foster Care Program
Appendix IV: Comments from the Department
of Health and Human Services




Page 50                                     GAO-12-312 Foster Care Program
Appendix IV: Comments from the Department
of Health and Human Services




Page 51                                     GAO-12-312 Foster Care Program
Appendix IV: Comments from the Department
of Health and Human Services




Page 52                                     GAO-12-312 Foster Care Program
Appendix V: GAO Contact and Staff
                  Appendix V: GAO Contact and Staff
                  Acknowledgments



Acknowledgments

                  Susan Ragland, (202) 512-8486 or raglands@gao.gov
GAO Contact
                  In addition to the contact named above, Carla Lewis, Assistant Director;
Staff             Betty Ward-Zukerman, Assistant Director; Sharon Byrd, Assistant
Acknowledgments   Director; Sophie Brown; Gabrielle Fagan; Vincent Gomes; and Nhi
                  Nguyen made key contributions to this report. Also contributing to this
                  report were Kay Brown; Wilfred Holloway; Francine DelVecchio; Jason
                  Kirwan; and Karen O’Conor.




(195161)
                  Page 53                                        GAO-12-312 Foster Care Program
GAO’s Mission         The Government Accountability Office, the audit, evaluation, and
                      investigative arm of Congress, exists to support Congress in meeting its
                      constitutional responsibilities and to help improve the performance and
                      accountability of the federal government for the American people. GAO
                      examines the use of public funds; evaluates federal programs and
                      policies; and provides analyses, recommendations, and other assistance
                      to help Congress make informed oversight, policy, and funding decisions.
                      GAO’s commitment to good government is reflected in its core values of
                      accountability, integrity, and reliability.

                      The fastest and easiest way to obtain copies of GAO documents at no
Obtaining Copies of   cost is through GAO’s website (www.gao.gov). Each weekday afternoon,
GAO Reports and       GAO posts on its website newly released reports, testimony, and
                      correspondence. To have GAO e-mail you a list of newly posted products,
Testimony             go to www.gao.gov and select “E-mail Updates.”

Order by Phone        The price of each GAO publication reflects GAO’s actual cost of
                      production and distribution and depends on the number of pages in the
                      publication and whether the publication is printed in color or black and
                      white. Pricing and ordering information is posted on GAO’s website,
                      http://www.gao.gov/ordering.htm.
                      Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
                      TDD (202) 512-2537.
                      Orders may be paid for using American Express, Discover Card,
                      MasterCard, Visa, check, or money order. Call for additional information.
                      Connect with GAO on Facebook, Flickr, Twitter, and YouTube.
Connect with GAO      Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.
                      Visit GAO on the web at www.gao.gov.
                      Contact:
To Report Fraud,
Waste, and Abuse in   Website: www.gao.gov/fraudnet/fraudnet.htm
                      E-mail: fraudnet@gao.gov
Federal Programs      Automated answering system: (800) 424-5454 or (202) 512-7470

                      Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-
Congressional         4400, U.S. Government Accountability Office, 441 G Street NW, Room
Relations             7125, Washington, DC 20548

                      Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800
Public Affairs        U.S. Government Accountability Office, 441 G Street NW, Room 7149
                      Washington, DC 20548




                        Please Print on Recycled Paper.