oversight

Legal Services Corporation: More Needs to be Done to Correct Case Service Reporting Problems

Published by the Government Accountability Office on 1999-09-29.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                     United States General Accounting Office

GAO                  Testimony
                     Before the Subcommittee on Commercial and
                     Administrative Law
                     Committee on the Judiciary
                     House of Representatives

For Release
on Delivery
2:00 p.m. EDT        LEGAL SERVICES
Wednesday
September 29, 1999   CORPORATION

                     More Needs to Be Done to
                     Correct Case Service
                     Reporting Problems
                     Statement of Laurie E. Ekstrand,
                     Director, Administration of Justice Issues,
                     General Government Division




GAO/T-GGD-99-185
Statement

Legal Services Corporation: More Needs to
Be Done to Correct Case Service Reporting
Problems
               Mr. Chairman and Members of the Committee:

               I am pleased to be here today to discuss two reviews that we have
               completed pertaining to case service reporting by the Legal Services
               Corporation (LSC). During the past year, both the LSC Office of the
               Inspector General (OIG) and we identified misreporting by grantees on
               both the number of cases they closed during calendar year 1997 and the
               number they had open at the end of that year. In a June 1999 report, we
               estimated that nearly 75,000 of the approximately 221,000 cases reported
                                                                      1
               to LSC by 5 of its largest grantees were questionable. In light of these
               findings, we were asked to determine (1) what efforts LSC and its grantees
               have made to correct case reporting problems, and (2) whether these
               efforts are likely to resolve the case reporting problems that occurred in
                                                           2
               1997. In a report issued earlier this month we found the following:

             • LSC revised its written case reporting guidance and issued a new
               handbook to its grantees to clarify case reporting requirements.
             • Grantees reported changing their policies and procedures to comply with
               LSC’s new reporting requirements.
             • Although most grantees indicated that LSC’s revised case reporting
               guidance has clarified reporting requirements for grantees, many grantees
               remained unclear about certain aspects of LSC’s reporting requirements.
             • Problems existed with LSC’s self-inspection, which sought to verify the
               accuracy of 1998 Case Service Reporting (CSR) data.

               We believe that more needs to be done to correct reporting problems at
               LSC. We made eight recommendations to help LSC improve the accuracy
               of future CSR reports.

               LSC was established in 1974 as a private, nonprofit, federally funded
Background     corporation to provide legal assistance to low-income people in civil
               matters. LSC provides the assistance indirectly, through grants to about
               260 competitively selected local programs. Grantees may receive
               additional funding from non-LSC sources. In fiscal years 1998 and 1999,
               LSC received appropriations of $283 million and $300 million, respectively.

               To qualify for LSC representation, clients must meet both financial and
               citizenship/alien eligibility requirements. With respect to financial

               1
                Legal Services Corporation: Substantial Problems in 1997 Case Reporting by Five Grantees
               (GAO/GGD-99-135R, June 25, 1999).
               2
                Legal Services Corporation: More Needs to Be Done to Correct Case Service Reporting Problems
               (GAO/GGD-99-183, September 20, 1999).




               Page 1                                        GAO/T-GGD-99-185 LSC Case Reporting Problems
                        Statement
                        Legal Services Corporation: More Needs to Be Done to Correct Case Service Reporting
                        Problems




                        eligibility, clients’ income, in general, is not to exceed 125 percent of the
                        federal poverty guidelines. LSC regulations require that grantees (1) adopt
                        a form and procedure to obtain eligibility information and (2) preserve that
                        information for audit by LSC. With respect to citizenship/alien eligibility,
                        only citizens and certain categories of aliens are eligible for services. For
                        clients who are provided services in person, a citizen attestation form or
                        documentation of eligible alien status is required. For clients who are
                        provided services via the telephone, documentation of the inquiry
                        regarding citizenship/alien eligibility is required.

                        LSC uses a Case Service Reporting system to gather quantifiable
                        information from grantees on the services they provide that meet LSC’s
                        definition of a case. The CSR Handbook is LSC’s primary official guidance
                        to grantees on how to record and report cases. LSC relies on such case
                        information in its annual request for federal funding.

                        Audit reports issued by LSC’s OIG between October 1998 and July 1999
                        reported that five grantees misreported the number of cases they had
                        closed during calendar year 1997 and the number of cases that remained
                        open at the end of that year. The OIG found that all five grantees
                        overstated the number of closed cases, while four overstated and one
                        understated open cases.

                        In June 1999, in response to Congress’ request for information on whether
Our Audit of Five LSC   the 1997 case data of other LSC programs had problems similar to those
Grantees Found That     reported by LSC’s OIG, we issued a report on our audit of five of LSC’s
One-Third of 1997       largest grantees: Baltimore, Chicago, Los Angeles, New York City, and
                                     3
                        Puerto Rico. We conducted a file review of a random sample of cases at
Cases Were              each of these grantees to determine the extent to which they made
Questionable            overreporting errors in reporting cases closed during 1997 and cases open
                        on December 31, 1997. We found similar types of reporting errors to those
                        the OIG found and estimated that, overall, about 75,000 (+/- 6,100) of the
                        approximately 221,000 cases that the five grantees reported to LSC for
                        1997 were questionable. Three grantees identified about 30,000 of their
                        cases as misreported prior to our case file review. The primary causes for
                        these self-identified overreporting errors were (1) improperly reporting to
                        LSC cases that were wholly funded by other sources, such as states, and
                        (2) problems related to case management reporting systems, such as
                        grantee staffs’ difficulty in transitioning to new automated systems.



                        3
                            GAO/GGD-99-135R, June 25, 1999.




                        Page 2                                  GAO/T-GGD-99-185 LSC Case Reporting Problems
                        Statement
                        Legal Services Corporation: More Needs to Be Done to Correct Case Service Reporting
                        Problems




                        Our case file review deemed approximately 45,000 additional cases
                        questionable for one of the following reasons:

                      • The grantee reported duplicate cases for the same legal service to the
                        same client.
                      • Some case files did not contain any documentation supporting the
                        grantee’s determination that the client was either a U.S. citizen or eligible
                        alien.
                      • For cases reported as closed in 1997, some case files showed no activity
                        during the 12 months before the case was closed. For cases reported as
                        open as of December 31, 1997, some cases showed no grantee activity
                        during calendar year 1997.
                      • Some case files did not contain any documentation that the grantee had
                        determined that the client was financially eligible for LSC services. LSC
                        regulations did not require specific documentation of these determinations
                        in all cases. However, they required that grantees (1) adopt a form and
                        procedure to obtain eligibility information and (2) preserve that
                        information for audit by LSC.

                        LSC officials and executive directors of the five grantees told us that they
                        had taken or were planning to take steps to correct these case reporting
                        problems.

                        LSC issued a new, 1999 CSR Handbook and distributed other written
LSC’s Clarified         communications intended to clarify reporting requirements to its grantees.
Reporting Guidance      The 1999 handbook, which replaced the 1993 edition, instituted changes to
Resulted in Program     some of LSC’s reporting requirements and provided more detailed
                        information on other requirements.
Changes, but Some
Requirements Remain     In responding to a GAO telephone survey, most grantees indicated that the
Unclear to Many         new guidance helped clarify LSC’s reporting requirements, and virtually all
Grantees                of them indicated that they had or planned to make program changes as a
                        result of the requirements. Many grantees, however, identified areas of
                        case reporting that remained unclear to them.

LSC Issued New CSR      The 1999 CSR Handbook included changes to (1) procedures for timely
                        closing of cases; (2) procedures for management review of case service
Guidance                reports; (3) procedures for ensuring single recording of cases; (4)
                        requirements to report LSC-eligible cases, regardless of funding source;
                        and (5) requirements for reporting cases involving private attorneys
                        separately.




                        Page 3                                  GAO/T-GGD-99-185 LSC Case Reporting Problems
                             Statement
                             Legal Services Corporation: More Needs to Be Done to Correct Case Service Reporting
                             Problems




                             On November 24, 1998, LSC informed its grantees that two of the changes
                             in the 1999 CSR Handbook were to be applied to the 1998 case data. The
                             two changes pertained to timely closing of cases and management review
                             of case service reports. The remaining new provisions of the 1999 CSR
                             Handbook were not applicable to 1998 cases. For example, for 1998, there
                             was no requirement for grantees to ensure that cases were not double
                             counted. For 1999, LSC is requiring the use of automated case management
                             systems and procedures to ensure that cases involving the same client and
                             specific legal problem are not reported to LSC more than once.

                             For 1998, grantees could report only those cases that were at least partially
                             supported by LSC funds. For 1999, LSC is requiring grantees to report all
                             LSC-eligible cases, regardless of funding source. LSC intends to estimate
                             the percentage of activity spent on LSC service by applying a formula that
                             incorporates the amount of funds grantees receive from other funding
                             sources compared with the amount they receive from LSC.

                             In addition to changing certain reporting requirements, the 1999 handbook
                             also provides more detailed guidance to grantees than the 1993 handbook.
                             For example, the 1999 handbook provides more specific definitions of
                             what constitutes a “case” and a “client” for CSR purposes. The 1999
                             handbook also addresses documentation requirements that were not
                             discussed in the 1993 handbook.

Grantee Directors Reported   Based on our survey of executive directors of 79 grantees, we estimate that
                                             4
                             over 90 percent of grantee executive directors viewed the changes in the
That They Are                1999 CSR Handbook as being clear overall, and virtually all of them
Implementing Changes to      indicated that they planned to or had made at least one change to their
Comply With Reporting        program operations as a result of the revised case reporting requirements.
Requirements                 These changes included revising policies and procedures, providing staff
                             training, modifying forms and/or procedures used during client intake,
                             implementing computer hardware and software changes, and increasing
                             reviews of cases.

Many Grantees Remain         Although most of the grantee executive directors reported that the new
                             LSC guidance helped clarify requirements, many of them also indicated
Unclear About Certain        that they were still unclear about certain requirements and that additional
Reporting Requirements       clarification was needed. Areas of confusion or uncertainty that executive
                             directors identified included requirements pertaining to asset and

                             4
                             We conducted telephone interviews with a random sample of executive directors of 79 grantees. All
                             percentage estimates from the results of this survey have 95 percent confidence intervals with a margin
                             of error of 10 percent or less.




                             Page 4                                         GAO/T-GGD-99-185 LSC Case Reporting Problems
                           Statement
                           Legal Services Corporation: More Needs to Be Done to Correct Case Service Reporting
                           Problems




                           citizenship/alien eligibility documentation, single recording of cases, and
                           who can provide legal services.

                           LSC sought to determine the accuracy of grantees’ case data by requiring
Most Grantees              that grantees complete self-inspections of their open and closed caseload
Certified the Accuracy     data for 1998. Grantees were required to determine whether the error rate
of Their 1998 CSR          in their data exceeded 5 percent. According to LSC, about three-fourths of
                           the grantees certified that the error in their data was 5 percent or less. LSC
Data, but Questions        used the results of the self-inspections to estimate the total number of case
About Data Accuracy        closings in 1998. Our review of LSC’s self-inspection process raised
and Interpretation         concerns about the accuracy and interpretation of the results, and what
Remain                     the correct number of certifying programs should be.

LSC Grantees Conducted     On May 14, 1999, LSC issued a memo to all grantees instructing them to
                           complete a self-inspection procedure by July 1, 1999. The purpose of the
Self-Inspections of 1998   self-inspection was to ensure that (1) grantees were properly applying
CSR Data                   instructions in the 1999 edition of the CSR Handbook that were applicable
                           to the 1998 data, and (2) LSC had accurate case statistical information to
                           report to Congress for calendar year 1998.

                           LSC provided detailed guidance to grantees on the procedures for the self-
                           inspection. Each grantee was to select and separately test random samples
                           of open and closed cases to determine whether the number of cases it
                           reported to LSC earlier in the year was correct. Grantees were to verify
                           that the case file contained a notation of the type of assistance provided,
                           the date on which the assistance was provided, and the name of the case
                           handler providing the assistance. Grantees were also to determine whether
                           assistance had ceased prior to January 1, 1998; was within certain service
                           categories as defined by the 1999 handbook; was provided by an attorney
                           or paralegal; and was not prohibited or restricted. Finally, grantees were to
                           verify that each case had eligibility information on household income, size,
                           assets, citizenship attestation for in-person cases, and indication of
                           citizenship/alien status for telephone-only cases.

                           If any single aspect of a case failed to meet LSC’s requirements, the case
                           was to be classified as an error for reporting purposes. If the grantees
                           found that their CSR case sampling had an error rate of 5 percent or less,
                           the program directors and policy board chairs were to sign a certification
                           form and return it to LSC. Grantees who could not certify to the
                           correctness of their 1998 CSR data were to submit a letter to LSC
                           describing (1) the problems they had identified during the self-inspection
                           process and (2) the corrective actions they had instituted to address the
                           problems. Grantees could resubmit their 1998 CSR data to LSC if they



                           Page 5                                  GAO/T-GGD-99-185 LSC Case Reporting Problems
                          Statement
                          Legal Services Corporation: More Needs to Be Done to Correct Case Service Reporting
                          Problems




                          identified one or more problems in the random sample and corrected their
                          entire 1998 database so that the problems no longer appeared. If, by
                          correcting the problems, the error rate in the data was reduced to 5
                          percent or less, the grantees could resubmit their 1998 data along with a
                          signed certification attesting to the substantial accuracy of the resubmitted
                          data. In this way, grantees who were unable to certify at one point in time
                          could certify at a later point in time.

Most Grantees Certified   According to LSC officials, about three-fourths of the grantees certified the
                          accuracy of their 1998 case data. As of August 26, 1999, LSC documents
Their 1998 CSR Data                                          5
                          indicated that 199 of 261 grantees (76 percent) reported substantially
                          correct CSR data to LSC. The remaining 62 grantees (24 percent) did not
                          certify to LSC that their CSR data were substantially correct. According to
                          LSC, 30 of the 50 largest grantees did not certify their 1998 data.

                          LSC officials told us they were surprised that such a large number of
                          grantees certified their 1998 CSR data. They attributed the results to the
                          following factors: (1) the self-inspection did not attempt to identify
                          duplicate cases; (2) grantees received the new 1999 handbook in
                          November 1998 and had already implemented some of the new
                          requirements; and (3) grantees were less likely to report as cases
                          telephone referrals in which no legal advice had been given and/or clients’
                          eligibility had not been determined because they were aware that the OIG
                          identified this as a problem.

                          On the basis of the self-inspection results, LSC estimated that grantees
                          closed 1.1 million cases in 1998.

Self-Inspection Results   Our review raised some concerns about LSC’s interpretation of the self-
                          inspection results and about the accuracy of the data provided to LSC by
Raised Concerns           grantees. As a result, we could not assess the accuracy of LSC’s estimate of
                          the number of certified programs and case closures for 1998.

                          LSC did not issue standardized procedures for grantees to use in reporting
                          the results of their self-inspections. Grantees that could not certify their
                          data wrote letters to LSC that contained varying degrees of detail about
                          data errors that they found. Since LSC did not have a standard protocol for
                          collecting the results of the self-inspections, LSC officials in some cases
                          had to rely on their own interpretations of grantees’ descriptions of the
                          problems they had discovered.

                          5
                           LSC funded 262 programs in 1998. Funding for one program was discontinued in 1999, and LSC has no
                          self-inspection results for this program.




                          Page 6                                       GAO/T-GGD-99-185 LSC Case Reporting Problems
Statement
Legal Services Corporation: More Needs to Be Done to Correct Case Service Reporting
Problems




We are uncertain how many programs should have been counted as
certified because we are uncertain if LSC applied a consistent definition of
“certification.” Most programs that were on LSC’s certification list
determined that they had error rates of 5 percent or less for both open and
closed cases. However, LSC placed some programs on the certified list if
the program’s overall error rate for closed cases was 5 percent or less,
even if the overall error rate actually was higher than 5 percent. In two
instances, executive directors told us that they did not certify their CSR
data because their overall error rate exceeded 5 percent. However, these
programs appeared on LSC’s list of certified programs. When we asked an
LSC official about this, he told us that they advised grantees that if their
closed case error rate did not exceed 5 percent, they should “partially
certify” their data. In response to our inquiry, the official reviewed the
certification letters submitted by nearly 200 grantees, and identified 5
certified programs whose error rates for open cases exceeded 5 percent.
Given that some grantees submitted only an overall estimate of data error,
we do not know how many programs qualified to be certified overall, just
for closed cases, or just for open cases.

We are also concerned that LSC’s instructions to grantees on how to
conduct the self-inspections may have led some of the smaller grantees to
select too few test cases to make a reliable assessment of the proportion of
error in their case data. Because these were smaller grantees, this
limitation would have had little effect on LSC’s estimate of the total closed
caseload. However, it could have affected LSC’s count of the number of
certified programs.

LSC does not know how well grantees conducted the self-inspection
process, nor how accurate the results are. We spoke with several executive
directors who did not correctly follow LSC’s reporting requirements.
Incorrect interpretations of LSC guidance may have resulted in some
programs certifying their 1998 data when they should not have, and other
programs not certifying their 1998 data when they should have. An LSC
official told us that, although they have conducted CSR training sessions
for grantee executive directors, thousands of case handlers in grantee
offices have not received such training. The official acknowledged that
written guidance and telephone contacts with grantees may not be
sufficient to ensure correct and consistent understanding of reporting
requirements, and that LSC plans to consider alternative ways of providing
training to staff.




Page 7                                  GAO/T-GGD-99-185 LSC Case Reporting Problems
                    Statement
                    Legal Services Corporation: More Needs to Be Done to Correct Case Service Reporting
                    Problems




                    LSC officials told us that the self-inspection was valuable and that LSC
                    plans to have grantees complete self-inspections again early next year as
                    part of the 1999 CSR reporting process.

                    LSC’s 1999 CSR Handbook and other written communications have
Conclusions         improved the clarity of reporting requirements for its grantees. However,
                    many grantees remained unclear about and/or misunderstood certain
                    aspects of the reporting requirements. LSC’s practice of disseminating
                    guidance primarily by written or telephone communications may not be
                    sufficient to ensure that grantees correctly and consistently interpret the
                    requirements.

                    LSC sought to determine the accuracy of grantees’ 1998 case statistics by
                    requiring grantees to conduct self-inspections. However, we do not know
                    the extent to which the results of the self-inspection process are accurate.
                    The validity of the results are difficult to determine because LSC did not
                    standardize the way that grantees were to report their results, some of the
                    grantees used samples that were too small to assess the proportion of
                    error in their data, some grantees did not correctly follow LSC’s reporting
                    guidance, and LSC had done no verification of the grantees’ self-inspection
                    procedures.

                    We do not believe that LSC’s actions, to date, have been sufficient to fully
                    resolve the case reporting problems that occurred in 1997.
                                                              6
                    In our September 20, 1999, report we recommended that the President of
Recommendations     LSC:

                  • clarify and disseminate information on the specific information on client
                    assets that grantees must obtain, record, and maintain;
                  • clarify and disseminate information on the types of citizenship/alien
                    eligibility information grantees must obtain, record, and maintain for
                    clients who receive legal assistance only over the telephone;
                  • clarify and disseminate LSC’s criteria for single recording of cases;
                  • clarify and disseminate LSC’s policy concerning who can provide legal
                    assistance to clients for the service to be counted as a case;
                  • explore options for facilitating correct and consistent understanding of
                    reporting requirements, including developing and disseminating a training
                    video for grantee staff;



                    6
                        GAO/GGD-99-183, September 20, 1999.




                    Page 8                                        GAO/T-GGD-99-185 LSC Case Reporting Problems
  Statement
  Legal Services Corporation: More Needs to Be Done to Correct Case Service Reporting
  Problems




• develop a standard protocol for future self-inspections to ensure that
  grantees systematically and consistently report their results for open and
  closed cases;
• direct grantees to select samples for future self-inspections that are
  sufficient to draw reliable conclusions about magnitude of case data
  errors; and finally,
• ensure that procedures are in place to validate the results of LSC’s 1998
  self-inspection, as well as of any future self-inspections.

  In a written response to a draft of our report, the President of LSC
  generally agreed with our findings and noted that he plans to implement
  our recommendations to the fullest extent possible.

  Mr. Chairman, this concludes my prepared statement. I would be pleased
  to answer any questions that you or other Members of the Committee may
  have.

  Contacts and Acknowledgement
  For further information regarding this testimony please contact Laurie E.
  Ekstrand or Evi Rezmovic on (202) 512-8777. Individuals making key
  contributions to this testimony included Mark Tremba and Jan
  Montgomery.




  Page 9                                  GAO/T-GGD-99-185 LSC Case Reporting Problems
Page 10   GAO/T-GGD-99-185 LSC Case Reporting Problems
Page 11   GAO/T-GGD-99-185 LSC Case Reporting Problems
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order made
out to the Superintendent of Documents, when necessary. VISA
and MasterCard credit cards are accepted, also. Orders for 100 or
more copies to be mailed to a single address are discounted 25
percent.

Order by mail:

U.S. General Accounting Office
P.O. Box 37050
Washington, DC 20013

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000 or by using
fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any list
from the past 30 days, please call (202) 512-6000 using a touch-
tone phone. A recorded menu will provide information on how to
obtain these lists.

For information on how to access GAO reports on the INTERNET,
send e-mail message with “info” in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                Permit No. G100
Official Business
Penalty for Private Use $300

Address Correction Requested




(182083)