oversight

Federal Bankruptcy Judges: Weighted Case Filings as a Measure of Judges' Case-Related Workload

Published by the Government Accountability Office on 2003-05-22.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                             United States General Accounting Office

GAO                          Testimony
                             Before the Subcommittee on Commercial
                             and Administrative Law, Committee on
                             the Judiciary, House of Representatives

For Release on Delivery
Expected at 11:00 a.m. EDT
Thursday, May 22, 2003       FEDERAL BANKRUPTCY
                             JUDGES
                             Weighted Case Filings as a
                             Measure of Judges' Case-
                             Related Workload
                             Statement of William Jenkins, Jr., Director
                             Homeland Security and Justice Issues




GAO-03-789T
This is a work of the U.S. government and is not subject to copyright protection in the
United States. It may be reproduced and distributed in its entirety without further
permission from GAO. However, because this work may contain copyrighted images or
other material, permission from the copyright holder may be necessary if you wish to
reproduce this material separately.
Mr. Chairman and Members of the Subcommittee:

I am pleased to be here today to discuss the results of our review and
assessment of bankruptcy court-weighted case filings, the workload
measure the Judicial Conference first considers in assessing the need for
additional bankruptcy judgeships. Weighted filings are a statistical
measure of the estimated judge time that specific types of bankruptcy
cases are expected to take. For example, a business chapter 7 bankruptcy
case with assets of $50,000 to $499,999 is expected to take about twice as
much judge time as a nonbusiness chapter 7 case with assets of $50,000 to
$499,999. You asked us to assess whether weighted case filings are a
reasonable means of measuring bankruptcy judges’ case-related workload
and to assess the methodology of any proposal to update the current case
weights.

My statement today focuses on the weighted case filings as a measure of
case-related bankruptcy judge workload. My testimony is based on the
results of our review of documentation provided by the Federal Judicial
Center (FJC) and the Administrative Office of the U.S. Courts (AOUSC)
and interviews with officials in each organization. My statement includes
the following major points:

•   The time demands on bankruptcy judges are largely a function of the
    number and complexity of the cases on their dockets. Not all cases
    necessarily take the same amount of judge time. Some types of cases
    may take more judge time than others.

•   In assessing the need for new bankruptcy judgeships, the Judicial
    Conference relies on the weighted case filings to be a reasonably
    accurate measure of case-related bankruptcy judge workload. Whether
    weighted case filings are a reasonably accurate workload measure
    rests in turn on the soundness of the methodology used to develop the
    case weights.

•   On the basis of the documentation provided for our review and
    discussions with FJC and AOUSC officials, we concluded that weighted
    case filings, as approved by the Judicial Conference in 1991 and
    amended in 1996, are likely to be a reasonably accurate means of
    measuring the case-related workload of bankruptcy judges.

•   The original case weights are now about 12 years old and were based
    on time data that are now about 15 years old. Changes in the
    intervening years in such factors as case characteristics and case


Page 1                                                         GAO-03-789T
                 management practices may have affected whether the case weights
                 continue to be a reasonably accurate measure of case-related judge
                 workload. Some of these changes may have increased the time
                 demands on bankruptcy judges and others reduced time demands. To
                 the extent that the case weights may now understate or overstate time
                 demands on bankruptcy judges, the weights could potentially result in
                 the Judicial Conference understating or overstating the need for new
                 bankruptcy judgeships. The Judicial Conference’s Committee on the
                 Administration of the Bankruptcy System has approved a revision of
                 the current weights whose methodological design is reasonable.

             •   The accuracy of the case weights is also dependent upon accurately
                 assigning each case filed in each bankruptcy court to the appropriate
                 case weight category. AOUSC said that its staff took a number of steps
                 to ensure that individual cases were assigned to the appropriate case
                 weight category. These are described in appendix I. We did not
                 evaluate how effective these measures may be in ensuring data
                 accuracy.

             Biennially, the Judicial Conference, the federal judiciary’s principal
Background   policymaking body, assesses the judiciary’s needs for additional
             judgeships.1 If the Conference determines that additional judgeships are
             needed, it transmits a request to Congress identifying the number, type
             (courts of appeals, district, or bankruptcy), and location of the judgeships
             it is requesting. In 2003, the Judicial Conference sent to Congress requests
             for 93 new judgeships—11 for the courts of appeals, 46 for the district
             courts, and 36 for the bankruptcy courts.

             The demands upon judges’ time are largely a function of both the number
             and complexity of the cases on their dockets. Some types of cases may
             demand relatively little time, and others may require many hours of work.
             The federal judiciary has developed workload measures for bankruptcy
             judges to estimate the national average amount of a judge’s time that
             different types of cases may require. Individual judges may actually spend
             more or less time than this average on specific cases within each type—
             such as personal chapter 7 bankruptcy cases with assets of less than
             $50,000 or chapter 13 cases with liabilities of $50,000 or more (see app. II).



             1
              The Chief Justice of the United States presides over the Conference, which consists of the
             chief judges of the 13 courts of appeals, a district judge from each of the 12 geographic
             circuits, and the chief judge of the Court of International Trade. The Conference meets
             twice a year.



             Page 2                                                                       GAO-03-789T
                       In assessing the need for additional bankruptcy judgeships in a bankruptcy
                       court, the Judicial Conference first considers the court’s weighted case
                       filings. The Judicial Conference has established 1,500 annual weighted
                       case filings per authorized judgeship as an indicator of a bankruptcy
                       court’s potential need for additional judgeships. This represents about
                       1,500 annual hours of case-related judge time. The Conference’s policy for
                       assessing bankruptcy judgeship needs recognizes that judges’ workloads
                       may be affected by factors not captured in the bankruptcy-weighted case
                       filings. Examples of such factors include historical caseload data and filing
                       trends; geographic, economic, and demographic factors in the bankruptcy
                       district; and the availability of alternative solutions and resources for
                       handling a court’s workload, such as assistance from judges outside the
                       district. However, our analysis focused solely on the weighted case filings
                       workload measure.

                       Each case filed in a bankruptcy court is assigned a case weight. The case
                       weight statistically represents the national average amount of judicial
                       time, in hours, each type of bankruptcy case would be expected to require.
                       The case weights are based on a 1988-1989 study in which bankruptcy
                       judges completed diaries on how many hours they spent on specific types
                       of cases and noncase-related work. Total annual weighted case filings for
                       any specific bankruptcy court is the sum of the weights associated with
                       each of the cases filed in the court in a year. Total annual weighted case
                       filings per judgeship represent the estimated average amount of judge time
                       that would be required to complete the cases filed in a specific bankruptcy
                       court in a year.

                       Weighted case filings per judgeship is the total weighted filings divided by
                       the number of authorized judgeships. For example, if a bankruptcy court
                       had 5,100 weighted case filings and three authorized judgeships, the
                       weighted case filings per judgeship would be 1,700. Because this exceeds
                       the 1,500 threshold, the Judicial Conference would consider this court for
                       an additional judgeship. However, it should be noted that the Judicial
                       Conference’s policy is to consider additional judgeships only for those
                       courts that request them. Thus, if a court would otherwise be eligible for
                       an additional judgeship, but did not request one, the Judicial Conference
                       would not request a judgeship for that court.


                       The Federal Judicial Center (FJC) developed the weights, adopted by the
How the Case Weights   Judicial Conference in 1991, based on a 1988-1989 time study in which 272
Were Developed         bankruptcy judges (97 percent of all bankruptcy judge in those years)
                       recorded the time they spent on specific cases for a 10-week period.

                       Page 3                                                           GAO-03-789T
Unlike the District Court time study, whose goal was to follow each
sample case from filing to disposition—a “case tracking” study—this study
was a “diary study” in which judges recorded in a time diary the hours
spent on each case in the study and for other judicial work for the 10-week
period. This period of time may or may not have covered the entire life of
the case from filing through disposition. Appendix III includes a more
detailed comparison of case-tracking and diary time studies as methods of
capturing judge time spent on specific cases.

The case weights were developed using a two-step process.2 First, time
data were collected from 272 judges (97 percent of the total of 280
bankruptcy judges at the time of the study). The judges recorded the time
they spent on a sample of cases and other judgeship work over a 10-week
period. The judges were subdivided into five groups and the recording
time period for each group was staggered over a 1-year period. Second, the
researchers assessed the relative impact on judicial workload of different
types of cases—that is, which types of cases seemed to take more or less
time—and developed individual case weights for specific case categories.
The basic case weight computations involved calculating the average
amount of time spent on cases of each type during each month of their life.
These averages were then summed to determine the total amount of time
for each case type.

Once the case weights had been created, total weighted case filings were
calculated for each bankruptcy court. Then, weighted caseloads were
transformed into initial estimates of required judgeships. These initial
estimates were adjusted to account for factors other than those covered
by the case weight calculation, such as the court’s case management
practices and the time required to travel to divisional offices. After all
adjustments, the study concluded that bankruptcy judges spent about
1,280 hours annually on direct case-related work and an average of 660
hours on matters not directly related to specific cases (e.g., on court and
chambers administration, work-related travel, and other matters related to
the judicial role).

When it approved the case weights in 1991, the Judicial Conference stated
that it expected that in addition to other judicial duties, a bankruptcy court


2
 The methodology is described in detail in Gordon Bermant, Patricia Lombard, and
Elizabeth Wiggins, A Day in the Life: The Federal Judicial Center’s 1988-1989
Bankruptcy Court Time Study, American Bankruptcy Law Journal, Vol. 65 (Lexington, SC:
1991).



Page 4                                                                  GAO-03-789T
                     should have at least 1,500 annual case-related hours per judgeship to
                     justify additional judgeships. The federal work year is 2,080 hours per
                     year, based on a 40-hour work week. Assuming that judges spent 1,500
                     hours annually on cases, there would remain 580 hours for federal
                     holidays, annual leave, training, and noncase-related administrative tasks.
                     Of course, the actual time that individual judges spend on case-related and
                     non case-related work will vary.


                     Overall, the methodology used to develop the bankruptcy case weights
Assessment of Case   appears to be reasonable. The methodology included a valid sampling
Weight Methodology   strategy, a very high participation rate among bankruptcy judges, and a
                     reasonable means of adjusting for such factors as missing data. A notable
                     strength of the methodology was the high participation rate by judges—97
                     percent of the bankruptcy judges at the time of the study. Thus,
                     participating judges represented almost the entire universe of bankruptcy
                     judges that could be included. The sampling period was not limited to a
                     single time of year, thus minimizing potential bias due to variations in case
                     filings by time of year. FJC researchers systematically used the reported
                     time data to develop the case weights and made an effort to address all
                     known limitations in the data. In computing the case weights,
                     assumptions, and adjustments needed to be made to account for time data
                     that were not linked to specific cases, missing data, and other factors.
                     Both the assumptions and the methods used to make these adjustments
                     appeared to be reasonable. It is important to note that the case weights
                     were designed to estimate the impact of case filings on the workload of
                     bankruptcy judges. Noncase-related time demands, such as time spent on
                     court administration tasks, are not included in the case weights. The
                     Judicial Conference focuses its analysis of the need for additional judges
                     primarily on the demands that result from caseload, not noncase-related
                     tasks and responsibilities.

                     Potential limitations of the methodology included the possibility of judges
                     using different standards and definitions to record their time. Although the
                     judges had written instructions on how to record their time, judges may
                     have varied in how they interpreted case-related and noncase-related
                     hours. To the extent this occurred, it may have resulted in the recording of
                     noncomparable time data among judges. Because some cases require
                     longer calendar time to complete than others, not all cases in the sample
                     were completed at the end of the 10 weeks in which judges recorded their
                     time. In particular, the study captured only a small portion of the total time
                     required for very large business bankruptcies. Where the cases were not
                     completed, it was necessary to estimate the judge time that would have

                     Page 5                                                           GAO-03-789T
                    been required to complete the case. However, the method used to make
                    these estimates was also reasonable.

                    The size and time demands of chapter 11 business bankruptcies vary
Amending the Case   considerably. The bankruptcy case weights, which the Judicial Conference
Weights—“Mega”      approved for use in 1991, included a weight of 11.234 hours for chapter 11
                    business filings involving $1 million or more and a weight of 4.021 hours
Chapter 11 Cases    for chapter 11 business filings with assets between $50,000 and $99,999.

                    In 1996, a new method was used for measuring the workload required for
                    very large (“mega”) chapter 11 business cases. This measure was also
                    developed by the FJC and approved by the Judicial Conference’s
                    Bankruptcy Committee. The mega cases were defined as “those involving
                    extremely large assets, unusual public interest, a high level of creditor
                    involvement, complex debt, a significant amount of related litigation, or a
                    combination of such factors.” The Administrative Office of the U.S. Courts
                    defines mega chapter 11 cases as a single case or set of jointly
                    administered or consolidated cases that involve $100 million or more in
                    assets and 1,000 or more creditors. Mega chapter 11 cases are distinct
                    from other large chapter 11 cases in that they generally involve a larger
                    number of associated filings and extend over a longer period of time.

                    The 1991 case weights did not fully reflect the judge time required for
                    these very large, complex bankruptcy filings. The weighting scheme was a
                    particular problem for the Southern District of New York and the District
                    of Delaware, both of which have a high number of mega cases. At the time
                    of the 1988-1989 bankruptcy time study, the highest value for chapter 11
                    cases in the bankruptcy administrative database was $1 million or more.
                    Subsequently, changes were made to the database, which now includes
                    several subcategories for cases above $1 million, the highest being $100
                    million and above. Also, the time study estimated the judge time required
                    by cases for the first 22 months after the case was filed, a period which
                    may not have encompassed the entire calendar time required to dispose of
                    the case. Both of these factors contributed to the inability to create case
                    weights for the mega chapter 11 cases.

                    Beginning in 1996, the adjustment of weighted case filings to account for
                    mega chapter 11 cases was implemented in the two districts where most of
                    these cases have been filed—first in the Southern District of New York and
                    later in the District of Delaware. FJC’s research suggested there was no
                    clear linear relationship between asset size and judge time in mega chapter
                    11 cases. Instead, FJC selected an adjustment method using data routinely
                    collected on docketed events in bankruptcy cases, such as docketed

                    Page 6                                                         GAO-03-789T
                          hearings. The method used to adjust the case weights for mega chapter 11
                          cases consists of a preliminary weighted caseload computation, followed
                          by a ratio adjustment step. The preliminary weighted caseload is the sum
                          of the bankruptcy case weights for each case filing associated with the
                          mega chapter 11 cases. For example, if a mega case consisted of two
                          consolidated cases, one with assets of between $50,000 and $99,999
                          (weight: 4.021) and one with assets greater than $1 million (weight:
                          11.234), the preliminary case weight would be 15.255 (4.021 plus 11.234).
                          In the Southern District of New York, this preliminary case weight is
                          adjusted by the ratio of docketed events per weighted case-hour for mega
                          chapter 11 cases to the docketed events per weighted case-

                          hour for nonmega chapter 11 cases involving more than $1 million in
                          assets.3 In the District of Delaware, where mega chapter 11 cases tended to
                          have a larger number of consolidated filings, several ranges of the number
                          of associated filings are used to classify mega chapter 11 cases. For each
                          range, a separate docketing ratio adjustment is calculated in the same
                          manner as it is for the District of Southern New York. In both districts, the
                          final step is to report these calculations over a period of several years and
                          use the average value across the years as the adjusted weighted caseload
                          for mega chapter 11 cases. The purpose of this final step is to moderate
                          the effect of fluctuations in the number of mega chapter 11 cases filed
                          from year to year.


Assessment of Mega Case   The methodology used to adjust the weighted caseload for mega chapter
Weighting Method          11 cases, specifically the ratio adjustment step, cannot be thoroughly
                          assessed because there are no objective time data to use for comparison.
                          The FJC selected this methodology after extensive research on other
                          possible methods. The overall strategy of applying a ratio adjustment using
                          auxiliary information, followed by use of a multiyear average, is a
                          reasonable approach.




                          3
                           This determines “how the level of docketing in mega cases differs from the docketing in
                          non-mega cases of one million dollars or more.”



                          Page 7                                                                      GAO-03-789T
                      In June 2002, the Judicial Conference Committee on the Administration of
Research Design for   the Bankruptcy System decided to begin a study to create new bankruptcy
Updating the          case weights. The preliminary design for the study has a two-phase
                      structure. In the first phase, a diary time study would be conducted, and
Bankruptcy Case       the time study data would be used to develop new case weights. In the
Weights               second phase, research is planned to assess whether it is possible to
                      develop “event profiles” that would allow future updating of the weights
                      without the necessity of conducting a time study for each update. Future
                      updating of the weights could include revision of case weight values
                      and/or developing case weights for new case categories. The data from the
                      time study can be used to validate the feasibility of the new approach.

                      The preliminary design for the study appears to be reasonable. In the first
                      phase, new weights would be constructed using objective data from the
                      time study. The second part represents experimental research to
                      determine if it is possible to make revisions to the weights in the future
                      without the requirement of conducting a time study. If the research
                      determines this is possible, it would be possible to update the case weights
                      more frequently with less cost than required by a time study.

                      If enacted, it is likely that the bankruptcy reform legislation passed by the
                      House of Representatives would probably affect the time bankruptcy
                      judges would need to devote to personal bankruptcy cases. Personal
                      bankruptcy filings accounted for 97 percent of the total 1,547,669
                      bankruptcy filings in fiscal year 2002. Currently, the great majority of those
                      who file for personal bankruptcy (70 percent in fiscal year 2002), file under
                      chapter 7, in which their eligible debts are discharged. Under the terms of
                      the proposed legislation, a greater proportion of those who file for
                      personal bankruptcy will be required to file under chapter 13 and enter
                      into a 5-year debt repayment plan. If the bankruptcy reform is enacted
                      during the course of the new bankruptcy time study, FJC officials said
                      they would recommend halting the time study and allowing some period
                      for the implementation of the new law before restarting the study. Because
                      personal bankruptcy filings represent the vast majority of bankruptcy
                      filings, this seems to be a prudent plan.


                      On the basis of the documentation provided for our review and
Conclusions           discussions with FJC and AOUSC officials, we concluded that weighted
                      case filings, as approved by the Judicial Conference in 1991 and amended
                      in 1996, was a reasonably accurate means of measuring the case-related
                      workload of bankruptcy judges. The 1991 bankruptcy case weights—
                      which cover all but mega chapter 11 business filings—are now about 12

                      Page 8                                                           GAO-03-789T
                  years old, and the data on which they were based are about 15 years old.
                  Changes since 1991 in such factors as case characteristics and case
                  management may have affected whether the weights continue to be a
                  reasonable measure of case-related bankruptcy judge workload. The
                  design for revising the current bankruptcy case weights seems reasonable.
                  The new weights would be based on the same type of objective time data
                  as are the current weights, and the time data from the new bankruptcy
                  case weight study can be used to validate the feasibility of using an event-
                  based approach for future updates of the weights.


                  Mr. Chairman, this concludes my prepared statement, I would be pleased
                  to answer any questions that you or other members of the Subcommittee
                  may have.


                  For further information regarding this testimony, please contact William
Contacts and      Jenkins, Jr., at (202) 512-8777. Individuals making key contributions to this
Acknowledgments   testimony included David Alexander, Kriti Bhandari, Chris Moriarity, and
                  R. Rochelle Burns.




                  Page 9                                                           GAO-03-789T
Appendix I: Quality Assurance Steps the
Judiciary Takes to Ensure the Accuracy of
Case Filing Data for Weighted Filings
              All current records related to bankruptcy filings that are reported to the
              Administrative Office of the U.S. Courts and used for the bankruptcy court
              case weights are generated by the automated case management systems in
              the bankruptcy courts. Filings records are generated monthly and
              transmitted to AOUSC for inclusion in its national database. On a quarterly
              basis, AOUSC summarizes and compiles the records into published tables,
              and for given periods, these tables serve as the basis for the weighted
              caseload determinations.

              In responses to written questions, AOUSC described numerous steps taken
              to ensure the accuracy and completeness of the filings data, including the
              following1:

              •   Built-in, automated quality control edits are done when data are
                  entered electronically at the court level. The edits are intended to
                  ensure that obvious errors are not entered into a local court’s database.
                  Examples of the types of errors screened for are the district office in
                  which the case was filed, the U.S. Code title and section of the filing,
                  and the judge code. Most bankruptcy courts have staff responsible for
                  data quality control.

              •   A second set of automated quality control edits are used by AOUSC
                  when transferring data from the court level to its national database.
                  These edits screen for missing or invalid codes that are not screened
                  for at the court level, such as dates of case events, the type of
                  proceeding, and the type of case. Records that fail one or more checks
                  are not added to the national database and are returned electronically
                  to the originating court for correction and resubmission.

              •   Monthly listings of all records added to the national database are sent
                  electronically to the involved courts for verification.

              •   Courts’ monthly and quarterly case filings are monitored regularly to
                  identify and verify significant increases or decreases from the normal
                  monthly or annual totals.

              •   Tables on case filings are published on the Judiciary’s intranet for
                  review by the courts.




              1
               Given the limited time for our review, AOUSC was unable to obtain input to our questions
              on data quality control procedures from individual courts.



              Page 10                                                                     GAO-03-789T
•   Detailed and extensive statistical reporting guidance is provided to
    courts for reporting bankruptcy statistics. This guidance includes
    information on general reporting requirements, data entry procedures,
    and data processing and reporting programs.

•   Periodic training sessions are conducted for bankruptcy court staff on
    measures and techniques associated with data quality control
    procedures.

In addition to the quality control procedures listed above, AOUSC
indicated that an audit was performed in 1997 by Clifton Gunderson L.L.C.,
a certified public accounting firm, to test the accuracy of the bankruptcy
statistical data maintained by bankruptcy courts and the AOUSC. The firm
compared individual case records in 11 courts nationwide with data in the
national database for cases filed in 1993, 1994, and 1995 for completeness
and accuracy. Excluding problems in one district, the overall match rate of
all statistical data elements captured exceeded 97 percent, and the fields
with most mismatches were not relevant to the bankruptcy weighted
caseload. AOUSC was unaware of any other efforts to verify the accuracy
electronic data to “hard copy” case records for bankruptcy courts. AOUSC
noted that it did not have time to seek detailed information from the
individual bankruptcy courts on this issue within the short time available
to respond to our questions.




Page 11                                                        GAO-03-789T
Appendix II: Bankruptcy Case Weights and
Confidence Intervals for All Cases Except
“Mega” Chapter 11 Business Filings

               Type of case                      Case weight in hours   Confidence interval
               Chapter 7—Business
               Assets less than $50,000                         0.335         0.312 - 0.359
               Assets $50,000-$499,999                          0.413         0.382 - 0.444
               Assets greater than $499,999                     1.704         1.426 – 1.982
               Chapter 7—Nonbusiness
               Assets less than $50,000                         0.089         0.079 - 0.099
               Assets $50,000-$499,999                          0.160         0.144 - 0.176
               Assets greater than $499,999                     0.302         0.239 – 0.365
               Chapter 11
               Assets less than $50,000                        5.372          5.054 – 5.690
               Assets $50,000-$99,999                          4.021          3.692 – 4.350
               Assets $100,000-$499,999                        4.285          3.991 – 4.579
               Assets $500,000-$999,999                        5.143          4.769 – 5.517
               Assets of $1 million or more                   11.234        10.397 – 12.071
               Chapter 12                                      4.040          3.558 – 4.522
               Chapter 13
               Liabilities less than $50,000                    0.310         0.269 - 0.351
               Liabilities at least $50,000                     0.457         0.410 - 0.504
               Other cases                                      0.194         0.074 - 0.314
               Adversary proceedings
               Dischargeability                                 1.346         1.232 – 1.460
               Other                                            2.016         1.722 – 2.310
              Source: Federal Judicial Center.




              Page 12                                                          GAO-03-789T
Appendix III: Measuring Judicial Workload
Using the Collection of Time Study Data

                        The current Bankruptcy Court and District Court workload measures were
                        developed using data collected from time studies. The District Court time
                        study took place between 1987 and 1993, and the Bankruptcy Court time
                        study took place between 1988 and 1989. Different procedures were used
                        in these two time studies. The Bankruptcy Court time study protocol is an
                        example of a “diary” study, where judges recorded time and activity details
                        for all of their official business over a 10 week period. The District Court
                        time study protocol is an example of a “case-tracking” study, where a
                        sample of cases were selected, and all judges who worked on a given
                        sample case recorded the amount of time they spent on the case. Time
                        studies, in general, have the substantial benefit of providing quantitative
                        information that can be used to create objective and defensible measures
                        of judicial workload, along with the capability to provide estimates of the
                        uncertainty in the measures.


                        At the conclusion of a case-tracking study, total time spent on each sample
Estimating Judge        case closed during the study period is readily available by summing the
Time in Diary and       recorded times spent on the case by each judge who worked on the case.
                        For a given case type, the summed recorded times can be averaged to
Case Tracking Studies   obtain an estimate of the average judicial time per case for that case type.

                        For a diary study, however, it is necessary to make estimates of judicial
                        workload for all cases that were not both opened and closed during the
                        data collection period. This estimation step requires information from the
                        caseload database, and thus the accuracy of estimates depends in part on
                        the accuracy of the caseload data. Two kinds of information are required
                        from the caseload database: case type and length of time the case has been
                        open.

                        With the diary approach, the total judicial time that is required for lengthy
                        case types is estimated by combining “snap shots” of the time required by
                        such cases of different ages. Thus, in theory, reducing accurate weights for
                        lengthy case types is not problematic. In practice, however, difficulties
                        may be encountered. For example, in the 1988-1989 bankruptcy time
                        study, the asset and liability information for cases older than 22 months
                        was inadequate and appropriate adjustments had to be made. In addition,
                        difficulties may arise if only a small number of cases of the lengthy type
                        are in the system. This is an issue FJC said it is considering as it finalizes
                        how to assess the judicial work associated with mega cases in the
                        upcoming bankruptcy case-weighting study.




                        Page 13                                                          GAO-03-789T
                             Each study type has advantages and disadvantages. The following outlines
Comparing Case-              the similarities and differences in terms of burden, timeliness of data
Tracking Studies and         collection, post-data collection steps, accuracy, and comprehensiveness.
Diary Studies

Burden on Participants       Each study type places burden on judicial personnel during data
                             collection. It is not clear that one study type is less burdensome than the
                             other. The diary study procedure requires more concentrated effort, but
                             data are collected for a shorter period of time.


Timeliness of Data           Data collection for a diary study can be completed more quickly than for
Collection                   a case-tracking study.


Post Data Collection Steps   More effort is needed to convert diary study data to judicial workload
                             estimates than case tracking study data. Also, the accuracy of estimates
                             from diary study data depends in part on the accuracy and objectivity of
                             the information in the caseload database.


Data Accuracy                It is not clear that one study type collects more accurate data than the
                             other study type. Some of the Bankruptcy Court case-related time study
                             data could not be linked to a specific case type due to misreporting errors
                             and/or errors in the caseload database. Some error of this type likely is
                             unavoidable because of the requirement to record all time rather than
                             record time for specific cases only. However, it is plausible that a diary
                             study collects higher quality data, on average, because all official time is to
                             be recorded during the study period; judicial personnel become
                             accustomed to recording their time. In contrast, the data quality for a case-
                             tracking study could decline over the study’s length; for example, after a
                             substantial proportion of the sample cases are closed, judicial personnel
                             could become less accustomed to recording time on the remaining open
                             cases.


Comprehensiveness and        In theory, a case-tracking study collects more comprehensive information
Efficiency                   about judicial effort on a given case than a diary study, because data for a
                             sampled case almost always are collected over the duration of the case.
                             (Data collection may be terminated for a few cases that remain open, or
                             are reopened, many years after initial filing.) For case types that

                             Page 14                                                           GAO-03-789T
           simultaneously stay open for a long period and require a substantial
           amount of judicial effort, it is possible that a diary study would not be able
           to produce suitable estimates of judicial workload due to a lack of data.




(440200)
           Page 15                                                           GAO-03-789T
                         The General Accounting Office, the audit, evaluation and investigative arm of
GAO’s Mission            Congress, exists to support Congress in meeting its constitutional responsibilities
                         and to help improve the performance and accountability of the federal
                         government for the American people. GAO examines the use of public funds;
                         evaluates federal programs and policies; and provides analyses,
                         recommendations, and other assistance to help Congress make informed
                         oversight, policy, and funding decisions. GAO’s commitment to good government
                         is reflected in its core values of accountability, integrity, and reliability.


                         The fastest and easiest way to obtain copies of GAO documents at no cost is
Obtaining Copies of      through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full-
GAO Reports and          text files of current reports and testimony and an expanding archive of older
                         products. The Web site features a search engine to help you locate documents
Testimony                using key words and phrases. You can print these documents in their entirety,
                         including charts and other graphics.
                         Each day, GAO issues a list of newly released reports, testimony, and
                         correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site
                         daily. The list contains links to the full-text document files. To have GAO e-mail
                         this list to you every afternoon, go to www.gao.gov and select “Subscribe to daily
                         E-mail alert for newly released products” under the GAO Reports heading.


Order by Mail or Phone   The first copy of each printed report is free. Additional copies are $2 each. A
                         check or money order should be made out to the Superintendent of Documents.
                         GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a
                         single address are discounted 25 percent. Orders should be sent to:
                         U.S. General Accounting Office
                         441 G Street NW, Room LM
                         Washington, D.C. 20548
                         To order by Phone:     Voice:    (202) 512-6000
                                                TDD:      (202) 512-2537
                                                Fax:      (202) 512-6061


                         Contact:
To Report Fraud,
                         Web site: www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in      E-mail: fraudnet@gao.gov
Federal Programs         Automated answering system: (800) 424-5454 or (202) 512-7470


                         Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800
Public Affairs           U.S. General Accounting Office, 441 G Street NW, Room 7149
                         Washington, D.C. 20548