oversight

Railroad Safety: FRA's Staffing Model Cannot Estimate Inspectors Needed for Safety Mission

Published by the Government Accountability Office on 1990-11-21.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                                               I




-GAO’   Report to the Chairman, Committee on
        Ihergy and Commerce, House of
        Reprctsentatives



        RAILROAD SAFETY
        F’FWs Staffing Model
        Cannot Estimate
        Inspectors Needed for
        Safety Mission


                                #IIIllllllllll
                                   142744
                   united
                     states
GAO                General Accounting Office
                   Washington, D.C. 20648

                   Resources, Commuuity, and
                   Economic Development Division

                   B-241384

                   November 21,199O

                   The Honorable John D. Dingell
                   Chairman, Committee on Energy
                     and Commerce
                   House of Representatives

                   Dear Mr. Chairman:

                   Concerned about railroad safety, you asked us to assess the effective-
                   ness of the Federal Railroad Administration’s (F’RA)safety inspection
                   program. This report, the fourth in a series addressed to you, focuses on
                   FRA’Scomputerized staffing model and its usefulness in developing
                   staffing standards to determine the number of inspectors needed to sat-
                   isfy FRA’Ssafety mission. In our prior reports,l we concluded that FRA
                   did not have enough hazardous materials inspectors and that staffing
                   levels in other disciplines might be inadequate.


                   FRAdoes not know how many inspectors it needs to adequately cover
Results in Brief   the railroad industry because it had not fully developed inspector
                   staffing standards for its inspector work force. Staffing standards are
                   derived from formulas or mathematical models used to determine the
                   number of employees needed to perform a function and to distribute
                   these employee resources.

                   FRAdeveloped a staffing model in 1986 to assist it in estimating how
                   many inspectors it needs and how to allocate its inspector work force
                   among the inspection disciplines and regional offices. To provide a
                   staffing standard that would be useful in determining the number of
                   inspectors needed to satisfy its safety mission, FRA’S staffing model
                   would need to provide for (1) calculating the number of inspectors it
                   needs and allocating the inspectors among its offices, (2) incorporating
                   inspection coverage standards that include information on the railroad
                   operations needing inspections, the time required to perform inspec-
                   tions, and the frequency of inspections, and (3) using available data to

                   LRailroadSafety:FRA Needsto CorrectDeficienciesin ReportingInjuries and Accidents
                   (GAOIRCED-89-109,Apr. 6,198Q).
                   RailroadSafety:DUI’ ShouldBetter ManageIts HazardousMaterials InspectionProgram
                   (GAO/RCED-90-43, Nov. 17,1989).
                   RailroadSafety:New ApproachNeededfor Effective FRASafetyInspectionProgram
                   (GAO/RCEDQO-194,  July 31,lQQO).



                   Page 1                                                     GAO/RCEB9132 Railroad Safety
             target routine inspections toward high-risk locations and railroads with
             poor safety records. The model provides for the first part of the staffing
             standard, although it bases its estimates of staff needed on historical
             data and budgetary factors, such as authorized positions, total inspec-
             tions performed, accidents, and funds allocated, rather than on mission
             factors. The model does not provide for the other elements.

             Without incorporating inspection coverage standards or an inspection
             strategy to target high-risk railroads into its current staffing model, a
             new model, or some other analytical method, FRAwill not have adequate
             staffing standards to determine the number of inspectors needed to sat-
             isfy its safety mission. In July 1990, F&I was in the process of designing
             safety inspector workload data needed to develop inspection coverage
             standards. Once these coverage standards are developed, FRAcould use
             these standards, along with its safety data on high-risk railroads, to
             better determine the size and makeup of the inspector work force
             needed to carry out its safety inspection mission.


             FRAis responsible for enforcing federal railroad safety regulations. In
Background   September 1990, FRA had a work force of 361 safety inspectors in eight
             regional offices to accomplish its mission. FRA groups inspectors into five
             disciplines: track; motive power and equipment; operating practices;
             signal and train control; and hazardous materials. Inspectors conduct
             routine safety inspections at the railroads and investigate accidents and
             complaints, When inspections or complaint investigations reveal non-
             compliance with the laws, inspectors list the condition as a defect on an
             inspection report. When inspectors identify defects that pose an imme-
             diate safety hazard, they prepare a violation report that is submitted to
             FRA’SOffice of Chief Counsel to be used to assess the railroad a civil
             penalty. Inspectors also periodically participate in broad system assess-
             ments of a single railroad which includes an evaluation of a railroad’s
             entire operation.

             Ebcecutive Order 12662, dated February Z&1986, established a govern-
             mentwide program to improve the quality, timeliness, and efficiency of
             services provided by the federal government. To achieve these goals, the
             executive order advocates the use of measurement systems and per-
             formance standards by government agencies. In response to the execu-
             tive order, FRA developed the Quality Improvement Program to gather
             and develop inspector workload data. In response to a congressional
             directive, FRA developed the staffing model to give it a methodology to
             determine the size of its inspector work force and a plan to allocate


             Page 2                                          GAO/RCED81-32Railroad Safety
                       those resources. The model was designed to enable FRA to make more
                       informed decisions about the size and deployment of its inspector
                       resources.


                       FRA’Sstaffing model has three distinct parts. The first part uses informa-
Staffing Model Based   tion on the previous number of authorized positions, number of acci-
on Historical Data     dents, number of casualties, revenue ton-miles,2 and total number of
                       inspections performed to estimate the number of inspectors available to
                       perform inspections. The second part allocates available inspectors to
                       each of FRA’Seight regional offices on the basis of each region’s share of
                       nine risk factors, such as accidents, population, density, and rail pas-
                       senger traffic. The third part of the model takes the estimated allocation
                       of inspector resources for each region and estimates how those
                       resources should be distributed among the five disciplines, on the basis
                       of previous work performed by each discipline.

                       FRA’Sstaffing model provides an estimate of staffing needs that is highly
                       dependent on historical data rather than on the staff needed to satisfy
                       its safety mission. Further, past budgetary constraints have limited the
                       size of F&I’s work force, and funding shortfalls have prevented FRA from
                       hiring enough staff to meet its authorized level. For example, for fiscal
                       year 1990, FRArequested and received authority for 34 additional
                       inspectors, but no funding was provided for these positions. Because the
                       staffing model bases its estimates of staff needed on historical data, the
                       model implicitly assumes that budgetary factors rather than mission
                       factors will continue to control staff levels.


                       We reviewed FRA’S staffing model as part of our management review of
A Staffing Model       the Department of Transportation. Our management review was a
Should Include         broad-based review of the management of the Department. One element
Inspection Coverage    included in the review was how FRAcould use productivity information
                       to set program goals, evaluate results, and support resource decisions.
Standards and          We concluded in our April 1987 report that the model was not based on
Inspection Targeting   the actual time spent completing inspection tasks or on standards
                       relating to the time those tasks normally should take. We recommended
                       that FRAuse current and accurate staffing standards in formulating its
                       safety program budget3
           w
                       2Revenueton-mile represents1 ton carried 1 mile for which a chargeis received.
                                                  on: EnhancingPolicy and ProgramEffectivenessThrough Improved
                                                 87-3, Apr. 13,1987).



                       Page 8                                                        GAO/WED-91-32Railroad Safety
                      0241884




                      In our July 1990 report on FRAinspections, we stated that FRA was not
                      basing the number of inspectors needed on inspection coverage stan-
                      dards that consider time spent completing inspections and the frequency
                      with which railroads’ track and equipment should be inspected. We also
                      found that FRAhad not developed an inspection strategy baaed on
                      existing safety data that would target inspection resources to high-risk
                      railroads. Agreeing that it needed to be more systematic in its approach
                      to inspections, FRA stated that it was making changes to use its safety
                      data more effectively in planning inspections to make the pattern of
                      inspections more strategic and that it was refining its management tools
                      to enhance the inspection program. Once these actions are completed,
                      FRAcould incorporate inspection coverage standards and its strategy to
                      inspect high-risk railroads into its current staffing model, a new model,
                      or some other analytical method. FRAcould then make staffing calcula-
                      tions based on the desired amount of inspection coverage, the amount of
                      time required to complete this work, and the amount of time inspectors
                      have available to perform inspections and other safety-related
                      activities.


Inspection Coverage   In our July 1990 report, we recommended that FRAdevelop inspection
Standards             coverage standards that prescribe (1) the frequency with which rail-
                      roads should be inspected, (2) the size of an inspector’s territory, (3) the
                      number of inspection locations an inspector can reasonably cover, and
                      (4) the frequency with which locations should be inspected. In response
                      to our recommendation, FRAstated that it was gathering the information
                      necessary to develop inspection coverage standards, and plans to use
                      coverage standards in its staffing model calculations.

                      FRAcurrently has two initiatives that should provide most of the data
                      needed to develop inspection coverage standards. In January 1989, FRA
                      initiated an effort called the Quality Improvement Program to gather
                      and develop inspector workload data. This program is designed to deter-
                      mine how long it takes inspectors to conduct routine inspections, write
                      reports, evaluate waiver petitions, and investigate accidents and com-
                      plaints. Under this program, inspectors are required to submit a daily
                      report that details their activities by the hour. In addition to providing
                      information on how long it takes inspectors to perform safety-related
                      activities, the program is intended to determine the amount of time
                      spent on other activities such as breaks and travel between inspection
                      points.




                      Page 4                                           GAO/RCED-91-32Railroad Safety
                        In the spring of 1990, FRAdesigned the Regional Inspection Points Pro-
                        gram. FRAofficials stated that this program is intended, in part, to pro-
                        vide FRAthe detailed information it now lacks about railroad activity.
                        The program requires inspectors to identify the inspection points in
                        their territory and record information on the volume of railroad activity
                        at, and the physical characteristics of, each point. For example, track
                        inspectors must identify the railroad class, route miles, total track miles,
                        and the number of switches and rail-highway crossings associated with
                        each point. Once this effort is complete, FRAwill have workload data
                        from the Quality Improvement Program as well as data on railroad
                        activity that are key to developing inspection coverage standards.


Targeting Inspections   In our July 1990 report on FRA’Sinspection program, we also stated that
                        FM does not use available data to target routine inspections toward
                        high-risk locations and railroads with poor safety histories. This has
                        occurred because inspectors independently schedule their inspection
                        efforts absent any analysis of existing safety or accident data. Because
                        of this, we found little relationship between changing accident trends
                        and FRAinspection activity. As a result, railroads with increasing num-
                        bers of accidents did not receive additional inspection activity. We found
                        in many cases inspections decreased for railroads with increasing num-
                        bers of accidents. This indicated a misallocation of inspection resources
                        since a worsening level of safety on a railroad should require more, not
                        less, inspection resources to determine the cause for the rise in
                        accidents.

                        In our report, we made several recommendations to improve FRA’Ssafety
                        inspection program, including using its safety data to target high-risk
                        railroads for routine inspections. We also recommended that FRArede-
                        fine the approach to system assessments by using existing inspection
                        data to detect known areas of weakness and assign inspector resources
                        to determine the underlying causes of the weaknesses. In response to
                        our report, the Administrator, FRA,stated in September 1990 that

                        FRA’s safety inspection program would be greatly improved by targeting our
                        resources more effectively on the basis of the wealth of statistics at our command.
                        Given the size of our inspector force in relation to the size of the railroad industry,
                        we can maximize the effect of our resources only by deploying them as scientifically
                        and strategically as possible.




                        Page 5                                                   GAO/WED-91-32Railroad &tfety
                  B2413s4




                  FRAdoes not know whether it has a sufficient number of safety inspec-
Conclusions       tors to carry out its safety mission because FRAhas not developed
                  inspection coverage standards. FRAhas initiated two efforts aimed at
                  gathering the data it needs to develop such standards. Once inspection
                  coverage standards are developed, FFUwould need to incorporate them,
                  along with an inspection strategy based on targeting high-risk railroads,
                  into its existing staffing model, a new model, or some other analytical
                  method to develop the type of staffing standards that would calculate
                  the number and types of safety inspectors it needs to fulfill its safety
                  mission.


                  We recommend that the Secretary of Transportation direct the Adminis-
Recommendation    trator, FRA,to develop staffing standards that determine the number of
                  safety inspectors it needs to carry out its safety mission. Such standards
                  should include (1) a method of calculating the number of inspectors it
                  needs and distributing them by discipline to FFU’Sregional offices; (2)
                  inspection coverage standards that include information on the railroad
                  operations needing inspections, the time required to perform inspec-
                  tions, and the frequency of inspections; and (3) a strategy of using avail-
                  able data to target routine inspections toward high-risk locations and
                  railroads with poor safety records.


                  We discussed the findings in this report with FFUofficials, who generally
Views of Agency   agreed with our findings. They said that in response to our July 1990
Officials         inspection report they have devised an extensive management program
                  that will be used to quantify the number of state and federal inspectors
                  needed to accomplish inspection coverage standards that are being
                  developed. Included in this management program will be data from both
                  the Quality Improvement and Regional Inspection Points programs, In
                  addition, FRA officials said that they are introducing an inspector
                  resource allocation plan that will consider, among other things, an
                  overall risk profile of each railroad. As requested, however, we did not
                  obtain official agency comments on a draft of this report.

                  We performed our work for this review from July 1989 to October 1990
                  in accordance with generally accepted government auditing standards.
                  Appendix I contains details of our objective, scope, and methodology. As
                  arranged with your office, unless you publicly announce its contents
                  earlier, we plan no further distribution of this report until 30 days from
                  the date of this letter. At that time we will send copies to the Secretary



                  Page 6                                          GAO/RCED-91-32Railroad Safety
8241884




of Transportation; the Administrator, FRA;and other interested parties.
We will make copies available to others upon request.

This work was performed under the direction of Kenneth M. Mead,
Director, Transportation Issues, who may be reached at (202) 275-1000.
Major contributors to this report are listed in appendix II.




J. Dexter Peach
Assistant Comptroller General




Page 7                                         GAO/WED-91-22Railroad Safety
Letter
Appendix I
Objective, Scope,and
Methodology
Appendix II                                                                                   11
Major Contributors to
This Report




                        Abbreviations

                        Do-r      Department of Transportation
                        FRA       Federal Railroad Administration
                        GAO       General Accounting Office


                        Page8                                       GAO/Rm91-32   Itdhad   Safety
Page 9   GAO/RCED-9132Railroad Safety
Appendix 1

Objective,Scope,and Methodology


              This report addresses the Federal Railroad Administration’s (FRA)
              efforts to determine its safety inspector work force needs. Our objective
              was to assess the usefulness of FRA’Sstaffing model in estimating
              inspector needs in relation to satisfying FRA’s safety mission. We
              reviewed F+RA’S  staffing model. We did not attempt to validate this model
              because it is a policy-assisting model. Models such as this cannot, by its
              very nature, be validated to the extent that its output can be relied upon
              as an exact predictor of the future. We reviewed the model to enhance
              our understanding of its output. We focused on the major assumptions
              of the model and the relationships between these assumptions. We
              reviewed all spreadsheet formulas, In this context, we observed no
              major problems; however, this does not attest to the validity of FRA’S
              model.

              FRA’Sexisting staffing model estimates staffing needs based on historical
              data and then allocates these estimated resources by region and disci-
              pline. Because this report focuses on satisfying FRA’s safety mission, we
              refer to only selected portions of FM’S staffing model in the body of this
              report.

              We reviewed reports and other documents discussing the staffing
              model’s development and usage. We interviewed FRA’SOffice of Safety
              officials to determine how they use the model to make staffing decisions
              and the sources of the data used in the model. We reviewed staffing
              decisions made by FRAwithout model input. We further reviewed and
              discussed planned program improvements with Office of Safety and
              regional officials.

              We conducted our review from July 1989 through October 1990 at FRA
              headquarters in Washington, DC.; region 2 headquarters in Philadel-
              phia; and region 3 headquarters in Atlanta. This review was performed
              in accordance with generally accepted government auditing standards.




              Page 10                                         GAO/RCED-91-32RaIlroad Safety
, Appendix II

 Major Contributors to This Report


 Resources,              Roy J. Kirk, Assistant Director
 Community, and          John S. Kalmar, Jr., Assignment Manager
                         Judy K. Pagano, Senior Operations Research Analyst
 Economic
 Development Division,
 Washington, D.C.

 Philadelphia Regional   LaVerne G. Tharpes, Evaluator-in-Charge
 Office                  Nancy D. Wagner, Evaluator




  (sraeo7)               Page 11                                       GAO/RCED-91-32RaIlroad safety
c   .
    -----...--.“.                --I .._-l-._(.l.-..-._.-
                                                       --_“l-ll..- -------




    _-.“_   ..-   -I ..-.   --

I                                                                            Ortl~*rinlg   Information

                                                                             ‘l’ht~ Iirsl. five copicss of each GAO report, are fret?. AddiGonal
                                                                             c*opicbs are $2 each. Ordctrs should be sent to the following
                                                                             atlclrthss, accompanied         by a check or money order made out. to
                                                                             thca Snpc~rint.c?tldt,nt, of IIocuments,     when necessary. Orders fur
                                                                              100 or more copies 1.0 lx* mailed to a single address are
                                                                             tlis~unt~~d       25 pr!rctent.

                                                                             1J.S. Gonc~ral Accounting   Office
                                                                             1’. 0. 130x 6015
                                                                             (;ait.hc~rsburg, MI) 201377

                                                                             Ortlcrs   may also be placed   by calling   (202)   275-624 1.