oversight

Office of Personnel Management: Better Performance Information Needed

Published by the Government Accountability Office on 1990-02-07.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

r

                                                       ”   ...”   ,_   ”
                                          .._..   -.
I__   _-_--_.-   -.-...   --_.   .-   ”
                      United States

GAO                   General Accounting  Office
                      Washington, D.C. 20548

                      General   Government   Division

                      B-232596

                      February 7, 1990

                      The Honorable Constance Newman
                      Director, Office of Personnel
                      Management

                      Dear Ms. Newman:

                      The Office of Personnel Management (OPM), as the agency with primary
                      responsibility for federal work force issues, provides a variety of ser-
                      vices to federal agencies, individuals seeking federal employment, and 9
                      million federal employees, retirees, and their dependents. These services
                      include providing information to, and examining the qualifications of,
                      applicants seeking federal employment; training current employees;
                      maintaining a health benefit program for employees, retirees, and their
                      survivors; and establishing and making changes to annuity payments to
                      retired federal employees. This report assesses OPM's use of performance
                      measures and standards to manage the delivery of these services.

                     The Civil Service Reform Act of 1978 recognizes the need for perform-
                     ance measures and emphasizes that, where feasible, organizational and
                     individual performance should be appraised in terms of timeliness, qual-
                     ity, and efficiency. Also, the,Office of Management and Budget (OMB),
                     promotes the establishment of performance measures for timeliness,
                     quality, efficiency, and customer satisfaction as part of its Federal Pro-
                     ductivity and Quality Improvement Program in which OPM participates.


                     We identified 24 key outputs and services for OPM's operations units.
Results in Brief     (See appendix I.) Although OPM already has one or more performance
                     measures and standards for most of these, certain improvements could
                     be made in the following areas:

                   9 Many key services lack the full range of potential performance measures
                     and standards, In particular, there are very few performance standards
                     for efficiency or customer satisfaction.
                   . Although many performance measures exist at lower management
                     levels, summarized performance information on only a limited number
                     of key services is provided to the directorate level. While not all meas-
                     ures for all services may be desired or needed at the directorate level,
                     there are relatively few key activities reported on in a manner that
                     allows the director to routinely and systematically track how well OPM is
                     doing in many key arcas.


                                                                                                        i
                     Page I                                   GAO/GGD4JO4   Performance   Information
                            B-232596




                        .   Senior Executive Service and General Management workplans, which
                            annually document expected individual and organizational achieve-
                            ments and are the basis for appraising performance, seldom identify the
                            need to meet existing performance standards and make only limited use
                            of performance information.

                            We recognize that the 24 services and 96 potential measures and stan-
                            dards we identified may not coincide directly with those that the direc-
                            tor wishes to track. In selecting areas to track performance and deciding
                            which measures to use, OPM needs to (1) prioritize the functions and ser-
                            vices it believes are the most important and then determine which per-
                            formance dimensions are critical for each of those services, and (2)
                            consider costs for routine tracking and reporting of performance.

                            Subsequent to our December 1, 1989, briefing on our results and recom-
                            mendations, CIPM initiated agency-wide efforts to improve its develop-
                            ment and use of performance measures and standards.


                            To help ensure that operations are managed properly and customers
Background                  served satisfactorily, an organization needs performance measures and
                            standards. A performance measure, such as the number of days
                            required to process retirement claims, can be matched against a stand-
                            ard to determine if management’s or customers’ expectations are being
                            met. Measures by themselves can identify whether performance is
                            improving or declining by comparing to past periods.

                            While line managers need detailed performance measures to ensure that
                            program objectives are being met and services are being appropriately
                            provided to customers on a daily basis, top management needs summa-
                            rized performance information to track mission achievement, identify
                            problems, and hold managers accountable.


                            In our review, we evaluated how well OPM uses appr0priat.e measures
Objective, Scope, and       and standards in assessing the quality, effectiveness, and efficiency of
Methodology                 services provided to customers, Our examination covered CPM’Smajor
                            service-providing organizations-those    involved in the retirement and
                            health insurance programs, employee background investigations: career
                            entry and employee development activities (recruiting and training),
                            and evaluations of other agencies’ personnel activities.




                            Page 2                                   GAO/GGD-9044   Performance   Information
                            B-232596




                            To achieve our objective, we identified (1) the key services of OPM'S
                            operational units and what performance measures and standards exist
                            for these services, (2) what information about delivery of these services
                            is routinely provided to the OPM director, and (3) how OPM uses perform-
                            ance expectations and standards to hold managers accountable for
                      -.    results. We examined 15 Senior Executive Service (sF~) and 7 General
                            Management (GM) workplans of managers having direct responsibility
                            for the services. Appendix II contains additional details on our
                            methodology.

                           We did our work between May 1989 and October 1989 at OPM headquar-
                           ters using generally accepted government auditing standards.


                           OPMhas developed some performance measures or standards for all the
Performance                24 services we examined, as well as for many other outputs and actions
Information for            not considered key services. There are, however, still significant gaps
Managing Operations        and limitations in the performance information provided to
                           management.
Could Be Improved

OPM’s Measures Are Both     Most ophf performance measures and standards are quantitative which,
Quantitative and            as a general rule, are easier for managers to use than qualitative meas-
                           ures. For example, Interim Retirement Annuity claims are to be com-
Qualitative                pleted in 12-14 days and with no more than 4.5-percent errors (14 days
                            are used during periods of seasonably high workloads). However, in
                           some instances, specific quantitative measures may not always be easily
                           developed or practical for OPM'S services. In some of these situations,
                           OPM is using alternative, qualit,ative measures and standards. Seven out
                           of 74 existing measures are qualitatively stated and 5 out of 35 stan-
                           dards are qualitative. (See tables III.1 and 111.2.)For example, the stand-
                           ard for the Agency Compliance and Evaluation Office’s Targeted
                           Installation Reviews (‘UK) states that “Reports reflect full coverage of all
                           compliance issues, specify violations and recommend appropriate cor-
                           rective actions. TIK reports are written in clear language and are organ-
                           ized logically.”


Many Potential Measures    Each of the 24 services could potentially have four measures (timeli-
and Standards Have Not     ness, quality, efficiency, and customer satisfaction) suggested by the
                           Federal Productivity and Quality Improvement Program. With four
Been Developed             measures for each of t.he 24 services, there are a possible 96 measures


                           Page 3                                    GAO/GGD-9044   Performance   Information
                              B232696




                              for the key services. Likewise, there are 96 possible standards of per-
                              formance. Our review showed that 73 (76 percent) of the total potential
                              measures and 35 (36 percent) of the possible standards have been devel-
                              oped. (See appendix III.) While it may not prove appropriate to develop
                              all the potential measures and standards, OPM needs to carefully deter-
                              mine its needs in this regard.

                              Measures and standards are missing primarily in the efficiency and cus-
                              tomer satisfaction areas. Slightly more than half the potential measures
                              of efficiency exist for the 24 services, and only one of the 24 services-
                              background investigations-has      an efficiency standard. There are no
                              standards for customer satisfaction for any of the services.

                              Also, there are no measures or standards for the services provided job
                              applicants or for the processing of health benefit claims. In particular,
                              the managers of the health benefits function pointed out that claims
                              processing is done by contract carriers and thus not directly controlled
                              by OPM. In spite of this, we included it in our list because the magnitude
                              of the program makes it a major OPM service, and OPM-not        the contrac-
                              tors-is ultimately responsible for the program, Moreover, we believe
                              measures and standards in the health benefits area would improve OPM’S
                              ability to assess contractor performance.

                              The Federal Health Benefits Program spent about $8.0 billion in fiscal
                              year 1989, covering 9 million employees, annuitants, and dependents.
                              We compared it with a similar program and noted that the Health Care
                              Financing Administration maintains performance measures and estab-
                              lishes standards for monitoring Medicare carriers-some of which are
                              the same firms that serve the Federal Health Benefits Program. For
                              example, one standard requires Medicare carriers to process 95 percent
                              of the physician claims within 18 days after receipt,


Many Measures Are             Performance measures help management identify successes and prob-
Limited in Ability to Track   lems through comparisons with either standards or past performance.
                              Consequently, performance information needs to be collected, organized,
and Report on                 and analyzed so that levels of performance and emerging problems can
Performance                   be clearly identified. Finally, to make measurement data fully useful to
                              management, it needs to be reported on a current basis and used with
                              past data to identify performance trends.

                              Some 0PM performance assessments-particularly     of customer satisfac-
                              tion-have not been fully developed and are limited in both the amount


                              Page 4                                    GAO/GGB904   Perfornxwwe   Information
    of documented analysis provided and in the availability of reports that
    can be tracked over time to assess changes in performance. The follow-
    ing are examples of such assessments.

w The Investigations Group meets with customer agencies regularly to
  assess service-related problems and concerns. Investigations Group offi-
  cials attending the meetings orally report these concerns to Investiga-
  tions Group top management who also attend some of the meetings.
  However, data are not systematically collected in writing to enable anal-
  yses of the extent of various performance problems. For example,
   agency officials are not surveyed in writing on performance issues to
  determine the breadth of problems or whether the level of performance
  is changing. Consequently, while performance data are analyzed, no
  written reports are developed and maintained, and performance over
  time is not determined and documented.
. The Retirement Group managers obtain customer satisfaction data from
  a customer survey form that is placed in an information rack in the
  Retirement Information Center in the OPM headquarters building and
  from retiree correspondence. However, the data are not systematically
  collected, hampering any potential analysis. For example, the customer
  survey form is available only to those who happen to see it in the rack
  and choose to complete it instead of being uniformly provided to all or to
  a statistically valid sample of customers. Further, data from the survey
  and correspondence are not organized to analyze major performance
  issues. Finally, reports are not written on current performance and
  trends in customer service are not determined.
l The Office of Agency Compliance and Evaluation maintains contacts
  with agencies’ officials through written requests for input to program
  changes, formal meetings with high level agency officials, and agency
  responses to compliance report recommendations. Program managers
  believe this type of regular input enables them to assess customer ser-
  vice and determine if the proper compliance issues are being addressed.
  However, as in the above two examples, the information gathered is not
  organized, summarized, or reported, and historical trends are not
  developed.

    Two      OPM   organizations have been working to achieve a more complete
    assessment of customer service through collection of performance infor-
    mation that can be organized, analyzed, and reported, and which will
    permit tracking of performance. One, the Career Entry and Employee
    Development Group, is currently assessing the results of a formal cus-
    tomer service survey in the career entry area.



    Page 6                                      GAO/GGD-9044   Performance   Information
                   R-232596




                   The second, the Washington Area Service Center, has a major ongoing
                   program to improve customer service, As a part of that program, the
                   Center surveyed agencies’ training and development managers to deter-
                   mine how the center could better serve them, and then administered the
                   same survey to over 500 students attending classes to gain additional
                   insights In the job information area, the Center made a limited customer
                   survey at the Job Information Center in OPM headquarters, The Center is
                   applying the information from these surveys in efforts to improve the
                   service delivery process. These efforts include employee participation in
                   improvement action teams.

                   We believe such surveys and other methods of systematically collecting
                   performance data are better than informal feedback as a mechanism for
                   determining customer satisfaction because they enable organizations to
                   systematically analyze, report, and track performance. In the late 1970s
                   a study for the White House Office of Consumer Affairs disclosed that
                   the average business never hears from 96 percent of its unhappy cus-
                   tomers, indicating that reliance on correspondence and informal feed-
                   back can result in missing or misinterpreting customer concerns.

                   It may not be possible or cost-effective to develop measures and stan-
                   dards for all 96 dimensions. However, because of both the large number
                   of performance dimensions without measures or standards and the
                   number of measures that are limited in their usefulness to management,
                   we believe that an assessment of information needs is appropriate. Such
                   an assessment should not only address which missing measures and
                   standards need to be developed, but also who OPM'S customers are and
                   what information is needed to assess services provided to them. Fur-
                   ther, there may be functions other than the ones we selected that man-
                   agement may wish to track in terms of performance. In selecting
                   functions and services to track, we believe particular consideration
                   needs to be given to measures and standards for timeliness, quality, effi-
                   ciency, and customer satisfaction for the part of the health benefits pro-
                   gram administered by contractors for OPM and services to federal
                   employment applicants.


                          top management regularly receives only limited performance
Performance        OPM'S
                   reports from operating units. As described to us by OPM managers, per-
Reporting to OPM   formance reporting is generally kept within the functional units.
Director Can Be    Detailed measures are reported to the first line operating managers, and
                   summarized performance reports are prepared for program managers.
Improved           Performance reports to the directorate level are limited to a few on a


                   Page 6                                   GAO/GGD-9044   Performance   Information
                       B-232596




                       regular basis+ For example, certain performance information has only
                       recently been provided to the director on a routine basis, such as the
                       files of customer correspondence and the monthly letter to the Chair-
                       man, House Committee on Post Office and Civil Service, on timeliness in
                       annuitant claims.

                       This increased information at the directorate level is useful but more
                       needs to be done. The information provided still addresses relatively
                       few key services and dimensions of performance and is not provided in
                       a manner that would allow routine and systematic tracking of how well
                       OPM is doing in many key areas.


                       Regular, summarized reports of performance in key areas of an organi-
                       zation can not only identify problem areas needing attention, but can
                       also identify whether attention to a problem is successful. For example,
                       we used OPM’S data to develop trend charts that illustrate the type of
                       summarized performance information that could be routinely provided
                       to the directorate level. (See appendix IV.) These charts show that man-
                       agement efforts to improve timeliness in the retirement annuity process
                       have been successful, but quality in that process has not met OPM stan-
                       dards. Although we did not validate the performance data, such per-
                       formance reporting may alert management that greater attention may
                       be needed for quality as well as timeliness.

                       We do not believe that the OPM director necessarily needs to routinely
                       receive specific performance information on all 96 dimensions, and as
                       previously stated, you may find that other functions and activities are
                       as important, or more important, to track than the 24 we listed. Rather,
                       the development and use of measures and standards should reflect man-
                       agement’s needs for performance and customer service information.
                       Appendix V contains more details on the issues we believe should be
                       considered in developing a more comprehensive and strategic approach
                       to measuring and reporting performance.

                                    ___-
                       Performance measures and standards should be used as part of the basis
Limited Use of         for appraising managers. In doing so, consideration should be given to
Performance            constraints beyond an individual manager’s control that can prevent
Information to Hold    performance targets from being achieved.
Managers Accountable   Performance standards and measures are used to a limited extent in
                       holding managers accountable for performance. In examining 15 SES
                       workplans, we found that 9 had timeliness performance identified in


                       Page 7                                  GAO/GGD-9044   Perfomance   Information
              B-232596




              general terms as an accountability item and 4 of these had quantitative
              timeliness standards for services and outputs. Similarly, 9 of 15 had
              quality identified and 2 had quantitative standards. Customer service
              and satisfaction was the least mentioned in the workplans-      managers
              had customer service identified as an accountability item in their
              workplans.

              First line managers are generally accountable for achieving the perform-
              ance expectations outlined in the workplans. Higher level program man-
              agers and senior executives are usually accountable for less explicitly
              defined program performance. For example, in the Investigations func-
              tion, the Chief of the Investigations Operations Division, as a first line
              (GM) manager, is responsible for achieving specific, quantitative timeli-
              ness and efficiency standards for that segment of the overall process.

              At the next higher level, accountability becomes less linked to perform-
              ance standards. The Assistant Director for Federal Investigations (SES) is
              accountable for achieving a lo-percent improvement in timeliness for
              investigations in general. At the program management level, the link to
              performance in delivering services is not specific. The fiscal year 1989
              workplan for the Associate Director for Investigations contained no ele-
              ments with specific timeliness, quality, or efficiency performance
              expectations.

              As another example, the Chief, Adjudication Division II, Office of Retire-
              ment Programs, a first line manager, is held accountable for specific
              goals in processing claims. That individual’s fiscal year 1989 workplan
              states that the unit should “process fully developed (survivor) cases
              within 30 days of receipt,” among other similar goals for various types
              of cases. At the next higher level, the Assistant Director for Retirement
              Programs is accountable for broader goals such as processing “claims
              accurately and timely” and minimizing “degradation in quality and
              processing times for CSHS initial and post adjudication workloads in the
              face of reduced staffing levels.” At the top program level, the Associate
              Director for Retirement and Insurance Programs is accountable for even
              broader goals such as to “identify and implement strategies for contain-
              ing growth for backlogs and processing times in key workloads.”


              We believe that the development and use of performance measures and
Conclusions   standards should be done in a manner that enables the director to use




              Page 8                                   GAO/GGD-90-44   Performance   Information
                            B-282596




                            these tools in the effective management of OPM, and that such perform-
                            ance information would be more useful if improvements were made in
                            the following three areas.

                        l   Expanding the availability and use of performance measures for OPM's
                            functions and activities. For example, consideration should be given to
                            the types of measures OPM management could use to assess performance
                            in the health benefits claims area.
                        l   More systematically and comprehensively compiling and reporting per-
                            formance information to the director for OPM'S key functions and
                            activities.
                    l       Linking performance standards and expectations for organizational
                            units more closely to the performance workplans of the units’ managers
                            to increase the managers’ accountability for results.


                            We recommend that you establish a more comprehensive and strategic
Recommendation to           approach to the development and use of performance measures and
the Director, OPM           standards. Such an approach would require:

                    9 Determining who OPM’Scustomers are, deciding which services provided
                       to these customers should be routinely tracked at the directorate level,
                       and developing the measures and standards needed to assess perform-
                       ance in those services. In following this approach, OPM needs to consider
                      the cost of developing measures and whether precise, quantitative meas-
                      ures are needed in all areas. Further, consideration should be given to
                      the need to have information on various services and outputs, not just
                      the 24 identified in this report.
                    l Developing a flexible performance reporting system for the directorate
                      level that will highlight areas needing attention. Such a reporting system
                      should assure that the director is routinely made aware of performance
                      in all areas of importance to good management of operations and cus-
                      tomer service. Further, the reporting system need not report all per-
                      formance dimensions of all key services on a frequent basis, but should
                      be tailored to the current needs of top management at any point in time.
                    . Making greater use of organizational performance information in hold-
                      ing SES and GMS accountable by specifying in their annual workplans spe-
                      cific key services and the performance expectations for these.




                            Page 9                                 GAO/GGD-9044   Performance   Information
                  B-232596




                  Based on discussions of the report with OPM, it is clear that OPM agrees
Agency Comments   with our findings and recommendations. OPM staff identified some tech-
                  nical changes to t,he data, which have been incorporated. They also com-
                  mented that areas where ongoing customer-oriented efforts have been
                  developed were not sufficiently discussed in the draft report; we have
                  added information to address these efforts appropriately.

                  Additionally, OPM discussed its current actions to improve the develop-
                  ment and use of performance information and said that our recommen-
                  dations have been helpful in these efforts. Information they provided
                  showed that (1) all UPM organizations have begun to identify customers
                  and determine the measures and standards that are available for judg-
                  ing performance in serving these customers, and (2) the Office of Policy
                  is establishing a reporting system that will provide key performance
                  measures to the directorate level. We believe that these are positive
                  steps t.hat should enhance the management of OPM.


                  John Leiteh, Assistant Director, Federal Human Resource Management
                  Issues, was the principal contributor to this assignment.

                  As you know, 3 1 I J.S.C.720 requires the head of a federal agency to
                  submit a written statement on actions taken on our recommendations to
                  the Senate Committee on Governmental Affairs and the House Commit-
                  tee on Government Operations not later than 60 days after the date of
                  the report and to the House and Senate Committees on Appropriations
                  with the agency’s first request for appropriations made more than 60
                  days after the date of the report.




                  Page 10                                 GAO/GGD9044   Performance   Information
B-232596




We are sending copies of this report to the Senate Committee on Govern-
mental Affairs; Subcommittee on Treasury, Postal Service and General
Government, Senate Committee on Appropriations; House Committee on
Post Office and Civil Service; and House Committee on Appropriations.

If you have any questions, please call me on 275-5074.

Sincerely yours,



                                                                                       i

Bernard L. Ungar
Director, Federal Human Resource
  Management Issues




                                                                                   i




Page 11                                 GAO/GGD-9044   Performance   Information
Contents


Letter
Appendix I
List of Functions and
Services
Appendix II                                                                                             17
Objective, Scope, and
Methodology
Appendix III                                                                                           21
OPM Performance
Measures and
Standards
Appendix IV                                                                                            25
Performance Trends
Appendix V                                                                                             30
Issues to Consider in
Developing a Strategic
Approach to
Measurement
Tables                   Table III. 1: OPM Operations: Summary of Measures by                          22
                             Service
                         Table 111.2:OPM Operations: Summary of Standards/                             23
                             Expectations by Service

Figures                  Figure II. 1: OPM Organization Chart                                          18
                         Figure IV. 1: Interim Annuity Timeliness: Processing Days                     26
                         Figure IV.2: Regular Recurring Annuity Timeliness:                            27
                              Processing Days
                         Figure IV.3: Combined Interim and Regular Annuity                             28
                              Quality: Error Rate


                         Page 12                                 GAO/GGD-9044   Performance   Information
Contents




Figure IV.4: Investigations Timeliness: Percent by Critical
     Date




Abbreviations

CZ3RS      Civil Service Retirement System
GM         General Management
OPM        Office of Personnel Management
SES        Senior Executive Service
TIR        Targeted Installation Review


Page 13                                  GAO/GGD90-44   Performance   Information
Appendix I

List of Functions and Services


Retirement Function         Interim annuity payment-the     first annuity payments federal retirees
                            receive immediately after retirement; provided as an interim annuity
                            during the lengthy process of computing the retiree’s proper annuity.
                            Generally amounts to about 95 percent of the regular recurring annuity.

                            Regular recurring annuity-the       full annuity for retires.

                            Survivor annuity, interim-the   first annuity payments the survivors
                            receive immediately after the death of a federal annuitant; provided as
                            an interim annuity during the lengthy process of computing the full
                            annuity.

                            Survivor annuity-regular   recurring-the      full annuity for survivors of
                            retirees.

                            Priority correspondence-inquiries       from members of Congress or con-
                            gressional committees.

                            Refund claims-requests for the refund of retirement funds by federal
                            employees leaving the federal service.

                            Open season changes-changes to health benefits programs by annui-
                            tants during the annual federal open season period.

                            Tax withholding actions-changes to annuitants’ taxes withheld from
                            annuities as a consequence of the annuitants’ change in tax status or as
                            requested by annuitant.

                            Address change-changes      of addresses as requested by annuitants.


Health Insurance Function   Open season information-brochures        and comparison charts that are
                            developed annually for when the federal employees, annuitants, and
                            survivors select specific carriers for their insurance needs. This infor-
                            mation is also used by new federal employees to select their carriers.

                            Disputed claims-claims that are in unresolved disagreement between
                            carriers and beneficiaries, and beneficiaries have requested that OPM
                            resolve the disagreements.

                            Priority correspondence-same        as retirement function.




                            Page 14                                     GAO/GGD-90-44   Performance   Information
                          Appendix I
                          List of Functions   and Services




                          Benefit claims-claims for payment of medical expenses submitted to
                          the carriers by either the medical service providers or by the health beti-
                          efit program enrollees.


Investigations Function   Kational agency check-national      agency cheek and inquiries are
                          searches of existing federal files, such as OPM personnel files and FBI
                          fingerprint files, for information on individuals seeking federal employ-
                          ment, or newly hired employees.

                          Investigations- background investigations on individuals, including
                          current federal employees, who are being considered for sensitive posi-
                          tions. Includes interviews with past employers and acquaintances, police
                          records searches, and searches of similar sources of information.


Career Entry and          Applicant services-those services provided to prospective federal
Employee Development      employees, such as information on available jobs and how to apply for
                          them.

                          Certification lists-lists of best qualified applicants for federal agency
                          positions, provided to the agencies for specific job openings.

                          Decentralization services -authorizations      to agencies for direct hire and
                          delegated examining activities.

                          Training courses- training courses that are provided to federal agencies
                          by or through OPM. OPM generally establishes the training courses on the
                          basis of agencies’ requirements, obtains facilities and instructors, and
                          coordinates all associated activities.


Agency Compliance and     Governmentwide reviews-multi-agency         reviews of significant person-
Evaluation                nel initiatives, trends, and concerns. Issues covered may include agency
                          use of delegated personnel authorities and agency efforts to deal with
                          problems of recruitment and retention.

                          Agency-specific reviews -single-agency,      multi-installation    reviews of
                          significant personnel initiatives.

                          Targeted installation reviews-single    installation reviews of significant
                          personnel problems.



                          Page 15                                     GAO/GGD-9044   Performance   Information
Appendix I
Lit of Functions   and Services




Off-site monitoring-monitoring   and analyses of Civilian Personnel
Data File information on approximately 600 federal installations.

Agency-led reviews-guiding, monitoring, and participating           in federal
agencies’ personnel management evaluations.




Page 16                                GAO/GGD90-44   Performance     Information
Appendix

Objective, Scope, and Methodology


                The objective of our examination was to determine what is known about
                performance in providing quality, effective, and efficient services to
                OPM'S customers. To do this, we identified (1) the key outputs and ser-
                vices of OPM'S operational units (figure II. 1 shows the units covered) and
                what performance measures and standards exist for these services,
                (2) what information is routinely provided to the OPM director on the
                operational units’ performance in delivering key services, and (3) how
                performance expectations and standards are used in OPM to hold mana-
                gers accountable for results,




               Page 17                                  GAO/GGIMKl-44   Performance   Information
                                                                      Appendix     II
                                                                      Objective,    Scope, and Methodology




Figure 11.1:OPM Organization                  Chart

                                                                                       Office of
                                                                                           the
                                                                                       DlPXlOr




                                                      I

                                                 Office of
                                                 Executive
                                               Admmtstratior




                                                                  I
                                                                                                          Career Entry
    Personnel Systems                                     Admin~strallon                                      and                 Retirement and
      and Oversight                                           Group                                        Employee
.---      Group                                                                                           Development
                                                                                                             GVNp



                                                                                                   _i

    Compliance     and




.1_1_____--------1--1_1____________11___---------

                                                            ---        Units Covered     by This Examination

                                                            Source:    From GGD-89-19.       Jan. 1989




                                                                      Page 18                                            GAO/GGD-90-44    Performance   Information
 Appendix     II
 Objective,    Scope, and Methodology




To determine key outputs and related measures and standards, we
asked the managers who were responsible for the five functions (retire-
ment, health insurance, background investigations, agency compliance,
and career entry and employee development) to identify the services
that they believe are the most important to their customers, and thus
the key services for judging the performance of their organizations. F’ur-
ther, to identify available information on organizational performance in
providing these key services, we asked managers to provide documenta-
tion on performance measures and standards that exist and are
reported.

Program managers identified 22 services and we added two additional
services that we believe are important on the basis of our review of
OPM’S operations. We included as a key service the processing of health
benefit claims because of the program’s high costs and large number of
customers, although the program manager believed that this service
should not be included because the process is under the direct control of
contract carriers. Further, we included job applicant services as a key
service in the Career Entry function. Although management in that
function has made a study that concluded that applicants were not their
customers, we noted that many services are provided to this group, such
as the telephone job information service that is available in all regions,
an therefore this service should be included as a key service.

We then obtained and reviewed OPM performance reports and studies
and interviewed OPM program managers and other staff on the sources
of performance data and the development of the reports.

To determine what information is routinely available to the OPM direc-
tor, we asked program managers how they use performance information
in managing their operations and what data they routinely provide to
the director. Further, we identified summarized reporting that could
now be made to the director from data currently available at the pro-
gram level. In addition, we used OPM performance information from cur-
rent and past program level reports to construct analyses that show
recent performance trends in key services. We did not validate the OPM
data before developing the Vends.

To determine how performance information is used to hold managers
accountable, we examined the performance workplans of OPM managers.
Specifically, we reviewed the workplans of all 11 headquarters and 4
Washington Area Service Center SFS members who had line management
responsibilities for the operations units to determine the extent to which


Page 19                                  GAO/GGD-9044   Performance   Information
Appendix     II
Objective,    Scope, and Methodology




organizational performance standards and measures were specified or
referred to in management expectations. We also examined the work-
plans of seven GM managers who had major line responsibility to deter-
mine if performance standards and measures were specified at this
lower level. These workplans contain the elements for which individual
managers-at both the SESand GM levels-are held accountable.




Page 20                                GAO/%&9044   Performance   Information
Appendix III

OPM Performmee Measures and Standards


               In our examination, we found that OPM has developed many perform-
               ance measures and standards. These have been established at various
               levels from overall program levels to individual sub-unit levels. For
               example, not only does the investigations function have national stan-
               dards, but the regions have individual standards as well.

               Measures and standards have been also established for a variety of out-
               puts, not only those outputs considered key services by OPM program
               managers, but also for outputs and services of more limited importance
               to the overall mission. For example, in the health insurance function, the
               performance of processing reconsiderations is measured. Reconsidera-
               tions, which deals with requests for reinstatement in a health benefit
               program, constitutes a minor workload for the health benefits function.

               This appendix focuses on the measures and standards that exist for the
               24 key output services that were identified by program managers or us.
               Table III. 1 identifies for the 24 key output services whether OPM has
               developed timeliness, quality, efficiency, and customer satisfaction
               measures for these services. Table III.2 identifies standards of perform-
               ance for these 24 services, where available.




               Page 21                                 GAO/GGD9O44   Performance   Information
                                                     Appendix III
                                                     OPM Performance          Measures     and Standards




Table 111.1:OPM Operations:       Summary     of Measures          by Service
                                                                            Measures     regularly   reported    to proaram      management
                                                                                                                                         Customer
Function/Service                                   Timeliness                     Quality                       Efficiency .-            satisfaction
Retirement                                                                        ~~--
   Interim annuity payment                         Process days
                                                              _....--.-~          Error rate                    Unit Cost/Time            LImiteda
                                                                                  Error                                                   Lrmited
   Regular recurring ~__
                       annuity ..-.-.-~
                                (fully developed) Process days           -.        .~_.--rate
                                                                                  __                            Unit Cost/Trme
   Survrvor annuity, interim                       Process days                   Error rate                    Unit Cost/Time            Limited
   Survrvor annuity regular recurring              Process days                   Error rate                    Unit Cost/Time            Limited
   Priority correspondence                         Process days                   Error rate                    Unit Cost/Time            Limited
   Refund claims                                   Process days                   Error rate                    Unit Cost/Time            Limited ---
   Open season changes                             Milestone                      None                          None                      Limited
   Tax withholding actrons                         Milestone                      Error rate                    Unit Cost/Trmeb           Limited
   Address chanqe                                  Milestone                      Error rate                    Unit Time                 Limited
Health Insurance
   Open season informatIon                         Milestone                      Qualitative                   None                      Limited
   Disputed Claims                                 Process davs                   None                          Unit CostiTrme            Limited
   Priority correspondence                         Process days                   Error rate                    None                      Limited
   Benefit Claims’                                 None                           None                          None                      Limited
Investigations                                    ~~~ ~~I
   National agency check                           None                           None                          Unit Cost                 Lrmited
   Investigations                                  Process days                   Quality rate                  Unit Cost/Time            Limited
Career entry and employee development                                                      -                                              .-
   Applicant services                              None                           None                          None                       Limrted
   Certification lists                             Process days-                  Certs/Select                  Unit Cost/Time             \Fp;j report by Jan.

  Decentralizatron   services                       None                          Errors                        None                      (1Fii;)report by Jan.

  Trainrng courses                                  None                          None                          Cost/Trnrng Day           Limited
Agency compliance and evaluation
  Governmentwide reviews                           Report process        days -.-.Qualitative                   None                      Lim’ied
                                                                                     ”._..-
  Agency-specrfrc reviews                          Report process        days      Qualitative                  None                      Limited
  Targeted installation reviews
                                                ~~ Report process        days
                                                                         .--_.-..- Qualitative                  None                      Limited
  Off-site monitoring                              Qualitative                     Qualitative                  None                      Limited
  Agency-led review                                None                            Qualitative                  None                      Limiteda
                                                     ‘LImited performance assessments: data not systematically organized, analyzed, reported, and tracked.
                                                     “Efficiency    measure IS not ~ndudually available. but combined with other services

                                                     ‘Contractor operations.




                                                     Page 22                                                         GAO/GGD-9044      Performance      Infommtion
                                                      Appendix Ill
                                                      OPM Performance      Measures      and Standards




Table 111.2:OPM Operations:     Summary       of Standards/Expectations              by Service
                                                                                      Performance    standards/expectations
                                                                                                                                                   Customer
Function/Service                                  Timeliness                     Quality --___              Efficiency                             satisfaction
ketirement                                                                     .__      _._____------~~-~                                     --      ~     ~~ -              -     ~
  lntenm Annuity Payment                          12-14 days                     4.5% error rate            None                                   None
  Regular recurring annuity (fully developed)     55-60 days                     4.5% error rate ..----     None                                   None
___-___                                                                          -.-~~--                               --
  Survivor annuity, Interim                       12-14 days interim             5% error rate              None                                   None
                                                  fw
      Survivor annuity regular recurring
  ~-___-____                                      55-60 days              5 -0% error rate                  None                            _-.-None
     Priority correspondence                      85% In 13 days          0.0% error rate                   None
                                                                                                         ______~                   -      --.-~-None                  -.--
 -Refund claims                                   12-14 days              1% error   rate                   None                               None
                                                                                _--___-
     Open season changes                          Jan. 1                  0.0%
                                                                             --- error rate                 None                               None
     Tax withholding actions                      20 days                 1% error rate                     None                               None
     Address change                               100% in 2 week cycle __ 1.O% ~-error rate                 None                               None
Health Insurance                                                                    ---
     Open season Information                      To GPO 9/30                    0.0% --.
                                                                                        error rate          None                                   None
     Disputed claims                              30 days after data             None        - _I           None                                   None
-__.                                              received                                                                                                            -        -.        ~
-___ Priority correspondence                      13 days           -_-          None ~ .-.__               None                                   None
                                                                                                                                                                             -.-
     Benefit claim9                                                              -                          -
 Investigations
~.                                              __.-.                               -   -..~~-__--.___                            -.-_.         -    ~_ ~~
-____National agency check                         None                          None                       None                                 None
     lnvestlgations                                75% by critical date          97 7%-__.
                                                                                        correct             $495/ease
                                                                   ___~-                                                    -~~~~
                                                                                                                                -             - None
                                                                                                                                                   -   ~  _                   ~_         ~~
Career
I__-         entry and employee development
     Applicant    services                        -                              -                          -                                      -
~~-
     Certification   lists                        None
-I__                                                                        -- None ~-__.-.________ None                          ___~__._ None
     Decentralization services                    None                          None_                None                                  None
                                                                               .__     ~~-~.-_________~
    Training courses                              None                          None
                                                                                 .--.                None                                  None
                                                                                                                     -.-~          ___      .----~                             __        -
Agency compliance and evaluation
-II-
I__-Governmentwide reviews                        Variable milestones            Qualitative                None                                None
-_I_Agency-specific reviews                       Variable milestones            Qualitative
                                                                                         ____               None
                                                                                                                            ---               __None__       -~~ __                 ~-
    Targeted installation reviews                 45 days after site             Qualitative                None                                None
                                                  work
                                                                           __-
  Off-site monitoring                             Regional milestones            Qualitative                None                                   None
                                                                                    __-                     _I--~                 __      -         -.-~          _           ~_.
  Agency-led review                               Agency milestones              Qualitative                None                                   None
                                                      aContractor operations
                                                      Notes:
                                                      1. Tables Ill.1 and 111.2
                                                                             idenhfy many of the measures In the longer cyc.le operations such as the agency
                                                      compliance reviews as being ‘qualltatlve ” These are measures that are not established In the quantita-
                                                      We Sense. such as error rates for quality, but rather as check lists of elements that must be adequately
                                                      covered in the work Although such measures are not as precise as quantitative measures, they are
                                                      sPeclfic, planned avaluatIOns of performance for which expectations are set before the work 1sbegun.
                                                      2 Some key Services have more than one standard of timeliness performance                For example, the annuity




                                                      Page 23                                                       GAO/GGD90-44          Performance         InPormation
Appendix III
OPM Performance       Measures   and Standards




claims may be received from agencies in a “fully developed” condition-which         means all of the material
provided is complete-or    they may be rn “undeveloped”      condition An undeveloped condition requires
obtatnlng the needed information from the agencies, thus both a dlfferent measure and standard are
established for these types of annuity cases. Table III 2 refers to only one of these types of cases in
order to demonstrate simply that standards have been estabkhed for the overall service.
3 Certain tlmellness standards In the Retirement function are given in terms of a range of values, e.g.,
12-14 days. Management established this range to recognize that during peak periods, such as at the
end of the year, abnormal backlogs will occur, thus slowing the processing ttme. Abnormal workload
peaks are consIdered beyond the managers’ control and thus the standards should be adjusted when
these peaks occur
4 “Certs/Select,”   a quality measure in the Career Entry function, refers to lists of certified applrcants
(Certs) per applicants selected and hlred (Selects). The Career Entry and Employee Development pro-
gram managers believe that reductions In the number of lists required to find and hire suitable candi-
dates reflects the improved quality of the Career Entry process.
5. “Errors” IS ldentlfled as the quality measure for decentralization services In the Career Entry function.
This IS a measure of the number of errors by agencies in carrying out delegated examining or direct hire
activities The method of detection for delegated examining is through periodic audits by Career Entry
speciallsts in OPM’s area offtces. OPM also reviews informatlon submitted by agencies on their direct
h/ring activtties




                           ,




Page 24                                                     GAO/GGD-90-44      Performance     Information
Appendix IV

Performmce Trends


               We used historical performance data from OPM reports to develop exam-
               ples of the type of summarized performance trend information that
               could be reported to the directorate level. From these, we selected four
               as illustrations, We did not verify the data.

              These figures show the relevant issues about which the director may
              wish to keep informed. For example, figures IV. 1 and IV.2 indicate that
              efforts to provide annuity payments to retirees in a short time after
              retirement are being successful. However, at the same time, as shown by
              figure IV.3, efforts to keep the quality within desired limits are not suc-
              cessful. One issue to consider, in our opinion, is whether the steps taken
              to improve timeliness have had any adverse effect on quality.

              Figure IV.4 addresses the continuing problem of timeliness in providing
              investigations for federal agencies. In 1987, we reported on problems in
              timeliness in providing background investigations to agencies.’ As the
              chart shows, OPM is still not achieving its own performance standard for
              the percentage of investigations completed by the critical date,




                                                                                   (GAO/




              Page 26                                  GAO/GGlHQ44   Performance   Information
                                                          Appendix IV
                                                          Performance Trends




Figure IV.l: Interim Annuity Timeliness:             Processing      Days

15      Processing     Days
14
13
12
11
10
 9
 8
 7
 6
 5
 4
 3
 2
 1



 II66          2/86           wa6      4166   1187      2fa7        3/66    4187   1166   2ta8   me    4168    1189      2189       3la9       4l89
 Fiscal Quatier

        -            Actual Performance
        I I - -      Performance Standard




                                                          Page 26                                     GAO/GGD-9044    Performance    Information
                                                                 Appendix Iv
                                                                 Performance Trends




Figure IV.2: Regular                 Recurring     Annuity Timeliness:    Processing      Days

120.0          Pmcetssing    Days


110.0


loo.0


 80.0


 80.0


 70.0


60.0


 50.0




        1166          2186          3186    4l86        1187     2l87     !a87     4187          t/86   2’68   3f88     4f88    1169       2189      389       4189
        Fiscal Quarter

               -         Actual Performance
               e--l      Performance Standard




                                                                Page 27                                               GAO/GGD-9044     Performance   Information
                                                         Appendix IV
                                                         Performance Trends




Figure W-3: Combined               Interim and Regular Annuity   Quality:   Error Rate

15’     Error Rate (Percent)
14
13
12
11
19
 9
 8
 7
 8
5




 1186             2196
 Fiscal Quarter

        -            Actual Performance
        I I I I      Performance Standard




                                                       Page 28                           GAO/GGD-90-44   Performance   Information
                                                           Appendix N
                                                           Performance Trends




Figure lV.4: Investigations          Timeliness:     Percent by Critical   Date
100      Percent By Critlcal Date



 80




20




 0


  1186         2/86       3~66      4186      II87       a87       3t87     4187   l/88   2l88   3188     4188    l/86       249       3189          4189
  Fiscal Quarter

         -         Actual Performance
         - - - I   Performance Standard




                                                         Page 29                                        GAO/GGD9044      Performance   Information
Appendix V

Issues to Consider in Developing a Strategic
Approach to Measurement

                   Taking a positive step, the director has recently requested more per-
                   formance information to be provided to the directorate level. At this
                   time, however, the information provided at that level is still limited to a
                   few key services and dimensions of performance, and more comprehen-
                   sive information is needed to fully assess and manage operational
                   performance.

                   Performance information designed to address a range of top manage-
                   ment needs should enhance the capacity to assess operational perform-
                   ance and hold managers accountable. We believe that developing a
                   performance measurement and reporting system for the directorate
                   level should include the following steps:

               l Ensuring that top level management and staff agree on who OPM's cus-
                 tomers are, which key services should be tracked, and what perform-
                 ance measures and standards should be used, keeping in mind that such
                 a list of key services can change over time as the environment and needs
                 of customers change.
               l Determining what, information should be given to the director and how
                 often. Too much information can make performance reporting systems
                 unwieldy and unusable, but too little information hampers management
                 decisionmaking. For example, a strategy for performance reporting may
                 require regular reports on (1) areas of chronic performance problems;
                 (2) activities that the director, the administration, or Congress consider
                 high priority; or (3) services that have high visibility or that require
                 large amounts of resources. Less frequent reporting may be desirable for
                 areas where measurement is costly, change is slow, or problems infre-
                 quent. Reporting may be desired for short periods of time where
                 improvement actions have recently been taken. Further, the report
                 should be dynamic and flexible, allowing for changes in the items to be
                 reported without undue difficulty. In this regard, the reporting system
                 should highlight for the director’s attention, when appropriate, services
                 where performance is moving in the wrong direction.
               . Including key services in SE;‘;and other managers’ workplans, as appro-
                 priate. Because SES managers can be responsible for several key services
                 at one time, it may not be necessary or desirable to specify all services
                 or dimensions of performance in a manager’s workplan, but rather to
                 selectively identify those services that require special attention. Some of
                 the same reasons for reporting performance on selective services to the
                 director may also apply to including these services in managers’ work-
                 plans. For example, falling performance in quality of a given service
                 may be a reason for having that service specified in a program mana-
                 ger’s workplan.


                   Page 30                                  GAO/GGD-904   Performance   Information
            Appendix V
            Issues to Consider in Developing   a Strategic
            Approach to Measurement




             We recognize that measures are only indicators and tools, and should not
             be used as the only basis to reach judgments on organizational or mana-
             gerial performance. Organizational performance data need to be supple-
             mented with other information and managerial judgment when
           . assessing success in achieving OPM’S missions. At least three critical
             aspects need to be considered in using performance measures.

            First, data are not always as accurate, complete, or current as desired.
            Further, the development of quantifiable measures and standards in cer-
            tain areas may be difficult because of the nature of the operations. Qual-
            itative measures and standards requiring judgments may be the best
            practical approach for some areas.

            Second, many factors beyond the managers’ control can influence opera-
            tional performance. These factors include the loss of key staff, budget
            constraints, rapid changes in legislation that change processes and ser-
            vices, and external factors that restrict the capability to hire and retain
            qualified employees.

            Third, not all dimensions of a manager’s individual performance are
            reflected promptly by an organization’s performance. For example,
            building the capability of an organization through improving the work
            force takes time, and the results of such efforts may not be seen quickly
            in the performance data.




I966385)   Page 3 1                                          GAO/GGD4044   Performance   Information
r
. ., _   --.. _
                  I
           .,.,
 ‘_,    ,._.
                                :

                    ,,     .:
   .F. :


I. -,          ‘.         ‘., r 1
   -,               ..’


  ‘.C 1-, ..i~
.“::“‘.‘~‘:‘.      ..:
              .-,-
                            ”
_. :._“_                 1.-. y