r ” ...” ,_ ” .._.. -. I__ _-_--_.- -.-... --_. .- ” United States GAO General Accounting Office Washington, D.C. 20548 General Government Division B-232596 February 7, 1990 The Honorable Constance Newman Director, Office of Personnel Management Dear Ms. Newman: The Office of Personnel Management (OPM), as the agency with primary responsibility for federal work force issues, provides a variety of ser- vices to federal agencies, individuals seeking federal employment, and 9 million federal employees, retirees, and their dependents. These services include providing information to, and examining the qualifications of, applicants seeking federal employment; training current employees; maintaining a health benefit program for employees, retirees, and their survivors; and establishing and making changes to annuity payments to retired federal employees. This report assesses OPM's use of performance measures and standards to manage the delivery of these services. The Civil Service Reform Act of 1978 recognizes the need for perform- ance measures and emphasizes that, where feasible, organizational and individual performance should be appraised in terms of timeliness, qual- ity, and efficiency. Also, the,Office of Management and Budget (OMB), promotes the establishment of performance measures for timeliness, quality, efficiency, and customer satisfaction as part of its Federal Pro- ductivity and Quality Improvement Program in which OPM participates. We identified 24 key outputs and services for OPM's operations units. Results in Brief (See appendix I.) Although OPM already has one or more performance measures and standards for most of these, certain improvements could be made in the following areas: 9 Many key services lack the full range of potential performance measures and standards, In particular, there are very few performance standards for efficiency or customer satisfaction. . Although many performance measures exist at lower management levels, summarized performance information on only a limited number of key services is provided to the directorate level. While not all meas- ures for all services may be desired or needed at the directorate level, there are relatively few key activities reported on in a manner that allows the director to routinely and systematically track how well OPM is doing in many key arcas. i Page I GAO/GGD4JO4 Performance Information B-232596 . Senior Executive Service and General Management workplans, which annually document expected individual and organizational achieve- ments and are the basis for appraising performance, seldom identify the need to meet existing performance standards and make only limited use of performance information. We recognize that the 24 services and 96 potential measures and stan- dards we identified may not coincide directly with those that the direc- tor wishes to track. In selecting areas to track performance and deciding which measures to use, OPM needs to (1) prioritize the functions and ser- vices it believes are the most important and then determine which per- formance dimensions are critical for each of those services, and (2) consider costs for routine tracking and reporting of performance. Subsequent to our December 1, 1989, briefing on our results and recom- mendations, CIPM initiated agency-wide efforts to improve its develop- ment and use of performance measures and standards. To help ensure that operations are managed properly and customers Background served satisfactorily, an organization needs performance measures and standards. A performance measure, such as the number of days required to process retirement claims, can be matched against a stand- ard to determine if management’s or customers’ expectations are being met. Measures by themselves can identify whether performance is improving or declining by comparing to past periods. While line managers need detailed performance measures to ensure that program objectives are being met and services are being appropriately provided to customers on a daily basis, top management needs summa- rized performance information to track mission achievement, identify problems, and hold managers accountable. In our review, we evaluated how well OPM uses appr0priat.e measures Objective, Scope, and and standards in assessing the quality, effectiveness, and efficiency of Methodology services provided to customers, Our examination covered CPM’Smajor service-providing organizations-those involved in the retirement and health insurance programs, employee background investigations: career entry and employee development activities (recruiting and training), and evaluations of other agencies’ personnel activities. Page 2 GAO/GGD-9044 Performance Information B-232596 To achieve our objective, we identified (1) the key services of OPM'S operational units and what performance measures and standards exist for these services, (2) what information about delivery of these services is routinely provided to the OPM director, and (3) how OPM uses perform- ance expectations and standards to hold managers accountable for -. results. We examined 15 Senior Executive Service (sF~) and 7 General Management (GM) workplans of managers having direct responsibility for the services. Appendix II contains additional details on our methodology. We did our work between May 1989 and October 1989 at OPM headquar- ters using generally accepted government auditing standards. OPMhas developed some performance measures or standards for all the Performance 24 services we examined, as well as for many other outputs and actions Information for not considered key services. There are, however, still significant gaps Managing Operations and limitations in the performance information provided to management. Could Be Improved OPM’s Measures Are Both Most ophf performance measures and standards are quantitative which, Quantitative and as a general rule, are easier for managers to use than qualitative meas- ures. For example, Interim Retirement Annuity claims are to be com- Qualitative pleted in 12-14 days and with no more than 4.5-percent errors (14 days are used during periods of seasonably high workloads). However, in some instances, specific quantitative measures may not always be easily developed or practical for OPM'S services. In some of these situations, OPM is using alternative, qualit,ative measures and standards. Seven out of 74 existing measures are qualitatively stated and 5 out of 35 stan- dards are qualitative. (See tables III.1 and 111.2.)For example, the stand- ard for the Agency Compliance and Evaluation Office’s Targeted Installation Reviews (‘UK) states that “Reports reflect full coverage of all compliance issues, specify violations and recommend appropriate cor- rective actions. TIK reports are written in clear language and are organ- ized logically.” Many Potential Measures Each of the 24 services could potentially have four measures (timeli- and Standards Have Not ness, quality, efficiency, and customer satisfaction) suggested by the Federal Productivity and Quality Improvement Program. With four Been Developed measures for each of t.he 24 services, there are a possible 96 measures Page 3 GAO/GGD-9044 Performance Information B232696 for the key services. Likewise, there are 96 possible standards of per- formance. Our review showed that 73 (76 percent) of the total potential measures and 35 (36 percent) of the possible standards have been devel- oped. (See appendix III.) While it may not prove appropriate to develop all the potential measures and standards, OPM needs to carefully deter- mine its needs in this regard. Measures and standards are missing primarily in the efficiency and cus- tomer satisfaction areas. Slightly more than half the potential measures of efficiency exist for the 24 services, and only one of the 24 services- background investigations-has an efficiency standard. There are no standards for customer satisfaction for any of the services. Also, there are no measures or standards for the services provided job applicants or for the processing of health benefit claims. In particular, the managers of the health benefits function pointed out that claims processing is done by contract carriers and thus not directly controlled by OPM. In spite of this, we included it in our list because the magnitude of the program makes it a major OPM service, and OPM-not the contrac- tors-is ultimately responsible for the program, Moreover, we believe measures and standards in the health benefits area would improve OPM’S ability to assess contractor performance. The Federal Health Benefits Program spent about $8.0 billion in fiscal year 1989, covering 9 million employees, annuitants, and dependents. We compared it with a similar program and noted that the Health Care Financing Administration maintains performance measures and estab- lishes standards for monitoring Medicare carriers-some of which are the same firms that serve the Federal Health Benefits Program. For example, one standard requires Medicare carriers to process 95 percent of the physician claims within 18 days after receipt, Many Measures Are Performance measures help management identify successes and prob- Limited in Ability to Track lems through comparisons with either standards or past performance. Consequently, performance information needs to be collected, organized, and Report on and analyzed so that levels of performance and emerging problems can Performance be clearly identified. Finally, to make measurement data fully useful to management, it needs to be reported on a current basis and used with past data to identify performance trends. Some 0PM performance assessments-particularly of customer satisfac- tion-have not been fully developed and are limited in both the amount Page 4 GAO/GGB904 Perfornxwwe Information of documented analysis provided and in the availability of reports that can be tracked over time to assess changes in performance. The follow- ing are examples of such assessments. w The Investigations Group meets with customer agencies regularly to assess service-related problems and concerns. Investigations Group offi- cials attending the meetings orally report these concerns to Investiga- tions Group top management who also attend some of the meetings. However, data are not systematically collected in writing to enable anal- yses of the extent of various performance problems. For example, agency officials are not surveyed in writing on performance issues to determine the breadth of problems or whether the level of performance is changing. Consequently, while performance data are analyzed, no written reports are developed and maintained, and performance over time is not determined and documented. . The Retirement Group managers obtain customer satisfaction data from a customer survey form that is placed in an information rack in the Retirement Information Center in the OPM headquarters building and from retiree correspondence. However, the data are not systematically collected, hampering any potential analysis. For example, the customer survey form is available only to those who happen to see it in the rack and choose to complete it instead of being uniformly provided to all or to a statistically valid sample of customers. Further, data from the survey and correspondence are not organized to analyze major performance issues. Finally, reports are not written on current performance and trends in customer service are not determined. l The Office of Agency Compliance and Evaluation maintains contacts with agencies’ officials through written requests for input to program changes, formal meetings with high level agency officials, and agency responses to compliance report recommendations. Program managers believe this type of regular input enables them to assess customer ser- vice and determine if the proper compliance issues are being addressed. However, as in the above two examples, the information gathered is not organized, summarized, or reported, and historical trends are not developed. Two OPM organizations have been working to achieve a more complete assessment of customer service through collection of performance infor- mation that can be organized, analyzed, and reported, and which will permit tracking of performance. One, the Career Entry and Employee Development Group, is currently assessing the results of a formal cus- tomer service survey in the career entry area. Page 6 GAO/GGD-9044 Performance Information R-232596 The second, the Washington Area Service Center, has a major ongoing program to improve customer service, As a part of that program, the Center surveyed agencies’ training and development managers to deter- mine how the center could better serve them, and then administered the same survey to over 500 students attending classes to gain additional insights In the job information area, the Center made a limited customer survey at the Job Information Center in OPM headquarters, The Center is applying the information from these surveys in efforts to improve the service delivery process. These efforts include employee participation in improvement action teams. We believe such surveys and other methods of systematically collecting performance data are better than informal feedback as a mechanism for determining customer satisfaction because they enable organizations to systematically analyze, report, and track performance. In the late 1970s a study for the White House Office of Consumer Affairs disclosed that the average business never hears from 96 percent of its unhappy cus- tomers, indicating that reliance on correspondence and informal feed- back can result in missing or misinterpreting customer concerns. It may not be possible or cost-effective to develop measures and stan- dards for all 96 dimensions. However, because of both the large number of performance dimensions without measures or standards and the number of measures that are limited in their usefulness to management, we believe that an assessment of information needs is appropriate. Such an assessment should not only address which missing measures and standards need to be developed, but also who OPM'S customers are and what information is needed to assess services provided to them. Fur- ther, there may be functions other than the ones we selected that man- agement may wish to track in terms of performance. In selecting functions and services to track, we believe particular consideration needs to be given to measures and standards for timeliness, quality, effi- ciency, and customer satisfaction for the part of the health benefits pro- gram administered by contractors for OPM and services to federal employment applicants. top management regularly receives only limited performance Performance OPM'S reports from operating units. As described to us by OPM managers, per- Reporting to OPM formance reporting is generally kept within the functional units. Director Can Be Detailed measures are reported to the first line operating managers, and summarized performance reports are prepared for program managers. Improved Performance reports to the directorate level are limited to a few on a Page 6 GAO/GGD-9044 Performance Information B-232596 regular basis+ For example, certain performance information has only recently been provided to the director on a routine basis, such as the files of customer correspondence and the monthly letter to the Chair- man, House Committee on Post Office and Civil Service, on timeliness in annuitant claims. This increased information at the directorate level is useful but more needs to be done. The information provided still addresses relatively few key services and dimensions of performance and is not provided in a manner that would allow routine and systematic tracking of how well OPM is doing in many key areas. Regular, summarized reports of performance in key areas of an organi- zation can not only identify problem areas needing attention, but can also identify whether attention to a problem is successful. For example, we used OPM’S data to develop trend charts that illustrate the type of summarized performance information that could be routinely provided to the directorate level. (See appendix IV.) These charts show that man- agement efforts to improve timeliness in the retirement annuity process have been successful, but quality in that process has not met OPM stan- dards. Although we did not validate the performance data, such per- formance reporting may alert management that greater attention may be needed for quality as well as timeliness. We do not believe that the OPM director necessarily needs to routinely receive specific performance information on all 96 dimensions, and as previously stated, you may find that other functions and activities are as important, or more important, to track than the 24 we listed. Rather, the development and use of measures and standards should reflect man- agement’s needs for performance and customer service information. Appendix V contains more details on the issues we believe should be considered in developing a more comprehensive and strategic approach to measuring and reporting performance. ___- Performance measures and standards should be used as part of the basis Limited Use of for appraising managers. In doing so, consideration should be given to Performance constraints beyond an individual manager’s control that can prevent Information to Hold performance targets from being achieved. Managers Accountable Performance standards and measures are used to a limited extent in holding managers accountable for performance. In examining 15 SES workplans, we found that 9 had timeliness performance identified in Page 7 GAO/GGD-9044 Perfomance Information B-232596 general terms as an accountability item and 4 of these had quantitative timeliness standards for services and outputs. Similarly, 9 of 15 had quality identified and 2 had quantitative standards. Customer service and satisfaction was the least mentioned in the workplans- managers had customer service identified as an accountability item in their workplans. First line managers are generally accountable for achieving the perform- ance expectations outlined in the workplans. Higher level program man- agers and senior executives are usually accountable for less explicitly defined program performance. For example, in the Investigations func- tion, the Chief of the Investigations Operations Division, as a first line (GM) manager, is responsible for achieving specific, quantitative timeli- ness and efficiency standards for that segment of the overall process. At the next higher level, accountability becomes less linked to perform- ance standards. The Assistant Director for Federal Investigations (SES) is accountable for achieving a lo-percent improvement in timeliness for investigations in general. At the program management level, the link to performance in delivering services is not specific. The fiscal year 1989 workplan for the Associate Director for Investigations contained no ele- ments with specific timeliness, quality, or efficiency performance expectations. As another example, the Chief, Adjudication Division II, Office of Retire- ment Programs, a first line manager, is held accountable for specific goals in processing claims. That individual’s fiscal year 1989 workplan states that the unit should “process fully developed (survivor) cases within 30 days of receipt,” among other similar goals for various types of cases. At the next higher level, the Assistant Director for Retirement Programs is accountable for broader goals such as processing “claims accurately and timely” and minimizing “degradation in quality and processing times for CSHS initial and post adjudication workloads in the face of reduced staffing levels.” At the top program level, the Associate Director for Retirement and Insurance Programs is accountable for even broader goals such as to “identify and implement strategies for contain- ing growth for backlogs and processing times in key workloads.” We believe that the development and use of performance measures and Conclusions standards should be done in a manner that enables the director to use Page 8 GAO/GGD-90-44 Performance Information B-282596 these tools in the effective management of OPM, and that such perform- ance information would be more useful if improvements were made in the following three areas. l Expanding the availability and use of performance measures for OPM's functions and activities. For example, consideration should be given to the types of measures OPM management could use to assess performance in the health benefits claims area. l More systematically and comprehensively compiling and reporting per- formance information to the director for OPM'S key functions and activities. l Linking performance standards and expectations for organizational units more closely to the performance workplans of the units’ managers to increase the managers’ accountability for results. We recommend that you establish a more comprehensive and strategic Recommendation to approach to the development and use of performance measures and the Director, OPM standards. Such an approach would require: 9 Determining who OPM’Scustomers are, deciding which services provided to these customers should be routinely tracked at the directorate level, and developing the measures and standards needed to assess perform- ance in those services. In following this approach, OPM needs to consider the cost of developing measures and whether precise, quantitative meas- ures are needed in all areas. Further, consideration should be given to the need to have information on various services and outputs, not just the 24 identified in this report. l Developing a flexible performance reporting system for the directorate level that will highlight areas needing attention. Such a reporting system should assure that the director is routinely made aware of performance in all areas of importance to good management of operations and cus- tomer service. Further, the reporting system need not report all per- formance dimensions of all key services on a frequent basis, but should be tailored to the current needs of top management at any point in time. . Making greater use of organizational performance information in hold- ing SES and GMS accountable by specifying in their annual workplans spe- cific key services and the performance expectations for these. Page 9 GAO/GGD-9044 Performance Information B-232596 Based on discussions of the report with OPM, it is clear that OPM agrees Agency Comments with our findings and recommendations. OPM staff identified some tech- nical changes to t,he data, which have been incorporated. They also com- mented that areas where ongoing customer-oriented efforts have been developed were not sufficiently discussed in the draft report; we have added information to address these efforts appropriately. Additionally, OPM discussed its current actions to improve the develop- ment and use of performance information and said that our recommen- dations have been helpful in these efforts. Information they provided showed that (1) all UPM organizations have begun to identify customers and determine the measures and standards that are available for judg- ing performance in serving these customers, and (2) the Office of Policy is establishing a reporting system that will provide key performance measures to the directorate level. We believe that these are positive steps t.hat should enhance the management of OPM. John Leiteh, Assistant Director, Federal Human Resource Management Issues, was the principal contributor to this assignment. As you know, 3 1 I J.S.C.720 requires the head of a federal agency to submit a written statement on actions taken on our recommendations to the Senate Committee on Governmental Affairs and the House Commit- tee on Government Operations not later than 60 days after the date of the report and to the House and Senate Committees on Appropriations with the agency’s first request for appropriations made more than 60 days after the date of the report. Page 10 GAO/GGD9044 Performance Information B-232596 We are sending copies of this report to the Senate Committee on Govern- mental Affairs; Subcommittee on Treasury, Postal Service and General Government, Senate Committee on Appropriations; House Committee on Post Office and Civil Service; and House Committee on Appropriations. If you have any questions, please call me on 275-5074. Sincerely yours, i Bernard L. Ungar Director, Federal Human Resource Management Issues i Page 11 GAO/GGD-9044 Performance Information Contents Letter Appendix I List of Functions and Services Appendix II 17 Objective, Scope, and Methodology Appendix III 21 OPM Performance Measures and Standards Appendix IV 25 Performance Trends Appendix V 30 Issues to Consider in Developing a Strategic Approach to Measurement Tables Table III. 1: OPM Operations: Summary of Measures by 22 Service Table 111.2:OPM Operations: Summary of Standards/ 23 Expectations by Service Figures Figure II. 1: OPM Organization Chart 18 Figure IV. 1: Interim Annuity Timeliness: Processing Days 26 Figure IV.2: Regular Recurring Annuity Timeliness: 27 Processing Days Figure IV.3: Combined Interim and Regular Annuity 28 Quality: Error Rate Page 12 GAO/GGD-9044 Performance Information Contents Figure IV.4: Investigations Timeliness: Percent by Critical Date Abbreviations CZ3RS Civil Service Retirement System GM General Management OPM Office of Personnel Management SES Senior Executive Service TIR Targeted Installation Review Page 13 GAO/GGD90-44 Performance Information Appendix I List of Functions and Services Retirement Function Interim annuity payment-the first annuity payments federal retirees receive immediately after retirement; provided as an interim annuity during the lengthy process of computing the retiree’s proper annuity. Generally amounts to about 95 percent of the regular recurring annuity. Regular recurring annuity-the full annuity for retires. Survivor annuity, interim-the first annuity payments the survivors receive immediately after the death of a federal annuitant; provided as an interim annuity during the lengthy process of computing the full annuity. Survivor annuity-regular recurring-the full annuity for survivors of retirees. Priority correspondence-inquiries from members of Congress or con- gressional committees. Refund claims-requests for the refund of retirement funds by federal employees leaving the federal service. Open season changes-changes to health benefits programs by annui- tants during the annual federal open season period. Tax withholding actions-changes to annuitants’ taxes withheld from annuities as a consequence of the annuitants’ change in tax status or as requested by annuitant. Address change-changes of addresses as requested by annuitants. Health Insurance Function Open season information-brochures and comparison charts that are developed annually for when the federal employees, annuitants, and survivors select specific carriers for their insurance needs. This infor- mation is also used by new federal employees to select their carriers. Disputed claims-claims that are in unresolved disagreement between carriers and beneficiaries, and beneficiaries have requested that OPM resolve the disagreements. Priority correspondence-same as retirement function. Page 14 GAO/GGD-90-44 Performance Information Appendix I List of Functions and Services Benefit claims-claims for payment of medical expenses submitted to the carriers by either the medical service providers or by the health beti- efit program enrollees. Investigations Function Kational agency check-national agency cheek and inquiries are searches of existing federal files, such as OPM personnel files and FBI fingerprint files, for information on individuals seeking federal employ- ment, or newly hired employees. Investigations- background investigations on individuals, including current federal employees, who are being considered for sensitive posi- tions. Includes interviews with past employers and acquaintances, police records searches, and searches of similar sources of information. Career Entry and Applicant services-those services provided to prospective federal Employee Development employees, such as information on available jobs and how to apply for them. Certification lists-lists of best qualified applicants for federal agency positions, provided to the agencies for specific job openings. Decentralization services -authorizations to agencies for direct hire and delegated examining activities. Training courses- training courses that are provided to federal agencies by or through OPM. OPM generally establishes the training courses on the basis of agencies’ requirements, obtains facilities and instructors, and coordinates all associated activities. Agency Compliance and Governmentwide reviews-multi-agency reviews of significant person- Evaluation nel initiatives, trends, and concerns. Issues covered may include agency use of delegated personnel authorities and agency efforts to deal with problems of recruitment and retention. Agency-specific reviews -single-agency, multi-installation reviews of significant personnel initiatives. Targeted installation reviews-single installation reviews of significant personnel problems. Page 15 GAO/GGD-9044 Performance Information Appendix I Lit of Functions and Services Off-site monitoring-monitoring and analyses of Civilian Personnel Data File information on approximately 600 federal installations. Agency-led reviews-guiding, monitoring, and participating in federal agencies’ personnel management evaluations. Page 16 GAO/GGD90-44 Performance Information Appendix Objective, Scope, and Methodology The objective of our examination was to determine what is known about performance in providing quality, effective, and efficient services to OPM'S customers. To do this, we identified (1) the key outputs and ser- vices of OPM'S operational units (figure II. 1 shows the units covered) and what performance measures and standards exist for these services, (2) what information is routinely provided to the OPM director on the operational units’ performance in delivering key services, and (3) how performance expectations and standards are used in OPM to hold mana- gers accountable for results, Page 17 GAO/GGIMKl-44 Performance Information Appendix II Objective, Scope, and Methodology Figure 11.1:OPM Organization Chart Office of the DlPXlOr I Office of Executive Admmtstratior I Career Entry Personnel Systems Admin~strallon and Retirement and and Oversight Group Employee .--- Group Development GVNp _i Compliance and .1_1_____--------1--1_1____________11___--------- --- Units Covered by This Examination Source: From GGD-89-19. Jan. 1989 Page 18 GAO/GGD-90-44 Performance Information Appendix II Objective, Scope, and Methodology To determine key outputs and related measures and standards, we asked the managers who were responsible for the five functions (retire- ment, health insurance, background investigations, agency compliance, and career entry and employee development) to identify the services that they believe are the most important to their customers, and thus the key services for judging the performance of their organizations. F’ur- ther, to identify available information on organizational performance in providing these key services, we asked managers to provide documenta- tion on performance measures and standards that exist and are reported. Program managers identified 22 services and we added two additional services that we believe are important on the basis of our review of OPM’S operations. We included as a key service the processing of health benefit claims because of the program’s high costs and large number of customers, although the program manager believed that this service should not be included because the process is under the direct control of contract carriers. Further, we included job applicant services as a key service in the Career Entry function. Although management in that function has made a study that concluded that applicants were not their customers, we noted that many services are provided to this group, such as the telephone job information service that is available in all regions, an therefore this service should be included as a key service. We then obtained and reviewed OPM performance reports and studies and interviewed OPM program managers and other staff on the sources of performance data and the development of the reports. To determine what information is routinely available to the OPM direc- tor, we asked program managers how they use performance information in managing their operations and what data they routinely provide to the director. Further, we identified summarized reporting that could now be made to the director from data currently available at the pro- gram level. In addition, we used OPM performance information from cur- rent and past program level reports to construct analyses that show recent performance trends in key services. We did not validate the OPM data before developing the Vends. To determine how performance information is used to hold managers accountable, we examined the performance workplans of OPM managers. Specifically, we reviewed the workplans of all 11 headquarters and 4 Washington Area Service Center SFS members who had line management responsibilities for the operations units to determine the extent to which Page 19 GAO/GGD-9044 Performance Information Appendix II Objective, Scope, and Methodology organizational performance standards and measures were specified or referred to in management expectations. We also examined the work- plans of seven GM managers who had major line responsibility to deter- mine if performance standards and measures were specified at this lower level. These workplans contain the elements for which individual managers-at both the SESand GM levels-are held accountable. Page 20 GAO/%&9044 Performance Information Appendix III OPM Performmee Measures and Standards In our examination, we found that OPM has developed many perform- ance measures and standards. These have been established at various levels from overall program levels to individual sub-unit levels. For example, not only does the investigations function have national stan- dards, but the regions have individual standards as well. Measures and standards have been also established for a variety of out- puts, not only those outputs considered key services by OPM program managers, but also for outputs and services of more limited importance to the overall mission. For example, in the health insurance function, the performance of processing reconsiderations is measured. Reconsidera- tions, which deals with requests for reinstatement in a health benefit program, constitutes a minor workload for the health benefits function. This appendix focuses on the measures and standards that exist for the 24 key output services that were identified by program managers or us. Table III. 1 identifies for the 24 key output services whether OPM has developed timeliness, quality, efficiency, and customer satisfaction measures for these services. Table III.2 identifies standards of perform- ance for these 24 services, where available. Page 21 GAO/GGD9O44 Performance Information Appendix III OPM Performance Measures and Standards Table 111.1:OPM Operations: Summary of Measures by Service Measures regularly reported to proaram management Customer Function/Service Timeliness Quality Efficiency .- satisfaction Retirement ~~-- Interim annuity payment Process days _....--.-~ Error rate Unit Cost/Time LImiteda Error Lrmited Regular recurring ~__ annuity ..-.-.-~ (fully developed) Process days -. .~_.--rate __ Unit Cost/Trme Survrvor annuity, interim Process days Error rate Unit Cost/Time Limited Survrvor annuity regular recurring Process days Error rate Unit Cost/Time Limited Priority correspondence Process days Error rate Unit Cost/Time Limited Refund claims Process days Error rate Unit Cost/Time Limited --- Open season changes Milestone None None Limited Tax withholding actrons Milestone Error rate Unit Cost/Trmeb Limited Address chanqe Milestone Error rate Unit Time Limited Health Insurance Open season informatIon Milestone Qualitative None Limited Disputed Claims Process davs None Unit CostiTrme Limited Priority correspondence Process days Error rate None Limited Benefit Claims’ None None None Limited Investigations ~~~ ~~I National agency check None None Unit Cost Lrmited Investigations Process days Quality rate Unit Cost/Time Limited Career entry and employee development - .- Applicant services None None None Limrted Certification lists Process days- Certs/Select Unit Cost/Time \Fp;j report by Jan. Decentralizatron services None Errors None (1Fii;)report by Jan. Trainrng courses None None Cost/Trnrng Day Limited Agency compliance and evaluation Governmentwide reviews Report process days -.-.Qualitative None Lim’ied ”._..- Agency-specrfrc reviews Report process days Qualitative None Limited Targeted installation reviews ~~ Report process days .--_.-..- Qualitative None Limited Off-site monitoring Qualitative Qualitative None Limited Agency-led review None Qualitative None Limiteda ‘LImited performance assessments: data not systematically organized, analyzed, reported, and tracked. “Efficiency measure IS not ~ndudually available. but combined with other services ‘Contractor operations. Page 22 GAO/GGD-9044 Performance Infommtion Appendix Ill OPM Performance Measures and Standards Table 111.2:OPM Operations: Summary of Standards/Expectations by Service Performance standards/expectations Customer Function/Service Timeliness Quality --___ Efficiency satisfaction ketirement .__ _._____------~~-~ -- ~ ~~ - - ~ lntenm Annuity Payment 12-14 days 4.5% error rate None None Regular recurring annuity (fully developed) 55-60 days 4.5% error rate ..---- None None ___-___ -.-~~-- -- Survivor annuity, Interim 12-14 days interim 5% error rate None None fw Survivor annuity regular recurring ~-___-____ 55-60 days 5 -0% error rate None _-.-None Priority correspondence 85% In 13 days 0.0% error rate None ______~ - --.-~-None -.-- -Refund claims 12-14 days 1% error rate None None _--___- Open season changes Jan. 1 0.0% --- error rate None None Tax withholding actions 20 days 1% error rate None None Address change 100% in 2 week cycle __ 1.O% ~-error rate None None Health Insurance --- Open season Information To GPO 9/30 0.0% --. error rate None None Disputed claims 30 days after data None - _I None None -__. received - -. ~ -___ Priority correspondence 13 days -_- None ~ .-.__ None None -.- Benefit claim9 - - Investigations ~. __.-. - -..~~-__--.___ -.-_. - ~_ ~~ -____National agency check None None None None lnvestlgations 75% by critical date 97 7%-__. correct $495/ease ___~- -~~~~ - - None - ~ _ ~_ ~~ Career I__- entry and employee development Applicant services - - - - ~~- Certification lists None -I__ -- None ~-__.-.________ None ___~__._ None Decentralization services None None_ None None .__ ~~-~.-_________~ Training courses None None .--. None None -.-~ ___ .----~ __ - Agency compliance and evaluation -II- I__-Governmentwide reviews Variable milestones Qualitative None None -_I_Agency-specific reviews Variable milestones Qualitative ____ None --- __None__ -~~ __ ~- Targeted installation reviews 45 days after site Qualitative None None work __- Off-site monitoring Regional milestones Qualitative None None __- _I--~ __ - -.-~ _ ~_. Agency-led review Agency milestones Qualitative None None aContractor operations Notes: 1. Tables Ill.1 and 111.2 idenhfy many of the measures In the longer cyc.le operations such as the agency compliance reviews as being ‘qualltatlve ” These are measures that are not established In the quantita- We Sense. such as error rates for quality, but rather as check lists of elements that must be adequately covered in the work Although such measures are not as precise as quantitative measures, they are sPeclfic, planned avaluatIOns of performance for which expectations are set before the work 1sbegun. 2 Some key Services have more than one standard of timeliness performance For example, the annuity Page 23 GAO/GGD90-44 Performance InPormation Appendix III OPM Performance Measures and Standards claims may be received from agencies in a “fully developed” condition-which means all of the material provided is complete-or they may be rn “undeveloped” condition An undeveloped condition requires obtatnlng the needed information from the agencies, thus both a dlfferent measure and standard are established for these types of annuity cases. Table III 2 refers to only one of these types of cases in order to demonstrate simply that standards have been estabkhed for the overall service. 3 Certain tlmellness standards In the Retirement function are given in terms of a range of values, e.g., 12-14 days. Management established this range to recognize that during peak periods, such as at the end of the year, abnormal backlogs will occur, thus slowing the processing ttme. Abnormal workload peaks are consIdered beyond the managers’ control and thus the standards should be adjusted when these peaks occur 4 “Certs/Select,” a quality measure in the Career Entry function, refers to lists of certified applrcants (Certs) per applicants selected and hlred (Selects). The Career Entry and Employee Development pro- gram managers believe that reductions In the number of lists required to find and hire suitable candi- dates reflects the improved quality of the Career Entry process. 5. “Errors” IS ldentlfled as the quality measure for decentralization services In the Career Entry function. This IS a measure of the number of errors by agencies in carrying out delegated examining or direct hire activities The method of detection for delegated examining is through periodic audits by Career Entry speciallsts in OPM’s area offtces. OPM also reviews informatlon submitted by agencies on their direct h/ring activtties , Page 24 GAO/GGD-90-44 Performance Information Appendix IV Performmce Trends We used historical performance data from OPM reports to develop exam- ples of the type of summarized performance trend information that could be reported to the directorate level. From these, we selected four as illustrations, We did not verify the data. These figures show the relevant issues about which the director may wish to keep informed. For example, figures IV. 1 and IV.2 indicate that efforts to provide annuity payments to retirees in a short time after retirement are being successful. However, at the same time, as shown by figure IV.3, efforts to keep the quality within desired limits are not suc- cessful. One issue to consider, in our opinion, is whether the steps taken to improve timeliness have had any adverse effect on quality. Figure IV.4 addresses the continuing problem of timeliness in providing investigations for federal agencies. In 1987, we reported on problems in timeliness in providing background investigations to agencies.’ As the chart shows, OPM is still not achieving its own performance standard for the percentage of investigations completed by the critical date, (GAO/ Page 26 GAO/GGlHQ44 Performance Information Appendix IV Performance Trends Figure IV.l: Interim Annuity Timeliness: Processing Days 15 Processing Days 14 13 12 11 10 9 8 7 6 5 4 3 2 1 II66 2/86 wa6 4166 1187 2fa7 3/66 4187 1166 2ta8 me 4168 1189 2189 3la9 4l89 Fiscal Quatier - Actual Performance I I - - Performance Standard Page 26 GAO/GGD-9044 Performance Information Appendix Iv Performance Trends Figure IV.2: Regular Recurring Annuity Timeliness: Processing Days 120.0 Pmcetssing Days 110.0 loo.0 80.0 80.0 70.0 60.0 50.0 1166 2186 3186 4l86 1187 2l87 !a87 4187 t/86 2’68 3f88 4f88 1169 2189 389 4189 Fiscal Quarter - Actual Performance e--l Performance Standard Page 27 GAO/GGD-9044 Performance Information Appendix IV Performance Trends Figure W-3: Combined Interim and Regular Annuity Quality: Error Rate 15’ Error Rate (Percent) 14 13 12 11 19 9 8 7 8 5 1186 2196 Fiscal Quarter - Actual Performance I I I I Performance Standard Page 28 GAO/GGD-90-44 Performance Information Appendix N Performance Trends Figure lV.4: Investigations Timeliness: Percent by Critical Date 100 Percent By Critlcal Date 80 20 0 1186 2/86 3~66 4186 II87 a87 3t87 4187 l/88 2l88 3188 4188 l/86 249 3189 4189 Fiscal Quarter - Actual Performance - - - I Performance Standard Page 29 GAO/GGD9044 Performance Information Appendix V Issues to Consider in Developing a Strategic Approach to Measurement Taking a positive step, the director has recently requested more per- formance information to be provided to the directorate level. At this time, however, the information provided at that level is still limited to a few key services and dimensions of performance, and more comprehen- sive information is needed to fully assess and manage operational performance. Performance information designed to address a range of top manage- ment needs should enhance the capacity to assess operational perform- ance and hold managers accountable. We believe that developing a performance measurement and reporting system for the directorate level should include the following steps: l Ensuring that top level management and staff agree on who OPM's cus- tomers are, which key services should be tracked, and what perform- ance measures and standards should be used, keeping in mind that such a list of key services can change over time as the environment and needs of customers change. l Determining what, information should be given to the director and how often. Too much information can make performance reporting systems unwieldy and unusable, but too little information hampers management decisionmaking. For example, a strategy for performance reporting may require regular reports on (1) areas of chronic performance problems; (2) activities that the director, the administration, or Congress consider high priority; or (3) services that have high visibility or that require large amounts of resources. Less frequent reporting may be desirable for areas where measurement is costly, change is slow, or problems infre- quent. Reporting may be desired for short periods of time where improvement actions have recently been taken. Further, the report should be dynamic and flexible, allowing for changes in the items to be reported without undue difficulty. In this regard, the reporting system should highlight for the director’s attention, when appropriate, services where performance is moving in the wrong direction. . Including key services in SE;‘;and other managers’ workplans, as appro- priate. Because SES managers can be responsible for several key services at one time, it may not be necessary or desirable to specify all services or dimensions of performance in a manager’s workplan, but rather to selectively identify those services that require special attention. Some of the same reasons for reporting performance on selective services to the director may also apply to including these services in managers’ work- plans. For example, falling performance in quality of a given service may be a reason for having that service specified in a program mana- ger’s workplan. Page 30 GAO/GGD-904 Performance Information Appendix V Issues to Consider in Developing a Strategic Approach to Measurement We recognize that measures are only indicators and tools, and should not be used as the only basis to reach judgments on organizational or mana- gerial performance. Organizational performance data need to be supple- mented with other information and managerial judgment when . assessing success in achieving OPM’S missions. At least three critical aspects need to be considered in using performance measures. First, data are not always as accurate, complete, or current as desired. Further, the development of quantifiable measures and standards in cer- tain areas may be difficult because of the nature of the operations. Qual- itative measures and standards requiring judgments may be the best practical approach for some areas. Second, many factors beyond the managers’ control can influence opera- tional performance. These factors include the loss of key staff, budget constraints, rapid changes in legislation that change processes and ser- vices, and external factors that restrict the capability to hire and retain qualified employees. Third, not all dimensions of a manager’s individual performance are reflected promptly by an organization’s performance. For example, building the capability of an organization through improving the work force takes time, and the results of such efforts may not be seen quickly in the performance data. I966385) Page 3 1 GAO/GGD4044 Performance Information r . ., _ --.. _ I .,., ‘_, ,._. : ,, .: .F. : I. -, ‘. ‘., r 1 -, ..’ ‘.C 1-, ..i~ .“::“‘.‘~‘:‘. ..: .-,- ” _. :._“_ 1.-. y
Office of Personnel Management: Better Performance Information Needed
Published by the Government Accountability Office on 1990-02-07.
Below is a raw (and likely hideous) rendition of the original report. (PDF)