oversight

Audit of the U.S. Office of Personnel Management's Performance Information

Published by the Office of Personnel Management, Office of Inspector General on 2013-04-01.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                                                     U.S. OFFICE OF PERSONNEL MANAGEMENT
                                                           OFFICE OF THE INSPECTOR GENERAL
                                                                            OFFICE OF AUDITS




Final Audit Report
Subject:


           ASSESSING THE RELEVANCE AND RELIABILITY OF THE
               U.S. OFFICE OF PERSONNEL MANAGEMENT’S
                      PERFORMANCE INFORMATION



                                           Report No. 4A-CF-00-12-066


                                          Date:                _April 1, 2013___




                                                            --CAUTION--
This audit report has been distributed to Federal officials who are responsible for the administration of the audited program. This audit
report may contain proprietary data which is protected by Federal law (18 U.S.C. 1905). Therefore, while this audit report is available
under the Freedom of Information Act and made available to the public on the OIG webpage, caution needs to be exercised before
releasing the report to the general public as it may contain proprietary information that was redacted from the publicly distributed copy.



                                                                    i
                                                     AUDIT REPORT




                 ASSESSING THE RELEVANCE AND RELIABILITY OF THE
                     U.S. OFFICE OF PERSONNEL MANAGEMENT’S
                             PEFORMANCE INFORMATION




                       Report No. 4A-CF-00-12-066                                  April 1, 2013
                                                                            Date: ________________




                                                                                     ________________________
                                                                                     Michael R. Esser
                                                                                     Assistant Inspector General
                                                                                       for Audits




                                                            --CAUTION--
This audit report has been distributed to Federal officials who are responsible for the administration of the audited program. This audit
report may contain proprietary data which is protected by Federal law (18 U.S.C. 1905). Therefore, while this audit report is available
under the Freedom of Information Act and made available to the public on the OIG webpage, caution needs to be exercised before
releasing the report to the general public as it may contain proprietary information that was redacted from the publicly distributed copy.
                                  EXECUTIVE SUMMARY




             ASSESSING THE RELEVANCE AND RELIABILITY OF THE
                 U.S. OFFICE OF PERSONNEL MANAGEMENT’S
                        PERFORMANCE INFORMATION




                      Report No. 4A-CF-00-12-066           Date: April 1, 2013


The Office of the Inspector General (OIG) has completed a performance audit Assessing the
Relevance and Reliability of the U.S. Office of Personnel Management’s Performance
Information (OPM). The objective of our audit was to determine if OPM has internal controls in
place over the collection, reviewing, and reporting of its performance information in the Annual
Performance Report (APR).

Our audit was conducted from October 23, 2012 through December 4, 2012 at OPM
headquarters in Washington, D.C. We determined that OPM needs to strengthen its controls
over the collection, review, and reporting of performance information in the APR. Our audit
identified five areas requiring improvement.

   1. Lack of Change Control Procedures for Changing Performance                    Procedural
      Measure Targets

       OPM does not have a change control process in place for the
       changes made to performance targets from the Congressional Budget
       Justification and reported in the APR.




                                               i
   2. Federal Investigative Services’ Performance Measures and                     Procedural
      Performance Indicators are not Consistent

       OPM’s reported performance indicators in the FY 2011 APR for
       the Federal Investigative Services investigative timeliness and
       deficient cases performance measures are not consistent. In
       addition, the data used to determine the performance result for
       the deficient cases measure is not aligned with the definition of
       the performance measure.

   3. Improved Internal Controls Needed for the Reporting of the                   Procedural
      Retirement Services’ Claims Backlog Status

       OPM does not have adequate controls in place to ensure that the
       tracking of claims received and claims processed for the retirement
       claims backlog are supported.

   4. Improved Internal Controls Needed for Reporting Performance                  Procedural
      Results for the Retirement Claims Processing Timeliness
      Performance Measure

       OPM does not have adequate controls in place to ensure that the
       retirement claims processing timeliness is accurately calculated.
       Specifically, OPM reported that it processed non-disability retirement
       claims in an average of 125 days; however, OPM actually processed these
       claims in an average of 131 days.

   5. Improved Internal Controls Needed for the Data Gathering,
                                                                            Procedural
                  Reviewing, and Reporting of the Retirement Services’ Customer
       Satisfaction Performance Measure

       OPM lacks adequate internal controls over the data gathering, reviewing,
       and reporting of performance data for the Retirement Services customer
       satisfaction performance measure.


A draft report was issued on January 29, 2013 to the Office of the Chief Financial Officer and
the Performance Improvement Officer for review and comment. The comments in response to
the draft were considered in preparing this final report and are included as an Appendix. OPM
concurred with two recommendations, partially concurred with two recommendations, and did
not concur with one recommendation.
                                                i
                             TABLE OF CONTENTS


                                                                            Page

     EXECUTIVE SUMMARY…………………………………….……..                                   i

 I. INTRODUCTION AND BACKGROUND..........................................   1

II. OBJECTIVE, SCOPE, AND METHODOLOGY…...........................           3

III. AUDIT FINDINGS AND RECOMMENDATIONS...........................          6

     1.     Lack of Change Control Procedures for Changing
            Performance Measure Targets…………………………………                        6
     2.     Federal Investigative Services’ Performance Measures and
            Performance Indicators are not Consistent……………………               7
     3.     Improved Internal Controls Needed for the Reporting of the
            Retirement Services’ Claims Backlog Status………………….              8
     4.     Improved Internal Controls Needed for Reporting Performance
            Results for the Retirement Claims Processing Timeliness
            Performance Measure…………………………………………                             9
     5.     Improved Internal Controls Needed for the Data Gathering,
            Reviewing, and Reporting of the Retirement Services’
            Customer Service Satisfaction Performance Measure…..........    10

IV. MAJOR CONTRIBUTORS TO THIS REPORT.................................      12


     APPENDIX
     OPM’s consolidated response dated February 21, 2013.
                         I. INTRODUCTION AND BACKGROUND

Introduction

This final audit report details the findings, conclusions, and recommendations resulting from our
performance audit Assessing the Relevance and Reliability of the U.S. Office of Personnel
Management’s (OPM) Performance Information. The audit was performed by OPM's Office of
the Inspector General (OIG), as authorized by the Inspector General Act of 1978, as amended.

Background

The Government Performance and Results Act of 1993 (GPRA) requires agencies to engage in
project management tasks such as setting goals, measuring results and reporting their progress.
In order to comply with the GPRA, agencies produce strategic plans, performance plans, and
reports on performance. Agencies must prepare annual performance reports that review the
agency's success or failure in meeting its targeted performance goals. Agencies may choose
either to produce a consolidated Performance and Accountability Report or produce a separate
Agency Financial Report or Annual Performance Report (APR). The APR provides information
on the agency’s progress in achieving the goals and objectives described in the agency’s
Strategic Plan and Annual Performance Plan.

On January 4, 2011, President Obama signed the GPRA Modernization Act of 2010
(Modernization Act). The Modernization Act helps agencies to focus on their highest priorities
and creates a culture where data and empirical evidence play a greater role in policy, budget, and
management decisions.

The Office of Management and Budget (OMB) issued Circular No. A-11 in August 2011, which
provides guidance to agencies for complying with the Modernization Act. Part 6 provides
specific instructions to government agencies on how to report on their performance measures to
OMB and Congress.

OPM’s Chief Financial Officer’s (CFO) Budget and Performance Office (BPO) is responsible
for producing OPM’s APR. Each September, the Associate CFO for BPO sends out a data call
for program performance results using a template. OPM program offices complete the template
by providing the actual performance result for each measure identified on the template; a
narrative statement highlighting the performance result; and a strategy for how to meet any
measure that was not met. The BPO transmitted the FY 2011 APR to Congress concurrent with
the President’s FY 2013 Congressional Budget Justification in February 2012.

On May 18, 2011, OPM appointed a Performance Improvement Officer (PIO), as required by the
Modernization Act. The PIO’s responsibilities are to: 1) establish and report on OPM’s High
Priority Performance Goals; 2) Conduct at least six program performance reviews annually;
3) Represent OPM at the Performance Improvement Council; and 4) Communicate progress on
performance goals to OPM employees through web-based dashboards and other means. OPM
also appointed two deputy PIO’s; one is responsible for information systems and the other is
responsible for innovations.

                                                1
We reviewed performance information as reported in the FY 2011 APR for the Federal
Investigative Service (FIS) and Retirement Services (RS).

FIS’s mission is to ensure the Federal Government has a suitable workforce that protects
National Security and is worthy of the Public Trust. FIS is responsible for providing
investigative products and services for over 100 Federal agencies to use as the basis for security
clearance or suitability decisions as required by Executive Orders and other rules and
regulations. Over 90 percent of the Federal Government’s background investigations are
provided by OPM. FIS’s performance measures fall under the OPM Strategic Goal to Hire the
Best.

RS is responsible for the administration of the Federal Retirement Program covering over
2.9 million active employees and 2.5 million annuitants. Processing of the approximately
100,000 retirement applications received by OPM annually is handled by operations staff in
Boyers, Pennsylvania and Washington, D.C. RS’s performance measures fall under the OPM
Strategic Goal to Honor Service.

This is our first audit Assessing the Relevance and Reliability of OPM’s Performance
Information. The preliminary results of our audit were discussed with the CFO, PIO, RS, and
FIS officials at an exit conference. A draft report was issued on January 29, 2013 to the CFO
and PIO for review and comment. The comments in response to the draft were considered in
preparing the final report and are included as an Appendix. OPM concurred with two
recommendations, partially concurred with two recommendations, and did not concur with one
recommendation.




                                                 2
                     II. OBJECTIVE, SCOPE, AND METHODOLOGY

Objective

The objective of our audit was to determine if OPM has internal controls in place over the
collection, review, and reporting of its performance information in the APR.

The recommendations included in this final report address this objective.

Scope and Methodology

We conducted this performance audit in accordance with generally accepted government
auditing standards as established by the Comptroller General of the United States. Those
standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to
provide a reasonable basis for our findings and conclusions based on our audit objectives.

The scope of our audit covered OPM’s performance measures reported by FIS and RS in
the FY 2011 APR, as well as the RS claims backlog addressed in the RS Strategic Plan,
dated January 2012.

FIS reported on the following performance measures in the FY 2011 APR:
       1. Average number of days to complete the fastest 90 percent of initial national security
           investigations, to meet requirements of the Intelligence Reform and Terrorism
           Prevention Act of 2004 (IRTPA), and
       2. Investigations determined to be deficient due to errors in investigation processing.

In addition, we reviewed the performance measure for FIS’s percent of customers satisfied with
the quality and service of FIS products, policies and guidance; however, these results were not
included in the FY 2011 APR because the survey results were still being collected when the APR
was published. At the time we performed our audit, the performance results were available for
review.

RS reported on the following performance measures in the FY 2011 APR:
         1. Retirement claims processing timeliness;
         2. Average unit cost for processing retirement claims;
         3. Number of retirement records OPM receives that are complete and require no
            development actions;
         4. Percent of customers satisfied with overall retirement services; and
         5. Rate of improper payments in the Retirement Program.

In addition, RS’s FY 2012 Strategic Plan focuses on reducing the backlog of pending claims.
RS’s goal is to eliminate the claims backlog by July 2013.

We did not audit the RS performance measure on the rate of improper payments in the
retirement program as this measure was reviewed during our audit of OPM’s Improper

                                                3
Payments Reporting for Compliance with the Improper Payments Elimination and Recovery
Act of 2010, report number 4A-RI-00-12-009, dated March 14, 2012.

We performed our audit fieldwork from October 23, 2012 through December 4, 2012 at
OPM headquarters located in Washington, D.C.

To accomplish the audit objective noted above, we:

   ■ Reviewed the performance reporting requirements for Federal agencies stated in the
     GPRA, as amended by the GPRA Modernization Act of 2010, OMB Circular A-11 part
     6, and other Federal Regulations that provide agencies guidelines on reporting
     performance measures;
   ■ Interviewed OPM officials to gain an understanding of OPM’s policies and procedures
     for performance reporting in the APR;
   ■ Reviewed supporting documentation, such as management reports and systems reports
     for FIS and RS performance measures reported in the FY 2011 APR;
   ■ Obtained data files extracted from the Annuity Roll Processing System to validate RS
     Hypershow management reports data used to calculate the claims processing timeliness
     and the unit cost performance measures, as well as to validate the reasonableness of the
     claims backlog reporting;
   ■ Reviewed supporting documentation for the inventory on the claims backlog as of
     December 2011 and June 2012;
   ■ Reviewed reporting requirements and PIO responsibilities for OPM’s reporting on its
     high-priority performance goals and measures; and,
   ■ Re-calculated performance results, as applicable.

In planning our work and gaining an understanding of the internal controls over OPM's
performance reporting process, we considered, but did not rely on, the internal control structure
to the extent necessary to develop our audit procedures. These procedures were mainly
substantive in nature, although we did gain an understanding of management procedures and
controls to the extent necessary to develop our audit objectives. The purpose of our audit was
not to provide an opinion on internal controls, but merely to evaluate controls over the processes
that were included in the scope of our audit. Our audit included such tests as validating
performance results to performance measure definitions, verifying performance result
calculations, and other procedures as we considered necessary under the circumstances. The
results of our review and tests indicate that with respect to the items tested, OPM needs to
strengthen its controls over the collection, review, and reporting of performance information in
the APR. We did not evaluate the effectiveness of the general and application controls over
computer processed performance data.

In performing our audit work, we selected samples for testing in order to accomplish our audit
objectives. Our sampling methodology was as follows:




                                                4
   •   For the customer service performance measures, we selected the entire universe of
       returned surveys for testing (246 surveys for FIS and 324 surveys for RS);
   •   For the FIS case timeliness performance measure, we obtained a data file of the entire
       universe of 634,342 national security cases processed during FY 2011 to recalculate the
       timeliness of investigative processing;
   •   For the FIS investigative case quality performance measure, we selected a random
       sample of 20 of the 943 national security cases reopened for quality deficiencies during
       FY 2011 and a random sample of 20 of the 1739 national security cases requested by
       agencies to be reopened, but that FIS determined should not be reopened because there
       was no quality deficiency in case processing by FIS or its contractors;
   •   For the RS claims completeness performance measure, we selected a judgmental sample
       of 10 out of 186 cases with errors for the month of September 2012 for review; and,
   •   For the RS claims backlog reporting, we judgmentally selected the starting December
       2011 inventory and the claims received and processed for the month of June 2012 for
       testing.

The results from our samples were not projected to the population reviewed.




                                               5
                  III. AUDIT FINDINGS AND RECOMMENDATIONS


1. Lack of Change Control Procedures for Changing Performance Measure Targets

   OPM does not have a change control process in place for changing performance targets from
   the Congressional Budget Justification (CBJ) and reported in the Annual Performance Report
   (APR). Specifically, OPM reported a performance target of $76.17 for the average unit cost
   for processing retirement claims in the FY 2011 CBJ. In the FY 2011 APR, OPM changed
   the performance target to $101.23. OPM did not have documentation for the justification and
   approval of the target change.

   The Government Accountability Office’s (GAO) Standards for Internal Controls in the
   Federal Government states that management is responsible for developing control activities,
   which are the policies, procedures, techniques and mechanisms that enforce management’s
   directives. Control activities occur at all levels and functions of an entity and include a wide
   range of activities, such as approvals, authorizations, verifications, reconciliations, and the
   creation and maintenance of related records which provide evidence of execution of these
   activities, as well as appropriate documentation.

   OPM does not have effective internal controls in place to track the approval and justification
   for changing performance targets from the CBJ to the APR.

   By not having internal controls in place for justifying and approving performance target
   changes from the CBJ to the APR, OPM performance reporting lacks transparency.

   Recommendation 1

   We recommend that OPM develop and implement change control procedures for explaining,
   justifying, and approving changes to performance targets.

   CFO’s Response:

   “The Chief Financial Officer (CFO) partially concurs with this recommendation. CFO
   believes it has adequate controls in place to document changes in performance targets.
   Nevertheless, CFO will take corrective actions as detailed below to improve the transparency
   of its process.…

   Although the CFO believes that sufficient internal controls currently exist to document any
   target changes, we concur with the OIG that transparency could be further enhanced.
   Therefore, CFO will address the OIG recommendation by doing the following:

      1) Creating a new form/template whose sole purpose will be to document the target
         change, associated rational, and identify approving officials.




                                                6
      2) Incorporate this template into the overall target change process.

   CFO plans to complete the above items by September 30, 2013.”
2. Federal Investigative Services’ (FIS) Performance Measures and Performance
   Indicators are not Consistent

   OPM’s reported performance indicators in the FY 2011 APR for FIS’s investigative
   timeliness and deficient cases performance measures are not consistent with the performance
   measures. In addition, the data used to determine the performance result for the deficient
   cases measure is not aligned with the definition of the performance measure. Specifically:

      •   For the investigative timeliness performance measure, the performance indicator
          reads “Increased percentage of security and suitability investigations that meet
          timeliness standards.” The performance measure result reported in the APR measures
          timeliness for security investigations only.
      •   For the deficient cases performance measure, the performance indicator states
          “Increased percentage of security and suitability investigations that meet quality
          standards.” The performance measure result reported in the APR measures case
          deficiencies for security investigations only.
      •   For the deficient cases performance measure, the FY 2011 APR performance measure
          definition reads, “The percent of investigations determined to be deficient due to
          errors in investigations processing.” FIS calculated the performance result on the
          initial national security investigations only, but the measure definition does not make
          this distinction.

   GAO Publication GAO/GGD-10.1.20, The Results Act. An Evaluator’s Guide to Assessing
   Agency Annual Performance Plans, states that “measures must clearly represent or be related
   to the performance they are trying to assess. … measures should have a clearly apparent or
   commonly accepted relationship to the intended performance or have been shown to be
   reasonable predictors of the desired behaviors or events. …”

   We therefore conclude that OPM does not have effective internal controls in place for the
   reporting of its performance measures to ensure performance measures, indicators, and
   results are consistent. As a result, OPM is at risk for reporting inaccurate or misleading
   performance information to Congress and the public.

   Recommendation 2

   We recommend that OPM develop and implement controls to ensure that performance
   indicators, measures, and results are consistent so that the measures validly represent the key
   aspects of performance.

   FIS’s Response:

   “FIS does not concur with the finding and recommendation.


                                                7
   The controls are sound and the measures are consistent. The legal mandates on the
   timeliness of national security investigations that originated with the Intelligence Reform and
   Terrorism Prevention Act (IRTPA) provided OPM with this High Priority Timeliness
   Goal…. The timeliness and quality performance measures that are applied to OPM FIS
   investigations are PMMSC [Performance Management and Measures Subcommittee]/PAC
   [Performance Accountability Council] approved measures.…
   For OPM it is appropriate to acknowledge that each of the investigations that are conducted
   for national security clearance purposes also support agency suitability determinations. The
   performance indicator/descriptor language (increased percentage of security and suitability
   investigations that meet timeliness/quality standards) is intended to make the point of the
   universality of the investigations being measured. However, it was always understood at [the
   Office of Management and Budget] OMB and across the Executive branch that high priority
   investigations subject to performance measurement are those distinguished by the IRTPA
   and subject to PAC oversight.
   To avoid future confusion, we wrote the timeliness Goal Statement in the FY12 APR to focus
   specifically on background investigations supporting the IRTPA mandate.”
   OIG’s Comment:

   Performance indicators, measures, and results need to be consistent in order to present a clear
   relationship of performance to desired behavior. We understand that the primary focus of
   FIS’s performance reporting has been in support of IRTPA. However, one of the objectives
   of performance reporting is to be transparent. FIS performs suitability and national security
   investigations as stated in the performance indicator. FIS’s performance indicators should
   match the performance measures reported in the APR. In addition, OPM should qualify the
   performance result for the deficient cases performance measure to indicate results for
   national security investigations only.

3. Improved Internal Controls Needed for the Reporting of the Retirement Services’
   Claims Backlog Status

   OPM needs to improve its controls over the reporting of the RS claims backlog status.
   During our audit, we sampled the June 2012 backlog status to validate the claims received
   and claims processed amounts reported on the claims backlog status report. RS provided a
   management report; however, we were unable to use the management report to recalculate
   the reported claims receipts and claims processed for June 2012.

   GAO’s Standards for Internal Controls in the Federal Government states that management is
   responsible for developing control activities, which are the policies, procedures, techniques
   and mechanisms that enforce management’s directives. Control activities occur at all levels
   and functions of an entity and include a wide range of activities, such as approvals,
   authorizations, verifications, reconciliations, and the creation and maintenance of related
   records which provide evidence of execution of these activities, as well as appropriate
   documentation.



                                                8
   We therefore conclude that OPM lacks adequate internal controls over the data gathering and
   reporting of its retirement claims backlog status. As a result, OPM is at risk for reporting
   incorrect status updates of its progress to reduce the retirement claims backlog.

   Recommendation 3

   We recommend that OPM develop and implement controls over the process to gather and
   report statistics related to its performance in reducing the retirement claims backlog. These
   controls should include the development and implementation of documented policies,
   procedures, methodology, and crosswalk to determine reported performance on the claims
   backlog.

   RS’s Response:

   “Retirement Services (RS) partially concur[s] with the finding. We still contend that we
   turned to components of long-established measures to define and report separately on this
   highly visible agency priority, yet agree that the new monthly report did not have the same
   level of documentation of other measures used historically. It remains that the measure was
   based on existing data already subject to controls and the IG confirms that RS ‘provided the
   verifiable source data’ to support our performance to reduce the retirement claims backlog.
   Since there is concurrence that the data was verified, and, also notable at this point, this
   remains a ‘temporary’ measure that is already waning in significance as we move to a new
   measure of processing effectiveness, we believe no new controls will add any value at this
   point.”

   OIG’s Comment:

   RS provided management reports as the source data for the claims backlog; however, we
   could not use the reports to recalculate all of the reported amounts on the backlog status
   report. We asked for support in the form of a crosswalk on how the reported claims receipts
   and claims processed for June 2012 were calculated, none could be produced, and we were
   unable to verify the reported amounts for the June 2012 retirement case backlog status. We
   understand that OPM expects to have reduced or eliminated the backlog by July 2013 and
   appears to be on track to achieve this goal. Given the high profile nature of this performance
   goal and OPM’s commitment to ensure that it processes retirement claims timely, we believe
   it will be critical for OPM to have effective controls in place for determining its claims
   processing inventory status at all times. Therefore, we continue to recommend that OPM
   develop and implement effective controls over the process to gather and report on its claims
   inventory performance. These controls should include the development and implementation
   of documented policies, procedures, methodology, and crosswalk to determine claims
   inventory status.

4. Improved Internal Controls Needed for Reporting Performance Results for the
   Retirement Claims Processing Timeliness Performance Measure

   In calculating OPM’s claims processing timeliness performance measure result, OPM
   omitted timeliness data for two of the four organization units used in the calculation, thereby
   underreporting the time it takes to process non-disability retirement claims. OPM reported in
                                                9
   the FY 2011 APR that its average claims processing time for non-disability retirement claims
   was 125 days; however, the correct result was 131 days.

   GAO’s Standards for Internal Controls in the Federal Government states that activities need
   to be established to monitor performance measures and indicators. Controls should be aimed
   at validating the propriety and integrity of performance measures and indicators.

   GAO Publication GAO/GGD-10.1.20, The Results Act. An Evaluator’s Guide to Assessing
   Agency Annual Performance Plans, states that agencies should have procedures in place for
   ensuring that data are free of significant levels of error and that bias is not introduced.
   Procedures should provide for the periodic review of data collection, sampling, and
   independent validation to ensure accuracy.

   We therefore conclude that OPM lacks effective controls to ensure that reported performance
   data is reported correctly. As a result, OPM is at risk for reporting incorrect performance
   information to Congress and the public.

   Recommendation 4

   We recommend that OPM develop and implement controls to ensure that performance results
   are correctly reported and include all aspects of performance information.

   RS’s Response:

   “RS concurs with this recommendation. In accordance with the OPM RS Strategic Plan, a
   new performance measure for timeliness (percent of retirement claims processed within 60
   days) will be established starting in August 2013. … The new measure will have improved
   internal controls, validation, and data integrity. …”

5. Improved Internal Controls Needed for the Data Gathering, Reviewing, and Reporting
   of the Retirement Services’ Customer Service Satisfaction Performance Measure

   OPM lacks adequate internal controls over the data gathering, reviewing, and reporting of the
   RS customer service satisfaction performance measure. Specifically, we found seven errors
   in RS’s performance data spreadsheet used to summarize the survey results. The errors noted
   were:
       • four instances where the customer did not answer Question #27; however, the
          performance data spreadsheet of customer responses indicates the response was
          marked “Satisfied”;
       • one instance where the customer marked a response of “Very Dissatisfied” for
          Question #27; however, the performance data spreadsheet of customer responses
          indicates the response was marked “Satisfied”;
       • one instance where RS excluded one Customer Service Survey (CSS). We were
          unable to locate the control number on RS’s performance data spreadsheet; and
       • one instance where RS could not provide the hard copy CSS booklet for review to
          validate responses recorded on the performance data spreadsheet.


                                              10
These errors did not change the overall percent of satisfaction reported in the APR.

The Government Accountability Office’s Standards for Internal Control in the Federal
Government states that management is responsible for developing control activities, which
are the policies, procedures, techniques and mechanisms that enforce management’s
directives. Control activities occur at all levels and functions of an entity and include a wide
range of activities, such as approvals, authorizations, verifications, reconciliations,
performance review, and the creation and maintenance of related records which provide
evidence of execution of these activities, as well as appropriate documentation. Some
examples of control activities are: reviews by management at the functional or activity
levels, controls over information processing, proper execution of transactions and events, and
appropriate documentation.

We therefore conclude that RS does not have adequate controls in place to ensure that
performance information is reliable. Without effective controls for data gathering, review,
and reporting of performance information, RS is at risk for reporting inaccurate performance
information to Congress and the public.

Recommendation 5

We recommend that RS develop and implement controls to ensure performance information
is reliable. These include processes to ensure management oversight and review of
performance data entry and output.

RS’s Response:

“RS concurs with the IG’s finding that OPM lacked adequate internal controls over the data
gathering, reviewing, and reporting of the RS customer service satisfaction performance
measure, and based on their advice will add a review component to this effort going
forward. …”




                                            11
                  IV. MAJOR CONTRIBUTORS TO THIS REPORT


Internal Audits Group




               A uditor-in-Charge

                    Team LeaQ,er
                    C hief




                                    12 

                                                                                                       APPENDIX


                         UNI TED STATES OFF1CE OF PERSO NNE L MANAGEMENT
                                                   Washington, DC 2(H 15

Men! Syslem Audl1
 and Comphance
                                                                                       fEB 2 1 2013

         MEMORANDUM 



         FROM:
                                   JANETL.BARNIEE~Se~~5t~~~::
                                   Director, Internal C Compliance

         SUBJECT:                  Management Response \0 the OIG Draft Report on the Audit of
                                   Assessi ng the Relevance and Reliability of the U,S. Office of Personnel
                                   Management's Pe rformance Informat ion (Report No. 4A-CF-OO-12­
                                   066)


         Thank you for the opportunity to respond to your draft report on the Audit of Assessi ng the
         Relevance and Reliability of the U.S. Office of Personnel Management's Perfo rmance
         Informati on. Below is the management response provided by the Program Offices to the
         fo ll owing issues noted in the report.


             1. Lack of Cha nec Co ntrol Procedures fo r Changing Perform ance Meas ure Ta rgets

         Fi nding: OrM does not have a change control process in place for the changes made to
         performance targets from the Congressiona l Budget Justification and reportcd in thc Annual
         Performance Report (AP R).

         Rcco mm cnd ati on 1: We recommend that OrM develop and implement change control
         procedures for exp laining, justifying, and approving changes to pe rformance targets.

         CFO Ma nagem ent Respon se: The Chief Financial Offi ce r (CFO) part ially concurs with
         thi s recommendati on. CFO be lieves it has adequate contro ls in place to document
         changes in perfonnance targets. Nevertheless, CFO will take corrective acti ons as
         detailed be low to improve the transparency of its process.

         Programs begin to develop performance targets during the annual budget fo rmulation
         process. From the beginning of budget fonnulation to the beginni ng of the budget
         execution year, almost two years has elapsed. During that time frame, performance
         targets for each performance measure are documented and approved on four separate
         occasiOns. For the performance measures reported in the FY 201 1 Annual Performance
         Report:
                  I) Performance targets were first documented in OPM 's FY 2011 OMB I?udget
                      submi ssion dated September 2009.

                                                               1

        www.opm.gov       Reuuit. Retain and Honor 3 World-Class WorHorcc to Sene the American Pcopk    www.usajobs_9OV
       2) Performance targets were updated in OPM’s FY 2011 Congressional Budget
          Justification/Annual Performance Plan dated February 2010.
       3) Performance targets were again updated in OPM’s FY 2012 OMB Budget
          submission dated September 2010.
       4) Performance targets were again updated in OPM’s FY 2012 Congressional
          Budget Justification dated February 2011.

For each of the four occasions mentioned above, any target changes are not only
approved by a CFO performance analyst (as mentioned in the OIG report), but also by
each affected Associate Director through the Agency’s Document Management System.
In addition to the approvals received within OPM, OMB must also concur on our budget
documents before they can be released to Congress and the Public.

Although the CFO believes that sufficient internal controls currently exist to document
any target changes, we concur with the OIG that transparency could be further enhanced.
Therefore, CFO will address the OIG recommendation by doing the following:

   1) Creating a new form/template whose sole purpose will be to document the target

      change, associated rational, and identify approving officials.

   2) Incorporate this template into the overall target change process.


CFO plans to complete the above items by September 30, 2013.


   2.	 Federal Investigative Services’ Performance Measures and Performance 

       Indicators are not Consistent


Finding: OPM's reported performance indicators in the FY 2011 APR for the Federal
Investigative Services (FIS)' investigative timeliness and deficient cases performance measures
are not consistent. In addition, the data used to determine the performance result for the deficient
cases measure is not aligned with the definition of the performance measure.

Recommendation 2: We recommend that OPM develop and implement controls to ensure that
performance indicators, measures, and results are consistent so that the measures validly
represent the key aspect of performance.

FIS Summary: OIG supports this finding as follows: “OPM does not have effective internal
controls in place for the reporting of its performance measures to ensure performance measures,
indicators, and results are consistent.” OIG found inconsistency between the description of the
measure and the actual measure metric. For investigative timeliness, the description was for
increased percentage of security and suitability investigation that meet timeliness standards;
however the timeliness measure metric is only applied to national security cases. The same
disconnect was identified for the quality standard, the description was for increased percentage
of security and suitability investigation that meet quality standards, while the quality measure is
only applied to national security case.



                                                 2

FIS Management Response: FIS does not concur with the finding and recommendation.

The controls are sound and the measures are consistent. The legal mandates on the timeliness of
national security investigations that originated with the Intelligence Reform and Terrorism
Prevention Act (IRTPA) provided OPM with this High Priority Timeliness Goal. The IRTPA also
provided for an Executive Branch structure for managing performance under this mandate. The
Office of Management and Budget first led the performance oversight effort and, working with
Executive branch agencies in the OMB, chaired “Security Clearance Oversight” working group
developed measures. That work was considered and adopted by the Performance Management and
Measures Subcommittee (PMMSC) of the Performance Accountability Council that followed
under EO 13467. The timeliness and quality performance measures that are applied to OPM FIS
investigations are PMMSC/PAC approved measures. OPM FIS has been consistently measuring
and reporting to these standards since 2006, first to Congress and OMB until February 2010, and
now to the Director of National Intelligence and OMB PAC when it meets.

For OPM it is appropriate to acknowledge that each of the investigations that are conducted for
national security clearance purposes also support agency suitability determinations. The
performance indicator/descriptor language (increased percentage of security and suitability
investigations that meet timeliness/quality standards) is intended to make the point of the
universality of the investigations being measured. However, it was always understood at OMB
and across the Executive branch that high priority investigations subject to performance
measurement are those distinguished by the IRTPA and subject to PAC oversight.

To avoid future confusion we wrote the timeliness Goal Statement in the FY12 APR to focus
specifically on background investigations supporting the IRTPA mandate.


    3.	 Improved Internal Controls Needed for the Reporting of the Retirement Services
        Claims Backlog Status

Finding: OPM does not have adequate controls in place to ensure that the retirement claims
backlog tracking of claims receipts and processed are supported.

Recommendation 3: We recommend that OPM develop and implement controls over the process to
gather and report on its performance to reduce the retirement claims backlog. These controls should
include the development and implementation of documented policies, procedures, methodology, and
crosswalk to determine reported performance on the claims backlog.

RS Management Response: Retirement Services (RS) partially concur with the finding. We
still contend that we turned to components of long-established measures to define and report
separately on this highly visible agency priority, yet agree that the new monthly report did not
have the same level of documentation of other measures used historically. It remains that the
measure was based on existing data already subject to controls and the IG confirms that RS
“provided the verifiable source data” to support our performance to reduce the retirement claims
backlog. Since there is concurrence that the data was verified, and, also notable at this point, this
remains a “temporary” measure that is already waning in significance as we move to a new
measure of processing effectiveness, we believe no new controls will add any value at this point.

                                                   3

   4.	 Improved Internal Controls Needed for Reporting Performance Results for 

       Retirement Claims Processing Timeliness


Finding: OPM does not have adequate controls in place to ensure that the retirement claims
processing timeliness is accurately calculated. Specifically, OPM reported that it processed non-
disability retirement claims in an average of 125 days; however, OPM actually processed claims
in an average of 131 days.

Recommendation 4: We recommend that OPM develop and implement controls to ensure that
performance results are correctly reported and include all aspects of performance information.

RS Management Response: RS concurs with this recommendation. In accordance with the
OPM RS Strategic Plan, a new performance measure for timeliness (percent of retirement claims
processed within 60 days) will be established starting in August 2013. As a result, the current
timeliness performance measure has been decommissioned effective FY 2012. The new measure
will have improved internal controls, validation, and data integrity. The plan of action for
implementing this new measure is below.

Percent of retirement claims processed within 60 days.

   •	 Plan and Develop the measure. Establish method for measuring percent of retirement
      claims processed within 60 days. Define what this measure is, how it will be calculated,
      what will be counted, how it will be tracked, what will be the future targets, etc. Create
      the program; develop report that automatically calculates the percent of retirement claims
      processed within 60 days. Verify the source data, reporting elements, and business rules.
      (FY 13 Q2)

   •	 Test the measure. Ensure all calculations and business rules are properly working. (FY
      13 Q3)

   •	 Implement the measure. Inventory reduction will be complete July 31, 2013.

Starting August 1, new measure will be implemented. (FY 13 Q4)


   5.	 Improved Internal Controls Needed for the Data Gathering, Reviewing, and
       Reporting for Retirement Services Customer Service Satisfaction Performance
       Measure

Finding: OPM lacks adequate internal controls over the data gathering, review, and reporting of
performance data for the Retirement Services (RS) customer satisfaction performance measure.

Recommendation 5: We recommend that RS develop and implement controls to ensure
performance information is reliable. These include processes to ensure management oversight
and review of performance data entry and output.

                                                4

RS Management Response: RS concurs with the IG's finding that OPM lacked adequate
internal controls over the data gathering, reviewing, and reporting of the RS customer service
satisfaction performance measure, and based on their advice will add a review component to this
effort going forward. Typically the initial input and analysis of the “Customer Satisfaction
Survey” is completed by September 30th of each year. Starting with the next survey (FY14 Q1),
Quality Assurance will randomly select a sample of the “Customer Satisfaction Survey” and
compare the date with the spreadsheet that is used to compile the survey results. This review
will be completed before results are officially reported to ensure the continued reliability of the
information.




                                                 5