U.S. OFFICE OF PERSONNEL MANAGEMENT OFFICE OF THE INSPECTOR GENERAL OFFICE OF EVALUATIONS Final Evaluation Report EVALUATION OF THE U.S. OFFICE OF PERSONNEL MANAGEMENT’S RETIREMENT SERVICES’ IMAGING OPERATIONS Report Number 4K-RS-00-17-039 March 14, 2018 EXECUTIVE SUMMARY Evaluation of The U.S. Office of Personnel Management’s Retirement Services’ Imaging Operations Report No. 4K-RS-00-17-039 March 14, 2018 Why Did We Conduct the Evaluation? What Did We Find? The Retirement Services’ (RS) RS began a federally staffed imaging operation in Boyers, Program’s Associate Director Pennsylvania, in January 2012, to image Federal employees’ requested that we conduct an retirement records. RS intended to achieve two specific results evaluation of their imaging from the imaging operation: operations. x Address the limited record storage space at their Boyers facility; and The objectives of our evaluation were to determine: (1) the efficiency x Position themselves to achieve their strategic objective of of RS’ process for imaging transitioning to a paperless retirement claims process documents into its Electronic system. Document Management System and, (2) the effectiveness of those imaged During our evaluation, we found that the RS’ process for imaging documents on end users. documents into its Electronic Document Management System was efficient and end users felt that the images were beneficial to accurately completing retirement application packages. However, we identified areas within the imaging operations where RS needs to take corrective actions: x RS’ policies and procedures does not reflect their current operations; x RS has not conducted a periodic quality assurance audit of its imaged documents since September 2012; and x RS has not developed performance measures to assess effectiveness of its imaging operations. _______________________ William W. Scott, Jr. Chief, Office of Evaluations i TABLE OF CONTENTS Page EXECUTIVE SUMMARY.......................................................................................... i INTRODUCTION....................................................................................................... 1 RESULTS OF EVALUATION .................................................................................. 4 1. Retirement Services Policies and Procedures Do Not Reflect Current Operations ............................................................................................................... 4 2. Quality Assurance Audits Are Not Conducted Periodically................................... 5 3. No Performance Measures to Assess Benefits of Imaging Efforts ......................... 5 APPENDIX A: Scope and Methodology ................................................................... 7 APPENDIX B: Management Comments ................................................................ 10 REPORT FRAUD, WASTE AND MISMANAGEMENT ................................... 12 INTRODUCTION This final evaluation report details the results of our evaluation of the U.S. Office of Personnel Management’s (OPM) Retirement Services’ (RS) Imaging Operations. The OPM’s Office of the Inspector General (OIG) conducted this evaluation, as authorized by the Inspector General Act of 1978, as amended. RS began a federally staffed imaging operation in Boyers, Pennsylvania, in January 2012, to image Federal employees’ retirement records. RS intended to achieve two specific results from the imaging operation: x Address the limited record storage space at their Boyers facility; and x Position itself to achieve their strategic objective of transitioning the Retirement Program to a paperless system. RS worked with OPM’s Office of the Chief Information Officer to develop the Retirement Operations Imaging Operational Plan that outlined how the imaging effort would operate, types of records scanned, quality control measures, and instructions on how to use the scanning equipment. The operational plan also detailed duties for imaging personnel: x Pulling non-case records from Federal Employees Retirement System Open File and resources permitting, non-case; related records and documents retained in the Civil Service Retirement System Open File; x Prepping documents (Doc Prep); x Bundling records; x Barcoding bundles; x Scanning bundles; x Quality review of bundles; x Exception processing (when required); x Managing boxes of imaged records; x Quality Control; and x Storing boxes of imaged records.1 Multiple users, both internal and external, have varying levels of access to view the over 4 million images stored in the Electronic Document Management System.2 OPM’s internal users 1 OPM’s Retirement Services, Retirement Operations Imaging Operating Plan, January 17, 2012, p. 4. 2 The Electronic Data Management System is a database that contains scanned images of Federal employees’ records collected over their careers. This system also assist with retirement benefits calculations and assists with long-term storage of hard copy documents. 1 Report Number 4K-RS-00-17-039 have access to these images for claim adjudication and other customer service purposes, and they have the ability to search imaged documents across Federal agencies. External users, Federal agencies, only have access to their specific employees’ imaged documents. Federal agencies use the Retirement Record Data Viewer, which is a separate system that allows its users to view the same information available within the Electronic Document Management System. As of August 2017, 793 external users from 44 Federal agencies have had access to these images to assist with retirement counseling and other retirement application preparation. Since implementation (August 2012), internal and external users have viewed over 978,000 imaged documents. (See Graph 1.1) See Graph 1.1 TotalImagedDocumentsViewedviaRetirementRecordDataViewer 1200000 TotalViewedDocuments 1000000 978150 932466 800000 662374 600000 437436 400000 200000 192880 16240 0 201220132014201520162017 During this evaluation, we conducted a survey to gain external users’ perspectives on viewing scanned images via the Retirement Record Data Viewer. This survey yielded positive results. While not all responders responded to every question, we found that for those who did respond: x 53.4% of responders used the images to assist with completing an employee’s retirement package four or more times per week. x 76.5% of responders were satisfied with the quality of the images that are available. 2 Report Number 4K-RS-00-17-039 x 82.4% of responders felt that the images were either “extremely beneficial” or “very beneficial” in accurately completing retirement application packages.3 3 Please refer to the Scope and Methodology section for details on our methodology used for conducting the survey and details of the results. 3 Report Number 4K-RS-00-17-039 RESULTS OF EVALUATION 1. Retirement Services’ Policies and Procedures Do Not Reflect Current Operations We found RS’ is not following the policies and procedures as outlined in its Retirement Operations Imaging Operational Plan. Specifically, (1) personnel position titles have changed, (2) personnel are no longer performing duties such as bundling records and conducting quality reviews of bundles, and (3) RS has not formalized their established quality assurance process for ensuring accuracy within the imaging process. Since the plan was developed, RS has changed its staffing positions and internal controls over operations to include establishing a quality assurance process for ensuring accuracy within the imaging process. This quality assurance process for barcoding and validating, includes having multiple personnel review the same record to ensure that the Customer Service Representative who barcodes a document also does not validate the same document. During our fieldwork, we obtained a random sample of fiscal year 2017’s Last Action Reports. These reports outline each action through the imaging process by a unique bundle control number to include the personnel responsible at the various stages. From these reports, we assessed whether RS was following its undocumented quality assurance process regarding barcoding and validating images. From our analysis, we found 6 out 200 instances (3%) where the customer service representative who had barcoded a document also validated the image into RS’ Electronic Document Management System. The United States Government Accountability Office’s Standards for Internal Control in the Federal Government (also known as the Green Book) defines the standards for internal controls in the Federal Government entities’ objectives relating to the operations, reporting, and compliance. These standards provide criteria for assessing the design, implementation, and operating effectiveness of internal controls in Federal Government entities to determine if an internal control system is effective. In accordance with the Green Book, Federal Government entities are required to establish and maintain an effective internal control system. Without RS formalizing its current policies and procedures to include their internal controls, operations, and staffing positions, there is an increased risk of imaging operation personnel not following the quality assurance process as identified in our analysis. Recommendation 1 We recommend that RS update their Retirement Operations Imaging Operational Plan policies and procedures to reflect their current operations, functions and staffing positions. Specifically, RS should outline its internal controls to ensure staff are up-to-date and ensure accurate and quality imaging. 4 Report Number 4K-RS-00-17-039 Management Response Management concurs with the recommendation and will review and update their policies and procedures to reflect the current operation. 2. Quality Assurance Audits Are Not Conducted Periodically We found RS has not conducted a periodic quality assurance audit of its imaged documents since September 2012. RS conducted the last quality assurance audit to address the National Archives Record Administration’s (NARA) requirement to conduct quality reviews on imaged records prior to securely destroying the hardcopies of these images after one year. In accordance with the Retirement Operations Imaging Operating Plan, RS should conduct periodic quality assurance audits to ensure documents are accurately imaged, and properly stored. RS has linked conducting quality assurance audits to NARA’s requirements to conduct a quality review of scanned records for destroying records as in the last audit they conducted in fiscal year 2012. However, RS should be conducting these audits periodically to ensure the imaged records’ accuracy and quality. By RS not conducting periodic quality assurance audits, they are not ensuring the accuracy of all imaged records. Recommendation 2 We recommend that RS conduct periodic quality assurance audits as specified in their Retirement Operations Imaging Operating Plan. Management Response Management concurs with the recommendation and states it is planning to conduct an audit this fiscal year. 3. No Performance Measures to Assess Benefits of Imaging Efforts RS’ implemented its imaging operations in January 2012, to prepare for implementation of the RS Information Technology Strategic Vision of a paperless retirement claims process and to reduce paper records storage space in its’ Boyers, Pennsylvania facility. However, RS has not 5 Report Number 4K-RS-00-17-039 developed any performance indicators that would allow it to measure the progress of its imaging operations in achieving its desired results. In accordance with OMB Circular No. A–11 and as outlined in the GPRA Modernization Act of 2010, Federal agencies are required to provide more frequent updates of actual performance on indicators that provide significant value to the Government.4 By not establishing performance measures to track the efforts of its imaging operations, RS increases the risk of wasting limited resources on a program that is not meeting its intended purpose. Recommendation 3 We recommend that RS develop performance measures to determine if its imaging operations is achieving its intended results. Management Response Management concurs with the recommendation and states they are planning to conduct an audit this fiscal year and will to determine from the results the appropriate performance measures. 4 Office of Management and Budget, Circular No. A–11 Preparation, Submission, and Execution of the Budget, July 2017, p. 675. 6 Report Number 4K-RS-00-17-039 APPENDIX A: SCOPE AND METHODOLOGY We conducted this evaluation in accordance with the Quality Standards for Inspection and Evaluation, January 2012, approved by the Council of the Inspectors General on Integrity and Efficiency. We performed our evaluation fieldwork from May 17, 2017, through September 27, 2017, at the OPM Headquarters in Washington, D.C, and RS’ offices in Boyers, Pennsylvana. The scope of this evaluation was for fiscal years 2012 – 2017. To assess RS’ Imaging Operations, we evaluated (1) the efficiency of RS’ process for imaging documents into its Electronic Document Management System, and (2) the effectiveness of the imaged documents on end users. As part of the planning phase for this evaluation, we identified and met with RS management and personnel. Using a standard set of questions, we interviewed those individuals to obtain information on strategic goals and anticipated benefits; roles and responsibilities; quality assurance controls; tracked data; maintenance and retention of hardcopies; and user activity related to imaging operations and use of scanned images in the Electronic Document Management System and viewable via the Data Viewer. We focused our fieldwork on the efficiency of the imaging process as well as the usage of the imaged records. We reviewed RS’ Annual Performance Reports to identify how they quantify their performance on goals related to imaging operations to determine if they were meeting these goals. To meet our objectives we reviewed the following documents related to RS’ process for imaging records: x Retirement Services’ Strategic Plan (August 2013); x Fiscal year 2014 – fiscal year 2018 OPM’s Strategic Plan; x Fiscal year 2015 – fiscal year 2016 OPM’s Annual Performance Report; x GRS 4.3 – General Records Schedule 4.3 Input Records, Output Records, and Electronic Copies; x Retirement Operations Imaging Operational Plan; x Imaging Procedures; x Customer Service Representative Job Descriptions; x Customer Service Representative Job Aids; 7 Report Number 4K-RS-00-17-039 x Electronic Document Management System Weekly Stats; x Quality Assurance Audit of Imaged Documents; x Retirement Record Data Viewer usage by Agency Activity report; x Package versus Storage Report; x Last Action Reports; and x Fiscal year 2012 Costs for Imaging Operation. Additionally, we reviewed the NARA’s General Records Schedule 4.3 Input Records, Output Records, and Electronic Copies to identify the requirements for retaining electronic records. We also reviewed OPM’s Strategic Plan to identify goals related to imaging operations. We conducted a site visit at the Retirement Operations Center in Boyers, Pennsylvana, to gain a better understanding of RS’ imaging process and to identify the internal controls RS has in place to verify the quality of scanned images. While there, we observed the imaging process, file storage space, and staffing levels. We also received a demonstration from OPM’s Enterprise Human Resources Integration personnel to gain an understanding of how internal and external users access documents. RS provided us a list of bundle control numbers for imaged documents scanned from January 2017 to July 2017. To assess RS’ quality control process within its imaging operations, we selected a random sample of 200 bundle control numbers out of 99,770. For each bundle control number in our sample, we obtained and reviewed a Last Action Report. To test RS’ imaging quality controls, we analyzed the sampled Last Action Reports to ensure that the customer service representative that conducted the barcoding stage of the imaging process did not also conduct the validation stage. Due to the nature of the evaluation, we did not verify the reliability of the list of bundle control numbers and Last Action Reports provided by RS. However, while analyzing these documents, nothing came to our attention to cause us to doubt their reliability. We believe that the information provided was sufficient to achieve our evaluation objectives. To gain external users’ perspectives on viewing scanned images via the Retirement Record Data Viewer; we developed and conducted a survey. While we identified 793 external users, we were only able to send the survey to 766 external users’ email addresses requesting their participation. Two hundred and sixty seven participants submitted their responses for analysis through a third- party website.5 Below are the questions we asked external users and the number of responses we received for each: 5 We utilized surveymonkey.com to conduct and facilitate our survey and analysis of the participants’ responses. 8 Report Number 4K-RS-00-17-039 x Do you utilize OPM’s Retirement Services’ scanned images via the Record Data Viewer to assist with completing an employee’s retirement application package? We received 267 responses to this question. x On average per week, how often do you access the Record Data Viewer to search for imaged documents to assist with completing an employee’s retirement package? We received 251 responses to this question. x How satisfied are you with the quality of the imaged documents that are available via the Record Data Viewer? We received 251 responses to this question. x When imaged documents are available in the Record Data Viewer for an employee, how beneficial are these images in assisting you with accurately completing their retirement application package? We received 255 responses to this question. 9 Report Number 4K-RS-00-17-039 APPENDIX B: MANAGEMENT COMMENTS 10 Report Number 4K-RS-00-17-039 11 Report Number 4K-RS-00-17-039 Report Fraud, Waste, and Mismanagement Fraud, waste, and mismanagement in the Government concerns everyone: Office of the Inspector General staff, agency employees, and the general public. We actively solicit allegations of any inefficient and wasteful practices, fraud, and mismanagement related to OPM programs and operations. You can report allegations to us in several ways: By Internet: http://www.opm.gov/our-inspector-general/hotline-to- report-fraud-waste-or-abuse By Phone: Toll Free Number: (877) 499-7295 Washington Metro Area: (202) 606-2423 By Mail: Office of the Inspector General U.S. Office of Personnel Management 1900 E Street, NW Room 6400 Washington, DC 20415-1100 12 Report Number 4K-RS-00-17-039
Evaluation of the U.S. Office of Personnel Management's Retirement Services' Imaging Operations
Published by the Office of Personnel Management, Office of Inspector General on 2018-03-14.
Below is a raw (and likely hideous) rendition of the original report. (PDF)