U.S. OFFICE OF PERSONNEL MANAGEMENT
OFFICE OF THE INSPECTOR GENERAL
OFFICE OF EVALUATIONS
Final Evaluation Report
EVALUATION OF THE U.S. OFFICE OF
PERSONNEL MANAGEMENT’S RETIREMENT
SERVICES’ IMAGING OPERATIONS
Report Number 4K-RS-00-17-039
March 14, 2018
EXECUTIVE SUMMARY
Evaluation of The U.S. Office of Personnel Management’s Retirement Services’
Imaging Operations
Report No. 4K-RS-00-17-039 March 14, 2018
Why Did We Conduct the
Evaluation? What Did We Find?
The Retirement Services’ (RS) RS began a federally staffed imaging operation in Boyers,
Program’s Associate Director Pennsylvania, in January 2012, to image Federal employees’
requested that we conduct an retirement records. RS intended to achieve two specific results
evaluation of their imaging from the imaging operation:
operations.
x Address the limited record storage space at their Boyers
facility; and
The objectives of our evaluation
were to determine: (1) the efficiency
x Position themselves to achieve their strategic objective of
of RS’ process for imaging
transitioning to a paperless retirement claims process
documents into its Electronic
system.
Document Management System and,
(2) the effectiveness of those imaged During our evaluation, we found that the RS’ process for imaging
documents on end users. documents into its Electronic Document Management System was
efficient and end users felt that the images were beneficial to
accurately completing retirement application packages. However,
we identified areas within the imaging operations where RS needs
to take corrective actions:
x RS’ policies and procedures does not reflect their current
operations;
x RS has not conducted a periodic quality assurance audit of
its imaged documents since September 2012; and
x RS has not developed performance measures to assess
effectiveness of its imaging operations.
_______________________
William W. Scott, Jr.
Chief, Office of Evaluations
i
TABLE OF CONTENTS
Page
EXECUTIVE SUMMARY.......................................................................................... i
INTRODUCTION....................................................................................................... 1
RESULTS OF EVALUATION .................................................................................. 4
1. Retirement Services Policies and Procedures Do Not Reflect Current
Operations ............................................................................................................... 4
2. Quality Assurance Audits Are Not Conducted Periodically................................... 5
3. No Performance Measures to Assess Benefits of Imaging Efforts ......................... 5
APPENDIX A: Scope and Methodology ................................................................... 7
APPENDIX B: Management Comments ................................................................ 10
REPORT FRAUD, WASTE AND MISMANAGEMENT ................................... 12
INTRODUCTION
This final evaluation report details the results of our evaluation of the U.S. Office of Personnel
Management’s (OPM) Retirement Services’ (RS) Imaging Operations. The OPM’s Office of the
Inspector General (OIG) conducted this evaluation, as authorized by the Inspector General Act of
1978, as amended.
RS began a federally staffed imaging operation in Boyers, Pennsylvania, in January 2012, to
image Federal employees’ retirement records. RS intended to achieve two specific results from
the imaging operation:
x Address the limited record storage space at their Boyers facility; and
x Position itself to achieve their strategic objective of transitioning the Retirement Program
to a paperless system.
RS worked with OPM’s Office of the Chief Information Officer to develop the Retirement
Operations Imaging Operational Plan that outlined how the imaging effort would operate, types
of records scanned, quality control measures, and instructions on how to use the scanning
equipment. The operational plan also detailed duties for imaging personnel:
x Pulling non-case records from Federal Employees Retirement System Open File and
resources permitting, non-case; related records and documents retained in the Civil
Service Retirement System Open File;
x Prepping documents (Doc Prep);
x Bundling records;
x Barcoding bundles;
x Scanning bundles;
x Quality review of bundles;
x Exception processing (when required);
x Managing boxes of imaged records;
x Quality Control; and
x Storing boxes of imaged records.1
Multiple users, both internal and external, have varying levels of access to view the over 4
million images stored in the Electronic Document Management System.2 OPM’s internal users
1
OPM’s Retirement Services, Retirement Operations Imaging Operating Plan, January 17, 2012, p. 4.
2
The Electronic Data Management System is a database that contains scanned images of Federal employees’
records collected over their careers. This system also assist with retirement benefits calculations and assists with
long-term storage of hard copy documents.
1 Report Number 4K-RS-00-17-039
have access to these images for claim adjudication and other customer service purposes, and they
have the ability to search imaged documents across Federal agencies. External users, Federal
agencies, only have access to their specific employees’ imaged documents. Federal agencies use
the Retirement Record Data Viewer, which is a separate system that allows its users to view the
same information available within the Electronic Document Management System. As of August
2017, 793 external users from 44 Federal agencies have had access to these images to assist with
retirement counseling and other retirement application preparation. Since implementation
(August 2012), internal and external users have viewed over 978,000 imaged documents. (See
Graph 1.1)
See Graph 1.1
TotalImagedDocumentsViewedviaRetirementRecordDataViewer
1200000
TotalViewedDocuments
1000000
978150
932466
800000
662374
600000
437436
400000
200000 192880
16240
0
201220132014201520162017
During this evaluation, we conducted a survey to gain external users’ perspectives on viewing
scanned images via the Retirement Record Data Viewer. This survey yielded positive results.
While not all responders responded to every question, we found that for those who did respond:
x 53.4% of responders used the images to assist with completing an employee’s retirement
package four or more times per week.
x 76.5% of responders were satisfied with the quality of the images that are available.
2 Report Number 4K-RS-00-17-039
x 82.4% of responders felt that the images were either “extremely beneficial” or “very
beneficial” in accurately completing retirement application packages.3
3
Please refer to the Scope and Methodology section for details on our methodology used for conducting the survey
and details of the results.
3 Report Number 4K-RS-00-17-039
RESULTS OF EVALUATION
1. Retirement Services’ Policies and Procedures Do Not Reflect Current Operations
We found RS’ is not following the policies and procedures as outlined in its Retirement
Operations Imaging Operational Plan. Specifically, (1) personnel position titles have changed,
(2) personnel are no longer performing duties such as bundling records and conducting quality
reviews of bundles, and (3) RS has not formalized their established quality assurance process for
ensuring accuracy within the imaging process.
Since the plan was developed, RS has changed its staffing positions and internal controls over
operations to include establishing a quality assurance process for ensuring accuracy within the
imaging process. This quality assurance process for barcoding and validating, includes having
multiple personnel review the same record to ensure that the Customer Service Representative
who barcodes a document also does not validate the same document.
During our fieldwork, we obtained a random sample of fiscal year 2017’s Last Action Reports.
These reports outline each action through the imaging process by a unique bundle control
number to include the personnel responsible at the various stages. From these reports, we
assessed whether RS was following its undocumented quality assurance process regarding
barcoding and validating images. From our analysis, we found 6 out 200 instances (3%) where
the customer service representative who had barcoded a document also validated the image into
RS’ Electronic Document Management System.
The United States Government Accountability Office’s Standards for Internal Control in the
Federal Government (also known as the Green Book) defines the standards for internal controls
in the Federal Government entities’ objectives relating to the operations, reporting, and
compliance. These standards provide criteria for assessing the design, implementation, and
operating effectiveness of internal controls in Federal Government entities to determine if an
internal control system is effective. In accordance with the Green Book, Federal Government
entities are required to establish and maintain an effective internal control system.
Without RS formalizing its current policies and procedures to include their internal controls,
operations, and staffing positions, there is an increased risk of imaging operation personnel not
following the quality assurance process as identified in our analysis.
Recommendation 1
We recommend that RS update their Retirement Operations Imaging Operational Plan policies
and procedures to reflect their current operations, functions and staffing positions. Specifically,
RS should outline its internal controls to ensure staff are up-to-date and ensure accurate and
quality imaging.
4 Report Number 4K-RS-00-17-039
Management Response
Management concurs with the recommendation and will review and update their policies and
procedures to reflect the current operation.
2. Quality Assurance Audits Are Not Conducted Periodically
We found RS has not conducted a periodic quality assurance audit of its imaged documents since
September 2012. RS conducted the last quality assurance audit to address the National Archives
Record Administration’s (NARA) requirement to conduct quality reviews on imaged records
prior to securely destroying the hardcopies of these images after one year.
In accordance with the Retirement Operations Imaging Operating Plan, RS should conduct
periodic quality assurance audits to ensure documents are accurately imaged, and properly
stored.
RS has linked conducting quality assurance audits to NARA’s requirements to conduct a quality
review of scanned records for destroying records as in the last audit they conducted in fiscal year
2012. However, RS should be conducting these audits periodically to ensure the imaged records’
accuracy and quality. By RS not conducting periodic quality assurance audits, they are not
ensuring the accuracy of all imaged records.
Recommendation 2
We recommend that RS conduct periodic quality assurance audits as specified in their
Retirement Operations Imaging Operating Plan.
Management Response
Management concurs with the recommendation and states it is planning to conduct an audit this
fiscal year.
3. No Performance Measures to Assess Benefits of Imaging Efforts
RS’ implemented its imaging operations in January 2012, to prepare for implementation of the
RS Information Technology Strategic Vision of a paperless retirement claims process and to
reduce paper records storage space in its’ Boyers, Pennsylvania facility. However, RS has not
5 Report Number 4K-RS-00-17-039
developed any performance indicators that would allow it to measure the progress of its imaging
operations in achieving its desired results.
In accordance with OMB Circular No. A–11 and as outlined in the GPRA Modernization Act of
2010, Federal agencies are required to provide more frequent updates of actual performance on
indicators that provide significant value to the Government.4
By not establishing performance measures to track the efforts of its imaging operations, RS
increases the risk of wasting limited resources on a program that is not meeting its intended
purpose.
Recommendation 3
We recommend that RS develop performance measures to determine if its imaging operations is
achieving its intended results.
Management Response
Management concurs with the recommendation and states they are planning to conduct an audit
this fiscal year and will to determine from the results the appropriate performance measures.
4
Office of Management and Budget, Circular No. A–11 Preparation, Submission, and Execution of the Budget, July
2017, p. 675.
6 Report Number 4K-RS-00-17-039
APPENDIX A: SCOPE AND METHODOLOGY
We conducted this evaluation in accordance with the Quality Standards for Inspection and
Evaluation, January 2012, approved by the Council of the Inspectors General on Integrity and
Efficiency.
We performed our evaluation fieldwork from May 17, 2017, through September 27, 2017, at the
OPM Headquarters in Washington, D.C, and RS’ offices in Boyers, Pennsylvana.
The scope of this evaluation was for fiscal years 2012 – 2017. To assess RS’ Imaging
Operations, we evaluated (1) the efficiency of RS’ process for imaging documents into its
Electronic Document Management System, and (2) the effectiveness of the imaged documents
on end users.
As part of the planning phase for this evaluation, we identified and met with RS management
and personnel. Using a standard set of questions, we interviewed those individuals to obtain
information on strategic goals and anticipated benefits; roles and responsibilities; quality
assurance controls; tracked data; maintenance and retention of hardcopies; and user activity
related to imaging operations and use of scanned images in the Electronic Document
Management System and viewable via the Data Viewer.
We focused our fieldwork on the efficiency of the imaging process as well as the usage of the
imaged records. We reviewed RS’ Annual Performance Reports to identify how they quantify
their performance on goals related to imaging operations to determine if they were meeting these
goals.
To meet our objectives we reviewed the following documents related to RS’ process for imaging
records:
x Retirement Services’ Strategic Plan (August 2013);
x Fiscal year 2014 – fiscal year 2018 OPM’s Strategic Plan;
x Fiscal year 2015 – fiscal year 2016 OPM’s Annual Performance Report;
x GRS 4.3 – General Records Schedule 4.3 Input Records, Output Records, and Electronic
Copies;
x Retirement Operations Imaging Operational Plan;
x Imaging Procedures;
x Customer Service Representative Job Descriptions;
x Customer Service Representative Job Aids;
7 Report Number 4K-RS-00-17-039
x Electronic Document Management System Weekly Stats;
x Quality Assurance Audit of Imaged Documents;
x Retirement Record Data Viewer usage by Agency Activity report;
x Package versus Storage Report;
x Last Action Reports; and
x Fiscal year 2012 Costs for Imaging Operation.
Additionally, we reviewed the NARA’s General Records Schedule 4.3 Input Records, Output
Records, and Electronic Copies to identify the requirements for retaining electronic records. We
also reviewed OPM’s Strategic Plan to identify goals related to imaging operations.
We conducted a site visit at the Retirement Operations Center in Boyers, Pennsylvana, to gain a
better understanding of RS’ imaging process and to identify the internal controls RS has in place
to verify the quality of scanned images. While there, we observed the imaging process, file
storage space, and staffing levels.
We also received a demonstration from OPM’s Enterprise Human Resources Integration
personnel to gain an understanding of how internal and external users access documents.
RS provided us a list of bundle control numbers for imaged documents scanned from January
2017 to July 2017. To assess RS’ quality control process within its imaging operations, we
selected a random sample of 200 bundle control numbers out of 99,770. For each bundle control
number in our sample, we obtained and reviewed a Last Action Report. To test RS’ imaging
quality controls, we analyzed the sampled Last Action Reports to ensure that the customer
service representative that conducted the barcoding stage of the imaging process did not also
conduct the validation stage. Due to the nature of the evaluation, we did not verify the reliability
of the list of bundle control numbers and Last Action Reports provided by RS. However, while
analyzing these documents, nothing came to our attention to cause us to doubt their reliability.
We believe that the information provided was sufficient to achieve our evaluation objectives.
To gain external users’ perspectives on viewing scanned images via the Retirement Record Data
Viewer; we developed and conducted a survey. While we identified 793 external users, we were
only able to send the survey to 766 external users’ email addresses requesting their participation.
Two hundred and sixty seven participants submitted their responses for analysis through a third-
party website.5 Below are the questions we asked external users and the number of responses we
received for each:
5
We utilized surveymonkey.com to conduct and facilitate our survey and analysis of the participants’ responses.
8 Report Number 4K-RS-00-17-039
x Do you utilize OPM’s Retirement Services’ scanned images via the Record Data Viewer
to assist with completing an employee’s retirement application package? We received
267 responses to this question.
x On average per week, how often do you access the Record Data Viewer to search for
imaged documents to assist with completing an employee’s retirement package? We
received 251 responses to this question.
x How satisfied are you with the quality of the imaged documents that are available via the
Record Data Viewer? We received 251 responses to this question.
x When imaged documents are available in the Record Data Viewer for an employee, how
beneficial are these images in assisting you with accurately completing their retirement
application package? We received 255 responses to this question.
9 Report Number 4K-RS-00-17-039
APPENDIX B: MANAGEMENT COMMENTS
10 Report Number 4K-RS-00-17-039
11 Report Number 4K-RS-00-17-039
Report Fraud, Waste, and
Mismanagement
Fraud, waste, and mismanagement in the
Government concerns everyone: Office of
the Inspector General staff, agency
employees, and the general public. We
actively solicit allegations of any inefficient
and wasteful practices, fraud, and
mismanagement related to OPM programs
and operations. You can report allegations to
us in several ways:
By Internet: http://www.opm.gov/our-inspector-general/hotline-to-
report-fraud-waste-or-abuse
By Phone: Toll Free Number: (877) 499-7295
Washington Metro Area: (202) 606-2423
By Mail: Office of the Inspector General
U.S. Office of Personnel Management
1900 E Street, NW
Room 6400
Washington, DC 20415-1100
12 Report Number 4K-RS-00-17-039
Evaluation of the U.S. Office of Personnel Management's Retirement Services' Imaging Operations
Published by the Office of Personnel Management, Office of Inspector General on 2018-03-14.
Below is a raw (and likely hideous) rendition of the original report. (PDF)