oversight

Veterans Benefits Administration: Better Collection and Analysis of Attrition Data Needed to Enhance Workforce Planning

Published by the Government Accountability Office on 2003-04-28.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

             United States General Accounting Office

GAO          Report to the Ranking Democratic
             Member, Committee on Veterans’
             Affairs, House of Representatives


April 2003
             VETERANS
             BENEFITS
             ADMINISTRATION

             Better Collection and
             Analysis of Attrition
             Data Needed to
             Enhance Workforce
             Planning




GAO-03-491
                                               April 2003


                                               VETERANS BENEFITS ADMINISTRATION

                                               Better Collection and Analysis of Attrition
Highlights of GAO-03-491, a report to the
Ranking Democratic Member, Committee           Data Needed to Enhance Workforce
on Veterans' Affairs, House of
Representatives                                Planning


By the year 2006, the Veterans                 About 16 percent of new examiners hired in fiscal year 2001 left VBA within
Benefits Administration (VBA)                  12 months of their hiring date, more than double the 6 percent rate for all
projects it will lose a significant            VBA employees who left that year. In general, new hire attrition tends to
portion of its mission-critical                exceed the rate for all other employees, and VBA’s 16 percent rate is similar
workforce to retirement. VBA has               to the attrition rate for all new federal employees hired in recent years, when
hired over 2,000 new employees to
begin to fill this expected gap.
                                               as many as 17 percent left within 12 months of being hired.
GAO was asked to review: (1) the
attrition rate at VBA, particularly            VBA does not have adequate data on the reasons why employees,
for new employees who examine                  particularly new employees, choose to leave the agency. VBA has
veterans’ claims, and the agency’s             descriptive data on whether employees leave the agency through
methods for calculating attrition;             resignation, termination, retirement, or transfer, but does not yet have
and (2) the adequacy of VBA’s                  comprehensive data on the reasons employees resign. While VBA collects
analysis of attrition data, including          some data on the reasons for attrition in exit interviews, these data are
the reasons for attrition. To                  limited because exit interviews have not been conducted consistently, and
answer these questions, GAO                    the data from these interviews are not compiled and analyzed. Without such
analyzed attrition data from VBA’s             data, VBA cannot determine ways to address why employees are leaving.
Office of Human Resources,
calculated attrition rates for VBA
                                               Furthermore, VBA has not performed analysis to determine whether it can
and other federal agencies using a             reduce its staff attrition. Despite recent steps to improve the collection and
governmentwide database on                     analysis of data on the reasons for attrition, an overall strategy for the
federal employment, and                        collection and analysis of attrition data could help guide workforce planning
interviewed VBA officials about                and determine the extent to which attrition and its costs could be reduced.
their efforts to measure attrition
and determine why new employees                Percentage of Examiners Who Left Within 2 Years of Their Hiring Date, FY 1998-2001
leave.
                                               Percentage who left within 12 and 24 months of hiring
                                               18     17.6
                                                                                                     16.3
                                                                                                                       15.8
                                               16
                                                                                14.7
To ensure that VBA collects and                14
analyzes information on the                    12
reasons for attrition, particularly
                                               10
for new hires, GAO recommends
that the Secretary of Veterans                  8
Affairs (VA) direct the Under                   6                                                                5.5
                                                                 4.5
Secretary for Benefits to develop a                                                         4.0
                                                4
strategy for the systematic                                                                                                   1.8
collection and analysis of attrition            2
data, including attrition rates,                0
reasons for leaving, and cost data;
and that VBA integrate the results                    1998                      1999                 2000              2001
of its attrition analysis into its                    Year of hire
workforce plan. VA concurred with
GAO’s recommendation.                                           0-12 months                       13-24 months

                                               Source: OPM's Central Personnel Data File.
www.gao.gov/cgi-bin/getrpt?GAO-03-491.
                                               Note: Data for fiscal year 2001 do not reflect a full 24-month time period.
To view the full report, including the scope
and methodology, click on the link above.
For more information, contact Cynthia A.
Bascetta at (202) 512-7101 or
bascettac@gao.gov.
Contents


Letter                                                                                              1
                       Results in Brief                                                             1
                       Background                                                                   3
                       Attrition at VBA Is Higher for Newly Hired Examiners Than for the
                         Agency Overall                                                             5
                       VBA Lacks Adequate Data on Reasons Employees Leave and
                         Analysis of Staff Attrition                                               9
                       Conclusions                                                                14
                       Recommendation                                                             14
                       Agency Comments                                                            14

Appendix I             Objectives, Scope, and Methodology                                         16
                       Objectives                                                                 16
                       Scope and Methodology                                                      16

Appendix II            Comments from the Department of Veterans Affairs                           18



Appendix III           GAO Contacts and Staff Acknowledgments                                     20
                       GAO Contacts                                                               20
                       Acknowledgments                                                            20

Related GAO Products                                                                              21
                       General Human Capital Reports                                              21
                       Department of Veterans Affairs                                             21
                       Veterans Benefits Administration                                           21


Table
                       Table 1: Overall Attrition Rates for VBA Examiners, Other VBA,
                                Other Department of Veterans Affairs, and Other Federal
                                Employees, Fiscal Years 2000-2002                                   6


Figures
                       Figure 1. Examiners Hired by VBA, Fiscal Years 1998-2002                     4
                       Figure 2: Percentage of Examiners Who Left VBA within 2 Years of
                                Their Hiring Date, Fiscal Years 1998-2001                           6



                       Page i                             GAO-03-491 Veterans Benefits Administration
Abbreviations

CPDF              Central Personnel Data File
OMB               Office of Management and Budget
OPM               Office of Personnel Management
SSA               Social Security Administration
VA                Department of Veterans Affairs
VBA               Veterans Benefits Administration




This is a work of the U.S. Government and is not subject to copyright protection in the
United States. It may be reproduced and distributed in its entirety without further
permission from GAO. It may contain copyrighted graphics, images or other materials.
Permission from the copyright holder may be necessary should you wish to reproduce
copyrighted materials separately from GAO’s product.




Page ii                                     GAO-03-491 Veterans Benefits Administration
United States General Accounting Office
Washington, DC 20548




                                   April 28, 2003

                                   The Honorable Lane Evans
                                   Ranking Democratic Member
                                   Committee on Veterans’ Affairs
                                   House of Representatives

                                   Dear Mr. Evans:

                                   The Veterans Benefits Administration (VBA) projects that 21 percent of its
                                   employees who examine veterans’ claims and are eligible to retire will do
                                   so by the year 2006. To prevent the potential disruptions in service to
                                   veterans applying for disability compensation that could result from these
                                   retirements, as well as to address a large claims backlog, VBA hired over
                                   2,000 new examiners between fiscal years 1998 and 2002. While VBA
                                   recognizes the importance of retaining its new employees, until 2001 it was
                                   not regularly calculating an attrition rate for its newly hired employees.

                                   Because of the concern that VBA maintain a sufficient workforce, you
                                   asked us to examine (1) the attrition rate at VBA, particularly for new
                                   employees who examine veterans’ claims, and the agency’s methods for
                                   calculating attrition; and (2) the adequacy of VBA’s analysis of attrition
                                   data, including the reasons for attrition.

                                   To do our work, we obtained and analyzed attrition data from VBA’s Office
                                   of Human Resources and interviewed VBA officials. We focused our
                                   analysis on new employees because of the investment in training they
                                   need to reach full productivity. We calculated VBA’s attrition rates and
                                   compared them to those for other federal new hires, using a
                                   governmentwide database on federal civilian employment. We also
                                   interviewed Office of Personnel Management (OPM) officials to identify
                                   generally accepted methods of calculating attrition and to determine how
                                   federal agencies develop and analyze data on attrition and the reasons for
                                   attrition. We conducted our work between October 2002 and February
                                   2003 in accordance with generally accepted government auditing
                                   standards.


Results in Brief                   Using OPM data for fiscal year 2001, we calculated the attrition rate at
                                   VBA for newly hired examiners at about 16 percent, more than double the
                                   6 percent rate for all VBA employees. The attrition rate for newly hired


                                   Page 1                               GAO-03-491 Veterans Benefits Administration
examiners is similar to the attrition rate for all new federal employees
hired in recent years. Specifically, between fiscal years 1998 and 2001, as
many as 17 percent left within 12 months of being hired. While it is typical
for new hire attrition to exceed overall attrition, the new hire attrition rate
was much higher in certain VBA regional offices located in major urban
areas than it was in other regional offices. For example, in the Newark
regional office, 40 percent of examiners hired in fiscal year 2001 left within
the first year of employment, while no newly hired examiners in Wichita
left. VBA calculates attrition by counting employees who leave the agency
and comparing that number to either total employees or a subgroup of
total employees. The methods VBA uses to calculate attrition are
consistent with those used by OPM and other federal agencies, and VBA’s
calculations are similar to those we used for this report.

While VBA has descriptive data on whether employees leave the agency
through resignation, termination, retirement, or transfer, it does not yet
have adequate analytic data on the reasons why employees, particularly
new examiners, leave the agency. VBA has also not conducted the types of
analysis that would help the agency determine whether its attrition,
particularly for newly hired examiners, is excessive. Efforts to collect
better data on the reasons for attrition, for example by using exit
interviews, are under way, but it will take some time before the results can
be fully analyzed. Without such analysis, VBA cannot determine ways to
address the reasons employees are leaving. Furthermore, VBA has not
fully analyzed the cost implications of its attrition, nor has it performed
the types of analysis, such as comparisons of its own attrition to that of
other federal employees who perform similar work, that would help the
agency determine the significance of its attrition rate, particularly for
newly hired examiners. However, recent requirements approved by the
Department of Veterans Affairs (VA) will begin to provide VBA the
opportunity to enhance the collection and analysis of attrition data. For
example, a VA-wide policy approved in January 2003 requires the use of
attrition data to guide workforce planning.

Because information on staff attrition is essential for effective workforce
planning, we are recommending that the Secretary of Veterans Affairs
direct the Under Secretary for Benefits to develop a strategy for the
systematic collection and analysis of attrition data at VBA. Furthermore,
we recommend that the results be integrated into the agency’s workforce
plan. VA concurred with our recommendation.




Page 2                                GAO-03-491 Veterans Benefits Administration
             VBA provides benefits to about 2.7 million veterans and about
Background   579,000 surviving spouses, children, and parents. These benefits and
             services include disability compensation and pension, education, loan
             guaranty, and insurance. VBA employs about 5,000 examiners,1 and they
             represent about 40 percent of the agency’s entire workforce. Most
             examiners are located at 57 regional offices and are responsible for
             reviewing and processing veterans’ disability claims. Typically, they begin
             service at GS-5 or GS-7 and can be promoted to GS-10.2

             Between 1998 and 2002, VBA hired over 2,000 new examiners (see fig. 1).
             According to VBA officials, this was the first time VBA had the authority to
             hire significant numbers of examiners. These examiners were hired in
             anticipation of a large number of future retirements. For example, in
             2000, VBA was expecting the retirement of 1,100 experienced examiners in
             the subsequent 5 years. The hiring of these new examiners coincided with
             a growth in the backlog of claims awaiting decisions. Between 1998 and
             2001, the backlog increased by 74 percent from about 241,000 to about
             420,000. VBA has since implemented an initiative to reduce this backlog.3




             1
              According to VBA, these positions carry the title of Veterans Service Representative. These
             positions and similar ones, such as rating specialists, are classified as job series 996,
             veterans claims examiner. For our analysis, we focused on the 996 job series. For this
             report, we are referring to jobs in this series as examiners.
             2
              According to a VBA official, in some cases, they can also start at GS-9. In 2003, basic
             starting salaries for GS-5 and GS-7 are about $23,400 and $29,000, respectively, not counting
             locality pay, and for those hired at the GS-9 level, the corresponding starting salary is about
             $35,500. VBA is planning to extend competitive promotion potential for this job series to
             GS-11.
             3
              VBA began to implement this initiative, called Claims Process Improvement, at all of its
             regional offices in July 2002. For more information, see U.S. General Accounting Office,
             Veterans’ Benefits: Claims Processing Timeliness Performance Measures Could Be
             Improved, GAO-03-282 (Washington, D.C.: Dec. 19, 2002).




             Page 3                                        GAO-03-491 Veterans Benefits Administration
Figure 1. Examiners Hired by VBA, Fiscal Years 1998-2002

New examiners hired
1,200



1,000



    800



    600



    400



    200



     0

          FY98   FY99   FY00   FY01    FY02
Source: VBA.



According to VBA, it takes 2 to 3 years for a newly hired examiner to
become fully productive. After being hired, new examiners receive a
combination of formal training in a central location and on-the-job training
in one of VBA’s regional offices. Once on the job, these workers perform a
variety of critical tasks, such as compiling medical evidence, assessing the
extent of the disability, determining the level of benefit, handling payment,
and considering appeals.

Workforce planning, which can be guided by different types of data, is a
key component to maintaining a workforce that can carry out the tasks
critical to an agency’s mission. Strategic workforce planning focuses on
developing and implementing long-term strategies—clearly linked to an
agency’s mission and programmatic goals—for acquiring, developing, and
retaining employees. In addition to data on attrition rates and the reasons
for attrition, data that can guide workforce planning include size and
composition of the workforce, skills inventory, projected retirement
eligibility and retirement rates, and feedback from exit interviews.4 These



4
For more information, see U.S. General Accounting Office, A Model of Strategic Human
Capital Management, Exposure Draft, GAO-02-373SP (Washington, D.C.: Mar. 15, 2002).




Page 4                                   GAO-03-491 Veterans Benefits Administration
                            data can be analyzed to identify gaps between an agency’s workforce and
                            its current and future needs. This insight can, in turn, become the basis for
                            developing strategies to build a workforce that accommodates those
                            needs.


                            We calculated the fiscal year 2001 attrition rate for new examiners at VBA
Attrition at VBA Is         at about 16 percent, more than twice as high as the 6 percent rate for all
Higher for Newly            employees who left that year. About 16 percent of new examiners hired in
                            fiscal year 2001 left the agency within 1 year of being hired. According to
Hired Examiners             human resources experts, it is typical for new employees to leave at higher
Than for the Agency         rates than all other employees. However, minimizing attrition is important
                            because of the high costs of hiring and training new examiners. VBA
Overall                     calculates attrition by counting employees who leave the agency and
                            comparing that number to either total employees or a subgroup of total
                            employees. The methods VBA uses to calculate attrition are consistent
                            with those used by OPM and other federal agencies.


Attrition for New           Attrition rates for new VBA examiners were generally higher than those
Employees at VBA Is More    for all VBA examiners and other employees. As shown in table 1, in fiscal
Than Twice as High as the   years 2000, 2001, and 2002, overall attrition rates for VBA examiners and
                            other VBA employees ranged from about 4 percent to 8 percent. However,
Agency’s Overall Rate of    among all new examiners hired in fiscal year 2001, about 16 percent left
About 6 Percent             the agency within 12 months, as shown in figure 2.5 These attrition rates,
                            for all employees as well as for newly hired examiners, reflect all types of
                            attrition—including resignation, retirement, and termination.6 New hire
                            attrition consists predominantly of resignations.




                            5
                             Since we issued our statement, Veterans Benefits Administration: Better Staff Attrition
                            Data and Analysis Needed GAO-03-452T (Washington, D.C.: Feb. 11, 2003), OPM issued
                            2002 data, which we used to update this analysis. For further information about our
                            methodology, see appendix I.
                            6
                             We did not include in our analysis of new hire attrition staff who left the examiner position
                            but remained in VBA, nor did we include transfers within VA.




                            Page 5                                       GAO-03-491 Veterans Benefits Administration
Table 1: Overall Attrition Rates for VBA Examiners, Other VBA, Other Department of
Veterans Affairs, and Other Federal Employees, Fiscal Years 2000-2002

                                                   VBA
                                                                                   All other
                                                All other                           federal
 Fiscal year          Examiners               employees Agencywide All other VA government
 2000                       4.6                       6.9      6.0           8.2         7.4
 2001                       6.0                       6.6      6.4           7.8         7.0
 2002                       7.0                       8.1      7.6           7.6         6.5
Source: OPM’s Central Personnel Data File.

Note: GAO performed these calculations by dividing separations by an average of the total workforce
on board at the beginning and end of each year. The averages could only be calculated for the years
shown. For all categories of employees shown, attrition rates were calculated based on white-collar
employees only.


Figure 2: Percentage of Examiners Who Left VBA within 2 Years of Their Hiring
Date, Fiscal Years 1998-2001

Percentage who left within 12 and 24 months of hiring

18     17.6

                                                         16.3
                                                                          15.8
16
                                 14.7

14

12

10

 8

 6                                                              5.5
                  4.5
                                             4.0
 4

                                                                                 1.8
 2

 0

       1998                      1999                    2000             2001
       Year of hire


                  0-12 months
                 13-24 months

Source: OPM's Central Personnel Data File.

Note: Data for fiscal year 2001 do not reflect a full 24-month time period. A comparable analysis could
not be done for fiscal year 2002 because comparable data were not available to reflect a full 24-
month time period.




Page 6                                                     GAO-03-491 Veterans Benefits Administration
According to human capital experts, in general, new employees tend to
leave at higher rates than all other employees, and some of this attrition
may even be desirable. Higher attrition among new hires has been the
experience for federal agencies historically and, according to our analysis
of OPM’s data, is generally the case governmentwide. The attrition rate for
all federal employees, both new hires and senior staff, was 7 percent in
fiscal year 2001.7 However, for all new federal employees—those hired in
fiscal years 1998, 1999, 2000, and 2001—as many as 17 percent left within
12 months of being hired. In some instances, according to a VBA official,
such attrition may even be desirable. For example, a newly hired employee
may turn out not to be a good fit for the agency.

The attrition rates we calculated for VBA’s newly hired examiners are
similar to those that the Social Security Administration (SSA) found
among its own new hires doing comparable work. As part of its own
workforce planning efforts, SSA has collected data on new hire attrition.
Of its employees who examine and process benefit claims, SSA found
attrition rates that ranged from about 15 percent for those hired in 1998 to
about 14 percent for those hired in 2000.8

VBA calculations show, and agency officials acknowledge, that attrition
for newly hired examiners is particularly high or particularly low in certain
locations.9 Specifically, VBA found attrition rates of 38 percent to
49 percent for new examiners hired over a 3-year period at four regional
offices—Baltimore (38 percent), Chicago (39 percent), Newark
(41 percent), and New York (49 percent). By contrast, some offices—such
as Phoenix, Louisville, Huntington, and Wichita—experienced no attrition
among new examiners hired during this period.




7
 These attrition rates represent employees at all federal agencies except VA.
8
 SSA’s analysis focused on claims representatives, service representatives, and teleservice
representatives. We could not duplicate this analysis because OPM’s Central Personnel
Data File does not break out job series into the kinds of subcategories that SSA was able to
use by accessing its own records. According to SSA officials, SSA plans to issue a report
including its attrition and other workforce analyses in spring 2003.
9
  According to VBA officials, attrition rates could also be calculated for certain subgroups of
newly hired examiners such as veterans or minorities. VBA has not calculated attrition
rates for these subgroups.




Page 7                                        GAO-03-491 Veterans Benefits Administration
VBA Uses Accepted          The two basic methods VBA uses to calculate attrition are consistent with
Methods to Calculate       methods used by OPM and other federal agencies. Each method, the
Attrition                  “annual calculation” and the “cohort calculation,” compares employees
                           who leave the agency to either total employees or a subgroup of total
                           employees. They provide different ways of looking at attrition trends. The
                           annual calculation indicates broad attrition patterns from year to year. In
                           contrast, the cohort calculation follows a particular group of employees
                           over time to see how many leave the agency, and the group or the
                           timeframe can vary to suit the needs of the analysis. VBA’s calculations
                           using this method are similar to ours. For example, VBA found that, of all
                           examiners hired in 2001, 18 percent had left within 2 years of their hiring
                           date, while our calculations using OPM data showed a corresponding rate
                           of 17.6 percent. The following are the two methods VBA uses:

                       •   Annual calculation. This method calculates attrition by dividing all
                           employees who left in a given year by an average of employees working at
                           the agency at the beginning of the year and at the end of the year.

                       •   Cohort calculation. This method calculates attrition by following a
                           specified group or “cohort” of employees to see how many leave the
                           agency. The cohort can be defined as all those hired (new hires only)
                           during a specific timeframe. These new hires are tracked for selected
                           intervals (3 months, 6 months, etc.). This method can be adapted by
                           defining the cohort differently (e.g., to track attrition among a subgroup of
                           new hires) and by using different timeframes for the tracking (e.g.,
                           12 months, 18 months, etc.). This calculation differs from the annual
                           calculation in that it does not use an average of the total workforce. VBA
                           used this method to determine the attrition rate of certain newly hired
                           examiners for a presentation in 2001 and for additional, more
                           comprehensive calculations in 2002. VBA plans to use this method to
                           calculate attrition rate for new examiners at least annually starting in 2003.

                           According to OPM officials, the annual method is a generally accepted
                           method used to calculate attrition by federal agencies. OPM officials also
                           recognized the value of the cohort method for calculations that require
                           specific time frames or groups of employees and added that tracking the
                           attrition of new employees is an important practice. OPM does not
                           mandate the use of a particular method for the calculation of attrition, but
                           officials said that any method used should be clearly explained.




                           Page 8                               GAO-03-491 Veterans Benefits Administration
                           While VBA has descriptive data on whether employees leave the agency
VBA Lacks Adequate         through resignation, termination, retirement, or transfer, it does not yet
Data on Reasons            have adequate information to analyze the reasons why employees,
                           particularly new employees who examine veterans’ claims, leave the
Employees Leave and        agency. While efforts to collect these data—for example, through an
Analysis of Staff          automated exit interview process—are under way, it will take some time
                           before the results can be fully analyzed. Furthermore, VBA has not
Attrition                  performed the types of analysis on its data, such as an analysis of the costs
                           of attrition, that would help the agency determine whether its attrition,
                           particularly for new examiners, is excessive. VBA is taking steps to ensure
                           that attrition data will be available to guide its workforce planning.


VBA Collects Some Data     While VBA systematically collects descriptive data on whether employees
on Types of Separations,   leave the agency through resignation, termination, retirement, or transfer,
but Data on Reasons Are    the data on the reasons employees leave have not been systematically
                           collected or analyzed. Without such analysis, VBA cannot determine the
Limited                    extent to which its attrition of newly hired examiners could be reduced. As
                           at other federal agencies, when employees leave VBA, a standard federal
                           “Form 52” is filled out.10 This form records whether the employee is leaving
                           due to a resignation, termination, retirement, or transfer. Because this
                           information appears on the form in discrete fields, VBA human resources
                           staff can easily enter it into the agency’s computer system to aggregate
                           information on the types of separations.

                           The Form 52 also includes a blank space for narrative comments on the
                           reasons for leaving. This space is primarily intended to be used in the case
                           of resignation, and its use is optional on the part of the employee.
                           According to VBA officials, this area is frequently left blank. When this
                           area is filled out, it is up to a human resources employee to decide how to
                           label an employee’s reason for leaving in the computer system. Several
                           “quit codes” exist to help in this labeling process. For example, reasons for
                           leaving can be coded as relating to pay and benefits, supervisory
                           relationship, opportunity for advancement, or personal reasons, including
                           family responsibilities, illness, or household relocation. All forms are sent
                           to one of four human resource centers to be entered into the agency’s
                           computer system. Human resources employees in these centers are
                           instructed to code the reasons for leaving to the best of their ability.
                           However, these staff members cannot clarify reasons when the


                           10
                                The Form 52, Request for Personnel Action, is used by all federal agencies, including VBA.




                           Page 9                                         GAO-03-491 Veterans Benefits Administration
    information is blank or ambiguous because they do not have access to
    either the separated employee or the regional human resources staff who
    actually processed the employee’s separation. Therefore, VBA officials do
    not consider the Form 52 to be a complete or reliable source of
    information on the reasons employees resign from VBA.

    Another source of information on reasons why examiners leave VBA is
    exit interviews. However, according to VBA officials, exit interviews have
    not been consistently conducted for all separating employees. VBA
    officials said that the downsizing of human resources staff in regional
    offices is at least partly responsible for the inconsistency with which exit
    interviews have been conducted in the past. In addition, the data from the
    interviews that were conducted have not been forwarded to national
    headquarters to be aggregated and analyzed. Existing VA policy has
    recognized the importance of exit interviews for determining the reasons
    an employee leaves, but it has not explicitly required the department to
    conduct them. To ensure more systematic collection of information about
    the reasons employees leave, as of February 2003, VA planned to change
    existing policy to require the department to offer exit surveys to
    separating employees, according to a VA official. Both VA and VBA told us
    they have plans to require that exit surveys be offered by spring 2003.

    Some offices and staff members within VBA have made special efforts to
    compile or collect information on the reasons examiners leave the agency
    by producing special studies or reports. These include the following:

•   High-Performing Young Promotable Employees (HYPE). In
    September 2002, a group of employees, representing six regional offices,
    prepared a report based on 72 exit interviews conducted at seven regional
    offices. The exit interviews had been conducted over 3 fiscal years:
    1999, 2000, and 2001. The report included recommendations that the
    agency develop a comprehensive strategic plan to address attrition and
    retention and improve and centralize its exit interview process.

•   Loss of New Hires in Veterans Service Centers. At the request of the
    head of VBA, the newly organized Office of Performance Analysis and
    Integrity (OPAI) issued a report in September 2002 that examined new hire
    attrition rates for regional offices individually. The report also looked at
    reasons for leaving, based on interviews with the directors of two regional
    offices.

•   Review of attrition data at certain regional offices. At least two
    regional offices have investigated the reasons for attrition in their own



    Page 10                              GAO-03-491 Veterans Benefits Administration
offices. For example, in October 2002, senior management at the Newark
regional office compiled information on the attrition of examiners over a
3-year period and the reasons these examiners left. This study was
prompted by concern about high attrition rates at the Newark office.
Portland did a similar review in September 2001.

These special efforts had several common findings. For example, three
reported that inadequate opportunity for training was one of the reasons
examiners left VBA. Two reported dissatisfaction with workload or
employees’ descriptions of the job as too difficult as reasons for leaving.
Two also identified instances in which examiners resigned as a result of
pending termination for poor performance or conduct. Reports associated
with these efforts touched on other reasons for resignation, including
inadequate opportunity for full utilization of skills and various personal
reasons.

The other source of information on reasons examiners left VBA was
anecdotal information provided by regional and other senior human
resources officials. For example, senior human resources officials said
that reasons for leaving included factors such as inadequate work space
and computer equipment as well as insufficient pay. According to a VBA
official, certain regional offices are aware of the types of employers with
whom they are competing. For example, some regional offices report
losing employees to a range of employers in both the public sector,
including other federal agencies (such as SSA and the Department of
Labor), and the private sector, including information technology firms. In
addition, these officials reported that some newly hired examiners left
when they discovered that the job tasks were not what they had expected.
Certain regional offices have taken steps to respond to some of these
findings. For example, some have offered job candidates opportunities to
observe the work place before being hired.

While all of these sources of information provide examples of reasons why
examiners left VBA, no comprehensive analysis of the reasons for attrition
has yet been conducted, due in large part to the inconsistency with which
exit interviews have been conducted and the fact that data on reasons for
leaving have not been compiled nationwide. Without such analysis, VBA
cannot yet determine the extent to which attrition among newly hired
examiners involves reasons that could be addressed by the agency, such as
work environment, or the reasons the agency cannot control, such as
personal reasons and market conditions. As a result, VBA has lacked
important information that could help the agency determine the extent to
which attrition could be reduced.



Page 11                             GAO-03-491 Veterans Benefits Administration
VBA Has Not Fully            VBA has not performed the types of analysis on its data that would help
Analyzed Data to             the agency determine the extent to which an attrition problem may exist.
Determine Whether            To better understand its own attrition, an agency can take advantage of
                             a range of analyses. These include the following:
Attrition Is Excessive
                         •   Comparisons. To understand the degree to which its attrition is a
                             problem, an agency can compare its own attrition to the attrition of other
                             federal agencies, especially to the attrition of agencies with employees
                             who do similar work. While one of VBA’s special reports did some broad
                             comparisons of VBA’s attrition to the attrition at other federal agencies,
                             VBA has not compared the attrition of newly hired examiners to the
                             attrition of employees in other parts of the federal government with
                             comparable job series, nor has it coordinated with other agencies to share
                             attrition data.

                         •   Attrition modeling. To understand the degree to which attrition is a
                             problem, an agency can estimate the attrition rates it expects in the future,
                             providing a baseline against which to measure the actual attrition it
                             experiences.11 This allows officials to determine if attrition rates are higher
                             or lower than expected. While VBA has projected retirement rates for
                             planning purposes, according to VBA officials, there was no formal or
                             informal process to estimate the expected attrition rates of the examiners
                             who joined the agency since 1998. In 2002, VA projected future attrition
                             trends for examiners in a restructuring plan submitted to the Office of
                             Management and Budget (OMB), and officials expect to compare these
                             projections to actual attrition rates for examiners in the future.

                         •   Cost analysis. To understand the degree to which attrition is a problem,
                             an agency can estimate the cost of recruiting and training new employees
                             who leave and the cost of recruiting and training their replacements. While
                             VBA’s Office of Human Resources conducted a partial estimate of attrition
                             costs in 2001, this estimate did not include all associated costs. For




                             11
                              For more information on attrition modeling, see U.S. General Accounting Office, Air
                             Traffic Control: FAA Needs to Better Prepare for Impending Wave of Controller Attrition,
                             GAO-02-591 (Washington, D.C.: June 14, 2002). For additional information on how attrition
                             data can be used by federal agencies, see Human Capital: A Self-Assessment for Agency
                             Leaders, GAO/OCG-00-14G (Washington, D.C.: Sept. 2000) and, for the importance of valid
                             and reliable data in assessing an agency’s workforce requirements, see A Model of Strategic
                             Human Capital Management, Exposure Draft, GAO-02-373SP (Washington, D.C.: Mar. 15,
                             2002).




                             Page 12                                     GAO-03-491 Veterans Benefits Administration
    example, the investment lost when a trained employee leaves was not
    included, although it is one of the most important and potentially
    expensive costs.

•   Labor market analysis. To understand the degree to which its attrition is
    a problem, an agency can evaluate labor market conditions in locations
    where it operates. Such an evaluation can provide context for
    understanding if an attrition rate is higher than might be expected in those
    locations. Using general labor market data, VBA has identified several
    locations where it faces significant competition from other employers,
    both public and private. This information could be used to better
    understand its attrition rate in those locations in the future. However, this
    information is not based on the actual employment plans of separating
    employees, and VBA does not routinely collect or document this
    information. According to a VBA official, collecting data on where VBA’s
    separating employees find employment after VBA would be useful for
    developing a more accurate understanding of the employers with whom
    VBA is competing.

    VBA is taking steps to ensure that attrition data will be available to guide
    workforce planning. First, according to a VBA official, VBA developed a
    workforce plan, following a workforce policy approved by VA in January
    2003.12 In a related document, VA stated its expectation that, in the current
    economy, attrition among examiners may stabilize. However, VBA cannot
    test that assumption without continued monitoring of attrition rates and
    improved data on reasons for attrition. Second, VBA has recently
    designated an official to head strategic planning efforts. While these
    efforts will address attrition and other human capital issues, a VBA official
    told us that its Office of Human Resources is expected to assume primary
    responsibility for human capital issues and to coordinate with the strategic
    planning office. Third, VA’s new automated exit survey, which VA officials
    expect to be available in spring 2003, has the potential to aid VBA in its
    attrition data gathering and analysis. Separating employees will be able to
    answer a series of questions about the reasons they decided to leave the
    agency. The survey will provide confidentiality for the employee,
    potentially allowing for more accurate responses. The survey will also
    facilitate electronic analysis that could be broken down by type of job and



    12
      The new VA policy requires workforce plans from all three of VA’s administrations—VBA,
    the Veterans Health Administration, and the National Cemetery Administration. VA first
    identified the need for a workforce policy following a workforce analysis required of all
    executive branch agencies by OMB in May 2001.




    Page 13                                    GAO-03-491 Veterans Benefits Administration
                  region. As of March 2003, VBA had created and filled a new position
                  intended to help VBA implement the automated exit survey, among other
                  responsibilities.


                  VBA’s ability to effectively serve veterans hinges on maintaining a
Conclusions       sufficient workforce through effective workforce planning. While attrition
                  data are just one part of workforce planning, the data are important
                  because they can be used to anticipate the number of employees and the
                  types of skills that need to be replaced. While VBA officials told us about
                  plans under way to better manage new hire attrition, the agency currently
                  lacks useful analysis of the reasons new employees leave. Furthermore,
                  while it is typical for new hires to leave their jobs at higher rates than
                  other employees, and this phenomenon is not unique to VBA, VBA has not
                  yet analyzed whether its new hire attrition is excessive, nor has it analyzed
                  the cost implications of such attrition to the agency. Such analysis can
                  help an agency determine the extent to which attrition could be reduced.
                  Sustained attention to both attrition rates and reasons for attrition,
                  particularly for new employees, can help VBA conduct more effective
                  workforce planning. Understanding the reasons for attrition could also
                  help the agency minimize the investment in training lost when a new
                  employee leaves.


                  To ensure that VBA collects and analyzes information on the reasons for
Recommendation    attrition, particularly for new employees, we recommend that the
                  Secretary of Veterans Affairs direct the Under Secretary for Benefits to
                  develop a strategy for the systematic collection and analysis of attrition
                  data. This could include the calculation of attrition rates, analysis of the
                  reasons for leaving, and estimation of the costs associated with new hire
                  attrition. Furthermore, we recommend that VBA integrate the results of its
                  attrition analysis into its workforce plan.


                  In its written comments on a draft of this report (see app. II), VA agreed
Agency Comments   with our findings and conclusions and concurred with our
                  recommendation. VBA is targeting July 1, 2003 for the implementation of
                  an exit survey process to develop data on reasons for employee turnover
                  and the costs of new hire attrition. VBA is also in the process of
                  developing its workforce plan as part of overall departmental efforts. This
                  should better position VBA to integrate the results of its analysis of
                  attrition into its workforce plan.




                  Page 14                              GAO-03-491 Veterans Benefits Administration
As agreed with your office, unless you publicly release its contents earlier,
we will make no further distribution of this report until 10 days after its
issue date. At that time, we will send copies of this report to the Secretary
of the Department of Veterans Affairs, appropriate congressional
committees, and other interested parties. We will also make copies of this
report available to others on request. The report will also be available at
no charge on GAO’s Web site at http://www.gao.gov.

If you or your staff have any questions about this report, please call me at
(202) 512-7101. Other contacts and contributors to this report are listed in
appendix III.

Sincerely yours,




Cynthia A. Bascetta
Director, Education, Workforce,
 and Income Security Issues




Page 15                              GAO-03-491 Veterans Benefits Administration
              Appendix I: Objectives, Scope, and
Appendix I: Objectives, Scope, and
              Methodology



Methodology

              The objectives of our review were to determine: (1) the attrition rate at the
Objectives    Veterans Benefits Administration (VBA), particularly for new employees
              who examine veterans’ claims, and VBA’s methods for calculating
              attrition; and (2) the adequacy of VBA’s analysis of attrition data, including
              the reasons for attrition. We conducted our work between October 2002
              and February 2003 in accordance with generally accepted government
              auditing standards.


              To determine the attrition rate for newly hired veterans claims examiners
Scope and     at VBA, we analyzed data from the Office of Personnel Management’s
Methodology   (OPM) Central Personnel Data File (CPDF). Using the CPDF data, we
              identified the newly hired veterans claims examiners and followed them
              over time to see how many left VBA. We identified all new hires for fiscal
              years 1998 through 2002 by using personnel action codes for accessions
              and conversions to career or career conditional positions. Accessions
              include new hires and hires of individuals returning to the government.
              Conversions are primarily temporary federal government employees
              whose positions are made permanent. Next, we determined whether these
              individuals had personnel actions indicating they had separated from VBA.
              Separation (attrition) included resignations, retirements, terminations, and
              deaths. We did not include a small percentage of individuals with
              inconsistent data such as multiple or different hiring or separation dates.
              The small percentage of employees with inconsistent data is congruent
              with the generally reliable data in the CPDF we have reported previously.
              (See U.S. General Accounting Office, OPM’s Central Personnel Data File:
              Data Appear Sufficiently Reliable to Meet Most Customer Needs,
              GAO/GGD-98-199 (Washington, D.C.: Sept. 30, 1998)). By subtracting the
              hire date from the separation date, we determined how long individuals
              worked before separating. We also noted those individuals who had not
              separated within specific time periods such as 0 to 12 months and 13 to
              24 months. We calculated the attrition rate for a specific time period by
              dividing the number of individuals who left within that time period by the
              total number of new hires tracked for that time period. Since we issued
              our statement,1 OPM issued 2002 data, which we used to update this
              analysis. The additional year of data resulted in a small number of
              additional employees who had multiple, different separation dates.
              Because we excluded all instances of employees with multiple records and
              because some of these instances corresponded to records previously


              1
               GAO-03-452T.




              Page 16                              GAO-03-491 Veterans Benefits Administration
Appendix I: Objectives, Scope, and
Methodology




included in our original analysis, the additional year of data resulted in
some revisions to our original analysis.

To determine overall attrition rates for veterans claims examiners at VBA
(not just new hires), we analyzed data from the CPDF for fiscal years 2000
through 2002. For each fiscal year, we counted the number of permanent
employees with personnel actions indicating they had separated from
VBA. We divided the total number of separations for each fiscal year by
the average of the number of permanent employees in the CPDF as of the
last pay period of the fiscal year before the fiscal year of the separations
and the number of permanent employees in the CPDF as of the last pay
period of the fiscal year of separations. To place the overall attrition rates
for VBA claims examiners in context, we compared VBA’s rates to those
for employees in other occupations and agencies (all other white-collar
VBA employees, all other white-collar employees in the Department of
Veterans Affairs, and all other white-collar employees in the executive
branch of the federal government).

To determine VBA’s methods for calculating attrition, we interviewed VBA
officials and reviewed VBA’s calculations. We also interviewed OPM
officials to identify generally accepted methods of calculating attrition. To
determine the adequacy of VBA’s data on the reasons for attrition and its
analysis of attrition data, we reviewed VBA reports with information about
the reasons for attrition, interviewed VBA officials to determine how VBA
collects data on the reasons employees leave the agency, and interviewed
a VA official about a new initiative to develop and implement an
automated exit survey. To determine the adequacy of VBA’s analysis of its
attrition, we reviewed VBA efforts to analyze attrition, interviewed VBA
officials, and interviewed OPM officials and relied on our prior reports on
federal human capital issues to determine how federal agencies develop
and analyze data on attrition and the reasons for attrition.




Page 17                               GAO-03-491 Veterans Benefits Administration
                   Appendix II: Comments from the Department
Appendix II: Comments from the Department
                   of Veterans Affairs



of Veterans Affairs




         Page 18                               GAO-03-491 Veterans Benefits Administration
          Appendix II: Comments from the Department
          of Veterans Affairs




Page 19                               GAO-03-491 Veterans Benefits Administration
                  Appendix III: GAO Contacts and Staff

Appendix III: GAO Contacts and Staff
                  Acknowledgments



Acknowledgments

                  Irene Chu, (202) 512-7102
GAO Contacts      Chris Morehouse, (202) 512-7214


                  In addition to those named above, others who made key contributions to
Acknowledgments   this report are Ronald Ito, Grant Mallie, Corinna Nicolaou, and Gregory
                  Wilmoth.




                  Page 20                                GAO-03-491 Veterans Benefits Administration
                    Related GAO Products
Related GAO Products


                    High-Risk Series: Strategic Human Capital Management. GAO-03-120.
General Human       Washington, D.C.: January 2003.
Capital Reports
                    Human Capital: Effective Use of Flexibilities Can Assist Agencies in
                    Managing Their Workforces. GAO-03-2. Washington, D.C.: December 6,
                    2002.

                    Air Traffic Control: FAA Needs to Better Prepare for Impending Wave of
                    Controller Attrition. GAO-02-591. Washington, D.C.: June 14, 2002.

                    A Model of Strategic Human Capital Management, Exposure Draft.
                    GAO-02-373SP. Washington, D.C.: March 15, 2002.

                    Federal Employee Retirements: Expected Increase Over the Next 5 Years
                    Illustrates Need for Workforce Planning. GAO-01-509. Washington, D.C.:
                    April 27, 2001.

                    Human Capital: A Self-Assessment Checklist for Agency Leaders.
                    GAO/OCG-00-14G. Washington, D.C.: September 2000.


                    Major Management Challenges and Program Risks: Department of
Department of       Veterans Affairs. GAO-03-110. Washington, D.C.: January 2003.
Veterans Affairs
                    High-Risk Series: An Update. GAO-03-119. Washington, D.C.: January
                    2003.


                    Veterans’ Benefits: Claims Processing Timeliness Performance Measures
Veterans Benefits   Could Be Improved. GAO-03-282. Washington, D.C.: December 19, 2002.
Administration
                    Veterans’ Benefits: Despite Recent Improvements, Meeting Claims
                    Processing Goals Will Be Challenging. GAO-02-645T. Washington, D.C.:
                    April 26, 2002.

                    Veterans’ Benefits: Training for Claims Processors Needs Evaluation.
                    GAO-01-601. Washington, D.C.: May 31, 2001.

                    Veterans Benefits Administration: Problems and Challenges Facing
                    Disability Claims Processing. GAO/T-HEHS/AIMD-00-146. Washington,
                    D.C.: May 18, 2000.




(130206)
                    Page 21                           GAO-03-491 Veterans Benefits Administration
                         The General Accounting Office, the audit, evaluation and investigative arm of
GAO’s Mission            Congress, exists to support Congress in meeting its constitutional responsibilities
                         and to help improve the performance and accountability of the federal
                         government for the American people. GAO examines the use of public funds;
                         evaluates federal programs and policies; and provides analyses,
                         recommendations, and other assistance to help Congress make informed
                         oversight, policy, and funding decisions. GAO’s commitment to good government
                         is reflected in its core values of accountability, integrity, and reliability.


                         The fastest and easiest way to obtain copies of GAO documents at no cost is
Obtaining Copies of      through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full-
GAO Reports and          text files of current reports and testimony and an expanding archive of older
                         products. The Web site features a search engine to help you locate documents
Testimony                using key words and phrases. You can print these documents in their entirety,
                         including charts and other graphics.
                         Each day, GAO issues a list of newly released reports, testimony, and
                         correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site
                         daily. The list contains links to the full-text document files. To have GAO e-mail
                         this list to you every afternoon, go to www.gao.gov and select “Subscribe to daily
                         E-mail alert for newly released products” under the GAO Reports heading.


Order by Mail or Phone   The first copy of each printed report is free. Additional copies are $2 each. A
                         check or money order should be made out to the Superintendent of Documents.
                         GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a
                         single address are discounted 25 percent. Orders should be sent to:
                         U.S. General Accounting Office
                         441 G Street NW, Room LM
                         Washington, D.C. 20548
                         To order by Phone:     Voice:    (202) 512-6000
                                                TDD:      (202) 512-2537
                                                Fax:      (202) 512-6061


                         Contact:
To Report Fraud,
                         Web site: www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in      E-mail: fraudnet@gao.gov
Federal Programs         Automated answering system: (800) 424-5454 or (202) 512-7470


                         Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800
Public Affairs           U.S. General Accounting Office, 441 G Street NW, Room 7149
                         Washington, D.C. 20548