oversight

Veterans' Employment and Training: Better Targeting, Coordinating, and Reporting Needed to Enhance Program Effectiveness

Published by the Government Accountability Office on 2012-12-13.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                United States Government Accountability Office

GAO             Report to Congressional Requesters




                VETERANS’
December 2012



                EMPLOYMENT AND
                TRAINING

                Better Targeting,
                Coordinating, and
                Reporting Needed to
                Enhance Program
                Effectiveness




GAO-13-29
                                               December 2012

                                               VETERANS’ EMPLOYMENT AND TRAINING
                                               Better Targeting, Coordinating, and Reporting
                                               Needed to Enhance Program Effectiveness
Highlights of GAO-13-29, a report to
congressional requesters




Why GAO Did This Study                         What GAO Found
In fiscal year 2011, the federal               The six federal veterans’ employment and training programs offer similar employment
government spent an estimated $1.2             services, but largely target different groups. Among these programs, the Department of
billion on six veterans’ employment            Labor’s (Labor) Disabled Veterans’ Outreach Program has the greatest potential for
and training programs, serving about           overlap with other veterans’ programs and Labor’s employment programs for the general
880,000 participants. Labor                    population. Federal law governing the Disabled Veterans’ Outreach Program makes all
administers five of these programs and         veterans who meet the broad definition of “eligible veteran” eligible for its services, but
VA administers one. Despite these              gives disabled veterans and economically and educationally disadvantaged veterans the
                                               highest priority for services. However, Labor’s guidance does not provide states—who
efforts, the unemployment rate for
                                               administer the program using federal funds—criteria for prioritizing services. The law also
veterans who have recently separated
                                               generally requires that program staff provide participants with intensive services (e.g.,
from the military is higher than that for      individual employment plans), but Labor’s data indicate that nationally 28 percent of
the civilian population. The number of         participants received such services in 2011. In explaining this statistic, Labor officials said
service members transitioning to the           one possible explanation was that staff are enrolling people who do not need intensive
civilian workforce is expected to              services. Labor said it plans to develop guidance on prioritizing services, and it also has a
increase. In response to a request, this       six-state pilot to improve monitoring, but neither of these efforts has been completed.
report examines (1) the extent to which
federal veterans’ employment and               In 2008, Labor and the Department of Veterans Affairs (VA) compiled a handbook
training programs vary in services they        intended to guide the roles of their respective staff in coordinating services to disabled
deliver and veterans who receive them;         veterans; however, they have not updated the handbook nor included related Department
                                               of Defense (DOD) employment initiatives in their interagency agreements. GAO’s
(2) the extent to which federal
                                               interviews with VA and Labor officials identified certain challenges with meeting desired
agencies coordinate programs; and (3)
                                               program outcomes resulting, in part, from sections of the handbook that provide
what is known about the performance            insufficient guidance or are subject to misunderstanding. For example, the handbook says
of these programs. To address these            Labor and VA are to coordinate to achieve “suitable employment”—employment that
objectives, GAO reviewed agency                follows the veteran’s rehabilitation plan and does not aggravate the disability. However, it
data, policy documents, and relevant           does not explicitly say how staff should navigate situations where a veteran’s financial
federal laws and regulations, reports,         need or preferences do not align with this goal. In such instances, program staff may work
and studies, and interviewed federal           at cross purposes and veterans may accept jobs that do not count as suitable
and regional officials and state officials     employment. Further, DOD is expanding its employment assistance, but does not have an
in six states selected to achieve              interagency agreement to coordinate with Labor and VA efforts. Absent an updated
geographic and demographic diversity.          handbook and integration of DOD into the coordination framework, there is increased risk
In examining coordination, GAO                 for poor coordination and program overlap.
included in its review employment              While available performance information shows that most programs’ outcomes are below
assistance DOD provides to Guard and           pre-2007 levels, the information Labor reports and the research it has conducted make it
Reserve members.                               difficult to know the extent to which each program is achieving its annual performance
                                               goals. Veterans’ employment outcomes for programs administered by both Labor and VA
What GAO Recommends                            have generally not regained levels seen before the recession that began in 2007, which is
GAO is making four recommendations             similar to employment programs for the general population. In reporting performance,
aimed at improving the guidance                Labor does not relate employment outcomes to individual program goals. In contrast,
provided to staff in the coordination          Labor reports outcomes and goals for its other workforce programs aimed at the general
handbook, integrating DOD into the             population. Moreover, while both agencies have studies completed or under way, neither
                                               has conducted impact evaluations that assess program effectiveness to determine
interagency coordination framework,
                                               whether outcomes are attributable to program participation and not other factors. As a
improving agency reporting on
                                               result, Congress and other key stakeholders lack essential information needed to assess
achievement of program performance             each program’s performance.
goals, and assessing program
effectiveness. Labor, VA, and DOD              Figure 1: Veterans’ Employment and Training Programs
generally agreed with our
recommendations.
View GAO-13-29. For more information,
contact Andrew Sherrill at (202) 512-7215 or
sherrilla@gao.gov.

                                                                                             United States Government Accountability Office
Contents


Letter                                                                                       1
               Background                                                                    4
               Veterans’ Programs Provide Similar Services and Largely Serve
                 Different Populations, but It Is Unclear If the Disabled Veterans’
                 Outreach Program Is Appropriately Serving Its Targeted
                 Population                                                                  8
               Labor and VA Have an Established Framework for Coordination,
                 but It Does Not Include New Employment Initiatives DOD Is
                 Pursuing                                                                  16
               While Most Programs’ Outcomes Are Below Pre-Recession Levels,
                 Labor Has Not Reported the Extent to Which Programs Are
                 Achieving Their Performance Goals                                         24
               Conclusions                                                                 32
               Recommendations for Executive Action                                        33
               Agency Comments                                                             34

Appendix I     Objectives, Scope, and Methodology                                          36



Appendix II    Summary of Veterans’ Program Performance                                    40



Appendix III   Comments from the Department of Labor                                       45



Appendix IV    Comments from the Department of Veterans Affairs                            47



Appendix V     Comments from the Department of Defense                                     51



Appendix VI    GAO Contact and Staff Acknowledgments                                       53



Tables
               Table 1: Federal Employment and Training Programs Targeted to
                        Veterans                                                             5



               Page i                               GAO-13-29 Veterans' Employment and Training
          Table 2: Studies Labor Conducted Related to Veterans’
                   Employment                                                       30
          Table 3: Studies VA Conducted Related to Veterans’ Employment             32


Figures
          Figure 1: Reported Services Provided by Programs Targeting
                   Veterans, Fiscal Year 2012                                         9
          Figure 2: Summary of Priority and Target Groups for Disabled
                   Veterans’ Outreach Program in Selected States in Our
                   Review                                                           12
          Figure 3: The Labor and VA Referral Process for Vocational
                   Rehabilitation Participants                                      19
          Figure 4: Veterans’ Entered Employment Rate, by Program,
                   Program Years 2006-2010                                          40
          Figure 5: Veterans’ Employment Retention Rate, by Program,
                   Program Years 2006-2010                                          40
          Figure 6: Veterans’ Average Earnings, by Program, Program Years
                   2006-2010                                                        41
          Figure 7: Employment Service Participants’ Entered Employment
                   Rate, Employment Retention Rate, and Average Earnings,
                   Program Years 2006-2010                                          42
          Figure 8: WIA Adult Program Participants’ Entered Employment
                   Rate, Employment Retention Rate, and Average Earnings,
                   Program Years 2006-2010                                          43
          Figure 9: VA Vocational Rehabilitation Program Participants
                   Rehabilitated to Employment, Fiscal Years 2006-2011              44




          Page ii                            GAO-13-29 Veterans' Employment and Training
Abbreviations

DOD               Department of Defense
ESGR              Employer Support of the Guard and Reserve
ES                Employment Service
ETA               Employment and Training Administration
Labor             Department of Labor
PTSD              post-traumatic stress disorder
TAP               Transition Assistance Program
USERRA            Uniformed Services Employment and Reemployment
                  Rights Act of 1994
VA                Department of Veterans Affairs
VETS              Veterans’ Employment and Training Service
VWIP              Veterans’ Workforce Investment Program
WIA               Workforce Investment Act of 1998


This is a work of the U.S. government and is not subject to copyright protection in the
United States. The published product may be reproduced and distributed in its entirety
without further permission from GAO. However, because this work may contain
copyrighted images or other material, permission from the copyright holder may be
necessary if you wish to reproduce this material separately.




Page iii                                     GAO-13-29 Veterans' Employment and Training
United States Government Accountability Office
Washington, DC 20548




                                   December 13, 2012

                                   The Honorable Max Baucus
                                   Chairman, Committee on Finance
                                   United States Senate

                                   The Honorable Jon Tester
                                   United States Senate

                                   In fiscal year 2011, the Department of Labor (Labor) and the Department
                                   of Veterans Affairs (VA) spent an estimated $1.2 billion on veterans’
                                   employment and training programs and served about 880,000
                                   participants. In addition, the Department of Defense (DOD) has begun
                                   providing additional employment assistance to Guard and Reserve
                                   members. In 2011, Congress enacted a new law, 1 and the Administration
                                   launched additional initiatives 2 to improve employment opportunities for
                                   veterans. While the overall unemployment rate for veterans is comparable
                                   to that of non-veterans, the unemployment rate for veterans who have
                                   more recently separated from the military is higher than that of civilians
                                   and other veterans. Moreover, more than 1 million service members are
                                   projected to separate from the military and transition to civilian life from
                                   2011 to 2016. In addition, the number of veterans with service-connected
                                   disabilities is on the rise.

                                   Over the last 20 years, we have periodically reported on individual
                                   employment and training programs as well as on specific populations of
                                   veterans who use them. 3 In January 2011, as a part of a larger review of



                                   1
                                       VOW to Hire Heroes Act of 2011, Pub. L. No. 112-56, 125 Stat. 711.
                                   2
                                     In November 2011, the White House announced several new initiatives related to
                                   veterans’ employment, including online tools to help veterans translate their military skills
                                   to the civilian workforce and find job postings from companies.
                                   3
                                    GAO’s most recent reports on veterans’ employment and training programs include
                                   Vocational Rehabilitation and Employment: Better Incentives, Workforce Planning, and
                                   Performance Reporting Could Improve Program, GAO-09-34 (Washington, D.C.: Jan. 26,
                                   2009); Disabled Veterans’ Employment: Additional Planning, Monitoring, and Data
                                   Collection Efforts Would Improve Assistance, GAO-07-1020 (Washington, D.C.: Sept. 12,
                                   2007); and Veterans’ Employment and Training Service: Labor Could Improve Information
                                   on Reemployment Services, Outcomes, and Program Impact, GAO-07-594 (Washington,
                                   D.C.: May 24, 2007).




                                   Page 1                                        GAO-13-29 Veterans' Employment and Training
all federal employment and training programs, we identified six
employment and training programs 4 administered by Labor and VA that
are targeted toward veterans. Labor oversees five of these programs: (1)
the Disabled Veterans’ Outreach Program, (2) the Homeless Veterans’
Reintegration Program, (3) the Local Veterans’ Employment
Representative Program (Employment Representative Program), (4) the
Transition Assistance Program (TAP), and (5) the Veterans’ Workforce
Investment Program (VWIP). VA oversees the sixth program: the
Vocational Rehabilitation & Employment Program (Vocational
Rehabilitation Program). 5 More specifically, our report identified services,
eligibility requirements, and outcome measures that these programs had
in common.

For this report, we focus in more detail on the six programs identified in
our January 2011 report and examine (1) the extent to which federal
veterans’ employment and training programs vary in terms of the services
they deliver and the veterans who receive them; (2) the extent to which
federal agencies coordinate these programs; and (3) what is known about
the performance of these programs.

To address our first objective, we analyzed Labor and VA data on
services provided and veteran participants, as well as agency annual
reports, budget justifications, guidance, and relevant federal laws and
regulations for the six programs listed above. To address our second
objective, we reviewed key agency agreements and guidance, and
interviewed agency officials and associations representing the interests of
veterans. In responding to this objective, we included not only the six
programs indicated above but also three Labor programs available to the



4
 GAO, Multiple Employment and Training Programs: Providing Information on Colocating
Services and Consolidating Administrative Structures Could Promote Efficiencies,
GAO-11-92 (Washington, D.C.: Jan. 13, 2011). We defined an employment and training
program as one specifically designed to enhance the specific job skills of individuals in
order to increase their employability, identify job opportunities, or help job seekers obtain
employment.
5
  In November 2011, Congress enacted the VOW to Hire Heroes Act of 2011, which
established the Veterans Retraining Assistance Program, a new employment and training
program for certain unemployed veterans between the ages of 35 and 60 years old. Pub.
L. No. 112-56, tit. II, 125 Stat. 711, 712. Through that program, veterans receive training
leading to an associate’s degree or similar certificate in a high-demand occupation from a
community college or technical school. VA administers this program. We did not include
this program in our review because it had not been fully implemented.




Page 2                                        GAO-13-29 Veterans' Employment and Training
general population: the Workforce Investment Act (WIA) Adult and
Dislocated Worker programs, and the Employment Service (ES)
Program. 6 We also included two DOD programs which have recently
begun providing employment services: (1) the Yellow Ribbon
Reintegration Program (Yellow Ribbon), which includes the Employment
Initiative Program, and (2) the Employer Support of the Guard and
Reserve (ESGR). 7 As part of answering our first two objectives, we also
conducted case studies in six selected states. For all 50 states, we first
determined whether each was high, medium, or low on the following
characteristics: the percentage of veteran population, amount of program
expenditures, program performance, and veterans’ unemployment rate.
We then selected one state from each of Labor’s six regions to achieve
variation on the above characteristics, as well as diversity in terms of
geography and state size. These states were Florida, Massachusetts,
Ohio, Oregon, Texas, and Virginia. Within these states we interviewed
federal officials assigned to the state and/or region 8 and the Director or
designee of the state workforce agency. State workforce agencies
operate employment programs using federal funds. Finally, within three
states, we interviewed the state government’s Director of Veterans
Affairs. For each state, we also reviewed state plans that specify how the
state will enact federal policies. To address our third objective, we
analyzed agency annual reports, budget justifications, and other agency
documents for the six programs indicated above. We also synthesized
findings from relevant past GAO reports and agency-sponsored program
evaluations. We used data from the Labor Exchange Reporting System,
Veterans’ Employment and Training Service (VETS) Operations and
Programs Activity Report data system, the VA Corporate Case
Management System, and Defense Manpower Data Center. We found
the data from these systems sufficiently reliable for our reporting



6
  Apart from the five Labor employment and training programs that target veterans, these
three programs account for 98 percent of the veterans who participated in other Labor
programs.
7
 These DOD programs were not included in our January 2011 review. For this report, we
decided to include nationally coordinated DOD programs focused on serving Guard and
Reserve members who may qualify as veterans for certain Labor and VA programs.
8
  The federal officials we interviewed who were assigned to a state or region included the
regional administrator and state director of Veterans’ Employment and Training, the
Vocational Rehabilitation Program officer, and the ESGR chairperson in each state.
ESGR chairpersons are federal representatives, but not federal employees, as they are
volunteers.




Page 3                                       GAO-13-29 Veterans' Employment and Training
                          purposes. Additional details regarding our methodology can be found in
                          appendix I.

                          We conducted this performance audit from September 2011 through
                          December 2012 in accordance with generally accepted government
                          auditing standards. Those standards require that we plan and perform the
                          audit to obtain sufficient, appropriate evidence to provide a reasonable
                          basis for our findings and conclusions based on our audit objectives. We
                          believe the information and data obtained, and the analysis conducted,
                          provide a reasonable basis for our findings and conclusions based on our
                          audit objectives.



Background
Labor and VA Employment   Labor and VA oversee six employment and training programs targeted to
and Training Programs     veterans (see table 1). Labor administers its programs through state
Serving Veterans          workforce agencies in each state. Within Labor, VETS administers five
                          employment programs targeted to veterans. 9 VETS provides grants to
                          states to support state workforce agency staff who serve veterans
                          through the Disabled Veterans’ Outreach Program, Employment
                          Representative Program, and TAP. Through the Homeless Veterans’
                          Reintegration Program and VWIP, VETS also provides funding to
                          organizations that serve eligible veterans, including nonprofits. Labor
                          oversees these programs through federal officials stationed in each
                          region, as well as a Director of Veterans’ Employment and Training
                          located in each state. Within VA, the Vocational Rehabilitation Program
                          provides employment services to certain veterans with disabilities. VA
                          offers the program in 56 regional offices and 169 satellite offices. The
                          program has about 1,000 staff, including vocational rehabilitation
                          counselors, employment coordinators, support staff, and managers.
                          Rehabilitation counselors determine entitlement to services.




                          9
                           VETS is also responsible for enforcing parts of the Uniformed Services Employment and
                          Reemployment Rights Act of 1994 (USERRA). Congress enacted USERRA in 1994, in
                          part, to minimize disruptions to the lives of persons performing service in the military as
                          well as to their employers. See 38 U.S.C. ch. 43.




                          Page 4                                       GAO-13-29 Veterans' Employment and Training
Table 1: Federal Employment and Training Programs Targeted to Veterans

Program and administering                                                                                                              Number of participants
agency                        Overview                                                                       2011 funding a               in fiscal year 2011b
Disabled Veterans’ Outreach   Formula grants to states to fund staff positions                                   $85,000,000                          319,274
Program                       in the state workforce agencies. These staff
(Labor)                       provide employment services to eligible
                              veterans. The law requires that to the greatest
                              extent possible Labor hire qualified veterans to
                                                    c
                              fill these positions.
Employment Representative     Formula grants to states to fund staff positions                                   $72,000,000                          292,506
Program                       in the state workforce agencies. These staff
(Labor)                       reach out to employers to find jobs for veterans.
                              The law requires that to the greatest extent
                              possible Labor hire qualified veterans or eligible
                                                               d
                              persons to fill these positions.
Homeless Veterans’            Competitive grants to state and local agencies,                                    $36,000,000                           15,956
Reintegration Program         for-profit/commercial entities, and nonprofit
(Labor)                       organizations to provide employment and
                              supportive services to veterans.
TAP                           Provides workshops to help service members                                           $7,000,000                         141,327
(Labor)                       prepare for civilian employment.
VWIP                          Competitive grants to state and local agencies,                                      $9,000,000                           4,269
(Labor)                       for-profit/commercial entities, and nonprofit
                              organizations to provide employment and
                              supportive services to veterans.
Vocational Rehabilitation     Provides funding for staff located in field offices                              $973,000,000                          107,925e
Program                       and subsistence allowances to veterans and
(VA)                          pays for tuition, books, and supplies for
                              veterans.
                                         Source: GAO analysis of Labor and VA annual budget justifications, performance reports, and fact sheets.
                                         a
                                           The Homeless Veterans’ Reintegration Program and VWIP numbers represent program year 2010,
                                         which began on July 1, 2010, and ended June 30, 2011, rather than fiscal year 2011. The funding for
                                         the other programs listed represent fiscal year 2011.
                                         b
                                           Veterans can be co-enrolled in more than one program and numbers do not represent unduplicated
                                         counts.
                                         c
                                           38 U.S.C. § 4103A(b).
                                         d
                                           38 U.S.C. § 4104(c).
                                         e
                                           Veterans generally have a 12-year period of eligibility from the time they leave the military and
                                         generally up to 48 months of benefits if enrolled in the program.


                                         In addition to its programs administered by VETS, Labor offers
                                         employment and training services to the general population—including
                                         veterans. These services are administered by the Employment and
                                         Training Administration (ETA). First, ETA administers the ES Program,
                                         which provides a national system of public employment services to all
                                         individuals seeking employment. ES provides services such as job
                                         search, labor market information, and job referrals to the public, including
                                         job seekers and employers. ETA carries out its ES Program through state



                                         Page 5                                                           GAO-13-29 Veterans' Employment and Training
workforce agencies. ETA also administers the WIA Adult and Dislocated
Worker programs, which provide a broad range of services including job
search assistance, skill assessment, and training for eligible individuals.
When funds are limited, the WIA Adult Program is to give priority for
intensive and training services to low income adults or those on public
assistance. 10 In program year 2010, 94,629 veterans exited from the WIA
Adult Program. 11 WIA’s Dislocated Worker Program generally targets
adults who have been terminated or laid off from employment and meet
other criteria. In program year 2010, 58,350 veterans exited from the WIA
Dislocated Worker Program.

Federal law requires VETS, ES, and WIA Adult and Dislocated Worker
programs to offer their services through the one-stop system—which
includes centers through which job seekers can access a range of
employment and training programs. 12 Two of VETS’ programs—Disabled
Veterans’ Outreach and Employment Representative programs—have
about 2,100 staff who work primarily in local one-stop centers. Federal
law also requires other Labor-funded programs—including ES and WIA
Adult and Dislocated Worker programs—to give veterans priority over the
general population when they seek services (referred to as priority of
service). 13 VETS and ETA jointly monitor compliance with this
requirement.

Most ETA and VETS programs report the same performance measures,
known as the common measures. They include

•     percentage of program exiters who have obtained employment
      (entered employment rate),




10
   The WIA Adult and Dislocated Worker programs have three levels of service—core,
intensive, and training. Core services include basic services such as job search and labor
market information; intensive services include activities such as comprehensive
assessments and case management; and training includes services such as occupational
or vocational training. ES is intended to only provide core services.
11
     The program year 2010 covers July 2010 to June 2011.
12
  Labor also has national electronic tools for recently separating veterans, such as My
Next Move for Veterans, and Veterans resources on the CareerOneStop Worker
ReEmployment Portal.
13
     38 U.S.C. § 4215.




Page 6                                      GAO-13-29 Veterans' Employment and Training
               •    percentage retaining employment for 6 months after exiting the
                    program (employment retention rate), and
               •    6-month average earnings of program exiters (average earnings). 14
               For each of these, Labor establishes annual performance goals.

               VA reports an employment rehabilitation rate as a measure of
               performance for the Vocational Rehabilitation Program. A “rehabilitated”
               veteran is one who successfully completes a rehabilitation plan and is
               equipped with the required skills and tools needed to obtain and maintain
               suitable employment (i.e., employment that is consistent with the
               veteran’s skills, aptitudes, and interests). 15


DOD Programs   DOD works with Labor and VA to provide transition assistance workshops
               as a part of TAP. 16 In addition, DOD helps Guard and Reserve members
               obtain civilian employment though its operation of several programs,
               including the Yellow Ribbon Program and ESGR. The Yellow Ribbon
               Program serves National Guard and Reserve members and their families
               by hosting events that provide information on employment opportunities,
               health care, education/training opportunities, finances, and legal
               benefits. 17 The ESGR is a nationwide network of volunteers who address




               14
                  These measures are calculated after the exit quarter, or 3 months after the participant
               exits the program. In addition, the 6-month average earnings measure is calculated for
               those participants who retain employment for 6 months after exiting the program.
               15
                  The employment rehabilitation rate tracks the percentage of program participants who
               are rehabilitated through attaining suitable employment as a proportion of program
               participants who left the program either through rehabilitation or because they
               discontinued their participation.
               16
                  The Department of Homeland Security also helps provide transition assistance
               workshops to U.S. Coast Guard service members.
               17
                  Under the Yellow Ribbon Program, DOD is required to conduct post-deployment
               activities and services that provide information on employment opportunities and
               resources and information necessary for successful reintegration of Guard and Reserve
               members. Pub. L. No. 110-181 § 582(g)(5), 122 Stat. 3, 125 (2008), as amended by Pub.
               L. No. 111-383 § 583(d), 124 Stat. 4137, 4228 (2011).




               Page 7                                       GAO-13-29 Veterans' Employment and Training
                           unemployment and underemployment of Guard and Reserve members
                           through participation in employment-related events. 18



Veterans’ Programs
Provide Similar
Services and Largely
Serve Different
Populations, but It Is
Unclear If the
Disabled Veterans’
Outreach Program Is
Appropriately Serving
Its Targeted
Population
Six Federal Programs       As shown in figure 1, the six federal employment and training programs
Provide Similar Types of   targeted to veterans offer similar types of employment and training
                           services. 19 For example, all programs offer employment counseling and
Employment Services but
                           assessment, job search or job placement activities, and job readiness
Largely Serve Different    skills training. Other services available from more than one of these
Populations                programs include the development of job opportunities, job referrals, and
                           occupational and vocational training, among others.




                           18
                              The ESGR was established in 1972 to promote cooperation and understanding
                           between reserve component service members and their civilian employers and to assist in
                           the resolution of conflicts arising from an employee’s military commitment. However,
                           within the last 2 years, the program has expanded its mission to focus on broader
                           employment needs of service members.
                           19
                                GAO-11-92.




                           Page 8                                     GAO-13-29 Veterans' Employment and Training
Figure 1: Reported Services Provided by Programs Targeting Veterans, Fiscal Year 2012




                                        Even though the programs generally offer similar types of services, they
                                        largely serve different populations. The following programs provide
                                        employment and training services solely for a specified group:

                                        •    TAP: transitioning service members and their spouses;
                                        •    Homeless Veterans’ Reintegration Program: homeless veterans; and
                                        •    Vocational Rehabilitation Program: service-connected, disabled
                                             veterans. 20
                                        The remaining programs serve a broader group of veterans. VWIP trains
                                        veterans with significant barriers to employment, among others. 21


                                        20
                                           38 U.S.C. § 3102(a). To receive Vocational Rehabilitation Program services, veterans
                                        generally must have at least a 20 percent disability rating and an employment handicap.
                                        Veterans with a 10 percent disability rating may also be entitled to receive services if they
                                        have a serious employment handicap.
                                        21
                                           29 U.S.C. § 2913. The program also serves veterans with service-connected
                                        disabilities; veterans who served on active duty in the armed forces during a war,
                                        campaign, or expedition for which a campaign badge has been authorized; and recently
                                        separated veterans.




                                        Page 9                                        GAO-13-29 Veterans' Employment and Training
                            However, Labor is seeking to defund VWIP, due to the increasingly high
                            cost per placement into employment for program participants. Of the two
                            remaining programs, the Employment Representative Program and the
                            Disabled Veterans’ Outreach Program can serve all eligible veterans. 22
                            Veterans generally obtain access to the Employment Representative
                            Program by first participating in the Disabled Veterans’ Outreach
                            Program. Consequently, the Disabled Veterans’ Outreach Program has
                            the most potential overlap with the other veterans’ employment and
                            training programs and Labor’s other workforce programs; we focused in
                            detail on this program’s target population and services.


It Is Unclear if Disabled   The law governing the Disabled Veterans’ Outreach Program defines who
Veterans’ Outreach          is eligible for services and who among those eligible should receive
Program Is Appropriately    priority for those services. Under the law, services are to be provided to
                            eligible veterans. However, the law says that among eligible veterans
Serving Its Targeted        priority should be given in the following order (1) special disabled
Population                  veterans, 23 (2) other disabled veterans, 24 and (3) other eligible veterans
                            as determined by the Secretary of Labor. 25 The law requires Labor to give



                            22
                               “Eligible veteran” is defined as a person who meets one of the following criteria: (1)
                            served on active duty for a period of more than 180 days and was discharged or released
                            with other than a dishonorable discharge; (2) was discharged or released from active duty
                            because of a service-connected disability; (3) as a member of a reserve component under
                            an order to active duty under certain circumstances, served on active duty during a period
                            of war or in a campaign or expedition for which a campaign badge is authorized and was
                            discharged or released from such duty with other than a dishonorable discharge; or (4)
                            was discharged or released from active duty by reason of a sole survivorship discharge.
                            See 38 U.S.C. § 4101(4), which incorporates the definition from 38 U.S.C. § 4211(4).
                            23
                               “Special disabled veteran” is defined to mean either of the following: (1) a veteran who
                            is entitled to compensation (or who but for the receipt of military retired pay would be
                            entitled to compensation) under laws administered by the Secretary for a disability (i) rated
                            at 30 percent or more, or (ii) rated at 10 or 20 percent in the case of a veteran who has
                            been determined to have a serious employment handicap; or (2) a person who was
                            discharged or released from active duty because of a service-connected disability. See 38
                            U.S.C. § 4101(1), which incorporates the definition from 38 U.S.C. § 4211(1).
                            24
                               “Disabled veteran” is defined to mean either (1) a veteran who is entitled to
                            compensation (or who but for the receipt of military retired pay would be entitled to
                            compensation) under laws administered by the Secretary, or (2) a person who was
                            discharged or released from active duty because of a service-connected disability. See 38
                            U.S.C. § 4101(3), which incorporates the definition from 38 U.S.C. § 4211(3).
                            25
                              Under the statute, in determining other eligible veterans, the Secretary is to take into
                            account rates of unemployment and other factors. 38 U.S.C. § 4103A(a)(1)(C).




                            Page 10                                       GAO-13-29 Veterans' Employment and Training
maximum emphasis to meeting the employment needs of economically or
educationally disadvantaged veterans in providing these services. 26 The
law also generally requires that program staff provide participants with
intensive services. 27

However, Labor’s 2010 guidance for the Disabled Veterans’ Outreach
Program indicates that Congress questioned how the program differed
from other programs and does not provide states with criteria for
determining which veterans fit two of the program’s priority groups:
economically and educationally disadvantaged veterans. This guidance
was intended to refocus the roles and responsibilities of Disabled
Veterans’ Outreach Program staff as a way to better reflect this program’s
legislative intent. 28 The 2010 guidance said program staff should ensure
that specific segments of the veteran population are served by the state
workforce agency and ensure that veterans are provided intensive
services—such as assessing skills and abilities, conducting in-depth
interviews to identify employment barriers, and developing individual
employment plans. The guidance indicates that Congress had questioned
how the program differed from other Labor programs. Although the
guidance identified disabled veterans among the program’s highest
priority, it also gave states the flexibility to identify other target groups.
Specifically, it said that states should select veteran populations who
require a significant cross-section of program services in order for these
populations to fully and successfully participate in the workforce. It goes
on to list a broad range of groups as examples of potential groups to
target for services without defining them. The list provided includes, for
example, older veterans and veterans residing in rural areas. It does not
identify these groups as subcategories of the disabled population. In
addition, Labor announced a new initiative in November 2011, called the
Gold Card initiative, which added another target population to be served
by the Disabled Veterans’ Outreach Program. This initiative provides
unemployed post-9/11 era veterans with enhanced intensive services,
including up to 6 months of follow-up. Importantly, neither the law, nor
Labor’s 2010 guidance referred to above define “economically


26
     38 U.S.C. § 4103A(a)(2).
27
   The law mentions Disabled Veterans’ Outreach Program staff as both carrying out
intensive services and also facilitating placements.
28
  Department of Labor, Veterans’ Employment and Training Service, Veterans’ Program
Letter, No. 07-10 (Washington, D.C.: June 29, 2010).




Page 11                                    GAO-13-29 Veterans' Employment and Training
disadvantaged” or “educationally disadvantaged.” As a result, states have
no standard method for identifying veterans based on these criteria.

Labor requires states to identify veteran populations to target for
specialized services in their state plans for veteran services. As shown in
figure 2, state plans for all of the six states in our review identified
veterans with service-connected disabilities and disabled veterans. Four
of the six states explicitly identified economically and educationally
disadvantaged veterans. Beyond that, states identified a broad range of
groups as target populations whose need for services would have to be
balanced against those of the priority groups identified by the law.

Figure 2: Summary of Priority and Target Groups for Disabled Veterans’ Outreach
Program in Selected States in Our Review




a
 Federal law states that, among eligible veterans, priority should be given in the following order: (1)
special disabled veterans, (2) other disabled veterans, and (3) other eligible veterans as determined
by the Secretary of Labor. The law also requires maximum emphasis be placed on meeting the
employment needs of economically or educationally disadvantaged veterans in providing these
services.
b
 This includes veterans who are enrolled in or completed the VA Vocational Rehabilitation Program,
returning wounded or injured service members, and REALifelines participants. REALifelines is a
Labor initiative that provides employment services for transitioning wounded and injured service
members and their families.




Page 12                                             GAO-13-29 Veterans' Employment and Training
A variety of state-level officials identified two issues affecting the
consistency of targeting for the Disabled Veterans’ Outreach Program.
The first is whether determinations of which veterans should receive
priority are consistent. Staff have flexibility in how to make this
determination. More importantly, local one-stop staff make the final
decision about who is served. Officials in half the states we reviewed said
this process can be very subjective. Secondly, all veterans, regardless of
priority, may want to use the program and program staff may want to
serve veterans regardless of priority. One state-level Labor official
acknowledged that veterans want to work with Disabled Veterans’
Outreach staff once they hear about services provided and acknowledged
that there may also be local one-stop centers that route veterans directly
to Disabled Veterans’ Outreach Program staff instead of assessing a
veteran’s need for intensive services. Another state-level official told us
his state decided to create a special position to manage Disabled
Veterans’ Outreach Program staff, in part, because staff wanted to help
all veterans, although they were supposed to see only veterans with
“barriers” to employment.

Further, Labor has limited data on who the program actually serves.
Labor collects data on program participants who are disabled or special
disabled. 29 In program year 2010, about 25 percent of participants in the
Disabled Veterans’ Outreach Program were disabled or special disabled.
However, Labor does not collect data on the other populations states
target, such as Guard and Reserve members or wounded service
members. Labor officials said the agency does not collect this information
because there are too many targeted groups and such reporting would
burden states. In addition, until June 2012, Labor had no state-level
method to collect data on veterans receiving services under the Gold
Card initiative.

Labor has initiated a pilot to improve how it monitors who the Disabled
Veterans’ Outreach Program serves. As part of its monitoring process,
Labor encourages federal VETS directors in each state to ensure that 20
percent of local areas within their state complete a self-assessment




29
  Labor also has data on VETS participants when they are co-enrolled in the WIA Adult
and Dislocated Worker programs.




Page 13                                    GAO-13-29 Veterans' Employment and Training
annually. 30 From this 20 percent, the directors randomly select a smaller
group of local areas for site visits. These random samples are not
generalizable, according to a Labor official. 31 Labor conducted a pilot of
new auditing protocols to improve monitoring of how Disabled Veterans’
Outreach Program staff conduct their duties, part of which involves
deciding who is served by the program. According to Labor officials,
Labor plans to review feedback from the six participating states and make
necessary changes before moving to full implementation of the new
auditing protocols, which is expected in early 2013.

National data indicate that about a third of veterans enrolled in the
Disabled Veterans’ Outreach Program receive intensive services. In
program year 2011, 32 the Disabled Veterans’ Outreach Program reported
that staff provided intensive services to 28 percent of program
participants nationally. (The precise percentage of veterans receiving
intensive services could be higher given measurement issues, which are
discussed in more detail below.) Further, according to Labor, Disabled
Veterans’ Outreach Program staff expended the majority of their effort
providing standard services to veterans that are available to all customers
through the ES Program.

Who the program is serving and staff’s other activities may, in part,
explain the relatively low percentage of Disabled Veterans’ Outreach
Program participants who receive intensive services. Federal and state
Labor officials and National Veterans’ Training Institute officials said the
percentage may be low because the program is serving some veterans
who do not need intensive services. Federal Labor officials also said if
Disabled Veterans’ Outreach staff are serving veterans regardless of
priority, then staff may have less time to provide intensive services to
those who most need them. Further, federal and state-level Labor officials
noted that Disabled Veterans’ Outreach staff are instructors for TAP




30
   These self-assessments, in part, reflect how well the current operations reflect the state
plan and the roles and responsibilities of Disabled Veterans’ Outreach staff. State-level
directors of VETS Programs are supposed to validate these self-assessments using
management reports that contain critical data on characteristics that make veterans
eligible and the services veterans receive.
31
     According to the official, local areas vary in how they deliver services and allocate staff.
32
     Program year 2011 reflects the reporting period July 2011 through June 2012.




Page 14                                          GAO-13-29 Veterans' Employment and Training
sessions and said this responsibility could take away some of their time
from providing intensive services. 33

Some federal and state-level Labor officials also said potential
underreporting could help explain the low percentage of participants
reported as receiving intensive services. In 2010, VETS stated that it was
adopting a definition of intensive services consistent with WIA. 34 Some
federal and state-level Labor officials as well as National Veterans’
Training Institute officials said state reporting systems were not
configured to capture certain activities that were part of the new
definition. 35 As mentioned previously, until June 2012, Labor had no
state-level method to collect data on veterans eligible to receive intensive
services under the Gold Card initiative.

Until recently, Labor had not set national targets for the percentage of
Disabled Veterans’ Outreach Program participants who should receive
intensive services. For fiscal year 2012 and fiscal year 2013, Labor has
set 32 percent and 35 percent as respective targets. These targets are
not based on the number of groups targeted or the nature of potential
barriers to employment participants might face. 36 Labor officials said they
are developing new guidance to clarify staff roles in providing intensive
services, but at the time of this review, the guidance was not completed.
Labor officials said they plan to implement the new guidance in early
2013.




33
 As of March 2013, contractors will conduct TAP sessions instead of Disabled Veterans’
Outreach staff.
34
   VETS guidance stated that the intensive services category was clarified to include the
following: comprehensive assessment of education, skills, and abilities; in-depth
interviewing and evaluation to identify employment barriers and appropriate employment
goals; group and individual career coaching; short-term, pre-vocational services that may
include development of learning and communication skills, interviewing skills, personal
maintenance skills, and professional conduct to prepare individuals for career goals; and
development of an individual employment plan that identifies employment goals, interim
objectives, and appropriate services that will enable the veteran to meet his or her
employment goals. Veterans’ Program Letter, No. 07-10.
35
  The Disabled Veterans’ Outreach Program uses the data system for the ES Program,
which differs from the data system used by WIA programs.
36
  According to Labor, the fiscal year 2012 target was established by increasing the
previous fiscal year outcome for intensive services by 22 percent.




Page 15                                     GAO-13-29 Veterans' Employment and Training
Labor and VA Have an
Established
Framework for
Coordination, but It
Does Not Include
New Employment
Initiatives DOD Is
Pursuing
Labor and VA Have an       Labor and VA have established a framework to coordinate their
Established Coordination   employment and training programs. In 2005, Labor and VA signed an
Framework and Are          interagency memorandum of agreement that outlines how the agencies
                           plan to coordinate the Vocational Rehabilitation and Disabled Veterans’
Taking Steps to            Outreach and Employment Representative programs to serve disabled
Periodically Review It     veterans, respectively. The agencies have also collaboratively created an
                           interagency handbook 37 that delineates roles and responsibilities and
                           establishes a referral process between the Disabled Veterans’ Outreach
                           and the Vocational Rehabilitation programs. To assist field staff, the
                           interagency handbook also provides standard language and guidance for
                           agreements between local Labor and VA offices. As a result, local offices
                           from both agencies can tailor the standard agreement language to meet
                           local situations. The handbook has not been updated since 2008.

                           Labor and VA have provided staff with training on the handbook and
                           formed a group to monitor coordination. Labor and VA conducted a
                           national training webinar based on the interagency handbook for both
                           agencies’ staff after it was published, have made virtual trainings
                           available since 2009, and provided technical assistance to staff. To
                           monitor the coordination activities outlined in the interagency handbook,
                           Labor and VA created a Joint Work Group. According to Labor and VA
                           officials, this group recently discussed and agreed on a plan to review




                           37
                              U.S. Department of Labor Veterans’ Employment and Training Service and U.S.
                           Department of Veterans Affairs Vocational Rehabilitation and Employment Service
                           Technical Assistance Guide (2008).




                           Page 16                                   GAO-13-29 Veterans' Employment and Training
                           one-third of local agreements made between Labor and VA field locations
                           annually. 38

                           Labor and VA have collected information that could be useful in updating
                           the handbook. The Joint Work Group recently conducted its first in-depth
                           review of states’ implementation of the handbook since it was established
                           in 2008. According to Labor and VA officials, the Joint Work Group
                           electronically surveyed the VA employment coordinators in all 56 VA
                           Regional Offices and the 52 state-level directors of the Disabled
                           Veterans’ Outreach and Employment Representative programs. 39 While
                           VA officials stated that they are currently reviewing the survey results to
                           determine if the handbook needs to be updated, Labor officials told us
                           told us they believe the handbook needs to be updated. We have
                           reported previously that agencies need to create means to monitor
                           agreements and related guidance periodically to keep them current and to
                           identify areas for improvement. 40


Some Coordination Issues   Our interviews with Labor and VA officials identified certain challenges
Have Arisen across         with meeting desired program outcomes resulting, in part, from sections
Veterans’ Employment and   of the handbook that are subject to misunderstanding or provide
                           insufficient guidance. They pertain to incorporating labor market
Training Programs          information into rehabilitation plans and finding “suitable employment” for
                           participants.

                           The first challenge with referrals as outlined in the handbook involved
                           ensuring that participants’ rehabilitation plans prepared them for jobs that
                           existed in their local area. According to the referral process outlined in the


                           38
                              In 2011, the work group reviewed existing agreements between local areas to
                           determine the extent to which agreements were in compliance with the interagency
                           handbook and found that a considerable number of the agreements lacked the information
                           required by the handbook or did not follow the format the handbook specified. Officials
                           noted that these deficiencies may be due to the fact that many agreements were written
                           before the interagency handbook was developed in 2008 and varied widely up to the
                           recent review. However, according to VA officials as of September 2012, all the local
                           agreements have been reviewed for compliance with the interagency handbook.
                           39
                             VA reported that Employment Coordinators in all 56 regional offices responded to this
                           survey. Labor received 51 of the 52 surveys administered, with one survey not completed
                           due to a vacancy in the position.
                           40
                             GAO, Results-Oriented Government Practices That Can Help Enhance and Sustain
                           Collaboration among Federal Agencies, GAO-06-15 (Washington, D.C.: Oct. 21, 2005).




                           Page 17                                    GAO-13-29 Veterans' Employment and Training
interagency handbook and by agency officials (see fig. 3), there are two
main referral points from the Vocational Rehabilitation Program to Labor’s
Disabled Veterans’ Outreach Program staff: (1) before the participant’s
rehabilitation plan is completed and (2) after the participant has
completed a rehabilitation plan and been deemed job-ready, or ready for
employment, by VA staff. Disabled Veterans’ Outreach Program staff may
provide participants with labor market information or other employment
assistance at the first referral point and are required to at the second
referral point. While VA officials in four of the six states we reviewed
reported that they connected participants with Labor staff to receive labor
market information and other employment consulting, only three of these
states reported that they did this early in the process before the
rehabilitation plan is completed. In two other states, VA officials reported
they understood that they were supposed to refer participants to Labor
only after they had completed rehabilitation plans and were job-ready,
essentially skipping the first step where labor market information may
have been useful. VA officials reported that labor market information may
be provided to participants through small group presentations with
Disabled Veterans’ Outreach staff. For their part, state-level Labor
officials noted that job placement was more challenging for Disabled
Veterans’ Outreach Program staff when participants’ rehabilitation plans
were developed without labor market information. In such cases,
according to Labor officials, Disabled Veterans’ Outreach Program staff
were sometimes working with plans focused on training in occupations
not available in the local labor market—in effect using programs’ funds to
prepare participants for jobs that do not exist in their local area. According
to Labor officials, this made it more difficult for participants to have
successful employment outcomes.




Page 18                               GAO-13-29 Veterans' Employment and Training
Figure 3: The Labor and VA Referral Process for Vocational Rehabilitation
Participants




Note: For the purposes of this graphic, VA refers to the Vocational Rehabilitation Program. Also, in
some locations where a Disabled Veterans’ Outreach staff may not be assigned, veterans could
receive these services from other appropriate one-stop staff. Also, the term “job-ready” refers to
veterans who are determined to be ready, willing, and able to participate in job development activities,
but who may still have issues or barriers to employment such as limited transportation or child care
needs.




Page 19                                            GAO-13-29 Veterans' Employment and Training
The second challenge with referrals as outlined in the handbook involved
ensuring that job-ready participants are directed to “suitable employment.”
When veterans are referred to the Disabled Veterans’ Outreach Program
at the job-ready stage, Disabled Veterans’ Outreach Program and VA
staff are supposed to coordinate to find “suitable employment,” or
employment that will not aggravate the participant’s disability and follows
the participant’s rehabilitation plan. State-level Labor officials noted that,
in some cases, veterans may choose to accept jobs they want or need
but that do not fit in their employment or rehabilitation plan. Such jobs do
not count as “suitable employment” for VA because the job may, in the
long run, aggravate the veteran’s disability. 41 While the handbook says
agencies are to coordinate to achieve “suitable employment,” it does not
explicitly say how Disabled Veterans’ Outreach Program and VA staff
should deal with situations where a veteran’s financial need or
preferences do not align with the goal of suitable employment. 42 Absent
guidance about how to navigate such situations, program staff may be
working at cross purposes and program participants may be taking
employment they cannot retain in the long run. This employment, in turn,
may make a veteran’s disability worse and may make finding future
employment more difficult. One official stressed that having labor market
information incorporated into rehabilitation plans early may help veterans
avoid taking a job that does not match their plans.




41
   According to the handbook, if the Vocational Rehabilitation staff determine that the job
is not suitable, the case is left open until suitable employment is found. If the veteran
declines further services from the Vocational Rehabilitation Program, Disabled Veterans’
Outreach Program staff continue to work with the veteran as needed.
42
   The handbook created a Labor representative role, co-located at VA offices, to help
facilitate communication between Labor and VA. This position was filled in five of the six
states in our review, according to state-level officials. We did not assess the direct role
that this position played in resolving coordination issues related to suitable employment.




Page 20                                       GAO-13-29 Veterans' Employment and Training
DOD Is Expanding Its        DOD is expanding its employment assistance to National Guard and
Employment Initiatives,     Reserve members, but does not have employment service agreements
                            with Labor or VA beyond an agreement for TAP. 43 In fiscal year 2011,
Which Are Not Included in
                            DOD launched an employment assistance initiative under the Yellow
the Labor-VA Coordination   Ribbon program, known as the Employment Initiative Program, that
Framework                   provides job workshops and job fairs to connect Guard and Reserve
                            members to employers. The Employment Initiative Program under the
                            Yellow Ribbon Program has hired and placed 56 “Employment Transition
                            Coordinators” covering all 50 states, territories, and the District of
                            Columbia who provide service members employment assistance among
                            other services. The Yellow Ribbon Program has also held 27 job fairs
                            since the beginning of fiscal year 2012. To support the job fairs, the
                            Yellow Ribbon Employment Initiative Program leverages the network of
                            4,900 volunteers who are affiliated with ESGR. These volunteers also
                            provide resume-building workshops, mock interviews, and career
                            counseling. DOD also recently testified that it is leading, per a White
                            House directive, a new Credentialing and Licensing Task Force to
                            address gaps between military occupational specialties and civilian
                            licensing requirements.

                            DOD reported that this additional employment assistance is needed to
                            support Guard and Reserve members who may not meet veteran status
                            requirements necessary for participating in Labor or VA programs. Most




                            43
                              For the purposes of this report, the term “employment assistance” does not include
                            activities related to USERRA. See 38 U.S.C. ch. 43. Labor and DOD do have an
                            agreement regarding coordination of USERRA responsibilities. As of September 2012,
                            DOD, Labor, VA, and other TAP partner agencies are also collaborating on a national
                            agreement for the newly redesigned TAP.




                            Page 21                                    GAO-13-29 Veterans' Employment and Training
of the ESGR representatives we spoke with also anticipated the program
would continue to provide employment-related services. 44

Although DOD has established these employment assistance services,
no agreement or formal mechanism has been established for coordinating
them with Labor’s and VA’s veterans’ employment efforts. 45 Specifically,
there is no interagency agreement for coordinating employment services
beyond DOD’s and Labor’s work on the Uniformed Services Employment
and Reemployment Rights Act of 1994 (USERRA) and TAP. 46 Although
DOD and VA have an agreement, it focuses on connecting service
members who are leaving the military for civilian life with vocational
rehabilitation services. Further, ESGR has no formal mechanism for
identifying and referring eligible veterans to the Disabled Veterans’
Outreach and Employment Representative programs. We have previously
reported that agencies with common goals and programs can enhance
and sustain collaboration by creating mutually agreed-upon strategies to
help align agencies’ activities and leverage resources to meet their
common goals. 47




44
   DOD reports that while the majority of Guard and Reserve members do not meet
Labor’s requirement of at least 180 days of active duty service necessary to be eligible for
Labor’s VETS Program, approximately 22 percent (284,916 out of 1,311,443) have met
this Labor requirement as of May 2012. For the VA Vocational Rehabilitation Program,
veterans must have a discharge other than dishonorable and a service-connected
disability rating of at least 10 percent to be eligible for the program. DOD reports that in
fiscal year 2011 there were 3,358 Navy Reservists; 7,749 Marine Corps Reservists; 565
Air Force Guard members; 3,021 Air Force Reservists; and 190 Coast Guard Reservists
that met these discharge and disability requirements after they had transitioned out of the
military. DOD reported that the Army Guard and Army Reserve did not have data available
to identify the number of “dishonorable” discharges in fiscal year 2011; therefore, we could
not determine how many individuals from these groups may be eligible for the Vocational
Rehabilitation Program.
45
   According to DOD officials, there is a White House-led taskforce of various federal
agencies who have drafted a Veterans Employment Initiative Taskforce Implementation
Plan, which is in the process of being reviewed and edited, and will be submitted through
a formal clearance process to the White House. However, DOD told us the report is not
available for public release.
46
   As noted above, DOD, Labor, VA, and other TAP partner agencies are also updating
the national TAP agreement, which may include an additional information-sharing
agreement.
47
     GAO-06-15.




Page 22                                      GAO-13-29 Veterans' Employment and Training
Currently, ESGR in the states we reviewed reported informal
coordination—such as meetings and co-participation in job fairs—with
Labor-funded programs. For example, a DOD official noted that the
Washington ESGR used a grant to hire 13 employment transition
counselors in areas that needed service not provided by the state
workforce agencies. According to this official, this ultimately increased
impact while saving funds.

However, this informal coordination may be affecting Labor resources and
confusing employers. According to Labor officials, Disabled Veterans’
Outreach Program staff participation at DOD job fairs reduces the amount
of time available for their primary duties, such as providing intensive
services to program participants. A variety of officials from the states we
reviewed also said that some employers were confused regarding which
agency was leading the initiatives to employ veterans.




Page 23                               GAO-13-29 Veterans' Employment and Training
While Most Programs’
Outcomes Are Below
Pre-Recession Levels,
Labor Has Not
Reported the Extent
to Which Programs
Are Achieving Their
Performance Goals
Most Programs’ Outcomes    Employment outcomes for veterans’ programs have generally not
Are Below Pre-Recession    regained levels attained prior to the recent recession. 48 (See appendix II
Levels, and Outcomes for   for performance outcomes for each veterans’ program as well as veteran
                           participants served by WIA Adult and ES programs over a 5-year period.)
Veterans Are Lower Than    From program years 2007 to 2009—which spanned July 2007 to June
They Are for the General   2010—most Labor veterans’ programs that have outcome measures 49
Population                 saw a decline in their entered employment rate and a slight decline in
                           their 6-month job retention rates. In program year 2010, 50 all programs
                           except VWIP had lower entered employment and employment retention
                           rates than in program year 2006, prior to the recession. The number of


                           48
                             We began our analysis with program year 2006 to obtain information on program
                           performance before the economic recession but after Labor had implemented its current
                           system of common measures, which began in 2005. According to the Business Cycle
                           Dating Committee of the National Bureau of Economic Research, the recession began in
                           December 2007 and ended in June 2009.
                           49
                              Labor collects outcome measures for the Disabled Veterans’ Outreach, Employment
                           Representative, Homeless Veterans’ Reintegration, and VWIP programs. Labor does not
                           track performance of TAP for separating service members using its common measures; it
                           only tracks the number of participants and TAP workshops delivered each fiscal year. As a
                           result of the VOW to Hire Heroes Act of 2011, Labor plans to track TAP participants
                           through its labor exchange reporting system. As part of the process, veterans must
                           identify themselves as having attended TAP. Labor will then be able to report outcomes
                           on the common measures for veterans who attended TAP. Labor officials stated that
                           ideally veterans would be tracked from the point of TAP attendance onward, but there are
                           challenges due to the time lag between TAP attendance and when a veteran may begin
                           searching for employment, as well as the lack of information about where TAP attendees
                           settle and seek employment.
                           50
                             Program year 2010 covers July 2010 to June 2011. These are the most current data
                           available.




                           Page 24                                     GAO-13-29 Veterans' Employment and Training
VA Vocational Rehabilitation Program participants who were rehabilitated
to employment has also declined from 9,225 participants in fiscal year
2006, to 7,975 participants in fiscal year 2011.

Officials at both VA and Labor attributed the declines to various causes.
For example, VA officials attribute some of the decline in the number of
participants rehabilitated to the establishment of the Post-9/11 GI Bill
Program. 51 The Post-9/11 GI Bill Program is an education benefit
administered by VA for individuals who served on active duty after
September 10, 2001. According to VA officials, VA’s Vocational
Rehabilitation Program lost some participants who had begun
rehabilitation efforts but switched to the Post-9/11 GI Bill Program. 52 They
switched, according to VA officials, because the GI Bill Program provided
a more generous living stipend than the Vocational Rehabilitation
Program. At the same time, Labor officials identified national economic
conditions as the primary reason for the drop in performance of its
programs. 53

In addition to the decline in outcomes for veterans’ programs, veterans
participating in broader workforce programs also achieved somewhat
lesser outcomes than those in the general population. (See app. II, figs. 7
and 8.) From program years 2007 to 2009, the WIA Adult and ES



51
     See Pub. L. No. 110-252, tit. V, 122 Stat. 2323, 2357.
52
   Veterans leaving the Vocational Rehabilitation Program reduced the universe of
program participants who could be rehabilitated to employment.
53
   Other factors might also explain the drop in performance. We have previously reported
that Iraq and Afghanistan veterans have a relatively higher incidence of adjustment
disorders, including post-traumatic stress disorder (PTSD), than veterans of other wars.
See GAO, VA Mental Health: Number of Veterans Receiving Care, Barriers Faced, and
Efforts to Increase Access, GAO-12-12 (Washington, D.C.: Oct. 14, 2011). Veterans’
future job prospects may be negatively affected if they develop PTSD. In fact, research
has shown that PTSD may affect service members’ return to employment, as well as their
future job prospects. For example, a Labor study of five states showed that veterans
suffering from PTSD or head trauma may need additional time before they are ready to re-
enter the workforce. The study also identified other veteran characteristics that could
affect employment outcomes, such as drug or alcohol addiction or mental illness, as well
as changes in careers requiring training, licensure, or certification. Other changes in
veteran characteristics—such as the education level of program participants—could also
affect their employment outcomes. See SRA International, An Assessment of the
Influence of the Jobs for Veterans Act and the Workforce Investment Act on the
Employment Outcomes of Veterans, prepared at the request of the Department of Labor,
July 2007.




Page 25                                        GAO-13-29 Veterans' Employment and Training
                            programs saw declines in measures for the percentage of participants
                            who entered employment and the percentage who retained their
                            employment for 6 months. These measures have generally rebounded
                            slightly in 2010, although they have generally not regained levels attained
                            prior to the recent recession. Since 2006, veterans have had slightly
                            lower entered employment outcomes than those for all participants using
                            the ES Program. 54 In the WIA Adult Program, veterans’ employment and
                            retention outcomes have been slightly lower than outcomes for all
                            participants since 2009. Further, between 2006 and 2010, employment
                            and retention outcomes were similar but slightly lower for veterans who
                            worked with Disabled Veterans’ Outreach and Employment
                            Representative program staff, in comparison with outcomes for veterans
                            in the WIA Adult Program.

                            According to Labor officials, some of these differences in outcomes may
                            be explained by differences in characteristics of the populations served.
                            They noted that veteran participants in the WIA Adult Program are more
                            likely to be over the age of 55 than nonveteran participants, and
                            historically, older workers have achieved lower outcomes in both the WIA
                            Adult and ES programs. In addition, Labor officials stated that because
                            the Disabled Veterans’ Outreach and Employment Representative
                            programs serve veterans who face barriers to employment, their
                            outcomes are likely to be lower than outcomes for veterans who are job-
                            ready and nonveterans served by the WIA Adult or ES programs.


Labor Has Not               While Labor reports some data on veterans’ program outcomes, it does
Consistently Reported the   not report the extent to which each of these programs is achieving its
Extent to Which Veterans’   established performance goals. Labor provides Congress an annual
                            veterans’ program report that provides certain performance information,
Programs Are Meeting        such as the number of disabled and recently separated veterans who
Performance Goals           received intensive services. For this annual report, however, Labor is not
                            required to report program outcomes in relation to performance goals.
                            Labor sets annual performance goals for its veterans’ programs, but it is
                            not reporting the results relative to those goals. In previous fiscal years,
                            Labor included some of this information for the Disabled Veterans’


                            54
                               Labor informally compares Disabled Veterans’ Outreach and Employment
                            Representative Program outcomes to those of the ES Program, rather than to the WIA
                            Adult Program. Officials told us that this is an appropriate comparison because unlike the
                            others, the WIA Adult Program provides funding for obtaining credentials.




                            Page 26                                      GAO-13-29 Veterans' Employment and Training
                           Outreach Program, Employment Representative Program, Homeless
                           Veterans’ Reintegration Program, and VWIP in its agencywide
                           performance report. However, in fiscal year 2011, it only reported
                           aggregate goals for three programs, rather than the separate outcomes
                           and goals it maintains for each of these veterans’ programs. 55 In contrast,
                           Labor’s website on general employment programs—WIA Adult and ES—
                           includes both performance goals and outcomes. This information includes
                           a national average for each measure comparing goals against
                           performance, as well as each state’s negotiated goals and performance
                           against those goals. 56 Further, VA reports both an employment outcome
                           and associated goal for the Vocational Rehabilitation Program. 57 We have
                           previously reported that relevant performance information should be
                           reported both internally and externally in order to maintain accountability
                           and transparency for achieving results. 58 Without information on how the
                           outcomes for each veterans’ program compare against their annual
                           performance goals, Congress and other key stakeholders lack essential
                           information needed to assess the performance of the program.


Labor and VA Are Working   Labor is working to implement new performance measures which have
to Collect Data for New    been mandated by the VOW to Hire Heroes Act of 2011 (VOW Act). 59
Performance Measures       Specifically, the act requires Labor to measure participants’ median
                           earnings 90 and 180 days after a participant stops using a veterans’
                           program. Prior to the VOW Act, Labor only measured participants’


                           55
                             Labor officials told us that ES, Disabled Veterans’ Outreach, and Employment
                           Representative programs’ performance targets may change during the program year.
                           They stated that since the targets are projections and are subject to economic conditions,
                           they may be adjusted as performance outcomes are analyzed each quarter. However,
                           Labor’s reports have not always indicated the targets were changed partway through the
                           program year.
                           56
                              Labor sets national goals for veterans programs, but each state negotiates its own
                           performance goals with the agency. For example, Labor negotiates with each state to
                           establish a goal for the percentage of veterans obtaining employment after assistance
                           from veterans program staff (the weighted entered employment rate).
                           57
                              In fiscal year 2010, VA began reporting the employment rehabilitation rate for the
                           Vocational Rehabilitation Program, and in fiscal year 2011 the agency also reported a goal
                           for this measure.
                           58
                              GAO, Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1
                           (Washington, D.C.: November 1999).
                           59
                                Pub. L. No. 112-56, § 238, 125 Stat. 711, 726.




                           Page 27                                       GAO-13-29 Veterans' Employment and Training
                           average earnings over 6 months after participants stop using a veterans’
                           program, for those who retained employment. 60 The VOW Act also
                           requires Labor to track the percentage of participants obtaining a
                           certificate, degree, diploma, licensure, or industry-recognized credential
                           after participating in its veterans’ programs.

                           VA also plans to collect additional information about its programs’
                           outcomes. VA officials said that they decided to track the number, in
                           addition to the rate, of veterans rehabilitated to employment, because the
                           employment rehabilitation rate can fluctuate based on a number of
                           factors. 61 VA has set a national goal to rehabilitate 9,000 veterans to
                           employment in fiscal year 2012. 62 VA officials said that this is the first
                           fiscal year this goal has been used. Consequently, VA has not yet
                           reported its performance against this goal.

                           In addition, the Vocational Rehabilitation Program has established a
                           working group to develop new national performance measures. According
                           to VA officials, the new measures will focus on the middle of the
                           rehabilitation process, because a veteran can be in the program from 1 to
                           6 years, with an average of 4 years. The measures that already exist
                           focus on the front-end (e.g., timeliness of services) and back-end (e.g.,
                           outcomes). Although the new measures have not been finalized, VA
                           plans to implement them in fiscal year 2014, contingent on resources to
                           make changes to the program’s database structure to capture data and
                           report on new measures.


Agencies Have Not          Labor has not conducted impact evaluations that would allow it to assess
Conducted Research That    veterans’ employment programs’ effectiveness, but has conducted
Links Veterans’ Outcomes   research on program outcomes. An impact evaluation attempts to assess
                           a program’s effectiveness by isolating the effect of the program from other
to the Programs That       factors. While many researchers consider impact evaluations to be the
Serve Them                 best method for determining the extent to which a program is causing


                           60
                             As a statistic, the median is less sensitive than the mean to individual earnings that are
                           extreme in relation to most others.
                           61
                              For example, the rehabilitation rate can be negatively affected by veterans who choose
                           to stop participating before completing a rehabilitation plan.
                           62
                             Each VA regional office is responsible for achieving a portion of this target, and each
                           vocational rehabilitation counselor has a target as well.




                           Page 28                                       GAO-13-29 Veterans' Employment and Training
participant outcomes, these studies can be difficult and potentially
expensive to conduct. Impact evaluations can be designed in several
ways, but fall into two basic design categories: experimental, using
random assignment, 63 and quasiexperimental. Quasiexperimental
designs use a comparison group that is not created with random
assignment. While Labor has not conducted impact evaluations, it has
conducted research that examines veterans’ outcomes in relation to their
characteristics and has other studies planned or under way (see table 2).
These studies, though, have limitations. For example, Labor’s 2007 study
of veterans’ outcomes covered five states, and its findings cannot be
generalized to all states. In addition, in the study conducted on the
Homeless Veterans’ Reintegration Program, researchers lacked access
to participant-level data and consequently could not determine whether
certain veterans’ characteristics were associated with positive or negative
employment outcomes for the program as a whole. 64 Labor is funding an
evaluation of the pilot of the redesigned TAP, but has not conducted any
studies or evaluations of VWIP in the last 10 years. 65




63
   There is ongoing debate about the role of randomized experimental methods in public
policy evaluation. For example, while some researchers argue that randomized
experiments are the most appropriate to use as a basis for making major resource
allocation decisions, others believe that this method is not always appropriate, in part
because of frequent policy and leadership changes to programs that can affect the nature
of the experiment.
64
   The evaluators recommended that Labor consider conducting a comprehensive study
of participant characteristics and outcomes, using a combination of data maintained on
participants and a direct survey of Homeless Veterans’ Reintegration Program current and
past participants.
65
   In its fiscal year 2013 budget justification, Labor stated that it seeks to defund VWIP,
due to the small size of the program and the increasingly high cost per placement into
employment for program participants. For program year 2010, Labor reported that VWIP
had a cost per placement of $4,700. For comparison, Labor cited a cost per placement of
$2,848 for the Homeless Veterans’ Reintegration Program.




Page 29                                      GAO-13-29 Veterans' Employment and Training
Table 2: Studies Labor Conducted Related to Veterans’ Employment

Name               Programs reviewed      Purpose                     Methodology                    Status       Key findings
An Assessment      Disabled Veterans’     To associate the            Site visits and interviews     Completed    Characteristics, such as
of the Influence   Outreach Program       characteristics of          with Labor and state           July 2007    drug and alcohol addiction
of the Jobs for    Employment             veterans with either        workforce agency staff in                   and mental illness
Veterans Act       Representative         positive or negative        five states                                 contribute to unsuccessful
and the            Program                employment                                                              employment outcomes.
Workforce                                 outcomes                                                                Unsuccessful veteran job
Investment Act     ES Program
                                                                                                                  seekers could benefit from
on the             WIA Adult Program                                                                              additional supportive
Employment                                                                                                        services such as
Outcomes of                                                                                                       transportation and housing
Veterans
(Phase I)


An Assessment      Disabled Veterans’     To assess veterans’ Survey of veterans with Canceled                    n/a
of Veteran         Outreach Program       employment          unsuccessful employment
Employment         Employment             outcomes            outcomes
Outcomes           Representative
(Phase II)         Program
                   ES Program
                   WIA Adult Program
Homeless           Homeless Veterans’     To assess how well          Interview with Homeless        Completed    Most common
Veterans’          Reintegration          program is                  Veterans’ Reintegration        December     characteristics of program
Reintegration      Program                achieving its mission       Program grantee staff;         2009         participants with positive
Program                                   and meeting goals,          analysis of grantee data                    employment outcomes
Effectiveness                             and to provide data                                                     include motivation; having a
Study                                     to inform future                                                        history of success such as
                                          programmatic                                                            relevant skills or
                                          decisions                                                               experience; and having a
                                                                                                                  support system.
Veterans           Disabled Veterans’     To examine (1) how          Interviews with state          Starting     n/a
Supplemental       Outreach Program       the workforce               workforce agency staff;        November
Study              Employment             system provides             analysis of workforce          2012;
                   Representative         services to veterans        program data on veteran        estimated
                   Program                and (2)                     characteristics and            completion
                                          characteristics of          common measure                 in 2014
                   ES Program             veterans served, the        outcomes
                   WIA Adult Program      services they
                                          receive, and their
                                          outcomes
                   Transition Assistance To evaluate pilot of         Site visits to military        Estimated    n/a
                   Program               revised Labor                bases; survey of               completion
                                         Employment                   workshop participants          in 2013
                                         Workshop                     and facilitators
                                         curriculum
                                            Source: GAO analysis of Labor documents.




                                            Page 30                                                GAO-13-29 Veterans' Employment and Training
While Labor has not conducted impact evaluations of its veterans’
employment and training programs, it is funding an impact evaluation of
the WIA Adult and Dislocated Worker programs, which is planned to be
completed in 2015. This study will include a supplemental study of
veterans using the public workforce system, but this portion of the study,
as described in the draft research plan for the study, is not an impact
evaluation and cannot determine the extent to which veterans’ outcomes
are due to the services they receive in the public workforce system.

Similar to Labor, VA has not conducted evaluations that allow it to
determine if veterans’ employment outcomes result from program
services or if they are the result of other factors. As shown in table 3, VA
has funded research that examines data related to the completion of
veterans’ rehabilitation plans and participant outcomes. 66 For example,
VA is funding a longitudinal study of Vocational Rehabilitation Program
participants and has issued two reports on the study. 67 The most recent
report begins to analyze VA administrative data to determine
characteristics associated with completing rehabilitation or discontinuing
the program within the first 2 years. However, the report states that its
findings thus far are only descriptive and may have little or no predictive
value. VA plans further study of emerging trends. VA also plans additional
follow-up of program participants in its case management process.
Specifically, the agency plans to send a questionnaire to collect
information on whether former participants are employed and whether
they need additional services.




66
  U.S. Department of Veterans Affairs, Vocational Rehabilitation and Employment
Program, Measures to Assist and Encourage Veterans in Completing Vocational
Rehabilitation (Washington, D.C.: April 2010).
67
   U.S. Department of Veterans Affairs, Vocational Rehabilitation and Employment
Program, Report to Congress: Vocational Rehabilitation and Employment Longitudinal
Study (Washington, D.C.: July 2011); and U.S. Department of Veterans Affairs, Vocational
Rehabilitation and Employment Program, Report to Congress: Vocational Rehabilitation
and Employment Longitudinal Study (Washington, D.C.: July 2012).




Page 31                                    GAO-13-29 Veterans' Employment and Training
Table 3: Studies VA Conducted Related to Veterans’ Employment

                 Programs
Name             reviewed       Purpose                      Methodology                  Status             Key findings
Measures to      Vocational     To determine factors         Literature review and        Completed          Identified factors that influence
Assist and       Rehabilitation that may prevent or          interviews with              2010               plan persistence and
Encourage        Program        preclude veterans            stakeholders                                    outcomes, such as greater
Veterans in                     from completing their        knowledgeable about                             satisfaction with the program,
Completing                      vocational                   vocational rehabilitation                       a strong working relationship
Vocational                      rehabilitation plans,        for veterans                                    with the veteran’s primary
Rehabilitation                  and identify actions to                                                      counselor and integration of
                                be taken to help                                                             vocational and medical
                                veterans overcome                                                            services were associated with
                                these identified                                                             higher rates of program
                                factors                                                                      completion.
Vocational       Vocational     To assess long-term          Analysis of administrative   Ongoing            Characteristics associated with
Rehabilitation   Rehabilitation employment and               data on individuals who      Reports            successful rehabilitation
and Employment   Program        other outcomes of            began participating in the   generally          include being over the age of
Longitudinal                    program participation        Vocational Rehabilitation    required each      60, having a primary (service-
Study                                                        Program in fiscal years      fiscal year;       connected) mental health
                                                             2010, 2012, and 2014         reports issued     diagnosis, and having a
                                                                                          for fiscal years   combined disability rating of 70
                                                                                          2010 and           percent or higher.
                                                                                          2011.              Characteristics associated with
                                                                                                             discontinuing the program
                                                                                                             within the first 2 years include
                                                                                                             having a serious employment
                                                                                                             handicap, having served in the
                                                                                                             Vietnam War era, and being of
                                                                                                             a junior rank.
                                          Source: GAO analysis of VA documents.



                                          Given that the number of service members transitioning to civilian
Conclusions                               employment is expected to increase and the number of veterans with
                                          service-connected disabilities is on the rise, Labor’s Disabled Veterans’
                                          Outreach Program is likely to see an increased demand for its services.
                                          Labor attempts to maximize the employment services for those veterans
                                          who need them most. However, we found that there is a need for clearer
                                          guidance to states on how to prioritize services and additional monitoring
                                          of their implementation of such guidance. Labor said it is developing such
                                          guidance but has not completed it, and has tested new monitoring
                                          protocols in six states but has not finalized them. It is encouraging that
                                          Labor has these efforts under way, and it will be important for the
                                          department to complete both efforts.

                                          Labor and VA both provide employment and training programs targeted to
                                          veterans. Although Labor and VA have a handbook governing their
                                          coordination with respect to employment and training for veterans, it has


                                          Page 32                                            GAO-13-29 Veterans' Employment and Training
                      not been updated since 2008. Our work identified sections of the
                      handbook that provided insufficient guidance, resulting in situations where
                      the practices of one department presented difficulties for the other in
                      meeting desired program outcomes. At the same time, DOD has begun
                      expanding employment assistance initiatives to segments of the veteran
                      population, such as National Guard and Reserve members, some of
                      whom may also meet Labor and VA veterans’ programs eligibility
                      requirements. However, Labor and VA’s agreement does not govern their
                      coordination with DOD’s programs. Without an agreement that includes
                      all three departments, efforts to help veterans find employment are at
                      greater risk of being fragmented or overlapping, and may not leverage
                      federal resources.

                      Finally, the federal investment in veterans’ employment and training
                      programs warrants greater transparency with regard to the extent to
                      which these programs are meeting their performance goals and whether
                      outcomes are attributable to program participation and not other factors.
                      Labor reports substantial information on outcomes for these programs.
                      However, Labor is not consistently reporting the extent to which outcomes
                      for each of its veterans’ programs are achieving the specific performance
                      goals that were established for these programs. This stands in contrast to
                      the level of performance reporting by Labor for its WIA Adult and ES
                      programs, which identifies the extent to which outcomes in these
                      programs are achieving performance goals. In addition, while the federal
                      government makes a substantial investment in Labor and VA programs to
                      achieve employment outcomes for veterans, neither agency has
                      conducted studies to see if these outcomes can be attributed to the
                      programs’ services, instead of other factors. As a result, Congress and
                      other stakeholders lack essential information to assess how well these
                      programs are performing and hold federal agencies accountable for
                      achieving results.


                      We are making the following four recommendations based on our review:
Recommendations for
Executive Action      •   To increase the effectiveness of coordination efforts, the Secretaries
                          of Labor and VA should incorporate additional guidance to address
                          the two problem areas we identified into any update to the interagency
                          handbook that governs their coordination for veterans’ employment
                          and training programs.
                      •   To ensure government resources are used efficiently, the Secretaries
                          of Labor, VA, and DOD should incorporate DOD’s employment




                      Page 33                              GAO-13-29 Veterans' Employment and Training
                      assistance initiatives into the agreements that guide interagency
                      coordination.
                  •   To enhance transparency and accountability for achieving results, the
                      Secretary of Labor should consistently report both performance goals
                      and associated performance outcomes for each of its veterans’
                      employment and training programs.
                  •   To assess veterans’ employment programs’ effectiveness, Secretaries
                      of Labor and VA should, to the extent possible, determine the extent
                      to which veterans’ employment outcomes result from program
                      participation or are the result of other factors.

                  We provided a draft of this report to the Department of Labor, the
Agency Comments   Department of Veterans Affairs, and the Department of Defense for
                  review and comment. Written comments from Labor, VA, and DOD
                  appear in appendixes III, IV, and V, respectively. In addition to the
                  comments discussed below, Labor, VA, and DOD provided technical
                  comments that we incorporated where appropriate.

                  All three agencies generally concurred with our recommendations. Both
                  Labor and VA said they would work to enhance coordination with each
                  other with respect to the guidance in their interagency handbook. All three
                  agencies said they would work to ensure interagency coordination
                  included DOD. In response to our recommendation on reporting program
                  performance, Labor said it will explore ways to increase consistency and
                  transparency of the information it reports. In response to our
                  recommendation to Labor and VA regarding assessing program
                  effectiveness, VA concurred and Labor did not specify whether or not it
                  agreed. Labor said that it is committed to robust program evaluation and
                  that each agency, including VETS, develops an annual evaluation agenda
                  and sets priorities. Labor said it has a multi-component agenda for
                  evaluating services to veterans and cited some current studies, such as
                  a study of the TAP program and a statistical analysis of services received
                  by veterans and their outcomes using the public workforce system. We
                  think obtaining information about the effectiveness of veterans' programs
                  is important because such information can assist Congress in assessing
                  program results and identifying areas where adjustments may be needed.
                  As Labor and VA conduct research on program outcomes, it will be
                  important for them to consider approaches that would enable them to
                  separate the impact of their programs from other factors that might
                  influence participants’ outcomes.




                  Page 34                              GAO-13-29 Veterans' Employment and Training
As agreed with your offices, unless you publicly announce the contents of
this report earlier, we plan no further distribution until 30 days from the
report date. At that time, we will send copies of this report to appropriate
congressional committees, the Secretary of Labor, the Secretary of
Veterans Affairs, the Secretary of Defense, and other interested parties.
In addition, this report will be available at no charge on GAO’s website at
http://www.gao.gov.

If you or your staff have any questions about this report, please contact
me at (202) 512-7215 or sherrilla@gao.gov. Contact points for our Offices
of Congressional Relations and Public Affairs may be found on the last
page of this report. GAO staff who made key contributions to this report
are listed in appendix VI.




Andrew Sherrill
Director
Education, Workforce, and Income Security Issues




Page 35                              GAO-13-29 Veterans' Employment and Training
Appendix I: Objectives, Scope, and
              Appendix I: Objectives, Scope, and
              Methodology



Methodology

              In a January 2011 report, 1 we identified six employment and training
              programs administered by the Department of Labor (Labor) and the
              Department of Veterans Affairs (VA) targeted toward veterans as a part of
              a larger review of all federal employment and training programs. We
              defined an employment and training program as one specifically designed
              to enhance the specific job skills of individuals in order to increase their
              employability, identify job opportunities, or help job seekers obtain
              employment. Labor oversees five of these programs for veterans: (1) the
              Disabled Veterans’ Outreach Program, (2) the Homeless Veterans’
              Reintegration Program, (3) the Local Veterans’ Employment
              Representative Program (Employment Representative Program), (4) the
              Transition Assistance Program (TAP), and (5) the Veterans’ Workforce
              Investment Program (VWIP). VA oversees the sixth program called the
              Vocational Rehabilitation & Employment Program (Vocational
              Rehabilitation Program). 2 Our 2011 report identified services, eligibility
              requirements, and outcome measures that these programs had in
              common.

              For this report, we focused on the six programs identified in our January
              2011 report in more detail and examined (1) the extent to which federal
              veterans’ employment and training programs vary in terms of the services
              they deliver and the veterans who receive them; (2) the extent to which
              federal agencies coordinate these programs; and (3) what is known about
              the performance of these programs.

              Our approach overall involved reviewing relevant literature, relevant
              federal laws and regulations, and analyzing Labor and VA data on
              veteran participants, services provided, and performance. We also
              interviewed federal Labor, VA, and Department of Defense (DOD) agency



              1
               GAO, Multiple Employment and Training Programs: Providing Information on Colocating
              Services and Consolidating Administrative Structures Could Promote Efficiencies,
              GAO-11-92 (Washington, D.C.: Jan. 13, 2011).
              2
                In November 2011, Congress enacted the VOW to Hire Heroes Act of 2011, which
              established a new employment and training program for certain unemployed veterans,
              referred to as the Veterans Retraining Assistance Program. Pub. L. No. 112-56, Title II,
              125 Stat. 711, 712. This program provides training for certain unemployed veterans
              between the age of 35 and 60 years old leading to an associate’s degree or similar
              certificate from a community college or technical school in a high-demand occupation. VA
              administers this program. We did not include this program in our review because it was
              under development.




              Page 36                                     GAO-13-29 Veterans' Employment and Training
Appendix I: Objectives, Scope, and
Methodology




officials who govern agency policy at the national level and key
stakeholder associations.

To determine the extent to which these programs vary in terms of the
services they deliver and the veterans receive them, we analyzed Labor
and VA data on services provided and veteran participants, agency
annual reports, budget justifications, and other agency policy documents.
To assess the reliability of Labor’s data on services to veterans in one-
stops, we reviewed available information about the data and conducted
interviews with officials knowledgeable about the data. We determined
that these data were sufficiently reliable for our purposes. We reviewed
relevant federal laws and regulations to determine program eligibility
requirements. We interviewed agency officials, veterans’ service
organizations, and workforce associations to better understand why
programs may serve similar populations with similar services. We
reviewed state plans and interviewed state-level Labor, VA, and DOD
staff for our case study states.

In addition, we conducted six case studies at the state level. In each
state, we reviewed state plans and interviewed Labor and VA officials
assigned to the state or the region. We also interviewed the Employer
Support of the Guard and Reserve (ESGR) chairperson operating in the
state. These are federal representatives of DOD, but not federal
employees, as they are volunteers. In addition, we interviewed the
directors of state workforce agencies, which carry out veterans’
employment and training programs using federal funds. Finally, within
three states, we interviewed the Director of Veterans Affairs in each state,
a state government official responsible for veterans’ programs and
services. To select states, for all 50 states, we determined whether each
was high, medium, or low on the following characteristics: the percentage
of veteran population, amount of program expenditures, program
performance (veterans’ entered employment rate), and veterans’
unemployment rate. We selected one state from each of Labor’s six
regions to achieve variation on the above characteristics, as well as
diversity in terms of geography and state size. These states were Florida,
Massachusetts, Ohio, Oregon, Texas, and Virginia.

To determine the extent to which federal agencies coordinate these
programs, we reviewed key agency agreements and guidance, and used
the same six case studies at the state level, and interviewed federal and
state agency officials and associations representing the interests of
veterans. In examining coordination we included not only the five
programs indicated above but also three Labor programs available to the


Page 37                              GAO-13-29 Veterans' Employment and Training
Appendix I: Objectives, Scope, and
Methodology




general population: the Workforce Investment Act (WIA) Adult and
Dislocated Worker and the Employment Service (ES) programs. 3 We also
included programs recently begun by DOD: the Yellow Ribbon and ESGR
programs. 4 We reviewed memoranda of understanding, agency guidance,
and other policy documents related to collaborative efforts among federal
agencies. We also interviewed National Veterans’ Training Institute 5
officials to discuss the extent to which required training for outreach
specialists and employment representatives includes instruction on how
to foster inter- and intra-agency coordination. In our case studies in six
states, we interviewed state-level officials from the Veterans’ Employment
and Training Services, as well as VA officials. We also interviewed ESGR
officials and Directors of State Offices of Veterans Affairs. 6 To understand
stakeholders’ views on coordination, we interviewed officials from
workforce associations and veterans’ service organizations. We also used
data from the Defense Manpower Data Center to determine the number
of Guard and Reserve members that may meet the eligibility
requirements for Labor veterans’ programs and VA’s Vocational
Rehabilitation Program. We assessed the reliability of information on
Guard and Reserve members’ length of service and disability status, and
determined that these data were sufficiently reliable for the purposes of
this report.

To determine what is known about program performance, we analyzed
relevant federal laws and regulations, and agency documents, and
interviewed agency officials and stakeholders. We reviewed agency
reports on veterans’ programs containing information on program



3
  Apart from the five Labor employment and training programs that target veterans, these
three programs account for 98 percent of the veterans who participated in other Labor
programs.
4
  These DOD programs were not included our January 2011 review. We decided to
include nationally coordinated DOD programs focused on serving Guard and Reserve
members who may qualify as veterans for certain Labor and VA programs.
5
  The National Veterans’ Training Institute was established to further develop and
enhance the professional skills of veterans’ employment and training service providers
throughout the United States. The program is funded by the Department of Labor’s
Veterans’ Employment and Training Service, and is administered by the University of
Colorado Denver with training conducted in Denver and other sites in the United States
and abroad.
6
 Three of the six State Directors of Veterans Affairs that we contacted did not respond to
our request.




Page 38                                      GAO-13-29 Veterans' Employment and Training
Appendix I: Objectives, Scope, and
Methodology




outcomes and agency goals established for these programs, such as
Labor’s annual report to Congress on veterans’ programs and
agencywide performance reports. We assessed Labor and VA data on
participant employment outcomes by reviewing available information
about the data and conducting interviews with officials knowledgeable
about the data. We determined that these data were sufficiently reliable
for our purposes. We reviewed the design and methodology of relevant
agency-sponsored program evaluations using GAO criteria on program
evaluation design. We also interviewed Labor and VA national and
regional officials.




Page 39                              GAO-13-29 Veterans' Employment and Training
Appendix II: Summary of Veterans’ Program
                                       Appendix II: Summary of Veterans’ Program
                                       Performance



Performance

Figure 4: Veterans’ Entered Employment Rate, by Program, Program Years 2006-2010




                                       Note: Information for WIA Adult and Employment Service programs represents outcomes specifically
                                       for veterans using these programs.



Figure 5: Veterans’ Employment Retention Rate, by Program, Program Years 2006-2010




                                       Note: Information for WIA Adult and Employment Service programs represents outcomes specifically
                                       for veterans using these programs.




                                       Page 40                                         GAO-13-29 Veterans' Employment and Training
                                       Appendix II: Summary of Veterans’ Program
                                       Performance




Figure 6: Veterans’ Average Earnings, by Program, Program Years 2006-2010




                                       Note: Information for WIA Adult and Employment Service programs represents outcomes specifically
                                       for veterans using these programs.




                                       Page 41                                         GAO-13-29 Veterans' Employment and Training
                                       Appendix II: Summary of Veterans’ Program
                                       Performance




Figure 7: Employment Service Participants’ Entered Employment Rate, Employment Retention Rate, and Average Earnings,
Program Years 2006-2010




                                       Page 42                                     GAO-13-29 Veterans' Employment and Training
                                       Appendix II: Summary of Veterans’ Program
                                       Performance




Figure 8: WIA Adult Program Participants’ Entered Employment Rate, Employment Retention Rate, and Average Earnings,
Program Years 2006-2010




                                       Page 43                                     GAO-13-29 Veterans' Employment and Training
Appendix II: Summary of Veterans’ Program
Performance




Figure 9: VA Vocational Rehabilitation Program Participants Rehabilitated to
Employment, Fiscal Years 2006-2011




Page 44                                     GAO-13-29 Veterans' Employment and Training
Appendix III: Comments from the
              Appendix III: Comments from the Department
              of Labor



Department of Labor




              Page 45                                      GAO-13-29 Veterans' Employment and Training
Appendix III: Comments from the Department
of Labor




Page 46                                      GAO-13-29 Veterans' Employment and Training
Appendix IV: Comments from the
             Appendix IV: Comments from the Department
             of Veterans Affairs



Department of Veterans Affairs




             Page 47                                     GAO-13-29 Veterans' Employment and Training
Appendix IV: Comments from the Department
of Veterans Affairs




Page 48                                     GAO-13-29 Veterans' Employment and Training
Appendix IV: Comments from the Department
of Veterans Affairs




Page 49                                     GAO-13-29 Veterans' Employment and Training
Appendix IV: Comments from the Department
of Veterans Affairs




Page 50                                     GAO-13-29 Veterans' Employment and Training
Appendix V: Comments from the Department
             Appendix V: Comments from the Department
             of Defense



of Defense




             Page 51                                    GAO-13-29 Veterans' Employment and Training
Appendix V: Comments from the Department
of Defense




Page 52                                    GAO-13-29 Veterans' Employment and Training
Appendix VI: GAO Contact and Staff
                  Appendix VI: GAO Contact and Staff
                  Acknowledgments



Acknowledgments

                  Andrew Sherrill, (202) 512-7215 or sherrilla@gao.gov
GAO Contact
                  In addition to the individual named above, Patrick Dibattista (Assistant
Staff             Director), Sheranda Campbell, Maria Gaona, and Dana Hopings made
Acknowledgments   key contributions to this report. In addition, key support provided by
                  James Bennett, David Chrisinger, Holly Dye, Rachel Frisk, David
                  Forgosh, Alexander Galuten, Kathy Leslie, Ashley McCall, and Walter
                  Vance.




(131103)
                  Page 53                              GAO-13-29 Veterans' Employment and Training
                      The Government Accountability Office, the audit, evaluation, and
GAO’s Mission         investigative arm of Congress, exists to support Congress in meeting its
                      constitutional responsibilities and to help improve the performance and
                      accountability of the federal government for the American people. GAO
                      examines the use of public funds; evaluates federal programs and
                      policies; and provides analyses, recommendations, and other assistance
                      to help Congress make informed oversight, policy, and funding decisions.
                      GAO’s commitment to good government is reflected in its core values of
                      accountability, integrity, and reliability.

                      The fastest and easiest way to obtain copies of GAO documents at no
Obtaining Copies of   cost is through GAO’s website (http://www.gao.gov). Each weekday
GAO Reports and       afternoon, GAO posts on its website newly released reports, testimony,
                      and correspondence. To have GAO e-mail you a list of newly posted
Testimony             products, go to http://www.gao.gov and select “E-mail Updates.”

Order by Phone        The price of each GAO publication reflects GAO’s actual cost of
                      production and distribution and depends on the number of pages in the
                      publication and whether the publication is printed in color or black and
                      white. Pricing and ordering information is posted on GAO’s website,
                      http://www.gao.gov/ordering.htm.
                      Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
                      TDD (202) 512-2537.
                      Orders may be paid for using American Express, Discover Card,
                      MasterCard, Visa, check, or money order. Call for additional information.
                      Connect with GAO on Facebook, Flickr, Twitter, and YouTube.
Connect with GAO      Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.
                      Visit GAO on the web at www.gao.gov.
                      Contact:
To Report Fraud,
                      Website: http://www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in   E-mail: fraudnet@gao.gov
Federal Programs      Automated answering system: (800) 424-5454 or (202) 512-7470

                      Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-
Congressional         4400, U.S. Government Accountability Office, 441 G Street NW, Room
Relations             7125, Washington, DC 20548

                      Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800
Public Affairs        U.S. Government Accountability Office, 441 G Street NW, Room 7149
                      Washington, DC 20548




                        Please Print on Recycled Paper.