oversight

Welfare Reform: Assessing the Effectiveness of Various Welfare-to-Work Approaches

Published by the Government Accountability Office on 1999-09-09.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                  United States General Accounting Office

GAO               Report to Congressional Committees




September 1999
                  WELFARE REFORM
                  Assessing the
                  Effectiveness of
                  Various
                  Welfare-to-Work
                  Approaches




GAO/HEHS-99-179
      United States
GAO   General Accounting Office
      Washington, D.C. 20548

      Health, Education, and
      Human Services Division

      B-282174

      September 9, 1999

      Congressional Committees

      The Personal Responsibility and Work Opportunity Reconciliation Act of
      1996 (P.L. 104-193), enacted in August 1996, significantly changed the
      nation’s cash assistance program for needy families with children. Title I
      of the law replaced the Aid to Families With Dependent Children (AFDC)
      cash assistance program with fixed block grants to states to provide
      Temporary Assistance for Needy Families (TANF) and ended families’
      entitlement to assistance. Several goals of the TANF program are specified
      in the law, including that of ending welfare dependence by promoting
      work over welfare and self-reliance over dependency. Over the years,
      states’ welfare-to-work programs have emphasized different goals and
      philosophies for moving individuals into work and have provided different
      types of services and activities to program participants to help them reach
      those goals. Programs with the goal of rapid employment emphasize quick
      exposure to and entry into the labor force, reflecting the belief that
      participants can best acquire employment-related skills when they are
      working, regardless of the quality of the job. These programs’ service
      strategies tend to rely heavily on job search activities but can make use of
      education and training to some extent. Other programs have the goal of
      skill building, often called an education-based approach, which usually
      involves a greater initial investment in participants’ education and
      occupational skills, so that when they do enter the labor market, they can
      obtain “good” jobs—those with higher pay, health benefits, and
      opportunity for advancement. The 1996 welfare reform law emphasizes the
      importance of moving welfare recipients into employment and gives states
      greater flexibility to tailor their programs to meet their own goals and
      needs.

      To help it assess how best to assist welfare recipients, the Congress
      directed us in the Higher Education Amendments of 1998 (P.L. 105-244) to
      review research on the effectiveness of various welfare-to-work
      approaches. Specifically, we were asked to examine the research findings
      on (1) the effectiveness of a rapid employment approach, an
      education-based approach (including adult vocational and postsecondary
      education), and a combination of these two approaches in improving
      employment-related outcomes for welfare recipients and other
      low-income women with children and (2) the effect of welfare recipients’
      educational attainment, including postsecondary education, on the
      educational attainment of their children. In performing this work, we




      Page 1                              GAO/HEHS-99-179 Welfare-to-Work Approaches
                   B-282174




                   reviewed studies that had been published from 1988 through mid-1999 that
                   assessed the effectiveness of welfare-to-work approaches by comparing
                   outcomes related to employment and earnings as well as declines in
                   welfare payments for those in the programs with those not in the
                   programs. Because some welfare-to-work programs may produce results
                   in the longer term rather than in the short term, we generally focused on
                   evaluations having relatively long-term results (5 years). In addition,
                   because many of the welfare recipients who participated in the
                   welfare-to-work programs evaluated had less than a high school
                   education, the education-based approaches evaluated in the studies we
                   assessed primarily provided basic education services rather than
                   postsecondary education. As a result, none of the studies evaluated the
                   effectiveness of a college education on improving employment-related
                   outcomes for welfare recipients. We also identified and reviewed studies
                   about the effect of welfare recipients’ educational attainment on their
                   children. (See app. I for a full discussion of our scope and methodology.)


                   Research conducted to date on the effectiveness of different
Results in Brief   welfare-to-work approaches suggests that programs with a combined
                   approach—including both job search assistance and some education and
                   training—tend to be more effective over a 5-year period than either
                   approach alone in increasing employment and earnings while reducing
                   welfare payments. Five evaluations begun in the 1980s with 5-year results
                   indicated that programs focusing on rapid employment and job search
                   activities combined with education and training activities more often
                   increased employment and earnings and reduced welfare payments,
                   compared with programs that focused solely on job search activities or
                   those that placed the greatest emphasis on education. In addition,
                   preliminary results (2-year findings) from a more recent ongoing
                   evaluation (started in 1992)—the only evaluation designed explicitly to
                   compare the effectiveness of a rapid-employment approach with an
                   education-based welfare-to-work approach—found that while each
                   approach has increased participants’ employment and earnings, so far,
                   neither approach has proven clearly better than the other. The rapid
                   employment approach did, however, cost about half as much per person
                   as the education-based approach. While these studies provide useful
                   information, more needs to be known about how well different
                   approaches are performing in the current environment created by the
                   enactment of welfare reform in 1996, which none of these evaluations
                   cover.




                   Page 2                              GAO/HEHS-99-179 Welfare-to-Work Approaches
             B-282174




             While research indicates that parents’ educational attainment has a
             positive effect on children’s educational attainment, little information is
             currently available on this relationship specifically within the welfare
             population. Recent studies have identified factors affecting cognitive
             development of children in welfare families. This research, while limited in
             scope, indicates that a mother’s higher level of educational attainment is
             one factor that may positively affect children’s development. In addition, a
             body of research that focuses on the effects of poverty on children’s
             educational attainment suggests a significant positive relationship
             between the educational attainment of parents and their children among
             both the welfare and the nonwelfare populations.


             AFDC  was created by the Social Security Act of 1935 to provide cash
Background   assistance to families with needy children who had been deprived of the
             support of one of their parents—at that time, mostly children living with
             widowed mothers. The program was not designed to promote
             employment, because, at the time, mothers were generally not expected to
             work outside the home. Over the past several decades, however, the
             public has come to believe that most welfare families should be at least
             partly self-supporting. Efforts to provide education, training, and job
             search assistance to help welfare recipients prepare for and find jobs can
             be traced back at least to the 1960s, when the Congress mandated that
             every state operate a Work Incentive (WIN) program to encourage AFDC
             recipients to become self-sufficient. WIN began primarily as a voluntary
             program focusing on job search assistance and immediate employment.
             Starting in 1981, WIN demonstration projects were established that gave
             states greater flexibility to design their own programs, and states could
             now require welfare recipients with children aged 6 and older to
             participate. Our reviews of the WIN program showed that it often served
             those most likely to find employment on their own rather than those less
             job-ready, who needed the most help to become employed.

             The Family Support Act of 1988 eliminated the WIN program and created
             the Job Opportunities and Basic Skills Training (JOBS) program. Research
             conducted before passage of that act showed that welfare recipients were
             a diverse group, making use of the AFDC program in different ways. While
             most who used AFDC did so for short periods of time, the majority of AFDC’s
             resources were devoted to providing benefits to long-term recipients. This
             research also identified several factors that were associated with
             long-term welfare dependence, including recipients’ low level of
             education, single-parent status, higher number of children, disability, and



             Page 3                              GAO/HEHS-99-179 Welfare-to-Work Approaches
B-282174




limited work experience.1 To better ensure that AFDC recipients received
assistance that would help them avoid long-term welfare dependence, JOBS
required state programs to include a broad range of services, including
education and training assistance,2 and to provide financial assistance with
support services such as child care and transportation. The population
that could be required to participate was changed from those with children
aged 6 and above to those with children aged 3 and above. In addition, for
the first time, states were required to place a specified minimum
percentage of nonexempt welfare recipients in education, training, and
work-related activities and to target resources to long-term recipients and
those considered at risk of long-term welfare dependence.

The Family Support Act also called for an evaluation with a random
assignment design, which would control for other factors that could affect
outcomes, to assess the effectiveness of various welfare-to-work
programs. The Department of Health and Human Services (HHS), with
support from the Department of Education, contracted with the
Manpower Demonstration Research Corporation (MDRC), a research
organization that analyzes education- and employment-related programs,
to conduct this evaluation, which focused on mandatory welfare-to-work
programs at seven sites. This evaluation is referred to as the National
Evaluation of Welfare-to-Work Strategies, formerly known as the JOBS
Evaluation. In the mid-1980s, California also contracted with MDRC to
conduct an evaluation of its state welfare-to-work program, called Greater
Avenues for Independence (GAIN), which became the state’s JOBS program
after 1988. At the beginning of the GAIN evaluation, California had about
one-sixth of the nation’s AFDC caseload, and GAIN expended about
13 percent of federal JOBS funds.

In response to JOBS’ increased emphasis on education and training and a
general belief that these activities could help improve welfare recipients’
financial well-being, as they do for the general population, many state and
local JOBS programs emphasized the provision of education and training. In
our 1991 report on the implementation of JOBS, we found that almost half
of the states reported a shift from an emphasis on immediate job

1
  David Ellwood, Targeting “Would-Be” Long-Term Recipients of AFDC (Princeton, N.J.: Mathematica
Policy Research, Inc., 1986).
2
 Under JOBS, states were to assess the needs and skills of welfare recipients, prepare them for
employment through education and training as needed, and place them in jobs. Federal rules specified
certain activities that each state’s JOBS program was required to offer. These included education
activities, job skills training, job-readiness activities, and job development and placement services.
States also had to offer at least two of the four WIN activities (job search, on-the-job training, work
supplementation programs, and community work experience programs). Postsecondary education was
optional under the federal JOBS rules, and states could assign participants to this activity on an
individual basis.


Page 4                                           GAO/HEHS-99-179 Welfare-to-Work Approaches
B-282174




placement under their previous welfare-to-work programs toward a new
emphasis on long-term education or training.3 In 1995, we reported that
JOBS participants nationwide were enrolled in a variety of education and
training activities—postsecondary education (17 percent), high school or
preparation for the general equivalency diploma (16 percent), job skills
training (13 percent), adult basic or remedial education (7 percent), and
English as a Second Language training (2 percent).4

Several years after the implementation of JOBS, our review of state JOBS
programs nationwide showed that only about 11 percent of welfare
recipients were involved in JOBS activities each month.5 In addition, AFDC
caseloads rose to their highest levels ever, peaking at 5 million families in
1994. Also during the 1990s, under waivers of the federal rules, several
states experimented with changes in their AFDC and JOBS programs. These
changes included encouraging welfare recipients to work by allowing
them to keep more of their earnings without losing welfare benefits,
strengthening and more strongly enforcing work requirements, and
imposing limits on the length of time a family could receive aid.

To encourage and facilitate innovation by the states and to address
continuing concerns among policymakers about growing welfare
dependence, the Personal Responsibility and Work Opportunity
Reconciliation Act of 1996, commonly referred to as welfare reform
legislation, overhauled the nation’s welfare system by abolishing the AFDC
program and establishing TANF block grants. Under TANF, which is
administered by HHS, states are provided up to $16.8 billion each year
through 2002 to provide aid to needy families with children. While the
states have great flexibility to design programs that meet their own goals
and needs, they must also meet several federal requirements designed to
emphasize the importance of work and the temporary nature of TANF aid.
TANF established stronger work requirements for those receiving aid than
the requirements of its predecessor program, and the population that can
be required to work now includes all parents, regardless of the ages of




3
 Welfare to Work: States Begin JOBS, but Fiscal and Other Problems May Impede Their Progress
(GAO/HRD-91-106, Sept. 27, 1991).
4
Welfare to Work: Participants’ Characteristics and Services Provided in JOBS (GAO/HEHS-95-93,
May 2, 1995).
5
Welfare to Work: Most AFDC Training Programs Not Emphasizing Job Placement (GAO/HEHS-95-113,
May 19, 1995).


Page 5                                         GAO/HEHS-99-179 Welfare-to-Work Approaches
B-282174




their children.6 In addition, states must enforce a 5-year limit (less at state
option) on the length of time a family may receive federal TANF assistance.7

Our June 1998 report and other studies of TANF implementation show that
many states and localities have taken steps to transform their welfare
offices into job placement centers and are encouraging or requiring those
seeking aid to engage in job search activities as soon as they apply.8 Along
with this increased emphasis on work, welfare offices and workers are
also focusing more on helping clients address and solve problems that
interfere with employment. States’ implementation of more work-focused
programs, undertaken under conditions of strong economic growth, has
been accompanied by a 45-percent decline in the number of families
receiving welfare—from a high of about 5 million families in 1994 to
2.7 million families as of December 1998.9 A nationally representative
survey of families who left welfare from 1995 to 1997 found that 61 percent
of former welfare recipients were working at the time of the survey,
although often at low-paying jobs.10




6
 The required minimum participation rate began at 25 percent in fiscal year 1997 and rises to 50
percent in fiscal year 2002. States receive credit for the degree to which their caseloads have declined
since fiscal year 1996. While states have considerable flexibility in designing their welfare-to-work
programs, the legislation prescribes the activities that states may count toward their work
participation rate. For example, vocational education is limited as a countable work activity both in
the percentage of recipients who can be engaged in vocational training and count toward the
participation rate (30 percent) and the length of time a recipient can be in vocational training (up to 12
months). Moreover, unless states include it in their definition of vocational education training, the
legislation does not allow postsecondary education to be counted as a work activity toward the states’
participation rate.
7
 States may exempt from time limits up to 20 percent of those receiving TANF aid and may use their
own funds to provide aid beyond the federal time limit.
8
 Welfare Reform: States Are Restructuring Programs to Reduce Welfare Dependence
(GAO/HEHS-98-109, June 17, 1998).
9
 Welfare Reform: States’ Implementation Progress and Information on Former Recipients
(GAO/T-HEHS-99-116, May 27, 1999).
10
  Pamela Loprest, “Families Who Left Welfare: Who Are They and How Are They Doing?” Urban
Institute Discussion Paper 99-02 (July 1999). See also Welfare Reform: Information on Former
Recipients’ Status (GAO-HEHS-99-48, Apr. 28, 1999).



Page 6                                             GAO/HEHS-99-179 Welfare-to-Work Approaches
                               B-282174




                               The current research on the relative merits of the rapid employment and
Research Shows That            education-based approaches does not conclusively show that one
a Variety of                   approach is more effective than the other in increasing welfare recipients’
Welfare-to-Work                employment and earnings and reducing their welfare payments. Of the six
                               evaluations of different approaches we identified and reviewed, five
Approaches Have                evaluations, with 5-year results, covered a range of programs, with some
Positive                       combining elements of a rapid employment approach with an
                               education-based approach. The results of these evaluations were mixed
Employment-Related             and, while not conclusive, indicated that programs that combined the
Outcomes                       approaches had more positive effects—and that these effects covered a
                               broader cross section of the welfare population—than programs that
                               focused more exclusively on providing only job search activities or only
                               education. Only one evaluation, part of the National Evaluation of
                               Welfare-to-Work Strategies, directly compared the results of a rapid
                               employment approach with those of an education-based approach. The
                               results for the first 2 years of this ongoing study showed that the outcomes
                               were roughly comparable for the two approaches, with both modestly
                               increasing participants’ employment and earnings and reducing welfare
                               payments. The rapid employment approach was only about half as costly
                               per participant as the education approach. None of the studies evaluated
                               the effectiveness of a college education in improving employment-related
                               outcomes for welfare recipients. All of the evaluations we reviewed
                               provided information on welfare-to-work programs operated in the 1980s
                               and 1990s; none included results on programs operated since welfare
                               reform was enacted in 1996. Currently, HHS is funding 23 studies in 20
                               states on welfare reforms that began under waivers of the AFDC program
                               but that are continuing in the new welfare environment. These studies will
                               provide more information on effective approaches for moving welfare
                               recipients into work.11


Little Research Is Available   We reviewed five evaluations conducted by MDRC that focused on
That Compares the              mandatory welfare-to-work programs and for which 5 years of follow-up
Effectiveness of Various       data were available. Because the results of the rapid employment and
                               education-based approaches are expected to unfold in different ways, with
Approaches                     more immediate results from the rapid employment approach and more
                               delayed results from the education-based approach, we focused only on
                               those evaluations with results from 5 years of follow-up. (Other studies we
                               identified that did not include 5-year results are listed in the bibliography.)
                               Four evaluations were begun in the early 1980s and covered a variety of

                               11
                                For more information, see Web sites http://www.acf.dhhs.gov/programs/opre/rd&e.htm and
                               http://aspe.os.dhhs.gov/hsp/hspwelfare.htm



                               Page 7                                        GAO/HEHS-99-179 Welfare-to-Work Approaches
B-282174




programs across the country; the fifth evaluation, begun in the late 1980s,
included six sites in California. As shown in table 1, these programs
generally used some combination of the rapid employment and
education-based approaches, although each tended to emphasize one
approach more than the other. By randomly assigning welfare recipients to
different groups—program participants and nonparticipants—evaluators
can determine which changes in people’s employment, earnings, and
welfare payments were due to their participation in the program. This
random assignment method cannot tell analysts which particular aspects
of the program caused the changes or definitively show that the program
effects were caused by the particular approach used, rather than other
program features. But such an evaluation can determine whether the way
a program was operated at a particular site was effective.

While evaluations of welfare-to-work programs have been conducted over
the years, they generally have not been designed to determine the
effectiveness of one particular approach compared with another. As part
of the National Evaluation of Welfare-to-Work Strategies, formerly the JOBS
Evaluation, required by the Family Support Act of 1988, MDRC started a
largely unprecedented effort to compare the effect of two distinct types of
welfare-to-work approaches—the rapid employment approach versus the
education-based approach.12 Under the rapid employment approach, it is
expected that individuals will move quickly into employment with
immediate payoffs in increased earnings and welfare savings and the
potential to earn more over time. Under the education-based approach, an
initial investment in education and training is expected to pay off in the
future, with increased earnings and welfare savings once the training is
completed. This study, the sixth MDRC study that we reviewed, will provide
up to 5 years of follow-up data and will analyze the effects of each
approach on a wider array of outcomes, including those for children, in
the future. Only the results from the first 2 years of this ongoing
comparison study are currently available.




12
  This comparison evaluation is part of the National Evaluation of Welfare-to-Work Strategies, which
also includes other components, such as a study, focusing on children who were between the ages of 3
and 5 when their mothers entered the study, to measure outcomes on children’s cognitive development
and academic achievement, safety and health, problem behavior and emotional well-being, and social
development. In the national evaluation, over 55,000 individuals at seven sites were randomly assigned
to groups that remained eligible for specific welfare-to-work programs or to groups that did not
participate in these programs. In addition to the 2-year findings reported in a study of three sites,
2-year impacts are available for one site in Portland, Oreg. Also, MDRC has completed evaluations at
the remaining three sites in Columbus, Ohio; Detroit, Mich.; and Oklahoma City, Okla.; however, these
studies had not been published at the time of our review.



Page 8                                           GAO/HEHS-99-179 Welfare-to-Work Approaches
                                   B-282174




Table 1: Evaluation Studies That
Assessed the Effectiveness of                                                     Number of programs Basic program
Welfare-to-Work Programs           Evaluation                 Start datea         with specific findings approach
                                   WIN-era programs
                                   Baltimore Options          1982                Combined results for      Mixed service strategy
                                   Program                                        10 of 18 city welfare
                                                                                  offices
                                   Arkansas WORK              1983                Combined results for      Rapid employment,
                                   Program                                        2 counties                with a primary focus
                                                                                                            on job search
                                                                                                            activities
                                   Virginia Employment        1983                Combined results for      Rapid employment,
                                   Services Program                               11 county welfare         with a primary focus
                                                                                  agencies                  on job search
                                                                                                            activities
                                   San Diego Saturation       1985                Combined results for      Mixed service strategy
                                   Work Initiative Model                          2 of 7 city welfare
                                   (SWIM)                                         offices
                                   JOBS-era program
                                   California’s Greater       1988                Separate results for      Mixed service
                                   Avenues for                                    six counties:             strategies that
                                   Independence (GAIN)                            Alameda; Butte;           emphasize rapid
                                   Program                                        Los Angeles;              employment or an
                                                                                  Riverside; San Diego;     education-based
                                                                                  and Tulare                approach to different
                                                                                                            degrees
                                   National Evaluation of 1992                    Separate results for      Comparison of rapid
                                   Welfare-to-Work                                three sites:              employment and
                                   Strategies: Evaluation                         Atlanta, Ga.;             education-based
                                   of Two Welfare-to-Work                         Grand Rapids, Mich.;      approaches
                                   Program Approaches                             Riverside, Calif.
                                   a
                                    All programs were evaluated for 5 years except for the JOBS-era evaluation of two
                                   welfare-to-work approaches—2-year results are available for this ongoing comparison study.



                                   The MDRC comparison study of two welfare-to-work approaches was
                                   conducted at three sites—Atlanta, Georgia; Grand Rapids, Michigan; and
                                   Riverside, California. At each site, welfare recipients were randomly
                                   assigned to the rapid employment program, the education-based program,
                                   or a control group. The study compared outcomes for those assigned to
                                   the program groups with outcomes for those assigned to the control
                                   group, and compared outcomes for those assigned to one program with
                                   outcomes for those assigned to the other program. Operating the two
                                   programs at the three sites simultaneously controlled for the economic
                                   and programmatic environments that could affect participants’ outcomes.
                                   Randomly assigning recipients to one of the two programs or a control
                                   group helped to eliminate any bias from, or effect of, differences in



                                   Page 9                                         GAO/HEHS-99-179 Welfare-to-Work Approaches
                                B-282174




                                recipients’ characteristics that could affect outcomes. Differences in
                                participant outcomes could then be attributed to the program in which
                                they participated. In this comparison study and in other evaluations we
                                reviewed, not all individuals assigned to a welfare-to-work program group
                                necessarily participated in program activities; also, some individuals not
                                assigned to a program group sought out and received services at their own
                                initiative. In this report, hereafter, we use the term “participants” to refer
                                to individuals assigned to a welfare-to-work program group and the term
                                “nonparticipants” to refer to those assigned to a control group.


Evaluations Showed That         Although they were not designed to assess the effect of one approach
Programs That Combine           compared with another, the four WIN-era and one JOBS-era evaluations
Approaches More Often           provide insights into the effectiveness of various approaches. The five
                                evaluations covered 10 programs—seven in California—with varying
Had Positive Effects Over 5     approaches for moving welfare recipients into work, and had mixed
Years Than Those That           results. Some programs had effects in all three areas—employment,
Emphasized One Approach         earnings, and welfare savings. In some cases, a program increased
                                earnings for those considered less job-ready but did not increase earnings
                                for those considered job-ready. Although not definitive, the studies show
                                that programs combining elements of both the rapid employment and
                                education-based approaches—having a rapid employment focus but
                                relying on education and training for some participants—tended to have a
                                greater effect on participants’ employment and earnings, and on welfare
                                savings, than approaches that emphasized just job search activities or just
                                longer-term education and training.

Five-Year Evaluations of Four   A study of the 5-year results of four programs operated during the WIN era
WIN-Era Programs                offers a look at two programs that relied primarily on job search activities
                                to move people into jobs in comparison with two programs that provided
                                education and training in addition to job search activities (see table 2). The
                                Arkansas and Virginia programs are generally considered examples of
                                programs focusing on job search only that were relatively low cost and
                                provided little in the way of education and training, although some Virginia
                                program participants did participate in education and training
                                opportunities but participated in such activities only slightly more than the
                                control group. The Baltimore Options program featured an initial
                                assessment of each participant’s needs and then provided participants a
                                choice of activities. SWIM, in San Diego, was designed to involve as many
                                welfare recipients as possible in ongoing activities, following a set
                                sequence of activities: job search, work experience, and then education or
                                training.



                                Page 10                               GAO/HEHS-99-179 Welfare-to-Work Approaches
                                       B-282174




Table 2: Emphasis and Other Features
of Four WIN-Era Programs                                                                                                   Percentage of
                                                                                                                         participants who
                                                                                                      Net program             were ever in
                                                         Program emphasis and                             cost per         education and
                                       Evaluated program service strategy                                  persona                training
                                       Arkansas WORK            Rapid employment with job
                                                                search first and 3-month
                                                                unpaid work assignments                         $220                           0
                                       Virginia Employment Rapid employment with job
                                       Services            search and some 3-month
                                                           unpaid work assignments                               598                          12
                                       Baltimore Options        After initial assessment,
                                                                participants are given some
                                                                choice among job search,
                                                                unpaid work assignments,
                                                                or education and training                      1,325                          17
                                       SWIM                     Involve as many
                                                                participants as possible in
                                                                activities for as long as
                                                                possible, starting with job
                                                                search, then 3-month
                                                                unpaid work assignment,
                                                                then some education and
                                                                training                                       1,212                          24
                                       a
                                        Represents the average cost per participant less the average cost per nonparticipant, adjusted
                                       to 1993 dollars. Note that net program costs per person are lower for these WIN-era programs
                                       compared with the JOBS-era programs because welfare recipients with younger children were
                                       not required to participate in WIN-era programs. Consequently, child care costs were lower.



                                       The SWIM and Baltimore Options programs significantly increased total
                                       earnings for participants over the 5-year study period, while the earnings
                                       increases for the Arkansas and Virginia programs were not statistically
                                       significant. Arkansas WORK did, however, have significant savings in
                                       welfare payments.13 As shown in figure 1, only the SWIM program both
                                       produced welfare savings and increased participant earnings. Over the 5
                                       years, the SWIM program increased total earnings of participants more than
                                       $2,000 above the total earnings increase for nonparticipants and reduced
                                       welfare payments almost $2,000 per participant. On the basis of a detailed
                                       analysis of these studies’ findings, researchers concluded that the
                                       Baltimore Options program was the only program that helped some

                                       13
                                         For our interpretations, we used a common significance level of 5 percent (.05) or less, which was
                                       stricter than that used in some evaluations. Total earnings impact represents the difference between
                                       the total average earnings between the program participants and nonparticipants. Welfare savings
                                       represent the differences between the total average welfare payments for participants and
                                       nonparticipants and do not take into account the costs of the program. In averaging the earnings or
                                       savings, those individuals with a zero value were included. The average earnings of only those with
                                       earnings would be higher.



                                       Page 11                                          GAO/HEHS-99-179 Welfare-to-Work Approaches
                                         B-282174




                                         participants find higher-paying jobs than the jobs nonparticipants found.
                                         The programs that relied primarily on job search generally resulted in
                                         those participants who would have worked eventually beginning to work
                                         sooner, and motivated some participants who would not have worked to
                                         get a job. For the near term, this resulted in participants relying more on
                                         their own earnings than on welfare, although it did not increase their
                                         overall financial well-being or increase their earnings capacity.14


Figure 1: Effect on Total Earnings and
Welfare Savings per Program              2500     Difference in Dollars
Participant Over a 5-Year Study Period
for Four WIN-Era Programs
                                         2000




                                         1500




                                         1000




                                          500




                                              0

                                                  Arkansas      Baltimore      SWIM      Virginia


                                                           Earnings Increase

                                                           Welfare Savings



                                         Note: Earnings differences between program participants and nonparticipants were statistically
                                         significant only for Baltimore Options and SWIM; welfare savings were statistically significant only
                                         for Arkansas WORK and SWIM.


Five-Year Evaluation of Six              In 1988, MDRC began a 5-year evaluation of California’s GAIN program in six
JOBS-Era Programs in                     counties. This six-site evaluation affords an opportunity to examine the
California                               performance of programs that varied in key ways, including program
                                         emphasis and service strategy. While all of the county programs used a
                                         mixed service strategy, including job search activities and education and
                                         training, the extent to which they emphasized rapid employment or
                                         longer-term education and training varied, as shown in table 3. The study

                                         14
                                          Daniel Friedlander and Gary Burtless, Five Years After: The Long-Term Effects of Welfare-to-Work
                                         Programs (New York, N.Y.: Russell Sage Foundation, 1995).



                                         Page 12                                           GAO/HEHS-99-179 Welfare-to-Work Approaches
B-282174




researchers noted that, among the six counties, the Riverside program
promoted the strongest message about the importance of moving quickly
into employment. The other counties’ programs placed more emphasis on
skill building, with the Alameda, Los Angeles, and Tulare programs placing
participants in education and training longer than the other counties.

The county programs and environments differed in other key ways that
can affect the operation and success of a welfare-to-work program. For
example, as shown in table 3, some counties had much higher percentages
of welfare recipients generally considered hard to employ (or less
job-ready) than other counties, as measured by the percentage of program
participants assessed by each program to be in need of basic education. A
key feature of the GAIN program at the time of the evaluation in these six
counties was its emphasis on adult basic education. Depending on
participants’ need for basic education, GAIN placed participants into groups
that were considered more job-ready and less job-ready. GAIN assigned the
job-ready participants to job search initially and, in general, assigned the
less job-ready to basic education, although sometimes these participants
were given the option to look for work or to look for work and participate
in basic education concurrently. As shown in table 3, the percentage of
participants who were enrolled in education and training ranged from 28
to 53 percent.




Page 13                             GAO/HEHS-99-179 Welfare-to-Work Approaches
                                            B-282174




Table 3: Program Emphasis and Other Features of the Six GAIN County Programs
                                                                    Participants
                                                               assessed to need Participants enrolled
                                                                     adult basic    in education and
                   Program emphasis and Net program cost              education    training activities
County program     service strategy               per persona      (percentage)         (percentage)                      Special features
Alameda             Emphasized longer-term                                                                                Targeted to
                    education and training;                                                                               long-term welfare
                    used mixed service                                                                                    recipients only
                    strategy                                 $5,597                      65                         53
Butte               Placed more emphasis                                                                                  Used more
                    on education and                                                                                      intensive case
                    training than on rapid                                                                                management than
                    employment; used mixed                                                                                other programs
                    service strategy                          2,904                      49                         28
Los Angeles         Emphasized longer-term                                                                                Targeted to
                    education and training;                                                                               long-term welfare
                    used mixed service                                                                                    recipients only
                    strategy                                  5,789                      81                         44
Riverside           Stressed rapid                                                                                        None
                    employment but
                    included short-term
                    education; used mixed
                    service strategy                          1,597                      60                         36
San Diego           Placed more emphasis                                                                                  None
                    on education and
                    training than on rapid
                    employment; used mixed
                    service strategy                          1,912                      56                         37
Tulare              Emphasized longer-term                                                                                None
                    education and training;
                    used mixed service
                    strategy                                  2,734                      65                         49
                                            Note: The information on these programs generally represents the programs and the status of
                                            their clients at the end of 3 years. The programs changed to some extent over the 5-year period.
                                            For example, some of the original nonparticipants were allowed to participate in some JOBS
                                            activities.
                                            a
                                             Represents the average cost per participant less the average cost per nonparticipant, adjusted
                                            to 1993 dollars.



                                            The six programs had mixed results for participants in increasing
                                            employment, earnings, and welfare savings, compared with
                                            nonparticipants. Of the sites with statistically significant employment
                                            increases, three—Los Angeles, Riverside, and San Diego—increased the
                                            percentage of participants who had been employed sometime during the 5
                                            years by at least 5 percentage points above the percentage for




                                            Page 14                                           GAO/HEHS-99-179 Welfare-to-Work Approaches
                                        B-282174




                                        nonparticipants. (See fig. 2.) The Riverside program had the greatest
                                        effect: the percentage of those who had held a job during the 5-year study
                                        period was 10 percentage points higher for participants than for
                                        nonparticipants (72 percent versus 62 percent).


Figure 2: Employment Rates for
Program Participants and                80   Employment Rate
Nonparticipants Over the 5-Year Study
Period in Six GAIN Counties             70


                                        60


                                        50


                                        40


                                        30


                                        20


                                        10


                                         0


                                             Alameda        Butte        Los Angeles Riverside   San Diego   Tulare


                                                       Participants

                                                       Nonparticipants




                                        Note: Employment rate differences between program participants and nonparticipants were
                                        statistically significant only in Los Angeles, Riverside, and San Diego.




                                        Two of the programs that had statistically significant effects on the
                                        employment of participants compared with nonparticipants—Riverside
                                        and San Diego—also had significant effects on earnings for their
                                        participants (see fig. 3). The Butte program did not result in increased
                                        employment but did result in increased earnings. For the programs with
                                        statistically significant differences, the total effect on earnings per
                                        participant compared with nonparticipants over the 5 years ranged from
                                        almost $3,000 in San Diego to just over $5,000 in Riverside. Four
                                        programs—Alameda, Los Angeles, Riverside, and San Diego—significantly




                                        Page 15                                          GAO/HEHS-99-179 Welfare-to-Work Approaches
                                        B-282174




                                        reduced welfare payments; these savings ranged from about $1,400 to
                                        about $2,700 per participant (see fig. 4).


Figure 3: Total Earnings of Program
Participants and Nonparticipants Over   20000     Earnings in Dollars
the 5-Year Study Period in Six GAIN
Counties                                18000

                                        16000

                                        14000

                                        12000

                                        10000

                                         8000

                                         6000

                                         4000

                                         2000

                                            0

                                                  Alameda        Butte        Los Angeles Riverside   San Diego    Tulare


                                                            Participants

                                                            Nonparticipants




                                        Note: Differences in earnings were statistically significant between program participants and
                                        nonparticipants only in Butte, Riverside, and San Diego.




                                        Page 16                                           GAO/HEHS-99-179 Welfare-to-Work Approaches
                                      B-282174




Figure 4: Average Total Welfare
Payments for Program Participants     30000     Welfare Payments in Dollars
and Nonparticipants Over the 5-Year
Study Period in Six GAIN Counties
                                      25000



                                      20000



                                      15000



                                      10000



                                       5000



                                          0

                                                Alameda        Butte          Los Angeles Riverside   San Diego   Tulare


                                                          Participants

                                                          Nonparticipants


                                      Note: Differences in welfare payments between program participants and nonparticipants were
                                      statistically significant only in Alameda, Los Angeles, Riverside, and San Diego.




                                      Because of the importance of understanding how a welfare-to-work
                                      program works for different segments of the welfare population, the GAIN
                                      evaluation also looked at results for two key subgroups: participants and
                                      nonparticipants who were job-ready (those determined not to be in need
                                      of basic education) and participants and nonparticipants who were less
                                      job-ready (those determined to be in need of education). Job-ready
                                      participants in three programs had statistically significant higher earnings
                                      or lower welfare payments when compared with job-ready
                                      nonparticipants, with the Riverside and San Diego programs having both
                                      higher earnings and lower welfare payments. These two programs had
                                      relatively large effects on earnings—almost $6,000 and more than $5,000,
                                      respectively—for the job-ready group.

                                      For those considered less job-ready, three programs—Butte, Riverside,
                                      and Tulare—had statistically significant increases in earnings, as shown in
                                      figure 5, with the effect on earnings ranging from about $2,700 in Tulare to




                                      Page 17                                             GAO/HEHS-99-179 Welfare-to-Work Approaches
B-282174




more than $5,000 in Butte. Five of the programs had significant welfare
savings, as shown in figure 6. Two programs—Butte and Riverside—had
both higher participant earnings and welfare savings. The Butte program
performed relatively well for the less job-ready with earnings of
participants $5,000 higher than those of nonparticipants and
per-participant welfare savings of about $3,900. While the Butte, Riverside,
and Tulare programs all had a statistically significant effect on participant
earnings for those needing basic education, the effects occurred later in
the study period in the Butte and Tulare programs than in the Riverside
program. This demonstrates that some programs that do not produce
results in a short time frame may do so in the longer term.

Among the six programs, participants determined in need of basic
education were generally assigned to adult basic education activities.15
The programs generally increased participation in these activities for
participants compared with nonparticipants, but a majority of those who
were in need of basic education did not obtain a general equivalency
diploma. For those programs that had higher earnings and welfare savings,
it is unclear the extent to which provision of basic education contributed
to these effects.16




15
 In the Riverside program, all participants—whether they needed education or not—were strongly
encouraged to start with job search as their first activity.
16
 For more information on adult education for welfare recipients, see Edward Pauly, The JOBS
Evaluation: Adult Education for People on AFDC: A Synthesis of Research (Washington, D.C.:
Department of Education and HHS, Dec. 1995). Also see Janet Quint, The JOBS Evaluation: Educating
Welfare Recipients for Employment and Empowerment: Case Studies of Promising Programs
(Washington, D.C.: Department of Education and HHS, 1997).



Page 18                                        GAO/HEHS-99-179 Welfare-to-Work Approaches
                                       B-282174




Figure 5: Total Earnings for Program
Participants and Nonparticipants       14000     Earnings in Dollars
Determined to Be in Need of Basic
Education Over the 5-Year Study
                                       12000
Period in Six GAIN Counties

                                       10000


                                        8000


                                        6000


                                        4000


                                        2000


                                           0

                                                 Alameda        Butte        Los Angeles Riverside   San Diego   Tulare


                                                           Participants

                                                           Nonparticipants


                                       Note: Differences between program participants and nonparticipants were statistically significant
                                       only in Butte, Riverside, and Tulare.




                                       Page 19                                            GAO/HEHS-99-179 Welfare-to-Work Approaches
                                      B-282174




Figure 6: Welfare Payments to
Program Participants and              30000     Welfare Payments in Dollars
Nonparticipants Determined to Be in
Need of Basic Education Over the
5-Year Study Period in Six GAIN       25000
Counties
                                      20000



                                      15000



                                      10000



                                       5000



                                          0

                                                Alameda        Butte          Los Angeles Riverside   San Diego   Tulare


                                                          Participants

                                                          Nonparticipants




                                      Note: Differences between program participants and nonparticipants were statistically significant
                                      in Alameda, Butte, Los Angeles, Riverside, and San Diego.




                                      Among the six programs, only the Riverside and San Diego programs had
                                      statistically significant effects in all three areas—total employment,
                                      earnings, and welfare savings—over the 5 years. In addition, the Riverside
                                      program, the least costly, achieved results in earnings and welfare savings
                                      for both the job-ready and less job-ready participants. The study
                                      researchers attributed the success of the Riverside program to a
                                      combination of factors: conveying a strong, consistent message about the
                                      importance of quick employment for participants, even for those who
                                      began the program with education and training; relying on a mixed
                                      strategy including job search, education and training, and other activities
                                      and services; enforcing participation requirements; devoting some staff to
                                      job development activities to help identify employment opportunities for




                                      Page 20                                             GAO/HEHS-99-179 Welfare-to-Work Approaches
                            B-282174




                            participants; and focusing staff on results by, for example, using
                            performance standards to measure their performance.17

                            The Riverside program is considered one of the most successful
                            large-scale mandatory welfare-to-work programs on the basis of its
                            significant effect on a range of outcomes and the fact that it reaches a
                            broad cross section of the welfare caseload. Even so, the program still did
                            not end participants’ dependence on welfare or lead to employment at
                            wages above poverty levels for many families. The 3-year results showed
                            that, after entering the program, 41 percent of participants were still
                            receiving welfare and 81 percent had income at or below the poverty line.
                            The percentage of those still receiving welfare is no doubt affected to
                            some extent by the relatively high maximum earnings levels for AFDC in
                            California at the time of the study. For example, in California in fiscal year
                            1991, a three-person family would have needed monthly earnings of $959
                            to become ineligible for AFDC.18 In a state with a relatively high earnings
                            limit such as California, many more welfare recipients are likely to
                            combine work and welfare, while in states with lower earnings limits, any
                            job, even at the minimum wage, can result in a family’s moving off welfare.


MDRC Comparison Study       In the ongoing comparison study conducted at three sites (Atlanta, Grand
Shows That Results From     Rapids, and Riverside) by MDRC as part of the National Evaluation of
Rapid Employment and        Welfare-to-Work Strategies, the welfare-to-work programs implemented at
                            the study sites were designed to provide “pure” examples of the rapid
Education-Based             employment and education-based approaches. In the rapid employment
Approaches Did Not Differ   approach, though, the program could assign participants to education and
in the Short Term           training activities if they were not able to find a job during the initial job
                            search period.

                            The rapid employment approach emphasizes quick job placement,
                            assuming that work habits and skills are better learned at a job than in a
                            classroom and that any job can be a stepping-stone to a better one.
                            Program staff only briefly assess participants assigned to this program
                            before they attend a 3- to 5-week job club. The job club includes classroom

                            17
                             The GAIN Riverside program discussed here differs somewhat from the Riverside program currently
                            being operated as a “pure” rapid employment approach as part of the MDRC comparison evaluation
                            under way. The researchers noted that the GAIN Riverside program emphasized a broader range of
                            activities for participants, placing more of them in education and training than the Riverside rapid
                            employment approach in place now.
                            18
                              We estimated that, in fiscal year 1991, the monthly amount of earnings needed for a family to no
                            longer be eligible for AFDC varied widely across the states, from a low of $385 to a high of $1,111, with
                            a median of $632. See Self Sufficiency: Opportunities and Disincentives on the Road to Economic
                            Independence (GAO/HRD-93-23, Aug. 6, 1993).



                            Page 21                                            GAO/HEHS-99-179 Welfare-to-Work Approaches
B-282174




instruction on how to look for a job and provides supervised job search.
Participants who have not found a job at the end of this period are more
fully assessed by staff to see what activities and services would help them
get a job. Depending on that assessment, participants are assigned to more
time in job club, individual job search, or short-term (up to 9 months)
basic education, vocational training, or work experience. A participant
who completes such an assignment without finding a job is assessed again
and assigned to the same or another of these activities.19

The education-based approach assumes that participants need to invest
some time in education or training before seeking employment so that
they can acquire skills that will help them get good jobs and thus leave
welfare permanently. Participants assigned to this approach first undergo
a detailed assessment by program staff to determine their job-related skills
and interests and to identify potential barriers that they face to getting
employment. Depending on the assessment, these participants are
assigned to up to 2 years of basic education, vocational training, college,
or work experience. A participant who remains unemployed after
completing the assignment is reassessed and assigned to the same or
another of these activities, a job club, or an individual job search.20

The 2-year earnings and welfare payment outcomes of both the rapid
employment and education-based program participants were significantly
better than outcomes for nonparticipants, but the outcomes of the
participants in the two approaches were not different enough from each
other to conclude that, overall, one approach is more effective than the
other. In Atlanta and Grand Rapids, there was no statistically significant
difference in the total earnings effects between the two program
approaches.21 Among participants with a high school diploma or

19
 During the study’s 2-year follow-up period, 41 percent (Riverside) to 69 percent (Atlanta) of
participants assigned to the rapid employment approach participated in job search and 8 percent
(Riverside) to 31 percent (Grand Rapids) participated in an education or training activity.
20
  There were some differences between the intended and actual sequence of program activities and
emphases. According to the researchers, assessments were generally not in-depth. For example, only
one program had an up-front assessment that was longer than a few hours—the program in Grand
Rapids, using an education-based approach, had an up-front assessment that lasted a week in a
classroom setting. In addition, at all three education-based sites, basic education was by far the most
commonly assigned first program activity, followed by vocational training. Assignments to work
experience or college were very rare. During the study’s 2-year follow-up period, 47 percent
(Riverside) to 58 percent (Grand Rapids) of participants assigned to the education-based approach
participated in an education or training activity and 12 percent (Atlanta and Riverside) to 14 percent
(Grand Rapids) of recipients participated in job search.
21
 A comparison of the outcomes of the full sample of the two groups can be made only in Atlanta and
Grand Rapids, because in Riverside the education-based approach was available only to participants
who did not have a high school diploma or equivalent or who achieved relatively low scores on basic
skills tests administered at orientation.



Page 22                                            GAO/HEHS-99-179 Welfare-to-Work Approaches
                        B-282174




                        equivalent, there were no differences between the two approaches in
                        terms of total earnings. The rapid employment approach did, however,
                        show greater effects on earnings than the education-based approach
                        among participants in Grand Rapids who did not have a high school
                        diploma or equivalent (42 to 44 percent of the total sample at the three
                        sites). In Grand Rapids, the rapid employment approach produced
                        significantly greater savings in total welfare payments than the
                        education-based approach, but that difference was not statistically
                        significant in Atlanta.

                        While both approaches produced positive outcomes, they did so at very
                        different program costs. Across the three sites, the average per-participant
                        cost for the rapid employment approach (above what was spent on
                        nonparticipants) was $1,550. The average per-participant cost for the
                        education-based approach was $3,077, nearly twice the cost of the rapid
                        employment approach.


                        While research indicates that parents’ educational attainment is positively
The Effect of Welfare   related to children’s educational attainment, little information is currently
Mothers’ Educational    available on this relationship specifically within the welfare population.
Attainment on Their     Recent studies have identified factors affecting the cognitive development
                        of children in welfare families. This research, while limited in scope,
Children’s              indicates that one factor that may positively affect children’s development
Educational             is the level of their mothers’ educational attainment. This issue has also
                        been analyzed within a body of research focusing on the effects of poverty
Attainment Is           on children’s educational attainment. In these studies, which did not
Currently Unknown       sample welfare recipients exclusively, analysis has included measuring
                        and controlling for welfare receipt in order to determine whether the
                        welfare population is different from the general population. In general,
                        these research results show a significant positive relationship between the
                        educational attainment of parents and their children among both the
                        welfare and nonwelfare populations.

                        Findings from research that focuses on the development of the children
                        within the welfare population are inconsistent. One longitudinal study of
                        614 children whose families received AFDC found that mothers’ prior
                        education corresponded to higher reading scores but not to higher math or
                        vocabulary scores for children.22 This study found no significant effect

                        22
                          Hirokazu Yoshikawa, “Welfare Dynamics, Support Services, Mothers’ Earnings, and Child Cognitive
                        Development: Implications for Contemporary Welfare Reform,” Child Development, Vol. 70, No. 3
                        (May/June 1999). The study examined mothers’ education level 1 year before the birth of the child in
                        the study.



                        Page 23                                          GAO/HEHS-99-179 Welfare-to-Work Approaches
               B-282174




               over the study period from a mother’s increasing her education over the
               first 5 years of her child’s life. As part of the National Evaluation of
               Welfare-to-Work Strategies currently being conducted, researchers will
               identify how the welfare-to-work experiences of mothers affect their
               children.23 A preliminary descriptive report on preschool-aged children in
               this evaluation found that a child’s development was associated with the
               mother’s educational attainment.24 Future reports from this evaluation will
               include school data for children approximately 5 years later. In addition to
               this evaluation, several states have included a component in evaluations of
               their welfare programs—partially funded by HHS—to look at the effects of
               various welfare reforms on children, including children’s school
               achievement.25 Although these studies will not measure the completion of
               schooling for these children, they will provide additional information on
               how mothers’ educational attainment affects the progress of their
               children’s education.

               Other applicable research, which has not analyzed welfare recipients
               exclusively, has attempted to determine whether and how the welfare
               population differs from the general population in terms of children’s
               educational attainment. These studies show that a parent’s various
               circumstances, such as economic status and educational level, have a
               significant effect on children’s educational attainment across the sample,
               even when analysis controls for welfare receipt. In all of the studies that
               provided detailed results, mothers’ educational attainment consistently
               had a significant effect on the educational attainment of their children.
               These findings suggest that a significant relationship between a mother’s
               and her children’s educational attainment may also hold true for the
               welfare population.


               Our review of research conducted over the past two decades shows that a
Concluding     welfare-to-work approach with a strong employment focus can have
Observations   positive effects on participant earnings and employment and on welfare
               costs. However, we do not yet definitively know, especially in the long
               term, whether a rapid employment or education-based approach works
               best for increasing the employment and earnings of welfare recipients and
               reducing their dependence on welfare. Future results from an ongoing

               23
                The three sites for this study are Atlanta, Ga.; Grand Rapids, Mich.; and Riverside, Calif. This study is
               contained within MDRC comparison study referred to earlier that compares the rapid employment and
               education-based welfare-to-work approaches.
               24
                Kristin A. Moore and others, The JOBS Evaluation: How Well Are They Faring, AFDC Families With
               Preschool-Aged Children in Atlanta at the Outset of the JOBS Evaluation (Washington, D.C.: HHS,
               Office of the Assistant Secretary for Planning and Evaluation, 1995).
               25
                 States include Connecticut, Iowa, Minnesota, Indiana, and Florida.
               Page 24                                            GAO/HEHS-99-179 Welfare-to-Work Approaches
                     B-282174




                     study designed specifically to compare the effectiveness of a rapid
                     employment approach with that of an education-based approach may shed
                     more light on this issue. In the meantime, our review of 10 evaluated
                     programs that had 5-year results indicates that those welfare-to-work
                     programs that combine elements of both approaches—emphasizing rapid
                     employment but tailoring services to some extent to meet the differing
                     needs of welfare recipients—may best meet the goals of increasing
                     employment and earnings for welfare recipients while at the same time
                     reducing welfare payments. These types of programs can play an
                     important role in moving welfare recipients into the labor force and
                     increasing the extent to which they rely on their own earnings rather than
                     government aid. Nevertheless, even the most successful program in
                     Riverside did not usually lead families to higher-paying jobs or move them
                     out of poverty during the time period studied. Future research will need to
                     focus on longer-term program outcomes, the effect of the 1996 welfare
                     reform legislation, and what works best for particularly hard-to-employ
                     populations.


                     We obtained comments on a draft of this report from HHS, which stated
Agency Comments      that, overall, it concurred with our findings. More specifically, HHS agreed
and Our Evaluation   that, of the welfare-to-work approaches that have been tested, programs
                     with a combined approach—emphasizing rapid employment but providing
                     education and training when appropriate—appear to be most effective.
                     HHS commended us in our selection of rigorous studies using random
                     assignment; however, HHS noted that this criterion eliminated any studies
                     regarding postsecondary education, since no rigorous evaluations have
                     been done of the effectiveness of such programs for welfare recipients.

                     HHS  made several suggestions to further clarify information contained in
                     the report. First, HHS pointed out that the costs for the WIN-era programs
                     would naturally be lower than the costs for the JOBS-era programs because
                     welfare recipients with younger children were exempt from participating
                     in the earlier programs and, consequently, costs for child care would be
                     lower. We agree with this suggestion and have included it in the report.
                     Second, HHS noted that while we focused on effects on cumulative
                     earnings over the 5-year follow-up period, which is a good measure of the
                     overall impact of a program, the effects on earnings at the end of the
                     follow-up period might be a better indicator of future earnings. We agree
                     with HHS that assessing the effects on earnings at the end of the follow-up
                     period can provide important information. However, our analysis of the
                     results of the six-county GAIN evaluation showed that the only counties



                     Page 25                              GAO/HEHS-99-179 Welfare-to-Work Approaches
B-282174




that showed statistically significant increases in earnings for the last year
of the follow-up period (Butte, Riverside, and San Diego) also showed
significant increases over the 5-year follow-up period; as a result, we did
not report separately on the effects at the end of the study time period.
Furthermore, we noted that for participants considered to be less
job-ready, effects occurred later in the study period for several counties,
indicating that some programs that do not produce results in a short time
frame may do so in the longer term. Finally, regarding the section about
the effect of welfare mothers’ educational attainment on their children’s
educational attainment, HHS noted that the report should more clearly
explain how studies documented a mother’s educational background. We
have made revisions to this section to clarify whether the education level
was documented at the onset of a study or during the study period.
However, additional information was not available on the percentage of
mothers in a study sample with increases in education during the study
period. In addition, HHS provided technical comments, which we
incorporated in the report where appropriate. HHS’ comments are included
in appendix II.

We provided a draft of this report for technical review to the Departments
of Education and Labor; we also provided a copy to two experts in welfare
issues. The Department of Education and the experts said we had
accurately characterized the research available in the field. They also
provided technical comments, which we incorporated as appropriate. The
Department of Labor had no comments.


We are sending copies of this report to the Honorable Donna E. Shalala,
Secretary of Health and Human Services; the Honorable Alexis M. Herman,
Secretary of Labor; the Honorable Richard W. Riley, Secretary of
Education; and state TANF directors. We will also make copies available to
others upon request.

If you or your staffs have any questions about this report, please call me
on (202) 512-7215. Other GAO contacts and staff acknowledgments are
listed in appendix III.




Cynthia M. Fagnoni
Director, Education, Workforce, and
  Income Security Issues
Page 26                              GAO/HEHS-99-179 Welfare-to-Work Approaches
B-282174




List of Addressees

The Honorable William V. Roth, Jr.
Chairman
The Honorable Daniel Patrick Moynihan
Ranking Minority Member
Committee on Finance
United States Senate

The Honorable James M. Jeffords
Chairman
The Honorable Edward M. Kennedy
Ranking Minority Member
Committee on Health, Education, Labor, and Pensions
United States Senate

The Honorable William F. Goodling
Chairman
The Honorable William L. Clay
Ranking Minority Member
Committee on Education and the Workforce
House of Representatives

The Honorable Bill Archer
Chairman
The Honorable Charles B. Rangel
Ranking Minority Member
Committee on Ways and Means
House of Representatives




Page 27                           GAO/HEHS-99-179 Welfare-to-Work Approaches
Contents



Letter                                                                                           1


Appendix I                                                                                      30

Scope and
Methodology
Appendix II                                                                                     32

Comments From the
Department of Health
and Human Services
Appendix III                                                                                    35

GAO Contacts and
Staff
Acknowledgments
Bibliography                                                                                    36


Related GAO Products                                                                            44


Tables                 Table 1: Evaluation Studies That Assessed the Effectiveness of            9
                         Welfare-to-Work Programs
                       Table 2: Emphasis and Other Features of Four WIN-Era Programs            11
                       Table 3: Program Emphasis and Other Features of the Six GAIN             14
                         County Programs

Figures                Figure 1: Effect on Total Earnings and Welfare Savings per               12
                         Program Participant Over a 5-Year Study Period for Four
                         WIN-Era Programs
                       Figure 2: Employment Rates for Program Participants and                  15
                         Nonparticipants Over the 5-Year Study Period in Six GAIN
                         Counties
                       Figure 3: Total Earnings of Program Participants and                     16
                         Nonparticipants Over the 5-Year Study Period in Six GAIN
                         Counties




                       Page 28                           GAO/HEHS-99-179 Welfare-to-Work Approaches
Contents




Figure 4: Average Total Welfare Payments for Program                      17
  Participants and Nonparticipants Over the 5-Year Study Period in
  Six GAIN Counties
Figure 5: Total Earnings for Program Participants and                     19
  Nonparticipants Determined to Be in Need of Basic Education
  Over the 5-Year Study Period in Six GAIN Counties
Figure 6: Welfare Payments to Program Participants and                    20
  Nonparticipants Determined to Be in Need of Basic Education
  Over the 5-Year Study Period in Six GAIN Counties




Abbreviations

AFDC       Aid to Families With Dependent Children
GAIN       Greater Avenues for Independence
HHS        Department of Health and Human Services
JOBS       Job Opportunities and Basic Skills Training
MDRC       Manpower Demonstration Research Corporation
SWIM       Saturation Work Initiative Model
TANF       Temporary Assistance for Needy Families
WIN        Work Incentive


Page 29                            GAO/HEHS-99-179 Welfare-to-Work Approaches
Appendix I

Scope and Methodology


                 To address the mandated objectives, we identified relevant evaluation
                 studies of welfare-to-work programs that help welfare recipients and other
                 low-income women with children become employed. To be included in our
                 review, evaluations had to meet the following criteria:

             •   A program could have started before 1988, but its evaluation had to have
                 been published since 1988, after the passage of the Family Support Act.
             •   A study had to measure the effect of welfare-to-work approaches on
                 employment-related outcomes such as employment, earnings, and welfare
                 payments.
             •   A study had to rigorously evaluate the program by controlling for factors
                 that could affect employment-related outcomes.

                 To identify the relevant evaluations as well as identify information on the
                 impact of welfare recipients’ educational attainment on the educational
                 attainment of their children, we searched several on-line bibliographic
                 databases. These databases included Sociological Abstracts, Social
                 SciSearch, ERIC, the Welfare Information Network, and ECONLIT. We
                 also reviewed bibliographies of research studies on these issues and
                 consulted with experts on welfare-to-work issues to identify other studies
                 we should consider. We met with officials at the Departments of Health
                 and Human Services, Labor, and Education to obtain further information
                 on pertinent evaluations.

                 We identified only one evaluation that compared the effectiveness of a
                 rapid employment approach with that of an education-based approach and
                 included this evaluation in our review. In selecting other welfare-to-work
                 evaluations for review, we included only evaluations with impacts for
                 follow-up periods of at least 5 years. Consequently, we identified for
                 review six evaluations, which evaluated a total of 13 programs. Because
                 many of the welfare recipients who participated in the welfare-to-work
                 programs evaluated had less than a high school education, the
                 education-based approaches evaluated in the studies we assessed
                 primarily provided basic education services rather than postsecondary
                 education. As a result, none of the studies evaluated the effectiveness of a
                 college education on improving employment-related outcomes for welfare
                 recipients. All the evaluations used research designs that controlled for
                 other factors that could affect outcomes. For example, participants were
                 randomly assigned to either a program group, which was subject to the
                 program being evaluated, or to a control group, which continued under a
                 previous program or no program. The experience of the control group
                 members—who, at their initiative, could use services elsewhere in the



                 Page 30                              GAO/HEHS-99-179 Welfare-to-Work Approaches
Appendix I
Scope and Methodology




community—indicates what would have happened to the program groups
in the absence of special intervention, providing a benchmark for
measuring program effects. The principal outcomes measured in the
evaluations were employment, earnings, and welfare savings. For each
outcome in each study, the researchers had compared results for the
participants receiving program services with those for participants in the
control group and identified statistically significant differences that were
deemed to be program impacts or effects. The evaluation reports
estimated the likelihood that these differences occurred by chance by
using standard tests of statistical significance. We did not independently
verify the information in the evaluation reports.

We conducted our work in accordance with generally accepted
government auditing standards between February 1999 and July 1999.




Page 31                              GAO/HEHS-99-179 Welfare-to-Work Approaches
Appendix II

Comments From the Department of Health
and Human Services




              Page 32      GAO/HEHS-99-179 Welfare-to-Work Approaches
Appendix II
Comments From the Department of Health
and Human Services




Page 33                                  GAO/HEHS-99-179 Welfare-to-Work Approaches
Appendix II
Comments From the Department of Health
and Human Services




Page 34                                  GAO/HEHS-99-179 Welfare-to-Work Approaches
Appendix III

GAO Contacts and Staff Acknowledgments


                  Gale C. Harris, (202) 512-7235
GAO Contacts      Sigurd R. Nilsen, (202) 512-7003


                  In addition to those named above, the following individuals made
Staff             important contributions to this report: Lara L. Carreon, Betty S. Clark,
Acknowledgments   Margaret A. Holmes, Denise D. Hunter, Susan A. Riedinger, and Megan V.
                  Smith.




                  Page 35                            GAO/HEHS-99-179 Welfare-to-Work Approaches
Bibliography


                        Freedman, Stephen, Daniel Friedlander, Winston Lin, and others. “The
Welfare-to-Work         GAIN Evaluation: Five-Year Impacts on Employment, Earnings and AFDC
Evaluations Reviewed    Receipt.” Working Paper 96.1, Manpower Demonstration Research
in This Report          Corporation, 1996.

                        Friedlander, Daniel. Subgroup Impacts and Performance Indicators for
                        Selected Welfare Employment Programs. New York, N.Y.: Manpower
                        Demonstration Research Corporation, 1988.

                        Friedlander, Daniel, and Gary Burtless. Five Years After: The Long-Term
                        Effects of Welfare-to-Work Programs. New York, N.Y.: Russell Sage
                        Foundation, 1995.

                        Friedlander, Daniel, and Gayle Hamilton. The Saturation Work Initiative
                        Model in San Diego: A Five-Year Follow-Up Study. New York, N.Y.:
                        Manpower Demonstration Research Corporation, 1993.

                        Hamilton, Gayle, Thomas Brock, Mary Farrell, and others. National
                        Evaluation of Welfare-to-Work Strategies: Evaluating Two
                        Welfare-to-Work Approaches, Two-Year Findings on the Labor Force
                        Attachment and Human Capital Development Programs in Three Sites.
                        Washington, D.C.: Departments of Education and Health and Human
                        Services, 1997.

                        Riccio, James, Daniel Friedlander, Stephen Freedman, and others. GAIN:
                        Benefits, Costs, and Three-Year Impacts of a Welfare-to-Work Program.
                        New York, N.Y.: Manpower Demonstration Research Corporation, 1994.


                        Auspos, Patricia, George Cave, and David Long. Maine: The Demonstration
Other Welfare-to-Work   of State Work/Welfare Initiatives: Final Report on the Training
Evaluations and         Opportunities in the Private Sector Program. New York, N.Y.: Manpower
Related Studies         Demonstration Research Corporation, 1988.

                        Bloom, Dan. After AFDC: Welfare-to-Work Choices and Challenges for
                        States. New York, N.Y.: Manpower Demonstration Research Corporation,
                        1997.

                        Bloom, Dan, Mary Farrell, James Kemple, and others. The Family
                        Transition Program: Implementation and Interim Impacts of Florida’s
                        Initial Time-Limited Welfare Program. New York, N.Y.: Manpower
                        Demonstration Research Corporation, 1998.



                        Page 36                            GAO/HEHS-99-179 Welfare-to-Work Approaches
Bibliography




Bos, Hans, Aletha Huston, Robert Granger, and others. New Hope for
People With Low Incomes: Two-Year Results of a Program to Reduce
Poverty and Reform Welfare. New York, N.Y.: Manpower Demonstration
Research Corporation, 1999.

Fein, David, Erik Beecroft, and John D. Blomquist. The Ohio Transitions to
Independence Demonstration: Final Impacts for JOBS and Work Choice.
Bethesda, Md.: Abt Associates, 1994.

Fein, David, Erik Beecroft, William Hamilton, and others. The Indiana
Welfare Reform Evaluation: Program Implementation and Economic
Impacts After Two Years. Cambridge, Mass.: Abt Associates, 1998.

Fein, David J., and Jennifer A. Karweit. The ABC Evaluation: The Early
Economic Impacts of Delaware’s A Better Chance Welfare Reform
Program. Cambridge, Mass.: Abt Associates, 1997.

Fraker, Thomas M., Lucia A. Nixon, Jonathan E. Jacobson, and others.
Iowa’s Family Investment Program: Two-Year Impacts. Washington, D.C.:
Mathematica Policy Research, Inc., 1998.

Freedman, Stephen, Jan Bryant, and George Cave. New Jersey: The
Demonstration of State Work/Welfare Initiatives, Final Report on the
Grant Diversion Project. New York, N.Y.: Manpower Demonstration
Research Corporation, 1988.

Freedman, Stephen, Marisa Mitchell, and David Navarro. “The Los Angeles
Jobs-First GAIN Evaluation: Preliminary Findings on Participation Patterns
and First-Year Impacts.” Working Paper, Manpower Demonstration
Research Corporation, 1998.

Gueron, Judith M., and Edward Pauly. From Welfare to Work. New York,
N.Y.: Russell Sage Foundation, 1991.

Kemple, James, Daniel Friedlander, and Veronica Fellerath. Florida’s
Project Independence: Benefits, Costs, and Two-Year Impacts of Florida’s
JOBS Program. New York, N.Y.: Manpower Demonstration Research
Corporation, 1995.

Long, Sharon K., Demetra Smith Nightingale, and Douglas A. Wissoker.
The Evaluation of the Washington State Family Independence Program.
Washington, D.C.: Urban Institute Press, 1994.



Page 37                             GAO/HEHS-99-179 Welfare-to-Work Approaches
Bibliography




Miller, Cynthia, Virginia Knox, Patricia Auspos, and others. Making
Welfare Work and Work Pay: Implementation and 18-Month Impacts of the
Minnesota Family Investment Program. New York, N.Y.: Manpower
Demonstration Research Corporation, 1997.

Nightingale, Demetra Smith, Douglas A. Wissoker, Lynn C. Burbridge, and
others. Evaluation of the Massachusetts Employment and Training (ET)
Program. Washington, D.C.: Urban Institute Press, 1991.

Olson, Jerome A., Deanna T. Schexnayder, Daniel P. O’Shea, and others.
Participation Patterns and Program Impacts of Hawaii’s JOBS WORKS!
Demonstration Project. Austin, Tex.: Lyndon B. Johnson School of Public
Affairs, University of Texas at Austin, 1997.

O’Neill, June. Work and Welfare in Massachusetts: An Evaluation of the ET
Program. Boston, Mass.: Pioneer Institute for Public Policy Research,
1990.

Orr, Larry L., Howard S. Bloom, Stephen H. Bell, and others. Does Training
for the Disadvantaged Work? Evidence From the National JTPA Study: An
Abt Associates Study. Washington, D.C.: Urban Institute Press, 1996.

Schexnayder, Deanna T., and Jerome A. Olson. Texas JOBS Program
Evaluation: Second Year Impacts. Austin, Tex.: Center for the Study of
Human Resources, University of Texas at Austin, 1995.

Scrivener, Susan, Gayle Hamilton, Mary Farrell, and others. National
Evaluation of Welfare-to-Work Strategies: Implementation, Participation
Patterns, Costs and Two-Year Impacts of the Portland (Oregon)
Welfare-to-Work Program. Washington, D.C.: Departments of Education
and Health and Human Services, 1998.

Werner, Alan, and Robert Kornfeld. Final Impact Report—The Evaluation
of To Strengthen Michigan Families. Cambridge, Mass.: Abt Associates,
1997.

Werner, Alan, David Rodda, Elsie Pan, and others. Final Report:
Evaluation of the Alabama Avenues to Self-Sufficiency Through
Employment and Training Services (ASSETS) Demonstration. Cambridge,
Mass.: Abt Associates, 1997.




Page 38                             GAO/HEHS-99-179 Welfare-to-Work Approaches
                         Bibliography




                         Brooks-Gunn, Jeanne, Greg J. Duncan, and Nancy Maritato. “Poor
Research Exploring       Families, Poor Outcomes: The Well-Being of Children and Youth,”
the Relationship         Consequences of Growing Up Poor, eds. Greg J. Duncan and Jeanne
Between Welfare          Brooks-Gunn. New York, N.Y.: Russell Sage Foundation, 1997.

Mothers’ Educational     Duncan, Greg J., Rachel Dunifon, Morgan Ward Doran, and W. Jean Yeung.
Attainment and That      “How Different ARE Welfare and Working Families? And Do Those
                         Differences Matter for Children’s Achievement?” Working Paper No. 2, The
of Their Children, and   Northwestern University/University of Chicago, Joint Center for Poverty
Related Studies          Research, July 1998.

                         Duncan, Greg J., and Wei-Jun J. Yeung. “Extent and Consequences of
                         Welfare Dependence Among America’s Children,” Children and Youth
                         Services Review, Vol. 17, Nos. 1 and 2 (1995), pp. 157-82.

                         Haveman, Robert, and Barbara Wolfe. “The Determinants of Children’s
                         Attainments: A Review of Methods and Findings,” Journal of Economic
                         Literature, Vol. 33 (Dec. 1995), pp. 1829-78.

                         _____. Succeeding Generations: On the Effects of Investments in Children.
                         New York, N.Y.: Russell Sage Foundation, 1994.

                         Haveman, Robert, Barbara Wolfe, and James Spaulding. “Childhood
                         Events and Circumstances Influencing High School Completion,”
                         Demography, Vol. 28, No. 1 (Feb. 1991), pp. 133-57.

                         Menaghan, Elizabeth, Susan Jekielek, Frank Mott, and Elizabeth Cooksey.
                         “Work and Family Circumstances and Child Trajectories: When (and For
                         What) Does AFDC Receipt Matter?” Working Paper No. 3, The Northwestern
                         University/University of Chicago, Joint Center for Poverty Research,
                         July 1998.

                         Moore, Kristin A., Martha J. Zaslow, Mary Jo Coiro, and others. The JOBS
                         Evaluation: How Well Are They Faring? AFDC Families With
                         Preschool-Aged Children in Atlanta at the Outset of the JOBS Evaluation.
                         Washington, D.C.: Department of Health and Human Services, Office of the
                         Assistant Secretary for Planning and Evaluation, 1995.

                         Teachman, Jay D., Kathleen M. Paasch, Randal D. Day, and Karen P.
                         Carver. “Poverty During Adolescence and Subsequent Educational
                         Attainment,” Consequences of Growing Up Poor, eds. Greg J. Duncan and
                         Jeanne Brooks-Gunn. New York, N.Y.: Russell Sage Foundation, 1997.



                         Page 39                            GAO/HEHS-99-179 Welfare-to-Work Approaches
Bibliography




Yoshikawa, Hirokazu. “Welfare Dynamics, Support Services, Mothers’
Earnings, and Child Cognitive Development: Implications for
Contemporary Welfare Reform,” Child Development, Vol. 70, No. 3
(May/June 1999), pp. 779-801.

Zill, Nicholas, Kristen A. Moore, Ellen Wolpow Smith, and others. “The
Life Circumstances and Development of Children in Welfare Families: A
Profile Based on National Survey Data,” Escape From Poverty: What
Makes a Difference for Children? eds. P. Lindsay Chase-Lansdale and
Jeanne Brooks-Gunn. New York, N.Y.: Cambridge University Press, 1995.




Page 40                           GAO/HEHS-99-179 Welfare-to-Work Approaches
Page 41   GAO/HEHS-99-179 Welfare-to-Work Approaches
Page 42   GAO/HEHS-99-179 Welfare-to-Work Approaches
Page 43   GAO/HEHS-99-179 Welfare-to-Work Approaches
Related GAO Products


              Welfare Reform: States’ Implementation Progress and Information on
              Former Recipients (GAO/T-HEHS-99-116, May 27, 1999).

              Welfare Reform: Information on Former Recipients’ Status (GAO/HEHS-99-48,
              Apr. 28, 1999).

              Welfare Reform: States’ Experiences in Providing Employment Assistance
              to TANF Clients (GAO/HEHS-99-22, Feb. 26, 1999).

              Welfare Reform: States Are Restructuring Programs to Reduce Welfare
              Dependence (GAO/HEHS-98-109, June 17, 1998).

              Welfare to Work: State Programs Have Tested Some of the Proposed
              Reforms (GAO/PEMD-95-26, July 14, 1995).

              Welfare to Work: Most AFDC Training Programs Not Emphasizing Job
              Placement (GAO/HEHS-95-113, May 19, 1995).

              Welfare to Work: Participants’ Characteristics and Services Provided in
              JOBS (GAO/HEHS-95-93, May 2, 1995).


              Welfare to Work: States Begin JOBS, but Fiscal and Other Problems May
              Impede Their Progress (GAO/HEHS-91-106, Sept. 27, 1991).




(205393)      Page 44                             GAO/HEHS-99-179 Welfare-to-Work Approaches
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order
made out to the Superintendent of Documents, when
necessary. VISA and MasterCard credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address
are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office
P.O. Box 37050
Washington, DC 20013

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any
list from the past 30 days, please call (202) 512-6000 using a
touchtone phone. A recorded menu will provide information on
how to obtain these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov




PRINTED ON    RECYCLED PAPER
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. G100
Official Business
Penalty for Private Use $300

Address Correction Requested