United States General Accounting Office GAO Report to Congressional Committees September 1999 WELFARE REFORM Assessing the Effectiveness of Various Welfare-to-Work Approaches GAO/HEHS-99-179 United States GAO General Accounting Office Washington, D.C. 20548 Health, Education, and Human Services Division B-282174 September 9, 1999 Congressional Committees The Personal Responsibility and Work Opportunity Reconciliation Act of 1996 (P.L. 104-193), enacted in August 1996, significantly changed the nation’s cash assistance program for needy families with children. Title I of the law replaced the Aid to Families With Dependent Children (AFDC) cash assistance program with fixed block grants to states to provide Temporary Assistance for Needy Families (TANF) and ended families’ entitlement to assistance. Several goals of the TANF program are specified in the law, including that of ending welfare dependence by promoting work over welfare and self-reliance over dependency. Over the years, states’ welfare-to-work programs have emphasized different goals and philosophies for moving individuals into work and have provided different types of services and activities to program participants to help them reach those goals. Programs with the goal of rapid employment emphasize quick exposure to and entry into the labor force, reflecting the belief that participants can best acquire employment-related skills when they are working, regardless of the quality of the job. These programs’ service strategies tend to rely heavily on job search activities but can make use of education and training to some extent. Other programs have the goal of skill building, often called an education-based approach, which usually involves a greater initial investment in participants’ education and occupational skills, so that when they do enter the labor market, they can obtain “good” jobs—those with higher pay, health benefits, and opportunity for advancement. The 1996 welfare reform law emphasizes the importance of moving welfare recipients into employment and gives states greater flexibility to tailor their programs to meet their own goals and needs. To help it assess how best to assist welfare recipients, the Congress directed us in the Higher Education Amendments of 1998 (P.L. 105-244) to review research on the effectiveness of various welfare-to-work approaches. Specifically, we were asked to examine the research findings on (1) the effectiveness of a rapid employment approach, an education-based approach (including adult vocational and postsecondary education), and a combination of these two approaches in improving employment-related outcomes for welfare recipients and other low-income women with children and (2) the effect of welfare recipients’ educational attainment, including postsecondary education, on the educational attainment of their children. In performing this work, we Page 1 GAO/HEHS-99-179 Welfare-to-Work Approaches B-282174 reviewed studies that had been published from 1988 through mid-1999 that assessed the effectiveness of welfare-to-work approaches by comparing outcomes related to employment and earnings as well as declines in welfare payments for those in the programs with those not in the programs. Because some welfare-to-work programs may produce results in the longer term rather than in the short term, we generally focused on evaluations having relatively long-term results (5 years). In addition, because many of the welfare recipients who participated in the welfare-to-work programs evaluated had less than a high school education, the education-based approaches evaluated in the studies we assessed primarily provided basic education services rather than postsecondary education. As a result, none of the studies evaluated the effectiveness of a college education on improving employment-related outcomes for welfare recipients. We also identified and reviewed studies about the effect of welfare recipients’ educational attainment on their children. (See app. I for a full discussion of our scope and methodology.) Research conducted to date on the effectiveness of different Results in Brief welfare-to-work approaches suggests that programs with a combined approach—including both job search assistance and some education and training—tend to be more effective over a 5-year period than either approach alone in increasing employment and earnings while reducing welfare payments. Five evaluations begun in the 1980s with 5-year results indicated that programs focusing on rapid employment and job search activities combined with education and training activities more often increased employment and earnings and reduced welfare payments, compared with programs that focused solely on job search activities or those that placed the greatest emphasis on education. In addition, preliminary results (2-year findings) from a more recent ongoing evaluation (started in 1992)—the only evaluation designed explicitly to compare the effectiveness of a rapid-employment approach with an education-based welfare-to-work approach—found that while each approach has increased participants’ employment and earnings, so far, neither approach has proven clearly better than the other. The rapid employment approach did, however, cost about half as much per person as the education-based approach. While these studies provide useful information, more needs to be known about how well different approaches are performing in the current environment created by the enactment of welfare reform in 1996, which none of these evaluations cover. Page 2 GAO/HEHS-99-179 Welfare-to-Work Approaches B-282174 While research indicates that parents’ educational attainment has a positive effect on children’s educational attainment, little information is currently available on this relationship specifically within the welfare population. Recent studies have identified factors affecting cognitive development of children in welfare families. This research, while limited in scope, indicates that a mother’s higher level of educational attainment is one factor that may positively affect children’s development. In addition, a body of research that focuses on the effects of poverty on children’s educational attainment suggests a significant positive relationship between the educational attainment of parents and their children among both the welfare and the nonwelfare populations. AFDC was created by the Social Security Act of 1935 to provide cash Background assistance to families with needy children who had been deprived of the support of one of their parents—at that time, mostly children living with widowed mothers. The program was not designed to promote employment, because, at the time, mothers were generally not expected to work outside the home. Over the past several decades, however, the public has come to believe that most welfare families should be at least partly self-supporting. Efforts to provide education, training, and job search assistance to help welfare recipients prepare for and find jobs can be traced back at least to the 1960s, when the Congress mandated that every state operate a Work Incentive (WIN) program to encourage AFDC recipients to become self-sufficient. WIN began primarily as a voluntary program focusing on job search assistance and immediate employment. Starting in 1981, WIN demonstration projects were established that gave states greater flexibility to design their own programs, and states could now require welfare recipients with children aged 6 and older to participate. Our reviews of the WIN program showed that it often served those most likely to find employment on their own rather than those less job-ready, who needed the most help to become employed. The Family Support Act of 1988 eliminated the WIN program and created the Job Opportunities and Basic Skills Training (JOBS) program. Research conducted before passage of that act showed that welfare recipients were a diverse group, making use of the AFDC program in different ways. While most who used AFDC did so for short periods of time, the majority of AFDC’s resources were devoted to providing benefits to long-term recipients. This research also identified several factors that were associated with long-term welfare dependence, including recipients’ low level of education, single-parent status, higher number of children, disability, and Page 3 GAO/HEHS-99-179 Welfare-to-Work Approaches B-282174 limited work experience.1 To better ensure that AFDC recipients received assistance that would help them avoid long-term welfare dependence, JOBS required state programs to include a broad range of services, including education and training assistance,2 and to provide financial assistance with support services such as child care and transportation. The population that could be required to participate was changed from those with children aged 6 and above to those with children aged 3 and above. In addition, for the first time, states were required to place a specified minimum percentage of nonexempt welfare recipients in education, training, and work-related activities and to target resources to long-term recipients and those considered at risk of long-term welfare dependence. The Family Support Act also called for an evaluation with a random assignment design, which would control for other factors that could affect outcomes, to assess the effectiveness of various welfare-to-work programs. The Department of Health and Human Services (HHS), with support from the Department of Education, contracted with the Manpower Demonstration Research Corporation (MDRC), a research organization that analyzes education- and employment-related programs, to conduct this evaluation, which focused on mandatory welfare-to-work programs at seven sites. This evaluation is referred to as the National Evaluation of Welfare-to-Work Strategies, formerly known as the JOBS Evaluation. In the mid-1980s, California also contracted with MDRC to conduct an evaluation of its state welfare-to-work program, called Greater Avenues for Independence (GAIN), which became the state’s JOBS program after 1988. At the beginning of the GAIN evaluation, California had about one-sixth of the nation’s AFDC caseload, and GAIN expended about 13 percent of federal JOBS funds. In response to JOBS’ increased emphasis on education and training and a general belief that these activities could help improve welfare recipients’ financial well-being, as they do for the general population, many state and local JOBS programs emphasized the provision of education and training. In our 1991 report on the implementation of JOBS, we found that almost half of the states reported a shift from an emphasis on immediate job 1 David Ellwood, Targeting “Would-Be” Long-Term Recipients of AFDC (Princeton, N.J.: Mathematica Policy Research, Inc., 1986). 2 Under JOBS, states were to assess the needs and skills of welfare recipients, prepare them for employment through education and training as needed, and place them in jobs. Federal rules specified certain activities that each state’s JOBS program was required to offer. These included education activities, job skills training, job-readiness activities, and job development and placement services. States also had to offer at least two of the four WIN activities (job search, on-the-job training, work supplementation programs, and community work experience programs). Postsecondary education was optional under the federal JOBS rules, and states could assign participants to this activity on an individual basis. Page 4 GAO/HEHS-99-179 Welfare-to-Work Approaches B-282174 placement under their previous welfare-to-work programs toward a new emphasis on long-term education or training.3 In 1995, we reported that JOBS participants nationwide were enrolled in a variety of education and training activities—postsecondary education (17 percent), high school or preparation for the general equivalency diploma (16 percent), job skills training (13 percent), adult basic or remedial education (7 percent), and English as a Second Language training (2 percent).4 Several years after the implementation of JOBS, our review of state JOBS programs nationwide showed that only about 11 percent of welfare recipients were involved in JOBS activities each month.5 In addition, AFDC caseloads rose to their highest levels ever, peaking at 5 million families in 1994. Also during the 1990s, under waivers of the federal rules, several states experimented with changes in their AFDC and JOBS programs. These changes included encouraging welfare recipients to work by allowing them to keep more of their earnings without losing welfare benefits, strengthening and more strongly enforcing work requirements, and imposing limits on the length of time a family could receive aid. To encourage and facilitate innovation by the states and to address continuing concerns among policymakers about growing welfare dependence, the Personal Responsibility and Work Opportunity Reconciliation Act of 1996, commonly referred to as welfare reform legislation, overhauled the nation’s welfare system by abolishing the AFDC program and establishing TANF block grants. Under TANF, which is administered by HHS, states are provided up to $16.8 billion each year through 2002 to provide aid to needy families with children. While the states have great flexibility to design programs that meet their own goals and needs, they must also meet several federal requirements designed to emphasize the importance of work and the temporary nature of TANF aid. TANF established stronger work requirements for those receiving aid than the requirements of its predecessor program, and the population that can be required to work now includes all parents, regardless of the ages of 3 Welfare to Work: States Begin JOBS, but Fiscal and Other Problems May Impede Their Progress (GAO/HRD-91-106, Sept. 27, 1991). 4 Welfare to Work: Participants’ Characteristics and Services Provided in JOBS (GAO/HEHS-95-93, May 2, 1995). 5 Welfare to Work: Most AFDC Training Programs Not Emphasizing Job Placement (GAO/HEHS-95-113, May 19, 1995). Page 5 GAO/HEHS-99-179 Welfare-to-Work Approaches B-282174 their children.6 In addition, states must enforce a 5-year limit (less at state option) on the length of time a family may receive federal TANF assistance.7 Our June 1998 report and other studies of TANF implementation show that many states and localities have taken steps to transform their welfare offices into job placement centers and are encouraging or requiring those seeking aid to engage in job search activities as soon as they apply.8 Along with this increased emphasis on work, welfare offices and workers are also focusing more on helping clients address and solve problems that interfere with employment. States’ implementation of more work-focused programs, undertaken under conditions of strong economic growth, has been accompanied by a 45-percent decline in the number of families receiving welfare—from a high of about 5 million families in 1994 to 2.7 million families as of December 1998.9 A nationally representative survey of families who left welfare from 1995 to 1997 found that 61 percent of former welfare recipients were working at the time of the survey, although often at low-paying jobs.10 6 The required minimum participation rate began at 25 percent in fiscal year 1997 and rises to 50 percent in fiscal year 2002. States receive credit for the degree to which their caseloads have declined since fiscal year 1996. While states have considerable flexibility in designing their welfare-to-work programs, the legislation prescribes the activities that states may count toward their work participation rate. For example, vocational education is limited as a countable work activity both in the percentage of recipients who can be engaged in vocational training and count toward the participation rate (30 percent) and the length of time a recipient can be in vocational training (up to 12 months). Moreover, unless states include it in their definition of vocational education training, the legislation does not allow postsecondary education to be counted as a work activity toward the states’ participation rate. 7 States may exempt from time limits up to 20 percent of those receiving TANF aid and may use their own funds to provide aid beyond the federal time limit. 8 Welfare Reform: States Are Restructuring Programs to Reduce Welfare Dependence (GAO/HEHS-98-109, June 17, 1998). 9 Welfare Reform: States’ Implementation Progress and Information on Former Recipients (GAO/T-HEHS-99-116, May 27, 1999). 10 Pamela Loprest, “Families Who Left Welfare: Who Are They and How Are They Doing?” Urban Institute Discussion Paper 99-02 (July 1999). See also Welfare Reform: Information on Former Recipients’ Status (GAO-HEHS-99-48, Apr. 28, 1999). Page 6 GAO/HEHS-99-179 Welfare-to-Work Approaches B-282174 The current research on the relative merits of the rapid employment and Research Shows That education-based approaches does not conclusively show that one a Variety of approach is more effective than the other in increasing welfare recipients’ Welfare-to-Work employment and earnings and reducing their welfare payments. Of the six evaluations of different approaches we identified and reviewed, five Approaches Have evaluations, with 5-year results, covered a range of programs, with some Positive combining elements of a rapid employment approach with an education-based approach. The results of these evaluations were mixed Employment-Related and, while not conclusive, indicated that programs that combined the Outcomes approaches had more positive effects—and that these effects covered a broader cross section of the welfare population—than programs that focused more exclusively on providing only job search activities or only education. Only one evaluation, part of the National Evaluation of Welfare-to-Work Strategies, directly compared the results of a rapid employment approach with those of an education-based approach. The results for the first 2 years of this ongoing study showed that the outcomes were roughly comparable for the two approaches, with both modestly increasing participants’ employment and earnings and reducing welfare payments. The rapid employment approach was only about half as costly per participant as the education approach. None of the studies evaluated the effectiveness of a college education in improving employment-related outcomes for welfare recipients. All of the evaluations we reviewed provided information on welfare-to-work programs operated in the 1980s and 1990s; none included results on programs operated since welfare reform was enacted in 1996. Currently, HHS is funding 23 studies in 20 states on welfare reforms that began under waivers of the AFDC program but that are continuing in the new welfare environment. These studies will provide more information on effective approaches for moving welfare recipients into work.11 Little Research Is Available We reviewed five evaluations conducted by MDRC that focused on That Compares the mandatory welfare-to-work programs and for which 5 years of follow-up Effectiveness of Various data were available. Because the results of the rapid employment and education-based approaches are expected to unfold in different ways, with Approaches more immediate results from the rapid employment approach and more delayed results from the education-based approach, we focused only on those evaluations with results from 5 years of follow-up. (Other studies we identified that did not include 5-year results are listed in the bibliography.) Four evaluations were begun in the early 1980s and covered a variety of 11 For more information, see Web sites http://www.acf.dhhs.gov/programs/opre/rd&e.htm and http://aspe.os.dhhs.gov/hsp/hspwelfare.htm Page 7 GAO/HEHS-99-179 Welfare-to-Work Approaches B-282174 programs across the country; the fifth evaluation, begun in the late 1980s, included six sites in California. As shown in table 1, these programs generally used some combination of the rapid employment and education-based approaches, although each tended to emphasize one approach more than the other. By randomly assigning welfare recipients to different groups—program participants and nonparticipants—evaluators can determine which changes in people’s employment, earnings, and welfare payments were due to their participation in the program. This random assignment method cannot tell analysts which particular aspects of the program caused the changes or definitively show that the program effects were caused by the particular approach used, rather than other program features. But such an evaluation can determine whether the way a program was operated at a particular site was effective. While evaluations of welfare-to-work programs have been conducted over the years, they generally have not been designed to determine the effectiveness of one particular approach compared with another. As part of the National Evaluation of Welfare-to-Work Strategies, formerly the JOBS Evaluation, required by the Family Support Act of 1988, MDRC started a largely unprecedented effort to compare the effect of two distinct types of welfare-to-work approaches—the rapid employment approach versus the education-based approach.12 Under the rapid employment approach, it is expected that individuals will move quickly into employment with immediate payoffs in increased earnings and welfare savings and the potential to earn more over time. Under the education-based approach, an initial investment in education and training is expected to pay off in the future, with increased earnings and welfare savings once the training is completed. This study, the sixth MDRC study that we reviewed, will provide up to 5 years of follow-up data and will analyze the effects of each approach on a wider array of outcomes, including those for children, in the future. Only the results from the first 2 years of this ongoing comparison study are currently available. 12 This comparison evaluation is part of the National Evaluation of Welfare-to-Work Strategies, which also includes other components, such as a study, focusing on children who were between the ages of 3 and 5 when their mothers entered the study, to measure outcomes on children’s cognitive development and academic achievement, safety and health, problem behavior and emotional well-being, and social development. In the national evaluation, over 55,000 individuals at seven sites were randomly assigned to groups that remained eligible for specific welfare-to-work programs or to groups that did not participate in these programs. In addition to the 2-year findings reported in a study of three sites, 2-year impacts are available for one site in Portland, Oreg. Also, MDRC has completed evaluations at the remaining three sites in Columbus, Ohio; Detroit, Mich.; and Oklahoma City, Okla.; however, these studies had not been published at the time of our review. Page 8 GAO/HEHS-99-179 Welfare-to-Work Approaches B-282174 Table 1: Evaluation Studies That Assessed the Effectiveness of Number of programs Basic program Welfare-to-Work Programs Evaluation Start datea with specific findings approach WIN-era programs Baltimore Options 1982 Combined results for Mixed service strategy Program 10 of 18 city welfare offices Arkansas WORK 1983 Combined results for Rapid employment, Program 2 counties with a primary focus on job search activities Virginia Employment 1983 Combined results for Rapid employment, Services Program 11 county welfare with a primary focus agencies on job search activities San Diego Saturation 1985 Combined results for Mixed service strategy Work Initiative Model 2 of 7 city welfare (SWIM) offices JOBS-era program California’s Greater 1988 Separate results for Mixed service Avenues for six counties: strategies that Independence (GAIN) Alameda; Butte; emphasize rapid Program Los Angeles; employment or an Riverside; San Diego; education-based and Tulare approach to different degrees National Evaluation of 1992 Separate results for Comparison of rapid Welfare-to-Work three sites: employment and Strategies: Evaluation Atlanta, Ga.; education-based of Two Welfare-to-Work Grand Rapids, Mich.; approaches Program Approaches Riverside, Calif. a All programs were evaluated for 5 years except for the JOBS-era evaluation of two welfare-to-work approaches—2-year results are available for this ongoing comparison study. The MDRC comparison study of two welfare-to-work approaches was conducted at three sites—Atlanta, Georgia; Grand Rapids, Michigan; and Riverside, California. At each site, welfare recipients were randomly assigned to the rapid employment program, the education-based program, or a control group. The study compared outcomes for those assigned to the program groups with outcomes for those assigned to the control group, and compared outcomes for those assigned to one program with outcomes for those assigned to the other program. Operating the two programs at the three sites simultaneously controlled for the economic and programmatic environments that could affect participants’ outcomes. Randomly assigning recipients to one of the two programs or a control group helped to eliminate any bias from, or effect of, differences in Page 9 GAO/HEHS-99-179 Welfare-to-Work Approaches B-282174 recipients’ characteristics that could affect outcomes. Differences in participant outcomes could then be attributed to the program in which they participated. In this comparison study and in other evaluations we reviewed, not all individuals assigned to a welfare-to-work program group necessarily participated in program activities; also, some individuals not assigned to a program group sought out and received services at their own initiative. In this report, hereafter, we use the term “participants” to refer to individuals assigned to a welfare-to-work program group and the term “nonparticipants” to refer to those assigned to a control group. Evaluations Showed That Although they were not designed to assess the effect of one approach Programs That Combine compared with another, the four WIN-era and one JOBS-era evaluations Approaches More Often provide insights into the effectiveness of various approaches. The five evaluations covered 10 programs—seven in California—with varying Had Positive Effects Over 5 approaches for moving welfare recipients into work, and had mixed Years Than Those That results. Some programs had effects in all three areas—employment, Emphasized One Approach earnings, and welfare savings. In some cases, a program increased earnings for those considered less job-ready but did not increase earnings for those considered job-ready. Although not definitive, the studies show that programs combining elements of both the rapid employment and education-based approaches—having a rapid employment focus but relying on education and training for some participants—tended to have a greater effect on participants’ employment and earnings, and on welfare savings, than approaches that emphasized just job search activities or just longer-term education and training. Five-Year Evaluations of Four A study of the 5-year results of four programs operated during the WIN era WIN-Era Programs offers a look at two programs that relied primarily on job search activities to move people into jobs in comparison with two programs that provided education and training in addition to job search activities (see table 2). The Arkansas and Virginia programs are generally considered examples of programs focusing on job search only that were relatively low cost and provided little in the way of education and training, although some Virginia program participants did participate in education and training opportunities but participated in such activities only slightly more than the control group. The Baltimore Options program featured an initial assessment of each participant’s needs and then provided participants a choice of activities. SWIM, in San Diego, was designed to involve as many welfare recipients as possible in ongoing activities, following a set sequence of activities: job search, work experience, and then education or training. Page 10 GAO/HEHS-99-179 Welfare-to-Work Approaches B-282174 Table 2: Emphasis and Other Features of Four WIN-Era Programs Percentage of participants who Net program were ever in Program emphasis and cost per education and Evaluated program service strategy persona training Arkansas WORK Rapid employment with job search first and 3-month unpaid work assignments $220 0 Virginia Employment Rapid employment with job Services search and some 3-month unpaid work assignments 598 12 Baltimore Options After initial assessment, participants are given some choice among job search, unpaid work assignments, or education and training 1,325 17 SWIM Involve as many participants as possible in activities for as long as possible, starting with job search, then 3-month unpaid work assignment, then some education and training 1,212 24 a Represents the average cost per participant less the average cost per nonparticipant, adjusted to 1993 dollars. Note that net program costs per person are lower for these WIN-era programs compared with the JOBS-era programs because welfare recipients with younger children were not required to participate in WIN-era programs. Consequently, child care costs were lower. The SWIM and Baltimore Options programs significantly increased total earnings for participants over the 5-year study period, while the earnings increases for the Arkansas and Virginia programs were not statistically significant. Arkansas WORK did, however, have significant savings in welfare payments.13 As shown in figure 1, only the SWIM program both produced welfare savings and increased participant earnings. Over the 5 years, the SWIM program increased total earnings of participants more than $2,000 above the total earnings increase for nonparticipants and reduced welfare payments almost $2,000 per participant. On the basis of a detailed analysis of these studies’ findings, researchers concluded that the Baltimore Options program was the only program that helped some 13 For our interpretations, we used a common significance level of 5 percent (.05) or less, which was stricter than that used in some evaluations. Total earnings impact represents the difference between the total average earnings between the program participants and nonparticipants. Welfare savings represent the differences between the total average welfare payments for participants and nonparticipants and do not take into account the costs of the program. In averaging the earnings or savings, those individuals with a zero value were included. The average earnings of only those with earnings would be higher. Page 11 GAO/HEHS-99-179 Welfare-to-Work Approaches B-282174 participants find higher-paying jobs than the jobs nonparticipants found. The programs that relied primarily on job search generally resulted in those participants who would have worked eventually beginning to work sooner, and motivated some participants who would not have worked to get a job. For the near term, this resulted in participants relying more on their own earnings than on welfare, although it did not increase their overall financial well-being or increase their earnings capacity.14 Figure 1: Effect on Total Earnings and Welfare Savings per Program 2500 Difference in Dollars Participant Over a 5-Year Study Period for Four WIN-Era Programs 2000 1500 1000 500 0 Arkansas Baltimore SWIM Virginia Earnings Increase Welfare Savings Note: Earnings differences between program participants and nonparticipants were statistically significant only for Baltimore Options and SWIM; welfare savings were statistically significant only for Arkansas WORK and SWIM. Five-Year Evaluation of Six In 1988, MDRC began a 5-year evaluation of California’s GAIN program in six JOBS-Era Programs in counties. This six-site evaluation affords an opportunity to examine the California performance of programs that varied in key ways, including program emphasis and service strategy. While all of the county programs used a mixed service strategy, including job search activities and education and training, the extent to which they emphasized rapid employment or longer-term education and training varied, as shown in table 3. The study 14 Daniel Friedlander and Gary Burtless, Five Years After: The Long-Term Effects of Welfare-to-Work Programs (New York, N.Y.: Russell Sage Foundation, 1995). Page 12 GAO/HEHS-99-179 Welfare-to-Work Approaches B-282174 researchers noted that, among the six counties, the Riverside program promoted the strongest message about the importance of moving quickly into employment. The other counties’ programs placed more emphasis on skill building, with the Alameda, Los Angeles, and Tulare programs placing participants in education and training longer than the other counties. The county programs and environments differed in other key ways that can affect the operation and success of a welfare-to-work program. For example, as shown in table 3, some counties had much higher percentages of welfare recipients generally considered hard to employ (or less job-ready) than other counties, as measured by the percentage of program participants assessed by each program to be in need of basic education. A key feature of the GAIN program at the time of the evaluation in these six counties was its emphasis on adult basic education. Depending on participants’ need for basic education, GAIN placed participants into groups that were considered more job-ready and less job-ready. GAIN assigned the job-ready participants to job search initially and, in general, assigned the less job-ready to basic education, although sometimes these participants were given the option to look for work or to look for work and participate in basic education concurrently. As shown in table 3, the percentage of participants who were enrolled in education and training ranged from 28 to 53 percent. Page 13 GAO/HEHS-99-179 Welfare-to-Work Approaches B-282174 Table 3: Program Emphasis and Other Features of the Six GAIN County Programs Participants assessed to need Participants enrolled adult basic in education and Program emphasis and Net program cost education training activities County program service strategy per persona (percentage) (percentage) Special features Alameda Emphasized longer-term Targeted to education and training; long-term welfare used mixed service recipients only strategy $5,597 65 53 Butte Placed more emphasis Used more on education and intensive case training than on rapid management than employment; used mixed other programs service strategy 2,904 49 28 Los Angeles Emphasized longer-term Targeted to education and training; long-term welfare used mixed service recipients only strategy 5,789 81 44 Riverside Stressed rapid None employment but included short-term education; used mixed service strategy 1,597 60 36 San Diego Placed more emphasis None on education and training than on rapid employment; used mixed service strategy 1,912 56 37 Tulare Emphasized longer-term None education and training; used mixed service strategy 2,734 65 49 Note: The information on these programs generally represents the programs and the status of their clients at the end of 3 years. The programs changed to some extent over the 5-year period. For example, some of the original nonparticipants were allowed to participate in some JOBS activities. a Represents the average cost per participant less the average cost per nonparticipant, adjusted to 1993 dollars. The six programs had mixed results for participants in increasing employment, earnings, and welfare savings, compared with nonparticipants. Of the sites with statistically significant employment increases, three—Los Angeles, Riverside, and San Diego—increased the percentage of participants who had been employed sometime during the 5 years by at least 5 percentage points above the percentage for Page 14 GAO/HEHS-99-179 Welfare-to-Work Approaches B-282174 nonparticipants. (See fig. 2.) The Riverside program had the greatest effect: the percentage of those who had held a job during the 5-year study period was 10 percentage points higher for participants than for nonparticipants (72 percent versus 62 percent). Figure 2: Employment Rates for Program Participants and 80 Employment Rate Nonparticipants Over the 5-Year Study Period in Six GAIN Counties 70 60 50 40 30 20 10 0 Alameda Butte Los Angeles Riverside San Diego Tulare Participants Nonparticipants Note: Employment rate differences between program participants and nonparticipants were statistically significant only in Los Angeles, Riverside, and San Diego. Two of the programs that had statistically significant effects on the employment of participants compared with nonparticipants—Riverside and San Diego—also had significant effects on earnings for their participants (see fig. 3). The Butte program did not result in increased employment but did result in increased earnings. For the programs with statistically significant differences, the total effect on earnings per participant compared with nonparticipants over the 5 years ranged from almost $3,000 in San Diego to just over $5,000 in Riverside. Four programs—Alameda, Los Angeles, Riverside, and San Diego—significantly Page 15 GAO/HEHS-99-179 Welfare-to-Work Approaches B-282174 reduced welfare payments; these savings ranged from about $1,400 to about $2,700 per participant (see fig. 4). Figure 3: Total Earnings of Program Participants and Nonparticipants Over 20000 Earnings in Dollars the 5-Year Study Period in Six GAIN Counties 18000 16000 14000 12000 10000 8000 6000 4000 2000 0 Alameda Butte Los Angeles Riverside San Diego Tulare Participants Nonparticipants Note: Differences in earnings were statistically significant between program participants and nonparticipants only in Butte, Riverside, and San Diego. Page 16 GAO/HEHS-99-179 Welfare-to-Work Approaches B-282174 Figure 4: Average Total Welfare Payments for Program Participants 30000 Welfare Payments in Dollars and Nonparticipants Over the 5-Year Study Period in Six GAIN Counties 25000 20000 15000 10000 5000 0 Alameda Butte Los Angeles Riverside San Diego Tulare Participants Nonparticipants Note: Differences in welfare payments between program participants and nonparticipants were statistically significant only in Alameda, Los Angeles, Riverside, and San Diego. Because of the importance of understanding how a welfare-to-work program works for different segments of the welfare population, the GAIN evaluation also looked at results for two key subgroups: participants and nonparticipants who were job-ready (those determined not to be in need of basic education) and participants and nonparticipants who were less job-ready (those determined to be in need of education). Job-ready participants in three programs had statistically significant higher earnings or lower welfare payments when compared with job-ready nonparticipants, with the Riverside and San Diego programs having both higher earnings and lower welfare payments. These two programs had relatively large effects on earnings—almost $6,000 and more than $5,000, respectively—for the job-ready group. For those considered less job-ready, three programs—Butte, Riverside, and Tulare—had statistically significant increases in earnings, as shown in figure 5, with the effect on earnings ranging from about $2,700 in Tulare to Page 17 GAO/HEHS-99-179 Welfare-to-Work Approaches B-282174 more than $5,000 in Butte. Five of the programs had significant welfare savings, as shown in figure 6. Two programs—Butte and Riverside—had both higher participant earnings and welfare savings. The Butte program performed relatively well for the less job-ready with earnings of participants $5,000 higher than those of nonparticipants and per-participant welfare savings of about $3,900. While the Butte, Riverside, and Tulare programs all had a statistically significant effect on participant earnings for those needing basic education, the effects occurred later in the study period in the Butte and Tulare programs than in the Riverside program. This demonstrates that some programs that do not produce results in a short time frame may do so in the longer term. Among the six programs, participants determined in need of basic education were generally assigned to adult basic education activities.15 The programs generally increased participation in these activities for participants compared with nonparticipants, but a majority of those who were in need of basic education did not obtain a general equivalency diploma. For those programs that had higher earnings and welfare savings, it is unclear the extent to which provision of basic education contributed to these effects.16 15 In the Riverside program, all participants—whether they needed education or not—were strongly encouraged to start with job search as their first activity. 16 For more information on adult education for welfare recipients, see Edward Pauly, The JOBS Evaluation: Adult Education for People on AFDC: A Synthesis of Research (Washington, D.C.: Department of Education and HHS, Dec. 1995). Also see Janet Quint, The JOBS Evaluation: Educating Welfare Recipients for Employment and Empowerment: Case Studies of Promising Programs (Washington, D.C.: Department of Education and HHS, 1997). Page 18 GAO/HEHS-99-179 Welfare-to-Work Approaches B-282174 Figure 5: Total Earnings for Program Participants and Nonparticipants 14000 Earnings in Dollars Determined to Be in Need of Basic Education Over the 5-Year Study 12000 Period in Six GAIN Counties 10000 8000 6000 4000 2000 0 Alameda Butte Los Angeles Riverside San Diego Tulare Participants Nonparticipants Note: Differences between program participants and nonparticipants were statistically significant only in Butte, Riverside, and Tulare. Page 19 GAO/HEHS-99-179 Welfare-to-Work Approaches B-282174 Figure 6: Welfare Payments to Program Participants and 30000 Welfare Payments in Dollars Nonparticipants Determined to Be in Need of Basic Education Over the 5-Year Study Period in Six GAIN 25000 Counties 20000 15000 10000 5000 0 Alameda Butte Los Angeles Riverside San Diego Tulare Participants Nonparticipants Note: Differences between program participants and nonparticipants were statistically significant in Alameda, Butte, Los Angeles, Riverside, and San Diego. Among the six programs, only the Riverside and San Diego programs had statistically significant effects in all three areas—total employment, earnings, and welfare savings—over the 5 years. In addition, the Riverside program, the least costly, achieved results in earnings and welfare savings for both the job-ready and less job-ready participants. The study researchers attributed the success of the Riverside program to a combination of factors: conveying a strong, consistent message about the importance of quick employment for participants, even for those who began the program with education and training; relying on a mixed strategy including job search, education and training, and other activities and services; enforcing participation requirements; devoting some staff to job development activities to help identify employment opportunities for Page 20 GAO/HEHS-99-179 Welfare-to-Work Approaches B-282174 participants; and focusing staff on results by, for example, using performance standards to measure their performance.17 The Riverside program is considered one of the most successful large-scale mandatory welfare-to-work programs on the basis of its significant effect on a range of outcomes and the fact that it reaches a broad cross section of the welfare caseload. Even so, the program still did not end participants’ dependence on welfare or lead to employment at wages above poverty levels for many families. The 3-year results showed that, after entering the program, 41 percent of participants were still receiving welfare and 81 percent had income at or below the poverty line. The percentage of those still receiving welfare is no doubt affected to some extent by the relatively high maximum earnings levels for AFDC in California at the time of the study. For example, in California in fiscal year 1991, a three-person family would have needed monthly earnings of $959 to become ineligible for AFDC.18 In a state with a relatively high earnings limit such as California, many more welfare recipients are likely to combine work and welfare, while in states with lower earnings limits, any job, even at the minimum wage, can result in a family’s moving off welfare. MDRC Comparison Study In the ongoing comparison study conducted at three sites (Atlanta, Grand Shows That Results From Rapids, and Riverside) by MDRC as part of the National Evaluation of Rapid Employment and Welfare-to-Work Strategies, the welfare-to-work programs implemented at the study sites were designed to provide “pure” examples of the rapid Education-Based employment and education-based approaches. In the rapid employment Approaches Did Not Differ approach, though, the program could assign participants to education and in the Short Term training activities if they were not able to find a job during the initial job search period. The rapid employment approach emphasizes quick job placement, assuming that work habits and skills are better learned at a job than in a classroom and that any job can be a stepping-stone to a better one. Program staff only briefly assess participants assigned to this program before they attend a 3- to 5-week job club. The job club includes classroom 17 The GAIN Riverside program discussed here differs somewhat from the Riverside program currently being operated as a “pure” rapid employment approach as part of the MDRC comparison evaluation under way. The researchers noted that the GAIN Riverside program emphasized a broader range of activities for participants, placing more of them in education and training than the Riverside rapid employment approach in place now. 18 We estimated that, in fiscal year 1991, the monthly amount of earnings needed for a family to no longer be eligible for AFDC varied widely across the states, from a low of $385 to a high of $1,111, with a median of $632. See Self Sufficiency: Opportunities and Disincentives on the Road to Economic Independence (GAO/HRD-93-23, Aug. 6, 1993). Page 21 GAO/HEHS-99-179 Welfare-to-Work Approaches B-282174 instruction on how to look for a job and provides supervised job search. Participants who have not found a job at the end of this period are more fully assessed by staff to see what activities and services would help them get a job. Depending on that assessment, participants are assigned to more time in job club, individual job search, or short-term (up to 9 months) basic education, vocational training, or work experience. A participant who completes such an assignment without finding a job is assessed again and assigned to the same or another of these activities.19 The education-based approach assumes that participants need to invest some time in education or training before seeking employment so that they can acquire skills that will help them get good jobs and thus leave welfare permanently. Participants assigned to this approach first undergo a detailed assessment by program staff to determine their job-related skills and interests and to identify potential barriers that they face to getting employment. Depending on the assessment, these participants are assigned to up to 2 years of basic education, vocational training, college, or work experience. A participant who remains unemployed after completing the assignment is reassessed and assigned to the same or another of these activities, a job club, or an individual job search.20 The 2-year earnings and welfare payment outcomes of both the rapid employment and education-based program participants were significantly better than outcomes for nonparticipants, but the outcomes of the participants in the two approaches were not different enough from each other to conclude that, overall, one approach is more effective than the other. In Atlanta and Grand Rapids, there was no statistically significant difference in the total earnings effects between the two program approaches.21 Among participants with a high school diploma or 19 During the study’s 2-year follow-up period, 41 percent (Riverside) to 69 percent (Atlanta) of participants assigned to the rapid employment approach participated in job search and 8 percent (Riverside) to 31 percent (Grand Rapids) participated in an education or training activity. 20 There were some differences between the intended and actual sequence of program activities and emphases. According to the researchers, assessments were generally not in-depth. For example, only one program had an up-front assessment that was longer than a few hours—the program in Grand Rapids, using an education-based approach, had an up-front assessment that lasted a week in a classroom setting. In addition, at all three education-based sites, basic education was by far the most commonly assigned first program activity, followed by vocational training. Assignments to work experience or college were very rare. During the study’s 2-year follow-up period, 47 percent (Riverside) to 58 percent (Grand Rapids) of participants assigned to the education-based approach participated in an education or training activity and 12 percent (Atlanta and Riverside) to 14 percent (Grand Rapids) of recipients participated in job search. 21 A comparison of the outcomes of the full sample of the two groups can be made only in Atlanta and Grand Rapids, because in Riverside the education-based approach was available only to participants who did not have a high school diploma or equivalent or who achieved relatively low scores on basic skills tests administered at orientation. Page 22 GAO/HEHS-99-179 Welfare-to-Work Approaches B-282174 equivalent, there were no differences between the two approaches in terms of total earnings. The rapid employment approach did, however, show greater effects on earnings than the education-based approach among participants in Grand Rapids who did not have a high school diploma or equivalent (42 to 44 percent of the total sample at the three sites). In Grand Rapids, the rapid employment approach produced significantly greater savings in total welfare payments than the education-based approach, but that difference was not statistically significant in Atlanta. While both approaches produced positive outcomes, they did so at very different program costs. Across the three sites, the average per-participant cost for the rapid employment approach (above what was spent on nonparticipants) was $1,550. The average per-participant cost for the education-based approach was $3,077, nearly twice the cost of the rapid employment approach. While research indicates that parents’ educational attainment is positively The Effect of Welfare related to children’s educational attainment, little information is currently Mothers’ Educational available on this relationship specifically within the welfare population. Attainment on Their Recent studies have identified factors affecting the cognitive development of children in welfare families. This research, while limited in scope, Children’s indicates that one factor that may positively affect children’s development Educational is the level of their mothers’ educational attainment. This issue has also been analyzed within a body of research focusing on the effects of poverty Attainment Is on children’s educational attainment. In these studies, which did not Currently Unknown sample welfare recipients exclusively, analysis has included measuring and controlling for welfare receipt in order to determine whether the welfare population is different from the general population. In general, these research results show a significant positive relationship between the educational attainment of parents and their children among both the welfare and nonwelfare populations. Findings from research that focuses on the development of the children within the welfare population are inconsistent. One longitudinal study of 614 children whose families received AFDC found that mothers’ prior education corresponded to higher reading scores but not to higher math or vocabulary scores for children.22 This study found no significant effect 22 Hirokazu Yoshikawa, “Welfare Dynamics, Support Services, Mothers’ Earnings, and Child Cognitive Development: Implications for Contemporary Welfare Reform,” Child Development, Vol. 70, No. 3 (May/June 1999). The study examined mothers’ education level 1 year before the birth of the child in the study. Page 23 GAO/HEHS-99-179 Welfare-to-Work Approaches B-282174 over the study period from a mother’s increasing her education over the first 5 years of her child’s life. As part of the National Evaluation of Welfare-to-Work Strategies currently being conducted, researchers will identify how the welfare-to-work experiences of mothers affect their children.23 A preliminary descriptive report on preschool-aged children in this evaluation found that a child’s development was associated with the mother’s educational attainment.24 Future reports from this evaluation will include school data for children approximately 5 years later. In addition to this evaluation, several states have included a component in evaluations of their welfare programs—partially funded by HHS—to look at the effects of various welfare reforms on children, including children’s school achievement.25 Although these studies will not measure the completion of schooling for these children, they will provide additional information on how mothers’ educational attainment affects the progress of their children’s education. Other applicable research, which has not analyzed welfare recipients exclusively, has attempted to determine whether and how the welfare population differs from the general population in terms of children’s educational attainment. These studies show that a parent’s various circumstances, such as economic status and educational level, have a significant effect on children’s educational attainment across the sample, even when analysis controls for welfare receipt. In all of the studies that provided detailed results, mothers’ educational attainment consistently had a significant effect on the educational attainment of their children. These findings suggest that a significant relationship between a mother’s and her children’s educational attainment may also hold true for the welfare population. Our review of research conducted over the past two decades shows that a Concluding welfare-to-work approach with a strong employment focus can have Observations positive effects on participant earnings and employment and on welfare costs. However, we do not yet definitively know, especially in the long term, whether a rapid employment or education-based approach works best for increasing the employment and earnings of welfare recipients and reducing their dependence on welfare. Future results from an ongoing 23 The three sites for this study are Atlanta, Ga.; Grand Rapids, Mich.; and Riverside, Calif. This study is contained within MDRC comparison study referred to earlier that compares the rapid employment and education-based welfare-to-work approaches. 24 Kristin A. Moore and others, The JOBS Evaluation: How Well Are They Faring, AFDC Families With Preschool-Aged Children in Atlanta at the Outset of the JOBS Evaluation (Washington, D.C.: HHS, Office of the Assistant Secretary for Planning and Evaluation, 1995). 25 States include Connecticut, Iowa, Minnesota, Indiana, and Florida. Page 24 GAO/HEHS-99-179 Welfare-to-Work Approaches B-282174 study designed specifically to compare the effectiveness of a rapid employment approach with that of an education-based approach may shed more light on this issue. In the meantime, our review of 10 evaluated programs that had 5-year results indicates that those welfare-to-work programs that combine elements of both approaches—emphasizing rapid employment but tailoring services to some extent to meet the differing needs of welfare recipients—may best meet the goals of increasing employment and earnings for welfare recipients while at the same time reducing welfare payments. These types of programs can play an important role in moving welfare recipients into the labor force and increasing the extent to which they rely on their own earnings rather than government aid. Nevertheless, even the most successful program in Riverside did not usually lead families to higher-paying jobs or move them out of poverty during the time period studied. Future research will need to focus on longer-term program outcomes, the effect of the 1996 welfare reform legislation, and what works best for particularly hard-to-employ populations. We obtained comments on a draft of this report from HHS, which stated Agency Comments that, overall, it concurred with our findings. More specifically, HHS agreed and Our Evaluation that, of the welfare-to-work approaches that have been tested, programs with a combined approach—emphasizing rapid employment but providing education and training when appropriate—appear to be most effective. HHS commended us in our selection of rigorous studies using random assignment; however, HHS noted that this criterion eliminated any studies regarding postsecondary education, since no rigorous evaluations have been done of the effectiveness of such programs for welfare recipients. HHS made several suggestions to further clarify information contained in the report. First, HHS pointed out that the costs for the WIN-era programs would naturally be lower than the costs for the JOBS-era programs because welfare recipients with younger children were exempt from participating in the earlier programs and, consequently, costs for child care would be lower. We agree with this suggestion and have included it in the report. Second, HHS noted that while we focused on effects on cumulative earnings over the 5-year follow-up period, which is a good measure of the overall impact of a program, the effects on earnings at the end of the follow-up period might be a better indicator of future earnings. We agree with HHS that assessing the effects on earnings at the end of the follow-up period can provide important information. However, our analysis of the results of the six-county GAIN evaluation showed that the only counties Page 25 GAO/HEHS-99-179 Welfare-to-Work Approaches B-282174 that showed statistically significant increases in earnings for the last year of the follow-up period (Butte, Riverside, and San Diego) also showed significant increases over the 5-year follow-up period; as a result, we did not report separately on the effects at the end of the study time period. Furthermore, we noted that for participants considered to be less job-ready, effects occurred later in the study period for several counties, indicating that some programs that do not produce results in a short time frame may do so in the longer term. Finally, regarding the section about the effect of welfare mothers’ educational attainment on their children’s educational attainment, HHS noted that the report should more clearly explain how studies documented a mother’s educational background. We have made revisions to this section to clarify whether the education level was documented at the onset of a study or during the study period. However, additional information was not available on the percentage of mothers in a study sample with increases in education during the study period. In addition, HHS provided technical comments, which we incorporated in the report where appropriate. HHS’ comments are included in appendix II. We provided a draft of this report for technical review to the Departments of Education and Labor; we also provided a copy to two experts in welfare issues. The Department of Education and the experts said we had accurately characterized the research available in the field. They also provided technical comments, which we incorporated as appropriate. The Department of Labor had no comments. We are sending copies of this report to the Honorable Donna E. Shalala, Secretary of Health and Human Services; the Honorable Alexis M. Herman, Secretary of Labor; the Honorable Richard W. Riley, Secretary of Education; and state TANF directors. We will also make copies available to others upon request. If you or your staffs have any questions about this report, please call me on (202) 512-7215. Other GAO contacts and staff acknowledgments are listed in appendix III. Cynthia M. Fagnoni Director, Education, Workforce, and Income Security Issues Page 26 GAO/HEHS-99-179 Welfare-to-Work Approaches B-282174 List of Addressees The Honorable William V. Roth, Jr. Chairman The Honorable Daniel Patrick Moynihan Ranking Minority Member Committee on Finance United States Senate The Honorable James M. Jeffords Chairman The Honorable Edward M. Kennedy Ranking Minority Member Committee on Health, Education, Labor, and Pensions United States Senate The Honorable William F. Goodling Chairman The Honorable William L. Clay Ranking Minority Member Committee on Education and the Workforce House of Representatives The Honorable Bill Archer Chairman The Honorable Charles B. Rangel Ranking Minority Member Committee on Ways and Means House of Representatives Page 27 GAO/HEHS-99-179 Welfare-to-Work Approaches Contents Letter 1 Appendix I 30 Scope and Methodology Appendix II 32 Comments From the Department of Health and Human Services Appendix III 35 GAO Contacts and Staff Acknowledgments Bibliography 36 Related GAO Products 44 Tables Table 1: Evaluation Studies That Assessed the Effectiveness of 9 Welfare-to-Work Programs Table 2: Emphasis and Other Features of Four WIN-Era Programs 11 Table 3: Program Emphasis and Other Features of the Six GAIN 14 County Programs Figures Figure 1: Effect on Total Earnings and Welfare Savings per 12 Program Participant Over a 5-Year Study Period for Four WIN-Era Programs Figure 2: Employment Rates for Program Participants and 15 Nonparticipants Over the 5-Year Study Period in Six GAIN Counties Figure 3: Total Earnings of Program Participants and 16 Nonparticipants Over the 5-Year Study Period in Six GAIN Counties Page 28 GAO/HEHS-99-179 Welfare-to-Work Approaches Contents Figure 4: Average Total Welfare Payments for Program 17 Participants and Nonparticipants Over the 5-Year Study Period in Six GAIN Counties Figure 5: Total Earnings for Program Participants and 19 Nonparticipants Determined to Be in Need of Basic Education Over the 5-Year Study Period in Six GAIN Counties Figure 6: Welfare Payments to Program Participants and 20 Nonparticipants Determined to Be in Need of Basic Education Over the 5-Year Study Period in Six GAIN Counties Abbreviations AFDC Aid to Families With Dependent Children GAIN Greater Avenues for Independence HHS Department of Health and Human Services JOBS Job Opportunities and Basic Skills Training MDRC Manpower Demonstration Research Corporation SWIM Saturation Work Initiative Model TANF Temporary Assistance for Needy Families WIN Work Incentive Page 29 GAO/HEHS-99-179 Welfare-to-Work Approaches Appendix I Scope and Methodology To address the mandated objectives, we identified relevant evaluation studies of welfare-to-work programs that help welfare recipients and other low-income women with children become employed. To be included in our review, evaluations had to meet the following criteria: • A program could have started before 1988, but its evaluation had to have been published since 1988, after the passage of the Family Support Act. • A study had to measure the effect of welfare-to-work approaches on employment-related outcomes such as employment, earnings, and welfare payments. • A study had to rigorously evaluate the program by controlling for factors that could affect employment-related outcomes. To identify the relevant evaluations as well as identify information on the impact of welfare recipients’ educational attainment on the educational attainment of their children, we searched several on-line bibliographic databases. These databases included Sociological Abstracts, Social SciSearch, ERIC, the Welfare Information Network, and ECONLIT. We also reviewed bibliographies of research studies on these issues and consulted with experts on welfare-to-work issues to identify other studies we should consider. We met with officials at the Departments of Health and Human Services, Labor, and Education to obtain further information on pertinent evaluations. We identified only one evaluation that compared the effectiveness of a rapid employment approach with that of an education-based approach and included this evaluation in our review. In selecting other welfare-to-work evaluations for review, we included only evaluations with impacts for follow-up periods of at least 5 years. Consequently, we identified for review six evaluations, which evaluated a total of 13 programs. Because many of the welfare recipients who participated in the welfare-to-work programs evaluated had less than a high school education, the education-based approaches evaluated in the studies we assessed primarily provided basic education services rather than postsecondary education. As a result, none of the studies evaluated the effectiveness of a college education on improving employment-related outcomes for welfare recipients. All the evaluations used research designs that controlled for other factors that could affect outcomes. For example, participants were randomly assigned to either a program group, which was subject to the program being evaluated, or to a control group, which continued under a previous program or no program. The experience of the control group members—who, at their initiative, could use services elsewhere in the Page 30 GAO/HEHS-99-179 Welfare-to-Work Approaches Appendix I Scope and Methodology community—indicates what would have happened to the program groups in the absence of special intervention, providing a benchmark for measuring program effects. The principal outcomes measured in the evaluations were employment, earnings, and welfare savings. For each outcome in each study, the researchers had compared results for the participants receiving program services with those for participants in the control group and identified statistically significant differences that were deemed to be program impacts or effects. The evaluation reports estimated the likelihood that these differences occurred by chance by using standard tests of statistical significance. We did not independently verify the information in the evaluation reports. We conducted our work in accordance with generally accepted government auditing standards between February 1999 and July 1999. Page 31 GAO/HEHS-99-179 Welfare-to-Work Approaches Appendix II Comments From the Department of Health and Human Services Page 32 GAO/HEHS-99-179 Welfare-to-Work Approaches Appendix II Comments From the Department of Health and Human Services Page 33 GAO/HEHS-99-179 Welfare-to-Work Approaches Appendix II Comments From the Department of Health and Human Services Page 34 GAO/HEHS-99-179 Welfare-to-Work Approaches Appendix III GAO Contacts and Staff Acknowledgments Gale C. Harris, (202) 512-7235 GAO Contacts Sigurd R. Nilsen, (202) 512-7003 In addition to those named above, the following individuals made Staff important contributions to this report: Lara L. Carreon, Betty S. Clark, Acknowledgments Margaret A. Holmes, Denise D. Hunter, Susan A. Riedinger, and Megan V. Smith. Page 35 GAO/HEHS-99-179 Welfare-to-Work Approaches Bibliography Freedman, Stephen, Daniel Friedlander, Winston Lin, and others. “The Welfare-to-Work GAIN Evaluation: Five-Year Impacts on Employment, Earnings and AFDC Evaluations Reviewed Receipt.” Working Paper 96.1, Manpower Demonstration Research in This Report Corporation, 1996. Friedlander, Daniel. Subgroup Impacts and Performance Indicators for Selected Welfare Employment Programs. New York, N.Y.: Manpower Demonstration Research Corporation, 1988. Friedlander, Daniel, and Gary Burtless. Five Years After: The Long-Term Effects of Welfare-to-Work Programs. New York, N.Y.: Russell Sage Foundation, 1995. Friedlander, Daniel, and Gayle Hamilton. The Saturation Work Initiative Model in San Diego: A Five-Year Follow-Up Study. New York, N.Y.: Manpower Demonstration Research Corporation, 1993. Hamilton, Gayle, Thomas Brock, Mary Farrell, and others. National Evaluation of Welfare-to-Work Strategies: Evaluating Two Welfare-to-Work Approaches, Two-Year Findings on the Labor Force Attachment and Human Capital Development Programs in Three Sites. Washington, D.C.: Departments of Education and Health and Human Services, 1997. Riccio, James, Daniel Friedlander, Stephen Freedman, and others. GAIN: Benefits, Costs, and Three-Year Impacts of a Welfare-to-Work Program. New York, N.Y.: Manpower Demonstration Research Corporation, 1994. Auspos, Patricia, George Cave, and David Long. Maine: The Demonstration Other Welfare-to-Work of State Work/Welfare Initiatives: Final Report on the Training Evaluations and Opportunities in the Private Sector Program. New York, N.Y.: Manpower Related Studies Demonstration Research Corporation, 1988. Bloom, Dan. After AFDC: Welfare-to-Work Choices and Challenges for States. New York, N.Y.: Manpower Demonstration Research Corporation, 1997. Bloom, Dan, Mary Farrell, James Kemple, and others. The Family Transition Program: Implementation and Interim Impacts of Florida’s Initial Time-Limited Welfare Program. New York, N.Y.: Manpower Demonstration Research Corporation, 1998. Page 36 GAO/HEHS-99-179 Welfare-to-Work Approaches Bibliography Bos, Hans, Aletha Huston, Robert Granger, and others. New Hope for People With Low Incomes: Two-Year Results of a Program to Reduce Poverty and Reform Welfare. New York, N.Y.: Manpower Demonstration Research Corporation, 1999. Fein, David, Erik Beecroft, and John D. Blomquist. The Ohio Transitions to Independence Demonstration: Final Impacts for JOBS and Work Choice. Bethesda, Md.: Abt Associates, 1994. Fein, David, Erik Beecroft, William Hamilton, and others. The Indiana Welfare Reform Evaluation: Program Implementation and Economic Impacts After Two Years. Cambridge, Mass.: Abt Associates, 1998. Fein, David J., and Jennifer A. Karweit. The ABC Evaluation: The Early Economic Impacts of Delaware’s A Better Chance Welfare Reform Program. Cambridge, Mass.: Abt Associates, 1997. Fraker, Thomas M., Lucia A. Nixon, Jonathan E. Jacobson, and others. Iowa’s Family Investment Program: Two-Year Impacts. Washington, D.C.: Mathematica Policy Research, Inc., 1998. Freedman, Stephen, Jan Bryant, and George Cave. New Jersey: The Demonstration of State Work/Welfare Initiatives, Final Report on the Grant Diversion Project. New York, N.Y.: Manpower Demonstration Research Corporation, 1988. Freedman, Stephen, Marisa Mitchell, and David Navarro. “The Los Angeles Jobs-First GAIN Evaluation: Preliminary Findings on Participation Patterns and First-Year Impacts.” Working Paper, Manpower Demonstration Research Corporation, 1998. Gueron, Judith M., and Edward Pauly. From Welfare to Work. New York, N.Y.: Russell Sage Foundation, 1991. Kemple, James, Daniel Friedlander, and Veronica Fellerath. Florida’s Project Independence: Benefits, Costs, and Two-Year Impacts of Florida’s JOBS Program. New York, N.Y.: Manpower Demonstration Research Corporation, 1995. Long, Sharon K., Demetra Smith Nightingale, and Douglas A. Wissoker. The Evaluation of the Washington State Family Independence Program. Washington, D.C.: Urban Institute Press, 1994. Page 37 GAO/HEHS-99-179 Welfare-to-Work Approaches Bibliography Miller, Cynthia, Virginia Knox, Patricia Auspos, and others. Making Welfare Work and Work Pay: Implementation and 18-Month Impacts of the Minnesota Family Investment Program. New York, N.Y.: Manpower Demonstration Research Corporation, 1997. Nightingale, Demetra Smith, Douglas A. Wissoker, Lynn C. Burbridge, and others. Evaluation of the Massachusetts Employment and Training (ET) Program. Washington, D.C.: Urban Institute Press, 1991. Olson, Jerome A., Deanna T. Schexnayder, Daniel P. O’Shea, and others. Participation Patterns and Program Impacts of Hawaii’s JOBS WORKS! Demonstration Project. Austin, Tex.: Lyndon B. Johnson School of Public Affairs, University of Texas at Austin, 1997. O’Neill, June. Work and Welfare in Massachusetts: An Evaluation of the ET Program. Boston, Mass.: Pioneer Institute for Public Policy Research, 1990. Orr, Larry L., Howard S. Bloom, Stephen H. Bell, and others. Does Training for the Disadvantaged Work? Evidence From the National JTPA Study: An Abt Associates Study. Washington, D.C.: Urban Institute Press, 1996. Schexnayder, Deanna T., and Jerome A. Olson. Texas JOBS Program Evaluation: Second Year Impacts. Austin, Tex.: Center for the Study of Human Resources, University of Texas at Austin, 1995. Scrivener, Susan, Gayle Hamilton, Mary Farrell, and others. National Evaluation of Welfare-to-Work Strategies: Implementation, Participation Patterns, Costs and Two-Year Impacts of the Portland (Oregon) Welfare-to-Work Program. Washington, D.C.: Departments of Education and Health and Human Services, 1998. Werner, Alan, and Robert Kornfeld. Final Impact Report—The Evaluation of To Strengthen Michigan Families. Cambridge, Mass.: Abt Associates, 1997. Werner, Alan, David Rodda, Elsie Pan, and others. Final Report: Evaluation of the Alabama Avenues to Self-Sufficiency Through Employment and Training Services (ASSETS) Demonstration. Cambridge, Mass.: Abt Associates, 1997. Page 38 GAO/HEHS-99-179 Welfare-to-Work Approaches Bibliography Brooks-Gunn, Jeanne, Greg J. Duncan, and Nancy Maritato. “Poor Research Exploring Families, Poor Outcomes: The Well-Being of Children and Youth,” the Relationship Consequences of Growing Up Poor, eds. Greg J. Duncan and Jeanne Between Welfare Brooks-Gunn. New York, N.Y.: Russell Sage Foundation, 1997. Mothers’ Educational Duncan, Greg J., Rachel Dunifon, Morgan Ward Doran, and W. Jean Yeung. Attainment and That “How Different ARE Welfare and Working Families? And Do Those Differences Matter for Children’s Achievement?” Working Paper No. 2, The of Their Children, and Northwestern University/University of Chicago, Joint Center for Poverty Related Studies Research, July 1998. Duncan, Greg J., and Wei-Jun J. Yeung. “Extent and Consequences of Welfare Dependence Among America’s Children,” Children and Youth Services Review, Vol. 17, Nos. 1 and 2 (1995), pp. 157-82. Haveman, Robert, and Barbara Wolfe. “The Determinants of Children’s Attainments: A Review of Methods and Findings,” Journal of Economic Literature, Vol. 33 (Dec. 1995), pp. 1829-78. _____. Succeeding Generations: On the Effects of Investments in Children. New York, N.Y.: Russell Sage Foundation, 1994. Haveman, Robert, Barbara Wolfe, and James Spaulding. “Childhood Events and Circumstances Influencing High School Completion,” Demography, Vol. 28, No. 1 (Feb. 1991), pp. 133-57. Menaghan, Elizabeth, Susan Jekielek, Frank Mott, and Elizabeth Cooksey. “Work and Family Circumstances and Child Trajectories: When (and For What) Does AFDC Receipt Matter?” Working Paper No. 3, The Northwestern University/University of Chicago, Joint Center for Poverty Research, July 1998. Moore, Kristin A., Martha J. Zaslow, Mary Jo Coiro, and others. The JOBS Evaluation: How Well Are They Faring? AFDC Families With Preschool-Aged Children in Atlanta at the Outset of the JOBS Evaluation. Washington, D.C.: Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation, 1995. Teachman, Jay D., Kathleen M. Paasch, Randal D. Day, and Karen P. Carver. “Poverty During Adolescence and Subsequent Educational Attainment,” Consequences of Growing Up Poor, eds. Greg J. Duncan and Jeanne Brooks-Gunn. New York, N.Y.: Russell Sage Foundation, 1997. Page 39 GAO/HEHS-99-179 Welfare-to-Work Approaches Bibliography Yoshikawa, Hirokazu. “Welfare Dynamics, Support Services, Mothers’ Earnings, and Child Cognitive Development: Implications for Contemporary Welfare Reform,” Child Development, Vol. 70, No. 3 (May/June 1999), pp. 779-801. Zill, Nicholas, Kristen A. Moore, Ellen Wolpow Smith, and others. “The Life Circumstances and Development of Children in Welfare Families: A Profile Based on National Survey Data,” Escape From Poverty: What Makes a Difference for Children? eds. P. Lindsay Chase-Lansdale and Jeanne Brooks-Gunn. New York, N.Y.: Cambridge University Press, 1995. Page 40 GAO/HEHS-99-179 Welfare-to-Work Approaches Page 41 GAO/HEHS-99-179 Welfare-to-Work Approaches Page 42 GAO/HEHS-99-179 Welfare-to-Work Approaches Page 43 GAO/HEHS-99-179 Welfare-to-Work Approaches Related GAO Products Welfare Reform: States’ Implementation Progress and Information on Former Recipients (GAO/T-HEHS-99-116, May 27, 1999). Welfare Reform: Information on Former Recipients’ Status (GAO/HEHS-99-48, Apr. 28, 1999). Welfare Reform: States’ Experiences in Providing Employment Assistance to TANF Clients (GAO/HEHS-99-22, Feb. 26, 1999). Welfare Reform: States Are Restructuring Programs to Reduce Welfare Dependence (GAO/HEHS-98-109, June 17, 1998). Welfare to Work: State Programs Have Tested Some of the Proposed Reforms (GAO/PEMD-95-26, July 14, 1995). Welfare to Work: Most AFDC Training Programs Not Emphasizing Job Placement (GAO/HEHS-95-113, May 19, 1995). Welfare to Work: Participants’ Characteristics and Services Provided in JOBS (GAO/HEHS-95-93, May 2, 1995). Welfare to Work: States Begin JOBS, but Fiscal and Other Problems May Impede Their Progress (GAO/HEHS-91-106, Sept. 27, 1991). (205393) Page 44 GAO/HEHS-99-179 Welfare-to-Work Approaches Ordering Information The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. Orders by mail: U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 or visit: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537. Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists. For information on how to access GAO reports on the INTERNET, send an e-mail message with "info" in the body to: firstname.lastname@example.org or visit GAO’s World Wide Web Home Page at: http://www.gao.gov PRINTED ON RECYCLED PAPER United States Bulk Rate General Accounting Office Postage & Fees Paid Washington, D.C. 20548-0001 GAO Permit No. G100 Official Business Penalty for Private Use $300 Address Correction Requested
Welfare Reform: Assessing the Effectiveness of Various Welfare-to-Work Approaches
Published by the Government Accountability Office on 1999-09-09.
Below is a raw (and likely hideous) rendition of the original report. (PDF)