oversight

Head Start: Research Provides Little Information on Impact of Current Program

Published by the Government Accountability Office on 1997-04-15.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                 United States General Accounting Office

GAO              Report to the Chairman, Committee on
                 the Budget, House of Representatives



April 1997
                 HEAD START
                 Research Provides
                 Little Information on
                 Impact of Current
                 Program




GAO/HEHS-97-59
      United States
GAO   General Accounting Office
      Washington, D.C. 20548

      Health, Education, and
      Human Services Division

      B-271866

      April 15, 1997

      The Honorable John R. Kasich
      Chairman, Committee on the Budget
      House of Representatives

      In the 30 years it has existed, Head Start has served over 15 million
      children at a total cost of $31 billion. Growing out of the War on Poverty in
      the mid-1960s, Head Start was created to provide comprehensive health,
      social, educational, and mental health services to disadvantaged preschool
      children. The program was built on the philosophy that effective
      intervention in the lives of children can best be accomplished through
      family and community involvement. Fundamental to program philosophy
      was the notion that communities be given considerable latitude to develop
      their own Head Start programs, an idea that has made variability a defining
      characteristic of the program. These philosophies and the general goals of
      the program remain virtually unchanged today.

      Although Head Start has long enjoyed both congressional and public
      support, opinions about the program’s impact have been divided and its
      effectiveness debated. Some maintain that no compelling evidence exists
      that Head Start makes any lasting difference in the lives of the population
      it serves. Others strongly support Head Start and maintain that research
      has conclusively established its value. Amid this debate, funding for Head
      Start has tripled in the past 10 years.

      Conflicting information on program impact and the focus on results-
      oriented program performance information required by the Government
      Performance and Results Act (GPRA) of 1993 have renewed interest in the
      impact of the current Head Start program. In light of this, you asked us to
      determine (1) what the studies conducted on current Head Start programs1
      suggest about Head Start’s impact and (2) what types of Head Start studies
      are planned by the Department of Health and Human Services (HHS).

      We defined impact for this study as differences in outcomes caused by
      Head Start participation.2 Implicit in this definition is the notion that


      1
       Because Head Start changed significantly in its early years, which could have affected program
      impact, we defined current Head Start programs as those in existence in 1976 or later. See app. I for a
      further discussion of this decision.
      2
       Our definition of impact is based on the concept of “net impact” as defined by the Office of
      Management and Budget (OMB) for agency use in developing performance measures for GPRA and is
      from “Selected Examples of Performance Measurement,” OMB Office Memorandum 95-37
      (Washington, D.C.: July 28, 1995).



      Page 1                                                       GAO/HEHS-97-59 Head Start Research
                   B-271866




                   differences found would not have occurred without program participation.
                   Impact research is designed to permit the assumption that differences
                   were caused by the program. Although impact studies are sometimes
                   difficult and expensive, they are the only way to answer the question, “Is
                   this program making a difference?” Thus, we included only studies in this
                   review that gave some information on program impact. See appendix I for
                   details on criteria we used to select studies.

                   To determine what research suggests about the impact of Head Start, we
                   searched many electronic databases to locate published and unpublished
                   manuscripts. We also spoke with early childhood researchers and
                   practitioners to identify research studies. Our search yielded nearly 600
                   citations and documents, which were screened for possible inclusion in
                   the study. Of these, we found 22 studies that fit our agreed-upon criteria
                   and are reviewed in this study. (See app. I for details on our scope and
                   methodology.) To obtain information about HHS’ studies of Head Start, we
                   reviewed HHS’ research plans for Head Start and other research documents
                   and spoke with HHS and National Head Start Association officials involved
                   with Head Start research.


                   Although an extensive body of literature exists on Head Start, only a small
Results in Brief   part of this literature is program impact research. This body of research is
                   inadequate for use in drawing conclusions about the impact of the national
                   program in any area in which Head Start provides services such as school
                   readiness or health-related services. Not only is the total number of studies
                   small, but most of the studies focus on cognitive outcomes, leaving such
                   areas as nutrition and health-related outcomes almost completely
                   unevaluated. Individually, the studies suffer to some extent from
                   methodological and design weaknesses, such as noncomparability of
                   comparison groups, which call into question the usefulness of their
                   individual findings. In addition, no single study used a nationally
                   representative sample so that findings could be generalized to the national
                   program.

                   Failing to find impact information in existing research, we examined HHS’
                   research plans for Head Start. Planned research will focus on new or
                   innovative service delivery strategies and demonstrations but will provide
                   little information on the impact of regular Head Start programs. HHS’
                   planned research includes descriptive studies; studies of program
                   variations, involving new and innovative service delivery strategies and
                   demonstration projects; and studies of program quality.



                   Page 2                                       GAO/HEHS-97-59 Head Start Research
             B-271866




             HHS officials, in explaining the agency’s research emphasis, stated that
             early research has proven Head Start’s impact. Such research, however,
             conducted over 20 years ago, may no longer apply to today’s program
             because of program changes and changes in the population served. HHS
             also noted some ethical and methodological difficulties of conducting
             impact research, especially studies that would produce national estimates
             of program effect. But neither ethical nor methodological issues present
             an insurmountable deterrent to conducting research on Head Start’s
             impact. Moreover, the size and cost of the program appear to warrant an
             investment in such research.


             Begun in 1965 as a part of the effort to fight poverty, Head Start is the
Background   centerpiece of federal early childhood programs. Head Start’s primary goal
             is to improve the social competence of children in low-income families,
             that is, their everyday effectiveness in dealing with both their present
             environment and later responsibilities in school and life. Social
             competence takes into account the interrelatedness of cognitive and
             intellectual development, physical and mental health, nutritional needs,
             and other factors. To support its social competence goal, Head Start has
             delivered a wide range of services to over 15 million children nationwide
             since its inception. These services consist of education and medical,
             dental, nutrition, mental health, and social services. Another essential part
             of every program is parental involvement in parent education, program
             planning, and operating activities.

             Head Start services are provided at the local level by public and private
             nonprofit agencies that receive their funding directly from HHS. These
             include public and private school systems, community action agencies,
             government agencies, and Indian tribes. In fiscal year 1996, grants were
             awarded to about 1,400 local agencies, called grantees. Head Start
             grantees are typically required to obtain additional funding from
             nonfederal sources to cover 20 percent of the cost of their programs. The
             Head Start program works with various community sources to provide
             services. For example, some programs coordinate with public health
             agencies to obtain health services, while other programs contract with
             local physicians. Although all programs operate under a single set of
             performance standards, local programs have a great deal of discretion in
             how they meet their goals, resulting in great variability among programs.

             Although the program is authorized to serve children at any age before the
             age of compulsory school attendance, most children enter the program at



             Page 3                                       GAO/HEHS-97-59 Head Start Research
                         B-271866




                         age 4. The law requires Head Start to target children from poor families,
                         and regulations require that 90 percent of the children enrolled in each
                         program be low income. By law, certain amounts are set aside for specific
                         subpopulations of children, including those with disabilities and Native
                         American and migrant children.

                         In addition to providing services to children and families, Head Start also
                         sees one of its roles as a national laboratory for child development.
                         Consequently, Head Start uses much of its discretionary research funding
                         for demonstrations and studies of program innovations. Although overall
                         funding has grown over the years, the amount of funds allocated to
                         research, demonstration, and evaluation3 has represented about 2 percent
                         or less of the Head Start budget. In fiscal year 1996, Head Start’s research,
                         demonstration, and evaluation budget totaled $12 million (see app. II).


Head Start Has Changed   Today’s Head Start is a much different program than it was 30 years ago.
Over the Years           Although the program’s goals have changed little since its inception, Head
                         Start changed considerably during its first decade. Begun as a summer
                         program, Head Start became largely a full-year program by the early 1970s.
                         In addition, in the early to mid-1970s, the program launched improvement
                         initiatives, including promulgation of performance standards and teacher
                         credentialing. Programs also had the option of providing home-based
                         services.

                         In the 1990s, the program continues to change. In 1990, the Congress
                         passed the Head Start Expansion and Quality Improvement Act, which
                         reauthorized Head Start and set aside funds for programs to use to
                         enhance and strengthen the quality of services.4 In 1994, the Congress
                         established a new program—called Early Head Start—to serve low-income
                         families with infants and toddlers. The program provides continuous,
                         intensive, and comprehensive child development and family support
                         services to low-income families with children under age 3.

                         In addition to changes to Head Start over the years, other changes
                         affecting the program relate to the children and families Head Start serves
                         and the amount appropriated to support the program. Head Start’s service
                         population has become increasingly multicultural and multilingual and is
                         confronted with difficult social problems such as domestic violence and

                         3
                          HHS makes little distinction between spending for research and evaluation, HHS officials told us.
                         4
                          Despite the emphasis on quality, some early childhood experts are still concerned about the uneven
                         quality of Head Start programs.



                         Page 4                                                      GAO/HEHS-97-59 Head Start Research
B-271866




drug abuse. Moreover, the number of children served by the program has
grown dramatically—from 349,000 children in 1976 to about 750,000 in
1995. The amount appropriated for the program, which totaled $3.5 billion
in 1995, has paralleled the growth in the number served (see fig. 1).




Page 5                                     GAO/HEHS-97-59 Head Start Research
                                 B-271866




Figure 1: Growth in Head Start




                                 Note: In the early years of Head Start, most programs were summer programs. In the early 1970s,
                                 summer programs were almost completely phased out.




                                 Page 6                                                   GAO/HEHS-97-59 Head Start Research
                         B-271866




Research on the Early    In the decade after Head Start’s inception, many studies of the program’s
Years of Head Start      impact were conducted. One of the first major studies was conducted for
                         the Office of Economic Opportunity by the Westinghouse Corporation in
                         1969. This study found that summer Head Start programs produced no
                         lasting gains in participants’ cognitive or affective development and that
                         full-year programs produced only marginal gains by grades one, two, and
                         three. Several researchers criticized this study because of its methodology.
                         Subsequently, many other studies investigated Head Start’s impact.

                         In 1981, HHS contracted with CSR, Inc., to synthesize the findings of Head
                         Start impact studies. CSR concluded that Head Start participants showed
                         significant immediate gains in cognitive test scores, socioemotional test
                         scores, and health status. Cognitive and socioemotional test scores of
                         former Head Start students, however, did not remain superior in the long
                         run to those of disadvantaged children who did not attend Head Start,
                         according to CSR. In addition, on the basis of a small subset of studies,
                         CSR reported that Head Start participants were less likely to be retained in
                         grade and less likely to be placed in special education.5

                         Because these research studies were conducted during Head Start’s
                         infancy, their findings provide little information on the effectiveness of the
                         current program. For instance, most of the programs included in the
                         Westinghouse study were summer programs. Almost all programs today
                         are full-year programs. Similarly, the great majority of studies in CSR’s
                         synthesis study were late 1960’s and early 1970’s programs and therefore
                         would not have reflected many significant program changes that took
                         place in the early to mid-1970s.


Interest in Impact       Interest in Head Start’s impact has grown with increased congressional
Research Has Increased   and public concern for substantiating federal program performance.
                         Traditionally, federal agencies have used the amount of money directed
                         toward their programs, the level of staff deployed, or even the number of
                         tasks completed as some of the measures of program performance. At a
                         time when the value of many federal programs is undergoing intense
                         public scrutiny, however, an agency that reports only these measures has
                         not answered the defining question of whether these programs have
                         produced real results. Because today’s environment is results oriented, the
                         Congress, executive branch, and the public are beginning to hold agencies



                         5
                           Ruth McKey and others, The Impact of Head Start on Children, Families, and Communities, HHS Pub.
                         No. (OHDS) 85-31193 (June 1985), p. 1.



                         Page 7                                                    GAO/HEHS-97-59 Head Start Research
                              B-271866




                              accountable for outcomes, that is, program results as measured by the
                              differences programs make.

                              The Congress’ determination to hold agencies accountable for their
                              performance lay at the heart of two landmark reforms of the 1990s: the
                              Chief Financial Officers Act of 1990 and GPRA. With these two laws, the
                              Congress imposed a new and more businesslike framework for
                              management and accountability on federal agencies. In addition, GPRA
                              created requirements for agencies to generate the information
                              congressional and executive branch decisionmakers need in considering
                              measures to improve government performance and reduce costs.


                              The body of research on current Head Start is insufficient to draw
Body of Research on           conclusions about the impact of the national program. Drawing such
Current Head Start            conclusions from a body of research would require either (1) a sufficient
Program Insufficient          number of reasonably well-designed individual studies whose findings
                              could appropriately be combined to provide information about the impact
to Draw Conclusions           of the national program or (2) at least one large-scale evaluation using a
About Impact                  nationally representative sample. Findings from the individual studies we
                              identified, however, could not be appropriately combined and generalized
                              to estimate program impact at the national level. In addition, no single
                              study used a nationally representative sample, permitting findings to be
                              generalized to the national program.


Findings Could Not Be         The body of studies was inadequate to assess program impact by
Combined to Produce           combining the findings of studies using similar outcome measures. The
National Estimates of         total number of studies found on Head Start impact was too small to
                              permit generalizing findings to the national program. Most of these studies
Impact                        targeted cognitive outcomes, leaving other outcome areas, such as health
                              and nutrition, scarcely examined. In addition, all the studies suffered to
                              some extent from methodological problems that weakened our confidence
                              in the findings of the individual studies.

Number of Studies Too Small   Although the body of literature on Head Start is extensive, the number of
                              impact studies was insufficient to allow us to draw conclusions about the
                              impact of the national Head Start program. Such an aggregation of findings
                              should be based on a large number of studies.6 The larger the number of
                              studies, the greater the chance that the variability in Head Start programs

                              6
                               John E. Hunter and Frank L. Schmidt, Methods of Meta-Analysis (Newbury Park, Cal.: Sage
                              Publications, 1990), p. 83.



                              Page 8                                                    GAO/HEHS-97-59 Head Start Research
                        B-271866




                        would be represented in the studies. Conversely, the smaller the number
                        of studies, the greater the risk that the aggregate findings from these
                        studies may not apply to Head Start in general. Most of the approximately
                        600 articles and manuscripts about Head Start that we identified could not
                        be used to answer questions about impact for various reasons. Much of
                        this literature consisted of program descriptions, anecdotal reports, and
                        position papers. Of those articles that were research studies, some (for
                        example, case studies) were not suitable for drawing general conclusions
                        about impact. Some studies examined change in outcome measures before
                        and after Head Start but did not control for other plausible explanations
                        for the change, for example, maturation. Other studies using a comparison
                        group to control for competing explanations of change did not provide
                        statistical information about the confidence that the differences found
                        were not chance occurrences.

                        Only 22 of the more than 200 manuscripts we reviewed met our criteria for
                        inclusion in our analysis. (See app. I for a detailed description of inclusion
                        criteria.) Of these, 16 investigated impact by comparing Head Start
                        participants with an unserved comparison group; 3 analyzed gains on
                        normed tests. Only three studies included comparisons of Head Start with
                        some other type of preschool or day care program. These studies
                        represent work by a variety of researchers, including college students,
                        college faculty, and contractors. Appendix III contains more detailed
                        information on each study.

No Outcome Area or      Although Head Start provides services in several outcome areas, such as
Population Adequately   health, nutrition, education, and the like, most of the studies we found
Researched              focused on educational/cognitive outcomes, and few made distinctions on
                        the basis of differing populations served by Head Start. For example, most
                        of the studies examined the impact of Head Start on grade retention and
                        other indicators of academic achievement, such as standardized reading
                        and math scores. Of the 22 studies included in our review, 16 included one
                        or more outcomes in the cognitive area. Conversely, only five studies
                        investigated health- or nutrition-related outcomes, and only five examined
                        family impacts.

                        Similarly, few studies analyzed impact by subpopulations. Because Head
                        Start is a multicultural program, serving children and families of varying
                        races, ethnic backgrounds, and socioeconomic levels, research that targets
                        these subpopulations may uncover differential effects.




                        Page 9                                        GAO/HEHS-97-59 Head Start Research
                            B-271866




Studies Suffered From       All of the studies had some methodological problems. Although research
Methodological Weaknesses   in field settings can rarely conform to rigorous scientific procedures, in
                            general, researchers place more confidence in findings of studies that
                            control for competing explanations for their results and that use large
                            samples.

                            One of the more serious of the methodological problems was
                            noncomparability of comparison groups. The most reliable way to
                            determine program impact is to compare a group of Head Start
                            participants with an equivalent group of nonparticipants. The preferred
                            method for establishing that the groups are equivalent at outset is to
                            randomly assign participants to either the Head Start group or the
                            comparison group. Only one of the studies we reviewed used random
                            assignment to form the Head Start and non-Head Start comparison groups.
                            Most of these studies formed a comparison group by selecting children
                            who were similar to the Head Start participants on some characteristic
                            thought to be important to the outcome under study. In most cases,
                            researchers matched participants on one or more demographic variables,
                            usually including some variable related to socioeconomic level. In other
                            cases, researchers did not match treatment and comparison groups but
                            tried to compensate statistically for any inequality between the groups.
                            Neither of these methods compensates completely for lack of random
                            assignment to group.

                            Some of the studies used no comparison group; instead, they compared
                            performance of Head Start participants with test norms. This approach to
                            evaluating program performance indicates the performance of Head Start
                            participants relative to the norming group. Because the norming group
                            may be unlike the Head Start group, however, conclusions about program
                            impact are unclear.

                            Finally, many of the studies also suffered from small samples, especially
                            those investigating intermediate and long-term effects. Some studies began
                            with relatively small samples; others, which began with larger samples,
                            ended up with smaller samples as the study progressed because of missing
                            data and attrition. Small samples present problems in research because
                            they adversely affect statistical procedures used in analyses. Some
                            procedures cannot appropriately be used with small samples; others are
                            rendered less able to detect differences, resulting in an underestimation of
                            program effects.




                            Page 10                                     GAO/HEHS-97-59 Head Start Research
                      B-271866




No National Program   No completed, large-scale evaluation of any outcome of Head Start that
Evaluation Found      used a nationally representative sample was found in our review. One
                      characteristic of Head Start is program variability, not only in the kind of
                      services delivered, but also in the quality of services. Making summary
                      statements about program impact requires that the sample of programs
                      studied represent all programs nationwide.

                      Although one evaluation had a study design that would have allowed
                      findings to be generalized to the national program, this study was never
                      completed. In the late 1970s, HHS contracted for a national evaluation of
                      the educational services component of basic Head Start. The design called
                      for a longitudinal study that would follow children and their parents from
                      preschool through the fourth grade. The evaluation was to compare the
                      Basic Educational Skills Program, regular Head Start, and a non-Head
                      Start control group. Thirty Head Start programs were to be randomly
                      selected, and Head Start-eligible children from these communities were to
                      be randomly assigned to Head Start or the control group. Many
                      methodological problems as well as funding problems occurred, however,
                      during the implementation of this study, and it was abandoned.

                      The 1990 act that reauthorized funding for Head Start directed the
                      Secretary of HHS to conduct “. . . a longitudinal study of the effects that the
                      participation in Head Start programs has on the development of
                      participants and their families and the manner in which such effects are
                      achieved.” The study, as described in the act, was to examine a wide range
                      of Head Start outcomes, including social, physical, and academic
                      development, and follow participants at least through high school. The
                      description also stipulated that, “To the maximum extent feasible, the
                      study . . . shall provide for comparisons with appropriate groups
                      composed of individuals who do not participate in Head Start programs.”
                      The act authorized the appropriation of funds to carry out this study for
                      fiscal years 1991 through 1996. According to HHS, however, funds were
                      never appropriated for the study, and it was not conducted.




                      Page 11                                       GAO/HEHS-97-59 Head Start Research
                          B-271866




                          Head Start’s planned research will provide little information about the
Research Planned by       impact of regular Head Start programs7 because it focuses on descriptive
HHS Focuses on            studies; studies of program variations, involving new and innovative
Program                   service delivery strategies and demonstration projects; and studies of
                          program quality. Although these types of studies are useful in evaluating
Improvement, Not          programs, they do not provide the impact information needed in today’s
Impact                    results-oriented environment and encouraged by GPRA.

HHS Focuses Research on   The primary focus of research, according to Head Start Bureau officials, is
Program Improvement       to improve the program by exploring ways to maximize and sustain Head
                          Start benefits. Thus, HHS studies evaluate which practices seem to work
                          best for the varying populations Head Start serves and ways to sustain
                          program benefits. Some of these studies are descriptive, providing
                          information on service delivery and the characteristics of populations
                          receiving services. For example, HHS is currently conducting a descriptive
                          study of the characteristics of families served by the Head Start Migrant
                          Program. Other descriptive studies have been conducted on health
                          services and bilingual/multicultural programs.

                          HHS also funds studies designed to answer questions about the
                          effectiveness of new or innovative service delivery strategies and
                          demonstrations and how effectiveness may relate to characteristics of the
                          population served. Such studies typically involve special program efforts
                          and demonstration projects conducted on a trial basis at a few Head Start
                          sites that focus on practices or services not typically found in regular Head
                          Start programs.8 For example, both Early Head Start and the
                          Comprehensive Child Development Program target infants and children
                          younger than those normally served by Head Start. Similarly, the Family
                          Service Center demonstrations place more emphasis on family services
                          and provide assistance in a variety of areas such as illiteracy, substance
                          abuse, and unemployment.

                          In addition, HHS funds research to explore program quality and to develop
                          instruments to assess program performance. In 1995-96, HHS funded
                          several Quality Research Centers and a Performance Measure Center to

                          7
                           “Regular” Head Start refers in this report to programs that operate within the scope of established
                          Head Start program options and under normal Head Start requirements. Regular programs are to be
                          distinguished from demonstrations and other special programs, which may serve populations or offer
                          services not normally found in Head Start.
                          8
                           Both special programs and demonstrations are innovative programs, implemented on a limited basis
                          and with program features not found in regular Head Start programs. Demonstrations, however, have a
                          predetermined end because their grants expire at the end of a specified period. Special programs may
                          continue to receive funding because programs may recompete for such grants at the end of the grant
                          period.



                          Page 12                                                    GAO/HEHS-97-59 Head Start Research
                          B-271866




                          develop and identify instruments for measuring the quality of Head Start
                          programs and to collect performance measure data on a nationally
                          representative sample of Head Start programs. The major purpose of this
                          effort, according to HHS officials, is to determine which program
                          characteristics relate to meeting program goals. Some of the performance
                          measure assessments use instruments for which national norms are
                          available, however, and HHS will be able to compare participant
                          performance to national norms for these measures.

                          Identifying performance measures is an important step in building a
                          research and evaluation base for Head Start. Because the program’s goals
                          are so broad and difficult to assess, precisely defining expected outcomes
                          and identifying appropriate instruments should produce a more valid,
                          useful body of research. But identifying standard performance measures is
                          also valuable because it provides a set of common measures upon which a
                          body of research could be built, including impact research.

                          Although descriptive studies, studies of new or innovative programs and
                          demonstrations, and studies of program quality provide information useful
                          both to HHS and the Congress, they do not provide full information on the
                          impact of regular Head Start. Even the performance measures study
                          already discussed will not provide clear-cut impact information because
                          no comparison group is being used. Over time, this type of study will
                          provide some useful information about program outcomes; however, such
                          a study can neither attribute effect nor estimate the precise effect size with
                          the level of confidence found in comparison group studies.


Research Planned by HHS   Research planned by HHS will provide little program impact information on
Will Provide Little       regular Head Start programs. HHS officials expressed concerns about using
Information on Program    their research dollars for impact research rather than program
                          improvement. The effectiveness of Head Start has been proven by early
Impact                    research, according to these officials, who also pointed to difficulties in
                          conducting impact studies. In addition, because Head Start is such a varied
                          program, averaging across local programs to produce national estimates of
                          effect is not appropriate, they said. Finally, HHS maintains that Head Start
                          is unique because of the comprehensiveness of services it offers and the
                          population it serves; therefore, comparing Head Start with other service
                          programs would be inappropriate, HHS officials believe.

                          Most of the research that HHS cited as evidence of Head Start’s impact is
                          outdated, however, and, as previously mentioned, insufficient research has



                          Page 13                                      GAO/HEHS-97-59 Head Start Research
                                B-271866




                                been done in the past 20 years to support drawing conclusions about the
                                current program. Furthermore, it appears that impact studies on Head
                                Start could be done and would provide valuable results-oriented
                                information. In addition, although research on programs that vary greatly
                                could be methodologically more challenging to producing national
                                estimates of impact, variation alone should not prevent developing such
                                estimates. Moreover, comparisons with other service programs, if
                                designed to answer questions about specific program outcomes, would
                                provide useful information about assessing program impacts.

HHS Believes Effectiveness of   HHS maintains that early research has proven the effectiveness of early
Head Start Is Already Proven,   childhood education, including Head Start, so impact research is not the
So Further Impact Research Is   most effective use of limited research funds. Findings from early studies,
Not Warranted                   however, do not conclusively establish the impact of the current Head
                                Start program because today’s program differs from that of the late 1960s
                                and early 1970s. Although program changes might be assumed to increase
                                positive impact, this assumption is largely unsubstantiated. In addition,
                                program impact may be affected by changes in the population served;
                                Head Start families today face different problems than those in the past
                                because of an increase in substance abuse, violence, and homelessness.
                                Furthermore, an increased availability of social services may have
                                lessened the impact of Head Start because families may get services from
                                other sources if not from Head Start. The net effect of these changes on
                                program impact is unknown.

                                Later studies offered to support Head Start’s impact do not provide
                                enough evidence to conclude that current Head Start is effective. Findings
                                in literature reviews cited by Head Start proponents to support its
                                effectiveness often involve only a few Head Start programs. For example,
                                HHS cited a review in a recent Packard Foundation report9 that reported
                                positive cognitive results of early childhood programs. This review,
                                however, had only five studies involving Head Start participation in 1976
                                or later, and two of the five studies combined Head Start and other public
                                preschools in the analyses. Authors of other studies of high-quality
                                preschool programs have sometimes warned against applying their
                                findings to Head Start. For instance, researchers in the Consortium for
                                Longitudinal Studies, which produced a major study reporting positive
                                long-term effects of preschool, explicitly stated that caution should be
                                used in generalizing their findings to Head Start and that the programs


                                9
                                Steven W. Barnett, “Long-Term Effects of Early Childhood Programs on Cognitive and School
                                Outcomes,” in The Future of Children: Long-Term Outcomes of Early Childhood Programs, Richard E.
                                Behrman, ed., Vol. 5, No. 3 (Los Altos, Cal.: 1995).



                                Page 14                                                  GAO/HEHS-97-59 Head Start Research
                          B-271866




                          were “. . . examples of what Head Start could be rather than what it has
                          been.”10

HHS Believes Conducting   HHS  believes conducting impact research would present methodological
Impact Studies Would Be   difficulties. Two types of research designs are commonly used in
Difficult                 conducting impact studies, experimental and quasi-experimental. HHS
                          officials mentioned difficulties with both types of designs in studying Head
                          Start’s impact. In addition, finding enough unserved children to form
                          comparison groups would be a problem with either kind of research
                          design, they said.

                          True experimental designs, also called randomized trials, are comparison
                          group studies that randomly assign study participants to either a treatment
                          or control group. In the case of Head Start, these studies would require
                          recruiting more eligible children than the program can serve. From these
                          recruits, some children would be randomly assigned to Head Start; the
                          rest, the unserved children, would constitute the control group. HHS
                          officials cited ethical considerations of assigning children to an unserved
                          control group as one of the difficulties in conducting randomized trials.

                          Randomized trials, however, could be appropriately applied to Head Start
                          research. In fact, the evaluation of the Early Head Start project, now under
                          way, has randomly assigned potential participants to Early Head Start or a
                          control group that has not received Early Head Start services.
                          Alternatively, a research design that delays, rather than withholds, services
                          could be used. This would involve selecting a study group and randomly
                          assigning some children to Head Start the first year, while the remainder
                          would serve as a control group. The control group would receive services
                          the following year. Another strategy that could be used to study specific
                          parts of the program would be to use an alternative treatment design. In
                          this case, some randomly assigned participants would receive the full
                          Head Start program, while others would receive partial services. For
                          example, if the study interest is in school readiness and cognitive issues,
                          the control group might receive only nutritional and health services.

                          Most researchers believe that randomized trials yield the most certain
                          information about program impact. Random assignment is an accepted
                          practice in virtually every area of research, including medicine,
                          economics, and social sciences. In some cases, the treatment of study
                          interest is simply withheld from the control group. In other cases, for

                          10
                            Sandra Condry, “History and Background of Preschool Intervention Programs and the Consortium
                          for Longitudinal Studies,” in As the Twig Is Bent . . . Lasting Effects of Preschool Programs, Lawrence
                          Erlbaum Associates (Hillsdale, N.J.: 1983), p. 27.



                          Page 15                                                      GAO/HEHS-97-59 Head Start Research
B-271866




example, when researchers suspect that withholding treatment would
have a profoundly negative impact, treatment may be delayed for awhile
or some lesser, alternative treatment offered. While acknowledging the
difficulties of random assignment, some early childhood researchers we
spoke with suggested that Head Start conduct randomized trials to study
regular Head Start programs because this type of study provides the most
conclusive information on program impact.

A common alternative to randomized trials, quasi-experimental designs,
uses a naturally occurring, unserved comparison group. In the case of
Head Start, some researchers have tried to identify other children in the
community who are like Head Start participants in ways thought to be
important (usually socioeconomic level) but who are not enrolled in Head
Start. This group became the comparison (control) group.

Quasi-experimental research is less rigorous than research that uses
random assignment, and less confidence can be placed in its conclusions.
Rarely are pre-existing groups equivalent. Even when statistical
adjustments are made to compensate for known nonequivalencies, some
questions always remain about the degree to which pre-existing
differences in the groups may have contributed to study results. When well
planned and well executed, however, such designs can provide some
indication of program impact.

Because Head Start strives to serve the neediest children, those in
quasi-experimental comparison groups would be less likely to be
disadvantaged than children in the Head Start group, according to HHS
officials. If true, this nonequivalency in groups would bias the outcome in
favor of the comparison group, resulting in underestimation of program
effects. Because investigating the characteristics of Head Start
participants was beyond the scope of this study, we do not know to what
extent, if any, Head Start children may be more disadvantaged than similar
children not attending Head Start. Even assuming that Head Start has
identified and is serving the neediest applicants, however, it seems
possible that a comparably disadvantaged, unserved group could be
identified from the applicants whom the program cannot serve and
nonapplicants in a community.

Regardless of which design is used, experimental or quasi-experimental,
finding enough truly unserved children for a comparison group would be
extremely difficult because of the growing number of public preschool
programs and the increased availability of child care, according to HHS



Page 16                                     GAO/HEHS-97-59 Head Start Research
                               B-271866




                               officials. Statistics on the percentage of children being served by
                               preschools suggest, however, that finding disadvantaged children
                               unserved by preschools is possible. In our report, Early Childhood
                               Programs: Many Poor Children and Strained Resources Challenge Head
                               Start (GAO/HEHS-94-169BR), we found that only 35 percent of poor 3- and
                               4-year-olds attended preschool in 1990. The Congressional Research
                               Service estimated that in fiscal year 1994, about 30 percent of eligible 3- to
                               4-year-olds were being served by Head Start.11 On the basis of these
                               estimates, it appears that some locations do exist where a control group of
                               children not attending preschool could be formed.

HHS Believes National          Estimating program impact at the national level is not appropriate because
Estimates of Program Impact    of the extreme variability of local programs, HHS officials said. Local Head
Are Not Appropriate            Start sites have great flexibility, and, even though all programs share
                               common goals, they may operate very differently. Therefore, on the advice
                               of HHS’ research advisory panel,12 HHS considers a single, large-scale,
                               national study of impact to be methodologically inappropriate. For this
                               same reason, HHS believes that summing across sites for an aggregate
                               estimate of effect is not justified in cases where sites are not basically
                               operating the same way.

                               Evaluating outcomes at the national program level is an accepted program
                               evaluation procedure, however, even for programs with a great deal of
                               variability. It is the only way to determine with certainty whether the
                               program is making an overall difference in any particular outcome area.
                               Aggregate analysis does not, however, replace the need for lower level
                               analyses, which provide insight into the summary finding. In cases where
                               effects are not uniform across sites, this lower level analysis provides
                               more understanding of which service areas and delivery approaches are
                               working for which subpopulations. Evaluations can be planned to answer
                               both the aggregate and disaggregate question in a single study.

HHS Believes Comparisons       Another way to evaluate Head Start’s impact is to compare its effects with
With Other Service Providers   some other types of preschool, for instance, state or local preschools.
Are Not Appropriate            When several programs exist that deliver similar services, studies
                               comparing programs in areas that have common goals can provide useful
                               information. For instance, Head Start and public preschools share the goal
                               of school readiness. A study might be conducted to compare Head Start


                               11
                                  Head Start in the 104th Congress, Congressional Research Service Report for Congress (Washington,
                               D.C.: Mar. 6, 1996), p. 2.
                               12
                                Since 1989, advisory panels of experts in early childhood education and research and evaluation have
                               helped HHS plan its research.



                               Page 17                                                    GAO/HEHS-97-59 Head Start Research
                  B-271866




                  and public preschool students on the basis of a measure of school
                  readiness. Such a study might compare the performance of program
                  participants, while describing relevant program differences that might
                  affect results, such as level of service in the area studied and program
                  costs.

                  Regarding a comparative study, HHS has maintained that Head Start is
                  unique in the comprehensiveness of the services it offers. Therefore,
                  according to the agency, any comparison of programs would be
                  misleading. In addition, HHS claims that children served by Head Start are
                  more disadvantaged than children in other types of preschools. The
                  agency also points out that in some places, other public preschools have
                  adopted the Head Start model, making such comparisons essentially Head
                  Start with Head Start.

                  Concerns about differences in populations served by the programs would
                  relate to the rigor of the study design, that is, whether it is experimental or
                  quasi-experimental. When quasi-experimental designs are used,
                  researchers frequently use statistical techniques to mitigate for pre-
                  existing differences; but these designs always suffer to some degree from
                  the limitations referred to earlier in our discussion of quasi-
                  experimental designs. Therefore, confidence in the study’s results would
                  vary depending on the study design used.

                  In the case of Head Start-like programs, one might reasonably expect a
                  difference in outcome on the basis of such factors as program
                  administration and context. For example, a preschool program operated
                  by a local school system might have different outcomes in school
                  readiness because of the possible advantage of transitioning its students
                  into kindergarten. Research that compares Head Start with alternative
                  ways of accomplishing a particular goal might provide insight into the
                  most effective and efficient way to provide services to needy children and
                  families.


                  Increasing demand for shrinking federal resources has raised the concerns
Conclusions and   of the Congress, the executive branch, and taxpayers about the impact of
Recommendations   multibillion dollar federal investments in federal programs such as Head
                  Start. In addition, GPRA requires agencies to be more accountable for
                  substantiating program results. Although research has been conducted, it
                  does not provide information on whether today’s Head Start is making a




                  Page 18                                       GAO/HEHS-97-59 Head Start Research
                  B-271866




                  positive difference in the lives of participants who live in a society that
                  differs vastly from that of the sixties and early seventies.

                  While we acknowledge the difficulties of conducting impact studies of
                  programs such as Head Start, research could be done that would allow the
                  Congress and HHS officials to know with more certainty whether the
                  $4 billion dollar federal investment in Head Start is making a difference.
                  For this reason, we recommend that the Secretary of HHS include in HHS’
                  research plan an assessment of the impact of regular Head Start programs.


                  In commenting on a draft of our report, HHS expressed the belief that the
Agency Comments   research base on the efficacy of Head Start is more substantial than
                  depicted in our report and that the strategy of the Department to extend
                  this base is appropriate to produce findings about both impact and
                  program quality. HHS also indicated plans to evaluate the feasibility of
                  conducting impact studies such as we recommended. The Quality
                  Research Centers are evaluating the feasibility of conducting randomized
                  trials in small-scale evaluations, and, on the basis of these experiences
                  may consider implementing larger scale studies. The full text of HHS’
                  comments appears in appendix IV.

                  HHS  supported the claim that the research base is more substantial than we
                  depict by pointing to the findings from the 1985 synthesis conducted by
                  CSR (cited as “McKey et al., 1985” in HHS’ comments) and two more recent
                  studies (the Currie and Thomas study and the Fosburg study). For reasons
                  discussed in this report, we do not agree that findings drawn from studies
                  more than 20 years old adequately support claims about the impact of the
                  current Head Start program. Similarly, the findings from the two more
                  recent studies mentioned fail to support conclusions about impact that
                  can be generalized to the national program. Even though these studies
                  were larger than others we found, both had significant methodological
                  limitations. The Currie and Thomas study examined information in a
                  database to reach conclusions about Head Start. This study used an
                  after-the-fact, post-test-only design. Although this design is frequently used
                  when researchers must rely on existing data as their only source, the
                  design is vulnerable to serious threats to validity, as discussed earlier in
                  this report. Because of these design limitations, neither positive
                  conclusions about Head Start (that is, that children’s test scores show
                  immediate positive effects) nor negative conclusions (that is, that these
                  effects quickly disappear for African American children) can be firmly
                  drawn from the findings of this study.



                  Page 19                                       GAO/HEHS-97-59 Head Start Research
B-271866




The second study, the Fosburg study, as HHS pointed out, used a much
stronger research design, which randomly assigned children in four Head
Start programs to either Head Start or a non-Head Start control group. The
site selection methodology, however, precluded generalizing these
findings to all Head Start programs. The four Head Start programs selected
were chosen from areas identified as underserved in medical and dental
services, and Head Start sites that were not in compliance with Head Start
performance standards were excluded from selection. In addition, attrition
was a significant problem in this study.

HHS also mentioned that on the basis of recommendations of leading
researchers, the Department is conducting a well-balanced, innovative set
of new studies of Head Start. It contends that our report does not
acknowledge the major longitudinal studies that HHS has planned or that
are being conducted by other agencies. Our report states that HHS’ planned
research focuses on program improvement, and we agree that such studies
are needed. We also support the studies of program impact that HHS has
under way in special program areas such as Early Head Start. Our work,
however, focused specifically on HHS’ research plans that address the
question of impact of the regular Head Start program. HHS’ current
research plans, however, do not include such research.

Finally, HHS maintained that it is building a substantial system of
innovative research, development, and management tools in response to
GPRA. The Department emphasized the role the Quality Centers play in
these efforts and said that these centers are currently evaluating possible
strategies for performing comparison group studies that use a random
assignment research design. HHS maintained that we overlooked the
importance of studying the quality of Head Start programs in assessing
impacts.

We fully support HHS’ plans to investigate the feasibility of conducting
randomized trials because these studies provide the clearest indication of
program impact. We also agree that the issue of quality is important in
assessing program impact and findings from studies need to include
information on program quality. The ultimate measure of program quality
is impact, however. Until sound impact studies are conducted on the
current Head Start program, fundamental questions about program quality
will remain.




Page 20                                      GAO/HEHS-97-59 Head Start Research
B-271866




We are sending copies of this report to the Secretary of Health and Human
Services, the Head Start Bureau, appropriate congressional committees,
the Executive Director of the National Head Start Association, and other
interested parties. Please call me at (202) 512-7014 if you or your staff have
any questions about this report. Major contributors to this report are listed
in appendix VI.

Sincerely yours,




Carlotta C. Joyner
Director, Education and
  Employment Issues




Page 21                                      GAO/HEHS-97-59 Head Start Research
Contents



Letter                                                           1


Appendix I                                                      24

Objectives, Scope,
and Methodology
Appendix II                                                     27

Research,
Demonstration, and
Evaluation Budgets
for the Head Start
Program
Appendix III                                                    28

Summaries of Studies
Included in the
Review
Appendix IV                                                     47

Comments From the
Department of Health
and Human Services
Appendix V                                                      53

Acknowledgments
Appendix VI                                                     54

GAO Contacts and
Staff
Acknowledgments
Related GAO Products                                            56




                       Page 22   GAO/HEHS-97-59 Head Start Research
         Contents




Figure   Figure 1: Growth in Head Start                                          6




         Abbreviations

         GPRA       Government Performance and Results Act
         HHS        Department of Health and Human Services
         PPVT-R     Peabody Picture Vocabulary Test-Revised
         WJ-R       Woodcock-Johnson Tests of Achievement


         Page 23                                 GAO/HEHS-97-59 Head Start Research
Appendix I

Objectives, Scope, and Methodology


              The Chairman of the Committee on the Budget, House of Representatives,
Objectives    asked us to examine existing research on Head Start programs and to
              determine what it suggests about the impact of the current Head Start
              program. Another objective was to determine what types of Head Start
              research HHS has planned.


              Although the bulk of research on Head Start was conducted in the early
Scope         years of the program, we focused on studies of Head Start participation in
              1976 or later for several reasons. First, HHS instituted quality initiatives and
              other important program changes in the early to mid-1970s that shaped the
              current Head Start program, including phasing out summer programs,
              implementing performance standards, and establishing teacher
              credentialing procedures. Second, findings from studies of early programs
              have limited generalizability to more stable programs. The early years of
              any program are not likely to represent a program in its maturity. This is
              especially true for Head Start, which was implemented quickly and on a
              large scale.

              Finally, earlier studies were thoroughly reviewed by the Head Start
              synthesis project and were reported in The Impact of Head Start on
              Children, Families, and Communities in 1985. Studies from the years
              before 1976 constituted the bulk of studies included in this synthesis. A
              short summary of these findings appears in the “Background” section of
              this report.

              After speaking with HHS research personnel, we anticipated that the body
              of studies usable for a research synthesis might be small. Therefore, in
              addition to comparison group studies, we included pretest/post-test-only
              designs in cases in which outcomes were discussed in relation to test
              norms. Although much less useful in providing information about program
              impact relative to comparison group designs, these studies provide a
              certain degree of valuable information.


              To report on what existing research says about Head Start’s impact, we
Methodology   identified studies meeting our basic selection criteria as outlined in the
              “Literature Review” section of this appendix. Because the number of
              studies found in the first phase was so small, we did not screen further for
              adequacy of information reported. To determine how HHS uses research,
              we reviewed HHS’ research plans and publications by their research
              advisory panel. We also spoke with HHS officials who direct Head Start



              Page 24                                        GAO/HEHS-97-59 Head Start Research
                    Appendix I
                    Objectives, Scope, and Methodology




                    research and with the director of research at the National Head Start
                    Association.


Literature Search   We began our search for studies with two bibliographies contracted for by
                    HHS. The first, An Annotated Bibliography of the Head Start Research Since
                    1965, was a product of the 1985 Head Start Evaluation, Synthesis and
                    Utilization Project. We also reviewed An Annotated Bibliography of Head
                    Start Research: 1985-1995. This bibliography was produced by Ellsworth
                    Associates, Inc., for the Head Start Bureau as a part of its contract to
                    maintain a library of Head Start and related research. Search strategies
                    used to compile these bibliographies are described in the introductions to
                    the documents.

                    In addition, we conducted our own search for studies. Our primary source
                    was the database maintained by the Education Resources Information
                    Center. However, we also searched a number of other databases, including
                    MEDLINE, AGRICOLA, Dissertation Abstracts, Government Printing
                    Office, Mental Health Abstracts, Psyc INFO, Federal Research in Progress,
                    Social SciSearch, Sociological Abstracts, IAC Business A.R.T.S., British
                    Education Index, Public Affairs Information Service International, and
                    National Technical Information Service.

                    We also interviewed people knowledgeable about early childhood
                    research. We attended the Head Start Third National Research Conference
                    and spoke with conference participants. We also mailed letters to every
                    conference participant asking for their assistance in locating relevant
                    research. We interviewed personnel in charge of research for the
                    Administration for Children Youth and Families and the Head Start Bureau
                    and spoke with other researchers whom they recommended. We also
                    talked with the executive director and the director of research and
                    evaluation of the National Head Start Association and addressed the state
                    and regional presidents of this organization at their annual meeting. In
                    addition, we announced our effort to locate research dealing with Head
                    Start effectiveness on several of the Internet forums sponsored by the
                    American Educational Research Association.


Literature Review   From these sources, we identified over 600 manuscripts that were
                    screened for relevance to our study. We acquired about 200 of these and
                    reviewed them carefully regarding the following selection criteria:




                    Page 25                                     GAO/HEHS-97-59 Head Start Research
    Appendix I
    Objectives, Scope, and Methodology




•   Head Start participation had occurred in 1976 or later;
•   studies had compared outcomes of Head Start participants with children
    not attending any preschool—or those attending some other type of
    preschool—or studies had compared Head Start outcomes with test
    norms; and
•   tests of statistical significance13 were reported to have been performed on
    the differences, except in cases in which outcomes were measured using
    normed instruments.

    We excluded studies of transition or follow-through programs that
    provided services beyond the Head Start years and studies that pooled
    Head Start and other kinds of preschool participants. We considered
    multiple articles or later follow-ups on the same study to be one study.
    This final screening yielded 22 impact studies that were evaluated in our
    review.

    We performed our work between April and December 1996 in accordance
    with generally accepted government auditing standards.




    13
      Tests of statistical significance produce a measure of the likelihood that a finding occurred as a result
    of sampling error. Such tests result in a value (usually denoted by “p”) that represents the probability
    that the outcome (for example, differences in scores) arose from a sampling error. Thus, “p< .01”
    means that the probability of the outcome occurring as a result of sampling variation is less than 1 in
    100. Although statistical significance may take on any value from .00 to 1.0, benchmark levels often
    used in research are .01 and .05. We excluded one study because of lack of evidence about the
    statistical significance of the findings.



    Page 26                                                       GAO/HEHS-97-59 Head Start Research
Appendix II

Research, Demonstration, and Evaluation
Budgets for the Head Start Program


                                                     Research,
                                               demonstration,      Percentage of
              Fiscal year   Appropriation   evaluation amount total appropriation
              1971          $360,000,000           $5,700,000                1.58
              1972            376,300,000           6,200,000                1.65
              1973            400,700,000           9,000,000                2.25
              1974            403,900,000           9,000,000                2.23
              1975            403,900,000           9,000,000                2.23
              1976            441,000,000           9,000,000                2.04
              1977            475,000,000           9,000,000                1.89
              1978            625,000,000           8,200,000                1.31
              1979            680,000,000          14,630,000                2.15
              1980            735,000,000          15,200,000                2.07
              1981            818,700,000          14,600,000                1.78
              1982            911,700,000          12,300,000                1.35
              1983            912,000,000           6,900,000                0.76
              1984            995,750,000           1,800,000                0.18
              1985          1,075,059,000           1,300,000                0.12
              1986          1,040,315,000             810,000                0.08
              1987          1,130,542,000           1,300,000                0.11
              1988          1,206,324,000           1,300,000                0.11
              1989          1,235,000,000           1,500,000                0.12
              1990          1,552,000,000           1,500,000                0.10
              1991          1,951,800,000           3,500,000                0.18
              1992          2,201,800,000           8,500,000                0.39
              1993          2,776,286,000           8,900,000                0.32
              1994          3,325,728,000          12,000,000                0.36
              1995          3,534,128,000          12,000,000                0.34




              Page 27                          GAO/HEHS-97-59 Head Start Research
Appendix III

Summaries of Studies Included in the
Review

Evaluation of the Process   Authors: Applied Management Sciences, Inc. (first study) and Roy
of Mainstreaming            Littlejohn Associates, Inc. (second study)
Handicapped Children Into
                            Outcome area studied: Cognitive and health
Project Head Start, Phase
II, Executive Summary,      Overview of study: Children receiving Head Start program services
and Follow-Up Evaluation    compared with children receiving services from other types of programs
of the Effects of           and with children receiving no special services
Mainstreaming
Handicapped Children in     Design: Pretest with post-test 6 months later
Head Start
                            Population: 55 randomly selected Head Start centers and 49 non-Head
                            Start programs

                            Sample: 391 Head Start children, 321 non-Head Start children, and 121
                            unserved children

                            Head Start program year(s): 1977-78

                            Measures/instrumentation: Various development indicators, including
                            physical development, self-help skills, cognitive development, social
                            development, communication skills, classroom social skills, and classroom
                            behavior and social integration

                            Findings: Developmental gains for Head Start and non-Head Start
                            children identified as physically handicapped, mentally retarded, and
                            health or developmentally impaired were generally not significantly
                            greater than those of unserved children. Developmental gains were
                            significant in physical, self-help, academic, and communications skills for
                            children identified as speech impaired in Head Start and non-Head Start
                            programs relative to unserved children.

                            Source: Applied Management Services, Inc., Evaluation of the Process of
                            Mainstreaming Handicapped Children Into Project Head Start, Phase II,
                            Executive Summary, Education Resources Information Center (ED
                            168291), 1978, and Roy Littlejohn Associates, Inc., Follow-Up Evaluation
                            of the Effects of Mainstreaming Handicapped Children in Head Start
                            (Washington, D.C.: 1985).




                            Page 28                                     GAO/HEHS-97-59 Head Start Research
                          Appendix III
                          Summaries of Studies Included in the
                          Review




A Longitudinal Study to   Author: Colleen K. Bee
Determine If Head Start
Has Lasting Effects on    Outcome area studied: Cognitive
School Achievement        Overview of study: Followed up Head Start participants in kindergarten,
                          first grade, and second grade

                          Design: Post-test only, comparison group selected from waiting list for
                          each respective year

                          Population: Head Start participants in Sioux Falls, South Dakota

                          Sample: 10 girls and 10 boys were selected for each Head Start year, 10
                          girls and 10 boys were selected each year for the comparison groups

                          Head Start program year(s): 1977-78, 1978-79, 1979-80

                          Measures/instrumentation: Metropolitan Reading Readiness Test,
                          special education placements, and grade retention

                          Findings: No significant differences were found at the .01 level of
                          confidence on reading readiness scores for any of the years studied.
                          Non-Head Start group retained in grade less than the Head Start group in
                          1977-78 (difference significant at .01 level). No significant difference was
                          found in special education placements for any of the years studied.

                          Source: Colleen K. Bee, A Longitudinal Study to Determine If Head Start
                          Has Lasting Effects on School Achievement, UMI Dissertation Services
                          (Ann Arbor, Mich.: 1981).


Evaluation of Public      Authors: Donna M. Bryant, Ellen S. Peisner-Feinberg, and Richard M.
Preschool Programs in     Clifford
North Carolina
                          Outcome area studied: Cognitive and socioemotional

                          Overview of study: Followed up public preschool graduates in
                          kindergarten

                          Design: Post-test only, comparison group comprised children from same
                          kindergarten classes




                          Page 29                                       GAO/HEHS-97-59 Head Start Research
                              Appendix III
                              Summaries of Studies Included in the
                              Review




                              Population: Public preschool programs in North Carolina

                              Sample: 97 children participated in Head Start, 99 in community day care,
                              and 120 in no group care

                              Head Start program year(s): 1992-93

                              Measures/instrumentation: Reading and math subscales of the
                              Woodcook-Johnson Tests of Achievement (WJ-R), Peabody Picture
                              Vocabulary Test-Revised (PPVT-R), developmental assessment on
                              communication development, and an assessment on social behavior
                              completed by a kindergarten teacher; adapted questionnaire form of the
                              Communication Domain of the Vineland Adaptive Behavior Scale used to
                              provide a measure of children’s cognitive development; and Social Skills
                              Questionnaire used to measure teachers’ ratings of children’s classroom
                              behaviors

                              Findings: Significant group effects were found for the PPVT-R, with all the
                              groups performing better than children in the non-day care group. For the
                              WJ-R reading scale, the preschool group showed no effects. Significant
                              main effects were found for preschool group on the WJ-R math scale, with
                              the community day care sample scoring higher than the four other groups.
                              Significant preschool group differences were found on the Vineland
                              Communication Domain, with community day care children rated higher
                              than the other four groups. Social skills of community child care children
                              were rated significantly higher by their kindergarten teachers than
                              children who attended the standard or the family-focused classes or
                              children who did not attend group day care and marginally higher than the
                              Head Start children. On the Academic Competence scale of the Social
                              Skills Questionnaire, children who previously attended community child
                              care scored significantly higher than those in the other four groups.

                              Source: Donna M. Bryant, Ellen S. Peisnar-Feinberg, and Richard M.
                              Clifford, Evaluation of Public Preschool Programs in North Carolina,
                              University of North Carolina (Chapel Hill, N.C.: 1993).


The Impact of Escalating      Authors: Mary Anne Chalkley and Robert K. Leik
Family Stress on the
Effectiveness of Head Start   Outcome area studied: Family
Intervention



                              Page 30                                     GAO/HEHS-97-59 Head Start Research
                            Appendix III
                            Summaries of Studies Included in the
                            Review




                            Overview of study: Explored the effects that declining conditions among
                            the U.S. poor may have on the potential for intervention programs to make
                            a difference in the lives of those receiving services

                            Design: Pretest/post-test, followed up Head Start Family Impact Project
                            participants in 1993; comparison group recruited from Head Start-eligible
                            families

                            Population: Head Start families in Minneapolis, Minnesota

                            Sample: 130 of the 190 families in the original study

                            Head Start program year(s): 1986-87, 1989-90

                            Measures/instrumentation: Mothers reported various measures on their
                            families, themselves, and their children. Children completed the pictorial
                            form of Perceived Competence and Acceptance.

                            Findings: An examination of the absolute amount of change in the
                            mother’s perception of the child was inconclusive on the impact of Head
                            Start.

                            Source: Mary Anne Chalkey and Robert K. Leik, The Impact of Escalating
                            Family Stress on the Effectiveness of Head Start Intervention, paper
                            presented at the National Head Start Association’s 22nd Annual Training
                            Conference (Washington, D.C.: April 1995).


Developmental Progress of   Author: Laurna Champ
Children Enrolled in
Oklahoma Head Start         Outcome area studied: Cognitive and social
Programs in 1987-1988       Overview of study: Head Start students were tested in the fall and again
                            in the spring in multiple developmental areas.

                            Design: Pretest/post-test

                            Population: Children in 15 Head Start programs in Oklahoma

                            Sample: 120 students

                            Head Start program year(s): 1987-88



                            Page 31                                     GAO/HEHS-97-59 Head Start Research
                         Appendix III
                         Summaries of Studies Included in the
                         Review




                         Measures/instrumentation: Brigance Diagnostic Inventory of Early
                         Development and Head Start Measures Battery

                         Findings: Gains on the Brigance ranged from 9 to 16 months. Similar
                         claims were made for results on the Head Start Measures Battery, but
                         findings were reported in raw scores with no intrinsic meaning.

                         Source: Laurna Champ, Developmental Progress of Children Enrolled in
                         Oklahoma Head Start Programs in 1981-1988, unpublished manuscript,
                         1988.


Does Head Start Make a   Authors: Janet Currie and Duncan Thomas
Difference?
                         Outcome area studied: Cognitive and health

                         Overview of study: Examined the impact of Head Start on school
                         performance, cognitive attainment, and various health and nutritional
                         measures

                         Design: Post-test only, comparison groups comprised participants in other
                         preschool or no preschool

                         Population: U.S. Head Start participants

                         Sample: National sample of data for nearly 5,000 children from the
                         National Longitudinal Survey of Youth and the National Longitudinal
                         Survey’s Child-Mother file

                         Head Start program year(s): 1986-90

                         Measures/instrumentation: Picture Peabody Vocabulary Test and grade
                         retention

                         Findings: Head Start had positive and persistent effects on test scores and
                         school attainment of white children relative to participation in either other
                         preschool or no preschool after controlling for family and background
                         effects. An increase in test scores was noted for African American
                         children, but these gains were quickly lost, and there appeared to be no
                         positive effects in school attainment. Greater access to preventive health
                         care was reported for white and African American children who attended
                         Head Start or other preschools.



                         Page 32                                      GAO/HEHS-97-59 Head Start Research
                           Appendix III
                           Summaries of Studies Included in the
                           Review




                           Source: Janet Currie and Duncan Thomas, “Does Head Start Make a
                           Difference?” The American Economic Review, Vol. 85, No. 3 (1995), pp.
                           341-64.


A Comparison of Head       Author: Maria D. Esteban
Start and Non-Head Start
Reading Readiness Scores   Outcome area studied: Cognitive
of Low-Income              Overview of study: Followed up Head Start participants in kindergarten
Kindergarten Children of
Guam                       Design: Post-test only, comparison group comprised low-income
                           kindergarten students that did not attend Head Start

                           Population: Head Start participants from six public schools on Guam

                           Sample: 35 male and 35 female Head Start children and 35 male and 35
                           female non-Head Start children

                           Head Start program year(s): 1985-86

                           Measures/instrumentation: Brigance K&I Screen for Kindergarten

                           Findings: Differences among the four groups were not significant at the p
                           = .05 level. Head Start to non-Head Start comparison was not significant.

                           Source: Maria D. Esteban, A Comparison of Head Start and Non-Head
                           Start Reading Readiness Scores of Low-Income Kindergarten Children of
                           Guam, UMI Dissertation Services (Ann Arbor, Mich.: 1987).


The Effectiveness of       Author: Barbara A. Facchini
Family Health Care in
Head Start: The Role of    Outcome area studied: Health
Parental Involvement       Overview of study: Relationship of the amount of parental involvement
                           in the Head Start program to the amount of health care received by both
                           Head Start-age children and their siblings

                           Design: Post-test only, comparison group selected from waiting list

                           Population: West Haven, Connecticut, Head Start program



                           Page 33                                    GAO/HEHS-97-59 Head Start Research
                             Appendix III
                             Summaries of Studies Included in the
                             Review




                             Sample: 40 Head Start children and 20 waiting-list children for
                             comparison group

                             Head Start program year(s): 1980-81

                             Measures/instrumentation: Immunizations, physical examinations,
                             health screenings, and dental examinations

                             Findings: Immunizations were up to date for about one-half of both the
                             Head Start children and the waiting-list children before the beginning of
                             the Head Start programs. All of the Head Start children were up to date
                             during the Head Start year, but only a few additional waiting-list children
                             were up to date. Head Start children were more likely to receive health
                             screenings and dental examinations (p < .001). Head Start children were
                             more likely to receive physical examinations, but only the difference for
                             children of highly involved parents was significantly different from the
                             waiting-list children (p < .05). No significant difference for immunizations
                             was found between the siblings of Head Start children and siblings of
                             waiting-list children. Head Start siblings were more likely to have received
                             health and dental screenings (p < .05).

                             Source: Barbara A. Facchini, The Effectiveness of Family Health Care in
                             Head Start: The Role of Parental Involvement, Quinnipiac College (West
                             Haven, Conn.: 1985).


The Effects of Head Start    Authors: Linda B. Fosburg and Bernard Brown
Health Services: Executive
Summary of the Head Start    Outcome area studied: Health
Health Evaluation            Overview of study: Longitudinal study of the Head Start health services

                             Design: Pretest/post-test, longitudinal experimental design, involving
                             random assignment of children to a Head Start and a non-Head Start group

                             Population: Four large Head Start programs

                             Sample: 208 children completed both pre- and post-tests, 609 received
                             post-tests only

                             Head Start program year(s): 1980-81




                             Page 34                                      GAO/HEHS-97-59 Head Start Research
                           Appendix III
                           Summaries of Studies Included in the
                           Review




                           Measures/instrumentation: Pediatric, dental, anthropometric,
                           hematology, developmental, speech and language, vision, and hearing
                           evaluations and nutritional observation; parent interview addressed the
                           health history of child, nutritional evaluation of child, and family
                           background

                           Findings: Head Start children were more likely to receive preventive and
                           remedial health services than other low-income children in their
                           community. Head Start children were more likely to receive medical and
                           dental examinations, speech evaluation and therapy services, and vision
                           screen or examination. Head Start children tested at both pretest and
                           post-test were less likely to have speech and language deficiencies at
                           post-test. Nutritional intake evaluation showed exceptionally positive
                           impacts of Head Start’s nutrition services on children and their families.

                           Source: Linda B. Fosburg and Bernard Brown, The Effects of Head Start
                           Health Services: Executive Summary of the Head Start Evaluation, Abt
                           Associates (Cambridge, Mass.: 1984).


Children Are a Wonderful   Authors: Mary Fulbright and others
Investment: A Study in
Preschool Education        Outcome area studied: Cognitive and family

                           Overview of study: To examine the effects of preschool education on
                           children of low-income families in Dallas, Texas

                           Design: Post-test only, followed up students who had attended Sunnyview
                           Head Start Center during the previous 5 years; comparison group selected
                           from children in district who did not attend Sunnyview Head Start Center
                           but were matched on demographic characteristics

                           Population: Dallas Independent School District students who had
                           attended Sunnyview Head Start Center during the previous 5 years

                           Sample: 83 former Sunnyview parents and 76 comparison group parents;
                           for the grade retention analysis, 43 Sunnyview and 41 comparison students

                           Head Start program year(s): 1984-89 (estimated)

                           Measures/instrumentation: Demographic, economic, home
                           environment, and educational experience/expectation information was



                           Page 35                                     GAO/HEHS-97-59 Head Start Research
                             Appendix III
                             Summaries of Studies Included in the
                             Review




                             collected from parents. Grade retention information was gathered from
                             school files.

                             Findings: Significantly fewer former Sunnyview students had repeated a
                             grade than comparison group students. Sunnyview parents reported
                             significantly more educational items in the home than comparison group
                             parents.

                             Source: Mary Fulbright and others, Children Are a Wonderful Investment:
                             A Study in Preschool Education, Community Services Development
                             Center, Graduate School of Social Work, University of Texas (Arlington,
                             Tex.: 1989).


Health Services and Head     Authors: Barbara A. Hale, Victoria Seitz, and Edward Zigler
Start: A Forgotten Formula
                             Outcome area studied: Health

                             Overview of study: Studied the impact of Head Start’s health services on
                             children and their siblings

                             Design: Post-test only, comparison groups from the waiting list and from
                             a nursery school serving middle-class families

                             Population: Head Start participants in two adjacent small cities in
                             Connecticut

                             Sample: 40 Head Start children, 18 children on the Head Start waiting list,
                             20 children enrolled in a nursery school, and 103 siblings of the nursery
                             school children

                             Head Start program year(s): 1984-85

                             Measures/instrumentation: Immunizations, physical examinations,
                             health screenings, and dental examinations

                             Findings: Head Start children received more age-appropriate health
                             screenings than middle-class children and waiting-list children. Head Start
                             children were more likely to receive dental examinations than middle-
                             class children and waiting-list children. Head Start siblings were less likely
                             than middle-class siblings to receive age-appropriate immunizations and
                             health screenings.



                             Page 36                                       GAO/HEHS-97-59 Head Start Research
                              Appendix III
                              Summaries of Studies Included in the
                              Review




                              Source: Barbara A. Hale, Victoria Seitz, and Edward Zigler, “Health
                              Services and Head Start: A Forgotten Formula,” Journal of Applied
                              Developmental Psychology, Vol. 11 (1990), pp. 447-58.


An Analysis of the            Author: Kathleen Hebbeler
Effectiveness of Head Start
and of the Performance of     Outcome area studied: Cognitive
a Low-Income Population       Overview of study: Examined the long-term effectiveness of Head Start
in MCPS                       by comparing the performance of Head Start graduates in elementary and
                              secondary school to that of students who had applied for Head Start but
                              did not attend

                              Design: Post-test only, followed up three cohorts of Head Start graduates;
                              one cohort, the 1978-79 group, was within the scope of our study;
                              comparison group selected from waiting list for each respective year

                              Population: Children continuously enrolled in Montgomery County,
                              Maryland, school system between 1980 and 1984 and currently in the
                              fourth grade

                              Sample: Head Start group comprised 411 children; the comparison group
                              had 89.

                              Head Start program year(s): 1978-79

                              Measures/instrumentation: California Achievement Test, Cognitive
                              Abilities Test, special education placements, and grade retention

                              Findings: Head Start group had a higher percentage of students who
                              scored above the 80th percentile on one of the subtests of the Cognitive
                              Abilities Test administered in the third grade.

                              Source: Kathleen Hebbeler, An Analysis of the Effectiveness of Head Start
                              and of the Performance of a Low-Income Population in MCPS,
                              unpublished manuscript, Education Resources Information Center
                              (ED281674), 1985.




                              Page 37                                     GAO/HEHS-97-59 Head Start Research
                              Appendix III
                              Summaries of Studies Included in the
                              Review




A Comparison of the           Author: Elva Williams Hunt
Academic Achievement of
Urban Second Grade            Outcome area studied: Cognitive
Pupils With Different         Overview of study: Comparison of the academic achievement of urban
Forms of Public Preschool     second grade students from low-income families with different forms of
Experience                    public preschool experience

                              Design: Post-test only, followed up Head Start participants in second
                              grade; comparison groups comprised students with public preschool (First
                              Step) or no preschool experience

                              Population: Three cohorts of second grade students from the Newport
                              News Public Schools

                              Sample: 74 former Head Start students, 92 former First Step preschool
                              students, and 92 students with no preschool experience

                              Head Start program year(s): 1980-81, 1981-82, 1982-83

                              Measures/instrumentation: Standardized test scores and grade
                              retention

                              Findings: Achievement test scores of the three groups were not
                              significantly different. No conclusion was reached about the performance
                              of Head Start students on the grade retention measure.

                              Source: Elva Williams Hunt, A Comparison of the Academic Achievement
                              of Urban Second Grade Pupils With Different Forms of Public Preschool
                              Experience, UMI Dissertation Services (Ann Arbor, Mich.: 1987).


A Head Start Program          Authors: Ron Iverson and others
Evaluation in Terms of
Family Stress and Affect: A   Outcome area studied: Family
Pilot Study                   Overview of study: Assessment of the effect of a local Minnesota Head
                              Start’s family services on family stress levels

                              Design: Pretest/post-test, comparison groups selected from waiting-list
                              families and from the local population




                              Page 38                                    GAO/HEHS-97-59 Head Start Research
                        Appendix III
                        Summaries of Studies Included in the
                        Review




                        Population: Families with children enrolled in the Clay-Wilkin
                        Opportunity Council Head Start program

                        Sample: 149 Head Start families were surveyed at the beginning of the
                        program year and completed a post-test in May and a 1-year follow-up the
                        following May. Twenty-one waiting-list families and 35 randomly selected
                        families with young children from the general population were surveyed as
                        comparison groups.

                        Head Start program year(s): 1991-92, 1992-93

                        Measures/instrumentation: Index of Family Stress and
                        Adjustment—Stress was measured by a correlated subscale of 13 of the
                        original 55 stress items, and affect was measured by a correlated subscale
                        of 9 of the original 30 affect items.

                        Findings: Head Start and waiting-list families were both significantly
                        higher in stress means and lower in affect means than the general
                        population families. Head Start families were both significantly lower in
                        stress and significantly higher in affect than the waiting-list families at
                        Head Start post-test. Head Start gains in both stress and affect measures
                        appeared to reverse at the 1-year follow-up, but the changes were not
                        significant.

                        Source: Ron Iverson and others, A Head Start Program Evaluation in
                        Terms of Family Stress and Affect: A Pilot Study, State of Minnesota
                        Department of Jobs and Training (Moorhead, Minn.: 1993).


Final Report-The Head   Authors: Robert K. Leik and Mary Anne Chalkley
Start Family Impact
Project                 Outcome area studied: Family

                        Overview of study: Studied family functioning and optimal involvement
                        of parents in Head Start

                        Design: Pretest/post-test of two treatment groups—regular Head Start and
                        an enriched program with a comparison group selected from the Head
                        Start waiting list

                        Population: Head Start participants in Minneapolis, Minnesota




                        Page 39                                      GAO/HEHS-97-59 Head Start Research
                              Appendix III
                              Summaries of Studies Included in the
                              Review




                              Sample: 51 families in regular Head Start, 30 families in the enriched
                              program, and 21 waiting-list families

                              Head Start program year(s): 1986-87

                              Measures/instrumentation: Various measures of family characteristics,
                              mother’s evaluation of her child’s behavior and competence, and
                              children’s feelings of competence and social acceptance

                              Findings: Head Start families exhibited large and significant changes in
                              family cohesion and adaptability. Mothers in both Head Start groups
                              increased their evaluation of their children’s competence. Children in all
                              samples increased their sense of competence and acceptance.

                              Source: Robert K. Leik and Mary Anne Chalkley, Final Report—The Head
                              Start Family Impact Project.


A Longitudinal Study to       Author: Paula J. Nystrom
Determine the Effects of
Head Start Participation on   Outcome area studied: Cognitive
Reading Achievement in        Overview of study: Review impact of the Head Start program on
Grades Kindergarten           academic achievement of children
Through Six in Troy Public
Schools                       Design: Post-test only, followed up Head Start participants in
                              kindergarten through sixth grade; comparison group comprised children
                              who had not attended Head Start, but who were similar to the Head Start
                              group on certain demographic variables

                              Population: Head Start participants from three schools in Troy, Michigan

                              Sample: 54 Head Start children and 54 comparison children

                              Head Start program year(s): 1980-81, 1981-82, 1982-83, 1983-84

                              Measures/instrumentation: Metropolitan Readiness Test,
                              Gates-MacGinite Reading Test, Iowa Tests of Basic Skills, Cognitive
                              Abilities Test, special education placements, and grade retention

                              Findings: Mean scores at kindergarten were higher for the Head Start
                              group compared with the comparison group at kindergarten; no significant



                              Page 40                                      GAO/HEHS-97-59 Head Start Research
                           Appendix III
                           Summaries of Studies Included in the
                           Review




                           differences were found at any of the other grade levels. An analysis of
                           change in scores over time showed a significant difference in favor of the
                           Head Start group. No significant differences were found between the Head
                           Start and comparison group for special education placement and grade
                           retention.

                           Source: Paula J. Nystrom, A Longitudinal Study to Determine the Effects
                           of Head Start Participation on Reading Achievement in Grades
                           Kindergarten Through Six in Troy Public Schools, UMI Dissertation
                           Services (Ann Arbor, Mich.: 1988).


A Comparison of Long       Author: Yvonne B. Reedy
Range Effects of
Participation in Project   Outcome area studied: Cognitive, socioemotional, and family
Head Start and Impact of   Overview of study: Investigated possible differences among groups of
Three Differing Delivery   children receiving Head Start through three different delivery models
Models
                           Design: Post-test only, followed up Head Start participants after 2 to 4
                           years in public schools to examine the long-range effects of different
                           delivery models; comparison group comprised children who might have
                           attended Head Start but did not

                           Population: Head Start participants in rural Pennsylvania

                           Sample: 18 were children for each of the three groups: classroom, mixed
                           model, and home based; and 18 children were included in the control
                           group

                           Head Start program year(s): Not specified

                           Measures/instrumentation: Woodcock-Johnson Psychoeducational
                           Battery - Part II, Tests of Achievement; PPVT-R; Child Behavior Checklist -
                           Parent Rating Scale; Child Behavior Checklist - Teacher Rating Scale;
                           Vineland Adaptive Behavior Scale - Survey Form; and Head Start
                           Follow-up Family Questionnaire

                           Findings: No differences among Head Start and non-Head Start children
                           in reading, math, written language, or receptive language. Levels were in
                           the average range when compared with national norms. Head Start
                           children obtained significantly higher mean scores on the measure of



                           Page 41                                      GAO/HEHS-97-59 Head Start Research
                            Appendix III
                            Summaries of Studies Included in the
                            Review




                            general knowledge. Head Start children had significantly lower mean
                            scores on both subscales and total scale on the measure of maladaptive
                            behavior. Correlations with teacher reports were significant. On the
                            socialization scale, differences were not significant at p = .05. On the
                            adaptive behavior measures, the non-Head Start children obtained
                            significantly higher means on the communication, daily living skills, and
                            social skills domains, as well as on the total adaptive behavior score.

                            On the parent questionnaire, non-Head Start parents reported they felt less
                            capable of providing a good learning environment, spent less time working
                            with the child on homework or other learning activities, were less likely to
                            seek information about age-appropriate expectations, were more likely to
                            resort to spanking as a form of discipline, were less able to find
                            community services when needed and to feel their involvement with their
                            child’s education had resulted in any noticeable accomplishments. On the
                            daily living skills, social skills, and total independent living scales, the
                            children in the classroom model obtained lower means than the two
                            groups who received home visits. Parents of children in the classroom
                            model reported they spent smaller amounts of time working with their
                            children at home, and they were less likely to seek out information about
                            age-appropriate information and to feel that their involvement in their
                            children’s education resulted in any noticeable accomplishments.

                            Source: Yvonne B. Reedy, A Comparison of Long Range Effects of
                            Participation in Project Head Start and Impact of Three Differing Delivery
                            Models, Pennsylvania State University (State College, Penn.: 1991).


A Study of Duration in      Author: Joyce Harris Roberts
Head Start and Its Impact
on Second Graders’          Outcome area studied: Cognitive and socioemotional
Cognitive Skills            Overview of study: Assessed the impact of Head Start programming on
                            later school success and the development of social competence in its
                            graduates

                            Design: Post-test only, compared 1-year Head Start participants with
                            2-year Head Start participants and a non-preschool comparison group of
                            second grade classmates

                            Population: Second grade students in four public schools in a large
                            suburban school district



                            Page 42                                     GAO/HEHS-97-59 Head Start Research
                              Appendix III
                              Summaries of Studies Included in the
                              Review




                              Sample: 30 children with 1 year of Head Start year, 22 children with 2
                              years of Head Start, and 33 children with no preschool

                              Head Start program year(s): 1978-79, 1979-80

                              Measures/instrumentation: Locus of Control Scale for Children -
                              Pre-School and Primary, Form A; Self-Concept Inventory; and Cognitive
                              Abilities Test (Primary Level)

                              Findings: No significant difference was found between groups on any
                              measure.

                              Source: Joyce Harris Roberts, A Study of Duration in Head Start and Its
                              Impact on Second Graders’ Cognitive Skills, UMI Dissertation Services
                              (Ann Arbor, Mich.: 1984).


Changes in Mental Age,        Author: Linda L.B. Spigner
Self-Concept, and Creative
Thinking in Ethnically        Outcome area studied: Cognitive
Different 3- and 4-Year-Old   Overview of study: Studied Head Start participants’ progress after 8
Head Start Students           months of Head Start participation and conducted home interviews for the
                              children who showed the highest gains and for the children who made the
                              least progress

                              Design: Pretest/post-test

                              Population: Head Start participants in a north Texas community

                              Sample: 37 Head Start participants

                              Head Start program year(s): Exact year not specified

                              Measures/Instrumentation: Bankson Language Screening Test,
                              Developmental Test of Visual-Motor Integration, Peabody Picture
                              Vocabulary Test, Self-Concept Adjective Checklist, and Torrance Tests of
                              Creative Thinking

                              Findings: Average mental age gain of almost 11 months was significant at
                              the .01 level. Gains in self-concept and creative thinking were significant at
                              the .01 level.



                              Page 43                                      GAO/HEHS-97-59 Head Start Research
                      Appendix III
                      Summaries of Studies Included in the
                      Review




                      Source: Linda L.B. Spigner, Changes in Mental Age, Self-Concept, and
                      Creative Thinking in Ethnically Different 3- and 4-Year-Old Head Start
                      Students, Texas Woman’s University (Denton, Tex.: 1985).


Learning by Leaps &   Author: Texas Instruments Foundation, Head Start of Greater Dallas, and
Bounds                Southern Methodist University

                      Outcome area studied: Cognitive

                      Overview of study: Followed up Margaret H. Cone Preschool Head Start
                      program participants; cohorts 4, 5, and 6 participated in a new Language
                      Enrichment Activities Program

                      Design: Post-test only, comparison group comprised classmates who did
                      not attend the Margaret H. Cone Preschool

                      Population: Six cohorts of children attending the Margaret H. Cone
                      Preschool Head Start program in Dallas, Texas

                      Sample: Cohorts ranged from about 30 to 58 children

                      Head Start program year(s): 1990-96

                      Measures/instrumentation: Battelle Developmental Inventory, PPVT-R,
                      Clinical Evaluation of Language Fundamentals - Preschool, and Iowa Test
                      of Basic Skills

                      Findings: Results of cohorts 2, 3, 4, 5, and 6 revealed a pattern of
                      improved performance in vocabulary, language skills, concept
                      development, and social-adaptive skills during the years of the language
                      enrichment program.

                      Source: Texas Instruments, Head Start of Greater Dallas, and Southern
                      Methodist University, Learning by Leaps & Bounds, paper presented at
                      Head Start’s Third National Research Conference (Washington, D.C.:
                      June 22, 1996).




                      Page 44                                     GAO/HEHS-97-59 Head Start Research
                             Appendix III
                             Summaries of Studies Included in the
                             Review




Early Childhood              Author: Marium T. Williams
Educational Intervention:
An Analysis of Nicholas      Outcome area studied: Cognitive
County, Kentucky, Head       Overview of study: Examined the impact of the Nicholas County Head
Start Program Impacts        Start Program over a 12-year period
From 1974-1986
                             Design: Post-test only, followed up Head Start participants in first grade
                             through sixth grade; comparison groups were selected from comparable
                             first grade enrollment

                             Population: Children who entered first grade in the years 1975, 1976,
                             1979, 1980, and 1981 in Nicholas County, Kentucky; the first three groups
                             are outside our period of study

                             Sample: 14 Head Start and 9 comparison children for 1979-80 and 11
                             Head Start and 10 comparison children for 1980-81

                             Head Start program year(s): 1979-80, 1980-81

                             Measures/instrumentation: Comprehensive Tests of Basic Skills,
                             Cognitive Skills Index, mathematics and reading/English grades, Kentucky
                             Essential Skills Test, special education placements, and grade retention

                             Findings: No significant differences were found for most comparisons.
                             Reading scores for the Head Start children were significantly better than
                             those of the comparison group for 3 of the 6 years at the .05 level of
                             significance.

                             Source: Marium T. Williams, Early Childhood Educational Intervention:
                             An Analysis of Nicholas County, Kentucky, Head Start Program Impacts
                             From 1974-1986, UMI Dissertation Services (Ann Arbor, Mich.: 1988).


Is an Intervention Program   Authors: Edward Zigler and others
Necessary in Order to
Improve Economically         Outcome area studied: Cognitive
Disadvantaged Children’s     Overview of study: Studied changes in intelligence quotient scores of
IQ Scores?                   children attending Head Start




                             Page 45                                     GAO/HEHS-97-59 Head Start Research
Appendix III
Summaries of Studies Included in the
Review




Design: Pretest/post-test, comparison group comprised Head Start-eligible
children not attending Head Start; testing was done at three points in the
Head Start year

Population: Preschool children from economically disadvantaged
families living in low-income, inner-city neighborhoods in New Haven,
Connecticut

Sample: 59 Head Start children and 25 comparison children

Head Start program year(s): Not stated

Measures/instrumentation: Stanford-Binet Intelligence Scale, Form L-M

Findings: Both groups increased from test to retest, a result which was
attributed to familiarity with the testing situation. Only the Head Start
group continued to show improvement on the post-test, which was
interpreted as reflecting changes in the children’s motivation from
attending a preschool intervention program.

Source: Edward Zigler and others, “Is an Intervention Program Necessary
in Order to Improve Economically Disadvantaged Children’s IQ Scores?”
Child Development, Vol. 53 (1982), pp. 340-48.




Page 46                                     GAO/HEHS-97-59 Head Start Research
Appendix IV

Comments From the Department of Health
and Human Services




              Page 47        GAO/HEHS-97-59 Head Start Research
Appendix IV
Comments From the Department of Health
and Human Services




Page 48                                  GAO/HEHS-97-59 Head Start Research
Appendix IV
Comments From the Department of Health
and Human Services




Page 49                                  GAO/HEHS-97-59 Head Start Research
Appendix IV
Comments From the Department of Health
and Human Services




Page 50                                  GAO/HEHS-97-59 Head Start Research
Appendix IV
Comments From the Department of Health
and Human Services




Page 51                                  GAO/HEHS-97-59 Head Start Research
Appendix IV
Comments From the Department of Health
and Human Services




Page 52                                  GAO/HEHS-97-59 Head Start Research
Appendix V

Acknowledgments


             Many researchers and early childhood experts provided valuable
             assistance and information used in producing this report. In particular, we
             wish to acknowledge the following individuals who reviewed the draft
             report: Dr. Richard Light, Harvard University; Dr. Mark Lipsey, Vanderbilt
             University; Greg Powell, National Head Start Association; and Dr. Edward
             Zigler, Yale University. Although these reviewers provided valuable
             comments, they do not necessarily endorse the positions taken in the
             report.




             Page 53                                     GAO/HEHS-97-59 Head Start Research
Appendix VI

GAO Contacts and Staff Acknowledgments


                  D. Catherine Baltzell, Assistant Director, (202) 512-8001
GAO Contacts      Deborah L. Edwards, Evaluator-in-Charge, (202) 512-5416


                  In addition to those named above, the following individuals made
Staff             important contributions to this report: Sherri Doughty managed the
Acknowledgments   literature search and co-wrote the report, Wayne Dow led the literature
                  review and screening, and Paula DeRoy performed the literature searches
                  and collected the manuscripts.




                  Page 54                                    GAO/HEHS-97-59 Head Start Research
Page 55   GAO/HEHS-97-59 Head Start Research
Related GAO Products


              Executive Guide: Effectively Implementing the Government Performance
              and Results Act (GAO/GGD-96-118, June 1996).

              Head Start: Information on Federal Funds Unspent by Program Grantees
              (GAO/HEHS-96-64, Dec. 29, 1995).

              Early Childhood Centers: Services to Prepare Children for School Often
              Limited (GAO/HEHS-95-21, Mar. 21, 1995).

              Program Evaluation: Improving the Flow of Information to the Congress
              (GAO/PEMD-95-1, Jan. 30, 1995).

              Early Childhood Programs: Parent Education and Income Best Predict
              Participation (GAO/HEHS-95-47, Dec. 28, 1994).

              Early Childhood Programs: Local Perspectives on Barriers to Providing
              Head Start Services (GAO/HEHS-95-8, Dec. 21, 1994).

              Early Childhood Programs: Multiple Programs and Overlapping Target
              Groups (GAO/HEHS-95-4FS, Oct. 31, 1994).

              Early Childhood Programs: Many Poor Children and Strained Resources
              Challenge Head Start (GAO/HEHS-94-169BR, May 17, 1994).

              Infants and Toddlers: Dramatic Increase in Numbers Living in Poverty
              (GAO/HEHS-94-74, Apr. 7, 1994).

              Poor Preschool-Age Children: Numbers Increase but Most Not in
              Preschool (GAO/HRD-93-111BR, July 21, 1993).




(104849)      Page 56                                    GAO/HEHS-97-59 Head Start Research
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order
made out to the Superintendent of Documents, when
necessary. VISA and MasterCard credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address
are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office
P.O. Box 6015
Gaithersburg, MD 20884-6015

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (301) 258-4066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any
list from the past 30 days, please call (202) 512-6000 using a
touchtone phone. A recorded menu will provide information on
how to obtain these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov




PRINTED ON    RECYCLED PAPER
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. G100
Official Business
Penalty for Private Use $300

Address Correction Requested