Impacts of Education Reform

Published by the Government Accountability Office on 1989-03-07.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                  United States General Accounting Office

For Release         Impacts   of Education    Reform
On Delivery
Expected at
lo:00 a.m. EDT
March 7, 1989

                    Statement   of
                    Eleanor   Chelimsky
                    Assistant   Comptroller    General for
                    Program Evaluation      and Methodology
                    Before the
                    Subcommittee    on Eiementary,    Secondary
                      and Vocational     Education
                    Committee    on Education    and Labor
                    United States House of Representatives

Mr.     Chairman            and Members of the                         Subcommittee:
          I am very            pleased             to     be here           today         to discuss             GAO's work                on
education            reforms          and their                impacts.                Specifically,               Mr.       Chairman,
you asked            us for         an evaluation                     of the           effects        of       recent        reforms            on
the     achievement                of disadvantaged                        students,             on their          dropout             rates
and on their                enrollments                  in    vocational                education.              My testimony
today       thus          addresses              a general            concern            about      how the         specific
reforms        that         have imposed                  stricter            new requirements                     in high             schools
might       have affected                   the      nation's              most        disadvantaged,               at-risk
          On balance,               our          study        shows that               in the       four        large-city                school
districts            we evaluated,                   education              reEorm         has been neither                        a
disaster           nor a boon for                    the       performance                of     low-achieving                    students.
Looking        at the          performance                    of all        students             in these          districts,                 we
did     not    find         that     the          reforms          we examined                  improved         education
outcomes           a great          deal.            Finally,              prior         progress          was not           maintained
for     some disadvantaged                         groups,            at least            over     the      short          term.           The
big     question,            then,          is     the        longer        term:         whether          performance                 will
improve        over         time.           It     is     important               to     note     that      the     students               we
studied        were the             first          to     experience               the      reforms'            effects.
         We are            completing              our        analyses            this     month         and the           full
findings           will      be included                  in a written                   report        later       this           year.         Our
study       offers          quasi-experimental                         evidence            on the          local        effects            of
state       reforms          in     four          major        city        school         districts             located            in four
geographically                 dispersed                 states.            Its        results        cannot        be
generalized                beyond      these             sites.


             The four             school            districts           in our            study      vary     in enrollment
from        40,000          to    230,000.               All      are       in     urban       areas.         One is           located         in
the     northeast,                two are            in the        southeast,               and the          fourth       is      in the
southwest.                  Black        and Hispanic                  students            are      in the     majority            in each
district.                  The legislative                     mandates            these       districts           implemented
included             varying           combinations                of
            --       requiring               specific           tests        for        graduation,
            --       increasing               academic            course           requirements,
            --       tightening               attendance               rules,
            --       setting           Ilno pass/no               play"          rules      governing           participation                  in
                     sports         or other             activities,                and
            --       requiring               or making            available               varying       degrees          of      remedial
                     classes           for      those          in difficulty.
Other        details             about        the       four      school           districts          we studied               can be
found            in table         1.

            The basic             data        for      answering             the        questions           came from            school
districts'                 computerized                student          records.               Our design             called       for
achievement                 tests        and other              data        on students              as far        back        as 1982,
so that            we could            analyze           students'               high      school       careers         before           and
after            reform.          Altogether,                  we traced            the     progress          of      61,500
students             as they           moved through                   secondary            schools.            I would           like      to
express            my appreciation                     here       to the           four     districts           for      their
cooperation                 in assembling                 the      large           data     files       we asked          for.           We
also        visited          each district                     and interviewed                    central      office           officials

    ‘Pdl,le    I :        Characteclstlcs                  ot         selected              Scnool             ulstricts

                                                                                                                                     School       Number      ot     cohort
                                    MAJOR reforms                      attectiny                                      city           district     students          entered             nace/etnn1c             enrolllnent
    Dlstrlct          Region        students                                                                          populatlo"     enrollment   studied           grade           Y   wnite
                                                                                                                                                                                        ___           Ulack
                                                                                                                                                                                                      -                tiispanlc

       A             SW             Protlclency                  test                                                 YUU ,uuu       13u.uuu      17,7nn           IY84-d5              lY#              50%                  ‘!ns

                                    More        academic               courses

                                    Stricter             attendance                      rules

                                    "U"        no longer               passing

                                    Hemedlatlon                  tar       low       test-

                                    "NO pdss/no                  play"            rule

       I3        NE                 Protlclency                  test                                                 38U ,uuu        2u,uuo         L,btiU        IYtli-tlb             1U              b4                   Lb

                                    Remedial             rrelp         reyulred
       c             SE             aore        credit           In     ,lldttI      b Science                             70,uuu     4u,uuo                                             44              '1 5

                                    Hetnedldl    nelp                  reyulred                  tar
                                    low test-scores

                                    "No     pass/no              play"            rule

                                    Fewec     vocdtl"nd,                     education

       D         SE                 tiusf       pass      proficlrncy                       test                           350,uoo   230,rJOU      36,000          1983-84

                                    Addltmndl               science                credit

                                    Extra        period           added            III      3rd         year

                                    Remedial             "elp          authorized                      but
                                    not        kunded
as well         as principals              and teachers                    in high            schools.          Before
presenting             our   findings,              let     me first              address           the     question       of
whether         and how educators                     in    our         four      school        districts             attended           to
the     needs      of disadvantaged                       students             during         the     period       in which            the
reforms         were      initiated.


         One hypothesis                about         current             education             reforms         is that,
although         clearly         intended             to    raise          educational                achievement          for         all
students,          they      could       actually             be harmful                for     disadvantaged,               low-
achieving          or at-risk            students             by creating                additional             barriers          to
school         completion          without           providing                 resources            and assistance               for
them to         meet      the    new standards.                         Did we see evidence                     that      this         was
happening          in our four             school           districts?                  We did         not.        In describing
their         implementation             of     their            states'          new requirements,                    educators
in these         cities         told     us of a wide                    variety         of     initiatives             to help
students         meet the          higher           standards.                  These         included,         for     example:
         --      offering          an optional                additional                period            in the      day even
                 when the          state        did        not      fund        it,     to     help        students       take         all
                 the      required         courses:
         --      altering          teaching               methods,             class     sizes,            and the      content
                 covered,          to help            students             learn        enough         to pass         required
         --       increasing           individual                 attention             by counselors                 to students
                 who may not             meet         increased                requirements;

           --      offering           special         remedial            classes            during      the       regular             year,
                  or in        special          periods         before           and after            school        or on
                  weekends            for     students          who fail               the    required            high        school
                  exit        examination;
           --      offering           summer work-study                    for         students        who failed                 the
                  exit        exam     during         the      year,       providing              remedial          classes             in
                  the     morning            and a job          in the           afternoons;
           --      requiring           teachers            to develop             special          individual
                   improvement               plans      for     low-achieving                    students          to target
                   instruction               on specific             skills            needed:
           --      reorganizing               school          to provide               special        self-contained
                  programs            and extra            attention             for     students            needing
                   extensive           help      in     all     basic         skills.
           We did        not    evaluate             the      adequacy,           in terms            of quantity                 or
quality,           of    each district's                    specific          education            programs              in
relation          to     students'            needs.           However,           it     appeared            to    us that             the
districts              were    in general             making         serious            efforts        to be fair                 in
helping          all     students            meet the          new requirements.
           The effect           of these             efforts         is    the         question        I turn            to    next.


           As I have           already          noted,         some people               argue        that        disadvantaged
students--those                 who reach             high      school           already          achieving              poorly         or
having          been held            back,      who are         limited           in their            English            ability,            or
who are          members of minority                        racial        or ethnic              groups--may               find        the
increased             requirements                 of education               reform          an insuperable                   barrier
to graduation.                      Thus,         one guess          about         the       likely       effects          of the
higher        hurdles              set     by reform         would           involve           student          failure          and
exit.         Achievement,                  while       perhaps            increasing             for     some      students,
might        not      rise         among those             who are           at-risk.             For these             students,
more vivid               failure           in classes           and on required                       tests      may be
accompanied               by restricted                 opportunities                   to    participate               in the
athletic           and extracurricular                       life        of the           school,         which         together
with       other         frustrations,               could          lead      to    increased             drop-out             rates.
           On the         other           hand,     many legislators                      and school             officials
believe           that       the         reforms     could          have their               intended           effect.
According            to this              hypothesis,           the        changed           legal       framework             of
increased             requirements                 would     be translated                    by teachers               into        higher
expectations                 for     all      students,             better         identification                  of     learning
gaps,        and increased                  provision           of extra            help,         such as the
initiatives               taken           in our     four       school           districts,              to help          low-
achieving             students             bridge       those        gaps.          In this             view,      at-risk
students           would           be even more likely                      to     benefit            from      reform         than          more
academically                 advantaged             students.
           The key question                       we asked          in comparing                student          achievement
before        and after              the      introduction               of the           reforms          is whether               there
are       benefits           for     low-achieving                  and minority                students           as well              as    for
higher        achievers              and white             students.               Data       from       our     four      districts
showed that,                 in general,             low-achievers                  did       not       universally              fall
behind        after          the     reforms,           as had been feared.                             Indeed,         we found
some       test       score         trends         showing          students            gaining          more from             school

after       reform          than       before,          and this            was true             for         both      low-
achieving           and for            higher-achieving                     students.                  However,          these             gains,
overall,          were        very       modest.          As for            instances             of markedly                   worse
drops       in    achievements                through        high           school         after             reform,          these         most
negative          results             actually          happened         more often                    for     the      higher-
achieving           group.
           Let    me      now turn           to a more detailed                       analysis                of these              findings.

Impacts          on At-Risk              or Low-Achieving                    Students

           We defined              at-risk        students          in       all      four        districts              as those                at
the      34th     percentile              or lower         on grade                 8 reading                achievement               tests.
We tracked             the      progress          of these          eighth            graders                through          high         school
in      terms     of their             performance           on reading                   and mathematics                       tests.            We
also       tracked          similar          results       for     ~higher-achieving--that                                    is,      not       at-
risk--students.                       We examined          the      performances                       of two groups                   of at-
risk       students:            a group          that     completed                high      school            before           the
reforms          were        in effect           (the     pre-reform                 group)            and the          first          group
to pass          through           school        under     the      full            reform        requirements                      (the
post-reform               group).             Thus,       the     principal                reforms              (such         as
increased           course            requirements           and graduation                       exams)            were        in effect
throughout             the      high      school         years        for      the        post-reform                  groups,             but
did      not     affect         the      pre-reform          groups            at all.
           We have           results,         then,       across            their         high         school          careers             to
compare          two cohorts              of at-risk             students             (pre-reform                   and post-

reform)            in    four      cities               on two achievement                          measures           each--reading
and mathematics--or                           a total             of eight              achievement                comparisons.
           Our findings                     are     negative             on five             of these             eight       outcomes,                 as
shown at the                   bottom         of        figure          1.        The most negative                         of these             is     in
District            B, where                we see a decline                         throughout             high       school           by both
the       pre-reform              and post-reform                        groups          in     reading             achievement.
Further,            the        decline             is    slightly              sharper          for        the      post-reform                  group.
The group               starts         grade            8 slightly                worse       off         than      the      pre-reform
group        (at        the      22nd percentile                       versus           the     23rd)            and drops           to the
15th percentile,                       thus         declining                 7 percentile                 points           or one more
than       their         predecessors.
           How do we assess                         the        other          four      cases         of no net              gain?          In
District            A, it         is        true        that      the        post-reform                  group       improved            its
percentile               standing                 in both         reading             and mathematics                       relative             to
the       norms.              In addition,                 the      post-reform                group             has higher             test
scores        by grade                 11    than        the      pre-reform                 group.              But the        growth                rate
of the        post-reform                    group         throughout                 high      school            was lower             than           that
of the        pre-reform                    group.             (This          finding         is      reflected              by the
somewhat            steeper             lines           shown in the                  graph         for     the      pre-reform                  group
in     District            A.)          Moreover,                the     post-reform                  group         had a higher
initial            grade         8 performance                    than         the      pre-reform                group.          These
initial            differences                 may result                    from     changes             in the          community              or in
testing            practices,                or could             represent               stronger               performance              at the
elementary               grades.                  In any case,                 the      post-reform                 group's            initial
advantage               fades      in high               school,              despite         reform             efforts.

                                Figure 1:
            Achievement of At-Risk Students: Comparisons of
       Pre-reform and Post-reform Groups from Grade 8 Through
                         High School Testing

                   Pattern of Net Gain for Post-Reform Students                                      Relative
                                  to Pre-Reform Students

a. Reversal from negative                        to positive trend

           District B -        Math                        District C -       Math                 District C -   Reading

                 Patterns         of No Net Gain for Post-Reform Students                                 Relative
                                          to Pre-Reform Students
a. Positive trends for                                                                     b. Negative trends for
   both cohorts; less                                                                         both cohorts; slightly
   gain after reform                                                                          greater decline after

           District D -          Math
                                                     o _____

                                                         District D -        Reading
                                                                                       ,   ;:q_l

                                                                                                   District B -   Reading

46                                                  46

           District A -          Math                     District A -       Reading

       -            Post-reform     cohort
        ------      Pre-reform     cohort                                9
         In addition             to the        District             A results,                we found          two more
examples          of positive            trends         for      both         the      pre-      and post-reform
groups      but      lower       rates        of gain           after         reform.            In District               D, the
post-reform             group      started           slightly           ahead of the                  pre-reform            group        in
both     mathematics             and reading,                 but      lost      those         advantages               and ended
up only         equal     to,      or slightly                lower         than,       its      predecessor              by the
final      testing        year.
         On the         other      hand,       we do have findings                            of net          gain      on some of
the     eight      outcomes.             The upper              part        of figure            1    shows three                clear
examples          of positive            impacts          of education                  reform          for     at-risk
students.            Note       that     in    all      three          cases,          the     post-reform               group
median      percentile             scores        increased              throughout               high         school,       while
the     pre-reform           group       had had declines.                            In short,           in these          three
cases,      the      post-reform              group       o f at-risk                 students          both         improved
their      performance             relative           to the           national           norms and improved                       more
than     the      pre-reform           group         (whi ch,          in     fact,      had declined                   rather       than
improved).              These three            positive             impacts            were      found         in     District           C
in both         reading         and mathematics                  and in          District             B in mathematics.
         These       achievement              results           do suggest              that         at-risk          students
have not          suffered         "disasters"                as a result               of education                  reform        in
these      four      cities.           But our          f ind ings            on the          eight       outcomes          are
both     mixed       and modest.

Impacts          on Black           and Hispanic                        Students

          We paid          special             attention                 in our       study         to the     impacts          of
education              reforms       on minority                        students.            I move now to the                  results
for      all     black       and Hispanic                    students,              not      just     those         with      lower         test
          For blacks,               the        results             are about           the        same as I have               reported
for      students          at-risk             due to             low      reading         achievement:              3 cases          of
overall          net      gain       (see       figure             2).         The only           difference           is .for
reading          in     District           D.         Those             results       showed a slight                 positive
trend          (though       no net            gain)         for         at-risk       students             (see     figure          1)     but
are      increasingly               negative                for         blacks      after         reform.           Figure      2 thus
shows two such cases                           of     increasingly                  negative          trends         for      black
students,              compared           to    only         one for              low-achieving              students          (in
figure          1).
          For Hispanics,                   the        results              were different.                   We had data              for
only      six     comparisons                  instead             of eight           due to the             very      small
Hispanic              student       population                    in District               C.      The results            showed no
instances              of net       gains            for     the         post-reEorm              group      and two cases                  of
increasing              negative           trends.                 Figure           3 shows the             reading        decline
for      Hispanics           in Districts                    B and D, which                      worsened      after          reform.
Two other              results--mathematics                              in    Districts            B and D--show              the
Hispanic          students'               performance                    relative          to     national          norms declined
somewhat          during           high        school             for     both       the     pre-reform             and post-reform
groups.           The results                  for         reading            and mathematics                in District              A