oversight

The Decennial Census: Potential Risks to Data Quality Resulting From Budget Reductions and Cost Increases

Published by the Government Accountability Office on 1990-03-27.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                       United States General Accounting OfPice / ‘!D 7 6 0

8 GAO                  Testimony



                                                                      140960

 For Release           The Decennial    Census:
 on Delivery           Potential   Risks To Data Quality
 Expected at
 lo:00   a.m.    EST   Resulting   From Budget Reductions      And
 Tuesday               Cost Increases
 March 27,      1990




                       Statement  of
                       L. Nye Stevens,  Director
                       Government Business Operations        Issues
                       General Government Division
                       Before the
                       Subcommittee  on Census and Population
                       Committee on Post Office  and Civil    Service
                       House of Representatives




 GAO/T-GGD-90-30
                                                                      GAO Form 160 (12/W
                           THE DECENNIAL CENSUS:
                 POTENTIAL RISKS TO DATA QUALITY RESULTING
                 FROM BUDGET REDUCTIONS AND COST INCREASES
                             SUMMARY STATEMENT OF
                                 L. NYE STEVENS
                         DIRECTOR, GOVERNMENTBUSINESS
                                OPERATIONS ISSUES
The Census Bureau has made program changes to accommodate about
$129 million    in congressional    budget reductions   in fiscal  years
1988-90.     The most substantial     reduction,  $57.6 million,  was
made in the Bureau's     fiscal   year 1990 budget request.
As a result     of this $57.6 million     reduction,     the Bureau initially
cancelled    the vacant/delete    program.       This program is used to
verify    the occupancy status    of housing units       reported   as vacant
or nonexistent     during   the Bureau's    follow-up    operation.    The
Bureau estimates      that this program added about 1.7 million
persons to the 1980 census counts and could add 2 million               persons
in 1990.
Although     the Department        of Commerce has decided to reinstate                the
vacant/delete       program,     funds are not currently           available      to
implement      it.   The Bureau plans to fund the program either
through    savings from a higher-than-expected                mail response rate or
a supplemental       appropriation.          According     to Bureau officials,          in
the event it does not obtain              the funding      as planned,      the Bureau
may need to scale back postcensus                operations    beginning       in
August 1990.        Under these circumstances,             the Bureau would
preserve     operations    aimed at developing           counts for apportionment
purposes,     preparing    redistricting        counts for the states,            and
coverage evaluation        activities        needed for the possible           adjustment
of census counts.         However, other operations            may be at risk,         and
if they are not fully          implemented      or delayed,      this could impair
data quality       more severely       than in the 1980 census.
In addition     to budget reductions,       the Bureau had to adjust
planned operations       to accommodate unanticipated      cost increases,
the 1990 budget sequestration,          and new program priorities.       To
accommodate these changes,         the Bureau reduced edit follow-up        for
most mail-out     questionnaires.       Bureau officials   recognize   this
may pose some risk to data quality;            however, the Bureau has not
determined    its full     effect.    We encourage the Bureau to evaluate
the impact of this change on data quality.
The Bureau may also be vulnerable                 to cost increases          as full-scale
operations    begin.     These vulnerable            areas include       difficulties        in
recruiting    temporary      workers      and a lower-than-expected               mail
response rate.       Although      the Bureau has taken some important
steps to better      position      itself     for 1990 recruiting            challenges,
the Bureau is having difficulty               meeting     its early recruiting
goals.     If the Bureau encounters             staffing     shortfalls,        operations
could be delayed,       contributing        to increased       costs.
Mr.      Chairman           and Members of                   the     Subcommittee:


We welcome            this         opportunity               to discuss            our observations                  on the
decennial            census             budget      issues.             The’ Subcommittee               asked        that        we
discuss        the        Bureau’s              actions       to accommodate                  the    $57.6      million
reduction            in     its         $1.3     billion       fiscal            year    1990 decennial               census
budget        request             and the         effect       of these            actions          on the      quality           of
census        data.          In addition,                  I will          discuss       other       adjustments              the
Bureau        has made to planned                          operations             to accommodate
unanticipated                cost         increases,           the         1990 budget           sequestration,                  and
new      program          priorities.


I will        also        discuss          areas       where        the Bureau              is particularly
vulnerable            to additional                   cost     increases             as full-scale              census
operations            are about                 to begin.           These         include        difficulties               in
recruiting            temporary                workers        and a lower-than-expected                          mail
response        rate.


My comments               today          are based           on our         review       of    the Bureau’s             cost
estimates,            its         budget         memoranda,             and discussions               with      Bureau
officials.                We did          not     attempt          to verify            the accuracy            of    the
Bureau’s        cost         estimates.                My comments                on personnel           issues         are
based        on our soon-to-be-released                                 report       on the Bureau’s
recruiting            for         the     1989 address              list      development             activities.




                                                                    1
ACTIONS TAKEN TO ADDRESS BUDGET SHORTFALLS
AND COST INCREASES MAY POSE RISK TO DATA QUALITY


The Bureau               has made program                  changes            to accommodate                     about         $129
million          in congressional                   budget              reductions              for     fiscal         years
1988-90,          a major              portion      of which              occurs          in fiscal              year     1990.
Delays          in starting              and completing                   several              early      1990 decennial
census          operations--for                   example,          address             list          development
activities               for     suburban          and rural              areas --may                 have contributed                to
a total          of about              $118.1      million              in carryover                  funds      from     the
Bureau's          fiscal          years          1987-89      appropriations.                           These carryover
amounts          may have contributed                        to congressional                          decisions          to reduce
the Bureau's                   budget      requests          for         fiscal        years           1988-90.


In addition               to congressional                   budget          reductions,                 to accommodate
cost      increases              for     other      operations               and to fund                 new program
priorities,               the Bureau             made a major                change             in planned             procedures.
According              to Bureau           officials,              this      change may pose some risk                                to
data      quality.               In addition,              we believe                 this       change          could      hamper
the    progress            and the precision                       of     coverage              evaluation             activities,
such as the               post         enumeration           survey.              This          survey        determines
whether          each person              was counted               correctly                or missed            in     the
census.           If      an adjustment                 to the           1990 census               counts         is     to be
made,      it     will          be based         on these           survey            results.




                                                                    2             .
Reductions Made
to Meet Budget Shortfalls

To meet the                $57.6     million          reduction                 in    its      fiscal        year      1990
appropriations                 request,          in    late             October        1989,          the Bureau
cancelled            its     most expensive                  coverage                improvement             program--the
vacant/delete                check,        most       recently              estimated                to cost        about          $64
million.             This      check       is    used        to verify                that         housing      units         on the
Bureau’s           address         list      were correctly                      classified              during
nonresponse                follow-up         as vacant              or nonexistent.                       During
nonresponse                follow-up,           census            enumerators                visit       households                that
did      not      return       a questionnaire                     to verify                their       occupancy           status.


In 1980 the Bureau,                       through           its     vacant/delete                     check,        found          that
768,000           of the       8.4 million             housing              units           that      were     initially
classified            as vacant            or nonexistent                       were        in fact       occupied.                 The
Bureau           estimated         that      this      check             added about                 1.7 million            persons
to      1980 census            counts,          including                a higher            proportion             of blacks
than       in the general                 population.                    This        group         has traditionally
been undercounted.                        For    1990,            the     Bureau            estimates          that        this
program           could      add as many as 2 million                                 persons           to the        count.


When the Bureau                 cancelled             the         1990 vacant/delete                      program,            it      said
that       it     would      continue           the    necessary                 preparatory              activities                to
implement           the      program        should           sufficient                funding          become available
later           in the census.              The Bureau                   specifically                 identified            possible
savings           from a higher-than-budgeted                                mail           response         rate      as a

                                                                    3
potential           source          of funding.                  On January               30,           1990,     and again                  on
February           28,     1990,         the    Department                of Commerce’s                    Undersecretary
for    Economic            Affairs          said         in congressional                     testimony                that           the
Bureau       would         proceed          with         the     vacant/delete                     program            as originally
planned.            The Undersecretary                         said       that      the     Department                      would
request        a dire            emergency          supplemental                  appropriation,                       if
necessary,            to fund            the    vacant/delete                    program.


In order           to fund         the      vacant/delete                   program         within              its         current
budget       authority,                 the Bureau             would        need a mail                   response               rate
several        percentage                points        higher            than     what        it        has assumed                   for
planning           and budgeting                purposes.                 For planning                    the     1990 census,
the    Bureau        has assumed a response                               rate     of 70 percent,                           5
percentage           points          below         the      response             rate      that           was obtained                      for
the    1980 census.                  While         we agree              that     a higher-than-expected                                     mail
response           rate         would     generate             significant               savings,               there            is     no
current        reason            to expect          such         an improvement.


According           to Bureau             officials,               the     Bureau         will           proceed             with           the
vacant/delete                   program        regardless               of whether                 it     obtains               a higher-
than-expected               mail        response            rate        or a supplemental                         appropriation.
In the       event         it     does not          obtain            funds       from      either              of these
sources,           to cover          these         costs,          Bureau         officials                said        they           would
most      likely          scale      back decennial                     operations              beginning                   in August
1990.   They said,                   however,             that      under         this      scenario,                  three            areas
     ‘p
would be protected                       from program               reductions--activities                                   associated

                                                                    4
with      delivering            apportionment                 counts              to the     President,              preparing
redistricting                 counts         for    the     states,               and completing               the      post
enumeration              survey.


In scaling           back operations,                      some postcensus                   activities              may be
cancelled           or delayed,               posing        some risk               to data        quality.              For
example,          Bureau        officials            said         that       the     postcensus           local          review
program        may be cancelled.                       This         cancellation              would       limit          the
opportunity              of    local      governments                   to assist          in ensuring               a complete
and accurate               census       count        by identifying                    instances          where          the
Bureau      did      not       locate         housing         units.


In addition,               Bureau       officials             said          that     delays        may also           occur           in
the    search/match               operation.                This         operation           attempts           to assure
that      certain          population              groups         are       not     missed      or double             counted
and are        counted          at     the     appropriate                  geographic          location.                For
example,          this        would     include            persons           who were not             at their            usual
residence           on Census Day.                   According               to the Bureau,               if      delays         are
significant,               results        from       the      search/match                 may be excluded                     from
the    apportionment                  counts        that     will           be provided            to the         President.


They also           said       that     delays         may occur                in the keying             and coding              of
certain        population              characteristic                    data       and thus        delay         the
publication              of these         data.            This        is    similar         to what happened                    in
the*1980        census          when,         because        of budget               shortfalls,               the    Bureau
reduced        the number of staff                         who coded               population         characteristic

                                                                   5
data         from questionnaires,                        contributing                 to a l-year               delay      in    the
publication               of some of              these          data.

Unanticipated    Cost Increases
Result    in Reduced Follow-up                                 Procedures

In our         March         1989 testimony                    we said         that      as the Bureau                  moves       to

contain         costs,             it    would        simultaneously                  face         additional           financial
pressures.          1        This        has in        fact       now happened.                     The Bureau           has had
to adjust           its       fiscal         year        1990 spending                 plans          to shift          resources
to accommodate                 unanticipated                    cost        increases              and to fund           new
program         priorities.


Some      examples            of        the unanticipated                    cost      increases            include         a $6
million         increase                in estimated              mileage           reimbursement                costs      and
about        $9.3      million            to fund             1990 federal             pay raises.                New funding
priorities              include           the     addition             of a $4.2             million        coverage
improvement               program          that        will       count        parolees             and probationers                 and
$5.2      million            in additional                    funding        for      the post          enumeration
survey.


To fund         cost         increases            and to accommodate                         the     $7.7    million            1990
budget         sequestration,                   the     Bureau             decided       to reduce           edit        follow-up
procedures             for     mail-out            short-form                questionnaires                 by $36 million.
Initially,             the Bureau               planned           to have all                mail-out        short-form
questionnaires     edited.   Questionnaires                                           with      inconsistent              or
     .
1Status    of Plans for the 1990 Decennial                                             Census:           An Update,
(GAO/T-GGD-89-15,      Mar. 23, 1989).
                                                                       6
missing        population              characteristic                   data       would     have been resolved
either        by telephone              or a personal                   visit        by a census         enumerator.


Rather        than     do follow-up               work           on the         entire      inventory         of   short-
form     questionnaires                 that      have           inconsistent              or missing         population
characteristic                data--for           example,              age,       race,     and sex data--the
Bureau        now plans             to do follow-up                    work      on 10 percent           of     these
questionnaires.                     The Bureau            will         use an automated              statistical
technique            known as imputation                     to assign               responses       to those
questionnaires                that      have missing                   or inconsistent              population
characteristic                data.        These          responses              are     assigned       using
questionnaire                data      from     a geographically                       proximate        housing         unit.


According            to Bureau          officials,                the    reduction           in    follow-up           may
pose     some risk            to data          quality;            however,            the Bureau        has not
quantified            that     risk.           In addition,                  as we said           in testimony
before        this     Subcommittee               in January                 1990,       reduced     follow-up           will
decrease         the     amount of data                   collected              from households,               may make
the    post      enumeration             survey        matching               more difficult             and time-
consuming            and potentially                 impair            the      precision         of data       that     could
be used        to adjust             census       counts.2




2CriYical   Issues for Census Adjustment:      Completing Post
Enumeration    Survey on Time While Protecting    Data Quality,
(GAO/T-GGD-90-15,     Jan. 30, 1990).
                                                                   7
Impact of Reduced Questionnaire                                      Edit
Follow-up Should be Evaluated


We are           concerned          that      the       Bureau           dec.ided         to reduce         short-form
questionnaire                edit      follow-up               without            having      thoroughly            studied
its      impact       on data         quality.                 As part            of the Bureau's                1990
research           and experimental                     program,            the     Bureau         plans       to estimate
the      errors       that      imputation               introduces                into     census         data.
According            to Bureau             officials,               this      study        will      incorporate           some
analysis           of the       impact          of      reduced            edit     follow-up             on data
quality.             We encourage               the      Bureau            and the Department                   of Commerce
to     follow        through         on the          Bureau's              evaluation             plans     because        of    the
implications             not        only      for       data     quality            for     the      1990 census           but
also       for     planning          for      future           decennial            censuses.

ADDITIONAL FACTORS THAT MAY
CONTRIBUTE TO HIGHER CENSUS COSTS


The Bureau's             ability            to recruit               and retain             a sufficient               number         of
temporary           workers          to take            the     census            and the         level     of voluntary
public           cooperation          in completing                      and returning               census
questionnaires                represent              significant                  unknowns,         with       major     cost

implications.


Recruitment


In*prior           decennial          censuses,               the        Bureau       experienced              problems
recruiting            and retaining                  employees.                   Meeting         recruiting           goals     iS

                                                                     8
important             not      only       to ensuring                      that         a sufficient               number of            staff
are      available             for      hiring          but          also         for     providing              managers        .with
flexibility                 to release             employees                    that      are       not meeting               production
requirements.                       Meeting        production                     goals         also       is    important            to
completing              operations               in a timely                      manner         and thereby              remaining
within        budget.                Even minor               deviations                  from productivity                     goals
could       cause           census        costs         to escalate                      dramatically.                   For example,
the      Bureau        estimates                that         for       each across-the-board                             percentage
decrease            in productivity,                      costs              increase            by $6.3 million.


During        the      1990 precensus                     activities,                     the Bureau              experienced
staffing            difficulties,                 most             recently              in 1989 when the Bureau
developed             its      urban       address                 list.           About         47 percent              of    the
Bureau’s            district            offices           did          not        meet recruiting                   goals       by the
time       this      field           activity           began.                  Staff          shortages           in 15 of           its       109
district            offices           were severe                   enough              to contribute               to delays               and
increased            costs           in completing                     field           activities               in most of these
offices.


The Bureau             has taken                some important                          steps       to better            position
itself        for      the          recruitment               challenges                  it     faces          during        1990.          For
example,            the Bureau             implemented                      a geographic                 pay system             that
should        enable           it     to better               compete              for         employees          needed        for
census        positions.                 Also,          at         the Bureau’s                  request,           the       Department
of    Health         and Human Services                             decided              to allow           states        to    request
waiv*ers          from provisions                      that         would          require           Aid        to Families            with

                                                                            9
Dependent              Children          recipients                to have         their      benefits           reduced                if
they      work on the              census.              In addition,                 last     year’s          legislation,
sponsored              by this       Subcommittee,                   enabling           federal         retirees             to work
on the          census        without        reductions                   to their          annuities          or salaries
should          also       increase         the    potential                 number of applicants                      for
census          employment.


Despite            these      efforts,        the           Bureau         faces      a considerable                challenge
in     recruiting             and retaining                  the     390,000          temporary          workers             it
will      need this             year.        Uncertainty                   about      the competitiveness                          of
its     geographic              pay rates              is    a major          concern.            These pay rates
were      not       based on local                market            analyses          and were not               tested            under
census-like                conditions.             The geographic                     pay plan          contains             seven
different            wage rates,             ranging               from      $5.00      to $8.00         per hour                 for
enumerators.                  The maximum pay rate                           was based on the                  results             of
the     pay increase               the Bureau                used     to attract              enumerator            staff               to
complete            the     1988 suburban                   and rural            address        list     development
activities              in six       East     Coast           areas.               The $8.00           rate     was
established                arbitrarily            in        1988,     according              to the Bureau,                  but
appears          to have assisted                  in attracting                     applicants.


The real            test      of the effectiveness                          of     the geographic               pay rate
will     come when the Bureau                           begins        hiring          for     peak activities,
which        are       scheduled          to run            from     April         to June.            However,          the
Bureau’s            early       1990 recruiting                     performance              indicates          difficulties
in Geeting              recruiting           goals.            For         example,          92 percent           of     the

                                                                    10
district           offices            had not           met      the Bureau’s                 March 9 recruiting
milestone             of 70 percent                    of the         1990 recruiting                  goal.            Nationally,
the     Bureau           has met            39 percent            of        its     goal.


Level       of public             cooperation


Another         significant                  determinant               of         census      costs        is    the     overall
mail       response          rate.            As I discussed                       earlier,          the Bureau           has
budgeted           for     a 70 percent                   mail        response             rate     and hopes            that      it
may     be higher.                However,               in     the    event          the     response           rate      is
lower,        follow-up               costs       will          be higher             than        anticipated.              The
Bureau        estimates               that       for      each percentage                     decrease           in the mail
response           rate,         follow-up              costs         increase           by $10 million.                    If     the
mail       response          rate        falls          significantly                  below         the planned            level,
the     Bureau           faces        the     potential               of substantial                  cost       increases.
Under       this         scenario,            the       Bureau         may        be forced           to request
additional            supplemental                     funding         or further                 reduce        or delay         census
operations,               which        could           result         in additional                 adverse        effects              on
data       quality.




That       concludes             my    prepared               statement,              Mr. Chairman.                My

colleagues            and I would                 be pleased                 to respond             to questions.




                                                                       11