oversight

Air Pollution: Reliability of EPA's Mobile Source Emission Model Could Be Improved

Published by the Government Accountability Office on 1990-05-14.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                                                                                                                                              ‘I
                       -,   ‘.i   .      _I”   .:                      z.    .‘i,   I’   .,       ,.   ^,   “,   ,Ir-   ‘-   ,   li,,   ,..
        ;      ..‘.,
                                                                                                                                                   i




                                      United        States   General        Accounting        Office
                                                                                                                                                       *’
i GAO                                 Report to the Chairman, Subcommittee
                                      on Oversight and Investigations,
                                      Committee on Energy and Commerce,
                                      House of Representatives

;., May 1990
                                      AIR POLLUTION
                                      Reliability of EPA’s
                                      Mobile Source
                                      Emission Model Could
                                      Be Improved
          .,         1
        ‘! :
         ,(          1                 ;’         .,’

 . I
  ” ,,::1     ;” ”                            I. :,
     “.:.&   *, “:
$,+..‘;,-                       ,_,,        -,.
     ,.          r’!...   .,’
GAO                United States
                   General Accounting  Office
                   Washington, D.C. 20648

                   Resources, Community,         and
                   Economic Development          Division

                   B-239214

                   May 14,199O

                   The Honorable John D. Dingell
                   Chairman, Subcommittee   on Oversight
                     and Investigations
                   Committee on Energy and Commerce
                   House of Representatives

                   Dear Mr. Chairman:

                   This report responds to your request that we examine the reliability               of
                   the Environmental      Protection  Agency’s (EPA) mobile source emission fac-
                   tor model (MOBILE4) and the effect of budget constraints          on its reliabil-
                   ity. The model is used in estimating       motor vehicle emissions of ozone
                   precursors   (hydrocarbons      and nitrogen oxides) and carbon monoxide
                   and in devising measures to reduce the atmospheric            concentrations      of
                   these pollutants.   In February    1989 WA made MOBILE4available to states
                   for use in developing     plans for improving    air quality.


                   The mobile source emission factor model has undergone continuous                   revi-
Results in Brief   sions* to more accurately    represent prevailing          conditions  and the model
                   year vehicles or fleet in use. According         to EPA,certain MOBILESassump-
                   tions need to be refined and revised to enhance the model’s reliability.
                   However, although the overall reliability            of MOBILEShas not been deter-
                   mined, EPAbelieves its statistical     reliability      should surpass that of the
                   previous version, MOBILE%Meanwhile,              EPAcontinues developmental
                   work to reflect changing conditions        in the vehicle fleet, address known
                   limitations, and update assumptions          used in the model. These actions
                   should improve the reliability     of the next version of the model. For now,
                   however, MOBILES remains an important              tool to understanding    efforts to
                   limit and decrease atmospheric       ozone and carbon monoxide pollution.

                   Although     work on model revisions has been continuous,       funding con-
                   straints have limited EPA'Sability to perform emission testing of later
                   model vehicles. Therefore,      ~0~1~~4’s estimates for later model vehicles
                   are statistically  less reliable than its projections for vehicles of earlier
                   model years, because of the smaller sample size for later model years.
                   However, EPAhas begun addressing past funding inadequacies             by pro-
                   viding increased funds in fiscal years 1989 and 1990 to increase the
                   sample size of later model vehicles. For example, funding increased from


                    ' MOBILE1
                            was made available   to states in 1978, MOBILE2 in 1981, and MOBILE3 in 1984.




                   Page 1                  GAO/RCEDW138        Reliability   of EPA’s Mobile   Source Emission   Model
              B-239214




              $807,000 in fiscal year 1988, to $1.5 million                 in fiscal year 1989, and to
              $2.1 million in 1990.


               The mobile source emission factor model is an integrated         collection of
Background     mathematical     equations that manipulate   certain variables-including,
               but not limited to, vehicle age and mileage, percentage of driving in dif-
               ferent conditions,   average vehicle speed, ambient temperature,          and rate
               of tampering    with emission control systems-to     estimate the grams of
               pollutant  (hydrocarbons,    nitrogen oxides, and carbon monoxide) emitted
               per mile driven. The estimate applies to the current fleet of vehicles,
               which consist of those models produced within the last 20 years.

               States use the MOBILES model in estimating       motor vehicle emissions that
               contribute   to ozone and carbon monoxide pollution.      These estimates are
               used in preparing    State Implementation     Plans, which outline pollution
               control measures designed to allow states to attain national air quality
               standards. The estimates are also being used by states and EPA to evalu-
               ate emission reduction strategies, such as vehicle inspection and mainte-
               nance programs; and the effectiveness        of potential mobile source
               regulations,   such as EPA’S fuel volatility  controls.

               MOBILE4 is the latest version of EPA’s mobile source emission factor
               model,* which has undergone periodic revision, updating, and refine-
               ment to more accurately     represent prevailing  conditions    and the fleet of
               vehicles in use. Each new version eliminates model years older than 20
               years, reflects recent advances in emission-control      technologies,  and
               tries to correct weaknesses in previous versions that have resulted in
               over- or underestimation     of emissions. The next version of the model,
               MOBILES, is due for release within 5 years.

               MOBILE4 estimates        four types of vehicle          emissions:

             . exhaust (from vehicle tailpipes),
             . nonrunning   evaporative    (from a vehicle that is parked with the engine
               turned off),
             . refueling (occurring    when a vehicle is being refueled), and
             . running-loss  evaporative    (occurring  while a vehicle is being driven).


                2 According to EPA,MOBILEX was issued as a general statement of policy and not a regulation sub
               ject to the notice and comment requirements under the Administrative Procedures Act. EPA did, how
                ever, hold a series of public workshops during the development of MOBILE4 to exchange information
                with the American Petroleum Institute, motor vehicle manufacturers, and others.



               Page 2                    GAO/RCED-9@138      Reliability   of EPA’s Mobile   !3ource Emission Mode
                 B-239214




                 According     to EPA,although the overall reliability         of MOBIL& has not been
Reliability of   determined,     its statistical   reliability  should surpass that of MOBILES,
MOBILE4          which was found to have an acceptable statistical               reliability   (within    k
                 22 percent at a 95-percent confidence level based on a Department                     of
                 Energy sponsored study). However, as a result of a reduced number of
                 vehicles tested in recent years, MOBILEA'semission estimates for later
                 model vehicles are less reliable than its projections            for earlier model
                 years. Even if a greater number of vehicles were tested, it still would not
                 address the validity        of many other assumptions        used in the model that
                 are crucial to the overall reliability        of its estimates. For example, there
                 is no way to quantify         the reliability  of model assumptions         about future
                 conditions-such         as the mix of emission control.technologies            in future
                 fleets.

                 A recent study by Southwest Research Institute3 questioned the overall
                 reliability of an adapted form of MOBILESdeveloped by the state of Cali-
                 fornia. California’s version has been modified to reflect that state’s more
                 stringent emission control requirements.   According   to this study, which
                 compared model estimates against emissions measured in a highway
                 tunnel, California’s  version of MOBILESunderestimated     hydrocarbon
                 emissions, on the average, by 74 percent (i.e., measured emissions were
                 about four times greater than model estimates) and carbon monoxide
                 emissions, on the average, by about 63 percent (measured emissions
                 were about three times greater than estimates).

                 According    to the Senior Project Manager for mobile models, EPAis cur-
                 rently trying to determine why the measured and estimated emission
                 rates differ so greatly. Specifically,   EPAis evaluating the Southwest
                 Research Institute     study methodology    and reassessing MOBILE~‘S
                 assumptions     concerning the prevalence in the fleet of super emitting
                 vehicles (vehicles with exceptionally     high emissions rates) and vehicles
                 with evaporative     system leaks. According     to EPA, if the underestima-
                 tions are real, then MOBILESwould also underestimate           these emissions by
                 large amounts, because MOBILESestimates for these emissions are not
                 dramatically    greater than those of California’s    MOBILESversion. In that
                 case, use of MOBILEd'Sestimates could result in overly optimistic         attain-
                 ment projections     with respect to hydrocarbon     and carbon monoxide
                 emissions.




                  3 Measurement of On-Road Vehicle Emission Factors in the California South Coast Air Basin, Volume
                  1, Regulated Emissions, Southwest Research Institute, June 1989.



                  Page 3                    GAO/‘RCEDW13l3      Beltability   of EPA’s Mobile   Source Emission   Model
                            5239214




                            Concerns were raised about the reliability     of MOBILESby the American
Additional Testing Is       Petroleum Institute   and the Motor Vehicle Manufacturers       Association
Planned to Increase         during the development     of the model. These concerns focused on the
Model Reliability           model’s running-loss   factor, limited testing of recent model vehicles, and
                            several other technical issues. EPAaddressed some of these concerns in
                            the final version of MOBILE4and plans additional      emissions tests to
                            address others.



Running-Loss Emissions      MOBILESis the first version      of the mobile-source   emissions model to
                            account for running-loss      emissions (quantities   of hydrocarbons  that
                            evaporate from a vehicle while it is being driven), because until about 2
                            to 3 years ago EPA did not know such emissions existed, Running-loss
                            emissions account for 25 percent or more of total hydrocarbons          emis-
                            sions from vehicles. However, that figure is based on extremely         limited
                            testing-only     39 vehicles-and       EPAintends to perform more tests for
                            the running-loss   factor in MOBILE5.



Limited Testing of Newer    According     to EPA, MOBILES is not based on a sufficient        number of tests of
                            recent model vehicles with high mileage. For example, only five 1986
Model V‘ehicles With High   model automobiles       with high mileage (more than 50,000 miles) and no
Mileage                     high mileage vehicles from later model years were tested. In-use vehicles
                            from the 1987 and 1988 model years had not accumulated                  high mileage
                            at the time EPA tested vehicles for MOBILE4’S data base. EPArecognizes
                            that additional    tests of such vehicles are particularly        important  because
                            (1) fuel injection systems are much more prevalent             in recent model vehi-
                            cles and little information     on the emissions performance         of these sys-
                            tems at high mileage exists and (2) manufacturers             claim to have
                            improved the durability       of pollution-control     equipment    over the past
                            several years. EPAplans additional          testing in these areas with the results
                            to be incorporated      into MOBILES.



Other Technical Issues      EPAplans several      other types of emission testing in developing       MOBILI%
                            These tests will address factors in MOBILE4 that are based on relatively
                            little test data. For example, because oxygenated          fuels are being used by
                            several cities to address carbon monoxide problems, EPAplans more tests
                            of automobiles     that run on oxygenated     fuels4 Testing of oxygenated-
                            fuel vehicles used to develop MOBILJZ~focused on 1983- to 1985-model
                            automobiles     equipped with multiport     fuel-injection    systems; EPAintends

                             4 Oxygenated fuels, such as gasohol, are a blend of gasoline with other components such as ethanol.



                             Page 4                     GAO/RCED-99-139     Reliability   of EPA’s Mobile Source Emission   Model
                        B-239214




                        to test newer models with these systems, as well as models introduced
                        since 1983 that use oxygenated  fuels in carbureted and throttle-body
                        fuel-injection systems.


                        EPA has sharply decreased the number of vehicles undergoing        emission
Funding Reduced         tests in the last decade. Specifically, EPA’S contract funds for such test-
EPA’s Emissions Tests   ing decreased from an estimated $2.9 million in 1977 to $807,000 in
                         1988, while the number of vehicles tested fell from a high of 2,209 vehi-
                        cles in 1977 to a low of 198 in 1988. Partly due to a reduced sample size
                        in recent years, ~0~1124’s estimates for later model vehicles are less reli-
                        able than its projections  for earlier model years-because   of less
                        information.

                        According   to the former Acting Director of EPA’S Mobile Source Office
                        (the office within EPA responsible     for developing mobile models), the
                        Mobile Source Office requested increased funding for 1983 but actually
                        received about $1 million less than in the previous year. The acting
                        director said that the request reflected his concerns that EPA had an
                        inadequate    basis for estimating  future motor vehicle emissions. None-
                        theless, EPA reduced funding for emissions testing because it was viewed
                        as less essential in the short term than competing programs such as the
                        development     of air quality standards and enforcement     activities.

                        However, while the number of vehicles being tested has fallen drasti-
                        cally, the number of tests performed per vehicle has risen, covering a
                        wider spectrum of test conditions.     This increase in the number of tests
                        performed     has been possible because in-house research funds were not
                        cut as drastically   as were the funds for leasing the vehicles to be tested5,
                        according to the Senior Project Manager for mobile models. This
                        expanded testing has enabled EPA to gather additional        data on emissions
                        behavior at different     speeds, temperatures,   and fuel volatility levels,
                        thereby improving      the accuracy of the model’s estimates at conditions
                        other than those of the standard test procedures.

                         Past funding inadequacies       are now being addressed. According        to EPA’S
                         Senior Project Manager, in fiscal year 1989 EPAwas allocated a $1.5 mil-
                         lion supplement     for contract testing, which was increased to $2.1 million
                         in fiscal year 1990. The funds are programmed          for additional  running-
                         loss tests, pilot studies of alternative  emission-testing    procedures,    and


                         “Each test vehicle is individually leased from its owner for the duration of the testing.



                         Page S                      GAO/RCED-90.138      Reliability   of EPA’s Mobile   Source Emission   Model
                        5239214




                        leasing additional   vehicles for the emission      tests needed to improve             the
                        reliability of future model versions.


                        While the precision and reliability    of MOBILE4needs to be improved, it
Conclusions             remains an important     tool for formulating    plans to reduce the atmos-
                        pheric concentrations    of ozone and carbon monoxide pollution.        When
                        the model assumptions      are revised to reflect changes in the vehicle fleet
                        and other refinements     resulting from additional     emissions tests, it
                        should produce more precise estimates of motor vehicle emissions. While
                        funding limitations   hindered the development        of MOBILEd,EPAappears to
                        have begun addressing this situation by increasing funding for emissions
                        testing in fiscal years 1989 and 1990.


                        As agreed with your office, we gathered information   on the reliability of
Objectives, Scope,and   the mobile-source emissions model and the effects that budget cuts may
Methodology             have had on that development.

                        To obtain information    on the model’s reliability,   we relied on evalua-
                        tions by the American Petroleum Institute        and the Motor Vehicle Manu-
                        facturers Association   and on interviews    with representatives    from EPA,
                        the Institute,  and the Association.   We did not independently    evaluate
                        the reliability  of MOBILEd.

                        Concerning the effects of budgetary   constraints  on model development,
                        we obtained documentation    on the funds available for vehicle testing
                        and the number of vehicles tested yearly since 1976. We relied on inter-
                        views with representatives  of EPA,the Institute,  the Association,   and
                        previous directors of EPA'SOffice of Mobile Sources to identify     any
                        effects.

                        We discussed information    contained in this report with EPAofficials who
                        generally agreed with the factual information    in this report, and we
                        have included their comments where appropriate.        However, as you
                        requested, we did not obtain official agency comments on a draft of this
                        report. We conducted our review between July 1989 and February           1990
                        in accordance with generally accepted government        auditing standards.


                        As arranged with your office, unless you publicly release the contents
                        earlier, we plan no further distribution   of this report until 30 days from
                        the date of this letter. At that time, copies of the report will be sent to



                         Page 6               GAO/RCED9@139   Reliability   of EPA’s Mobile   Source Emission    Model
B-239214




appropriate   congressional committees, the Administrator                of EPA,and the
Director of the Office of Management    and Budget.

Major contributors  to this report are listed in appendix I. You may con-
tact me at (202) 275-5489 should you or your staff have any questions.

Sincerely   yours,




Richard L. Hembra
Director, Environmental    Protection
   Issues




Page 7               GAO/BCED-9@139     lteliabili~   of EPA’s Mobile Source Emission   Model
Appendix    I

Major Contributors to This Report


                        Peter F. Guerrero, Associate   Director,    (202) 252-0600
Resources,              William F. McGee, Assistant    Director
Community, and
Economic
Development Division,
Washington, DC.

Norfolk Regional        Everett 0. Pace, Evaluator-in-Charge
Office                  Robert R. Floren, Site Supervisor




 (069461)                Page 8               GAO/RCED9@138    Reliability   of EPA’s Mobile   !3ource Emission   Model
United States                         First-Class Mail
General Accounting  Office          Postage fk Fees Paid
Washington, D.C. 20548                       GAO
                                      Permit No. GlOO
ofticial   Business
Penalty    for Private   Use $300
Requests for copies of GAO reports     should be sent to:

U.S. General Accounting    Office
Post Office Box 6016
Gaithersburg, Maryland     20877

Telephone   202-276-6241

The first five copies of each report   are free. Additional   copies are
$2.00 each.

There is a 26% discount    on orders for 100 or more copies mailed to a
singleaddress.

Orders must be prepaid by cash or by check or money order made
out to the Superintendent of Documents.