‘I -, ‘.i . _I” .: z. .‘i, I’ ., ,. ^, “, ,Ir- ‘- , li,, ,.. ; ..‘., i United States General Accounting Office *’ i GAO Report to the Chairman, Subcommittee on Oversight and Investigations, Committee on Energy and Commerce, House of Representatives ;., May 1990 AIR POLLUTION Reliability of EPA’s Mobile Source Emission Model Could Be Improved ., 1 ‘! : ,( 1 ;’ .,’ . I ” ,,::1 ;” ” I. :, “.:.& *, “: $,+..‘;,- ,_,, -,. ,. r’!... .,’ GAO United States General Accounting Office Washington, D.C. 20648 Resources, Community, and Economic Development Division B-239214 May 14,199O The Honorable John D. Dingell Chairman, Subcommittee on Oversight and Investigations Committee on Energy and Commerce House of Representatives Dear Mr. Chairman: This report responds to your request that we examine the reliability of the Environmental Protection Agency’s (EPA) mobile source emission fac- tor model (MOBILE4) and the effect of budget constraints on its reliabil- ity. The model is used in estimating motor vehicle emissions of ozone precursors (hydrocarbons and nitrogen oxides) and carbon monoxide and in devising measures to reduce the atmospheric concentrations of these pollutants. In February 1989 WA made MOBILE4available to states for use in developing plans for improving air quality. The mobile source emission factor model has undergone continuous revi- Results in Brief sions* to more accurately represent prevailing conditions and the model year vehicles or fleet in use. According to EPA,certain MOBILESassump- tions need to be refined and revised to enhance the model’s reliability. However, although the overall reliability of MOBILEShas not been deter- mined, EPAbelieves its statistical reliability should surpass that of the previous version, MOBILE%Meanwhile, EPAcontinues developmental work to reflect changing conditions in the vehicle fleet, address known limitations, and update assumptions used in the model. These actions should improve the reliability of the next version of the model. For now, however, MOBILES remains an important tool to understanding efforts to limit and decrease atmospheric ozone and carbon monoxide pollution. Although work on model revisions has been continuous, funding con- straints have limited EPA'Sability to perform emission testing of later model vehicles. Therefore, ~0~1~~4’s estimates for later model vehicles are statistically less reliable than its projections for vehicles of earlier model years, because of the smaller sample size for later model years. However, EPAhas begun addressing past funding inadequacies by pro- viding increased funds in fiscal years 1989 and 1990 to increase the sample size of later model vehicles. For example, funding increased from ' MOBILE1 was made available to states in 1978, MOBILE2 in 1981, and MOBILE3 in 1984. Page 1 GAO/RCEDW138 Reliability of EPA’s Mobile Source Emission Model B-239214 $807,000 in fiscal year 1988, to $1.5 million in fiscal year 1989, and to $2.1 million in 1990. The mobile source emission factor model is an integrated collection of Background mathematical equations that manipulate certain variables-including, but not limited to, vehicle age and mileage, percentage of driving in dif- ferent conditions, average vehicle speed, ambient temperature, and rate of tampering with emission control systems-to estimate the grams of pollutant (hydrocarbons, nitrogen oxides, and carbon monoxide) emitted per mile driven. The estimate applies to the current fleet of vehicles, which consist of those models produced within the last 20 years. States use the MOBILES model in estimating motor vehicle emissions that contribute to ozone and carbon monoxide pollution. These estimates are used in preparing State Implementation Plans, which outline pollution control measures designed to allow states to attain national air quality standards. The estimates are also being used by states and EPA to evalu- ate emission reduction strategies, such as vehicle inspection and mainte- nance programs; and the effectiveness of potential mobile source regulations, such as EPA’S fuel volatility controls. MOBILE4 is the latest version of EPA’s mobile source emission factor model,* which has undergone periodic revision, updating, and refine- ment to more accurately represent prevailing conditions and the fleet of vehicles in use. Each new version eliminates model years older than 20 years, reflects recent advances in emission-control technologies, and tries to correct weaknesses in previous versions that have resulted in over- or underestimation of emissions. The next version of the model, MOBILES, is due for release within 5 years. MOBILE4 estimates four types of vehicle emissions: . exhaust (from vehicle tailpipes), . nonrunning evaporative (from a vehicle that is parked with the engine turned off), . refueling (occurring when a vehicle is being refueled), and . running-loss evaporative (occurring while a vehicle is being driven). 2 According to EPA,MOBILEX was issued as a general statement of policy and not a regulation sub ject to the notice and comment requirements under the Administrative Procedures Act. EPA did, how ever, hold a series of public workshops during the development of MOBILE4 to exchange information with the American Petroleum Institute, motor vehicle manufacturers, and others. Page 2 GAO/RCED-9@138 Reliability of EPA’s Mobile !3ource Emission Mode B-239214 According to EPA,although the overall reliability of MOBIL& has not been Reliability of determined, its statistical reliability should surpass that of MOBILES, MOBILE4 which was found to have an acceptable statistical reliability (within k 22 percent at a 95-percent confidence level based on a Department of Energy sponsored study). However, as a result of a reduced number of vehicles tested in recent years, MOBILEA'semission estimates for later model vehicles are less reliable than its projections for earlier model years. Even if a greater number of vehicles were tested, it still would not address the validity of many other assumptions used in the model that are crucial to the overall reliability of its estimates. For example, there is no way to quantify the reliability of model assumptions about future conditions-such as the mix of emission control.technologies in future fleets. A recent study by Southwest Research Institute3 questioned the overall reliability of an adapted form of MOBILESdeveloped by the state of Cali- fornia. California’s version has been modified to reflect that state’s more stringent emission control requirements. According to this study, which compared model estimates against emissions measured in a highway tunnel, California’s version of MOBILESunderestimated hydrocarbon emissions, on the average, by 74 percent (i.e., measured emissions were about four times greater than model estimates) and carbon monoxide emissions, on the average, by about 63 percent (measured emissions were about three times greater than estimates). According to the Senior Project Manager for mobile models, EPAis cur- rently trying to determine why the measured and estimated emission rates differ so greatly. Specifically, EPAis evaluating the Southwest Research Institute study methodology and reassessing MOBILE~‘S assumptions concerning the prevalence in the fleet of super emitting vehicles (vehicles with exceptionally high emissions rates) and vehicles with evaporative system leaks. According to EPA, if the underestima- tions are real, then MOBILESwould also underestimate these emissions by large amounts, because MOBILESestimates for these emissions are not dramatically greater than those of California’s MOBILESversion. In that case, use of MOBILEd'Sestimates could result in overly optimistic attain- ment projections with respect to hydrocarbon and carbon monoxide emissions. 3 Measurement of On-Road Vehicle Emission Factors in the California South Coast Air Basin, Volume 1, Regulated Emissions, Southwest Research Institute, June 1989. Page 3 GAO/‘RCEDW13l3 Beltability of EPA’s Mobile Source Emission Model 5239214 Concerns were raised about the reliability of MOBILESby the American Additional Testing Is Petroleum Institute and the Motor Vehicle Manufacturers Association Planned to Increase during the development of the model. These concerns focused on the Model Reliability model’s running-loss factor, limited testing of recent model vehicles, and several other technical issues. EPAaddressed some of these concerns in the final version of MOBILE4and plans additional emissions tests to address others. Running-Loss Emissions MOBILESis the first version of the mobile-source emissions model to account for running-loss emissions (quantities of hydrocarbons that evaporate from a vehicle while it is being driven), because until about 2 to 3 years ago EPA did not know such emissions existed, Running-loss emissions account for 25 percent or more of total hydrocarbons emis- sions from vehicles. However, that figure is based on extremely limited testing-only 39 vehicles-and EPAintends to perform more tests for the running-loss factor in MOBILE5. Limited Testing of Newer According to EPA, MOBILES is not based on a sufficient number of tests of recent model vehicles with high mileage. For example, only five 1986 Model V‘ehicles With High model automobiles with high mileage (more than 50,000 miles) and no Mileage high mileage vehicles from later model years were tested. In-use vehicles from the 1987 and 1988 model years had not accumulated high mileage at the time EPA tested vehicles for MOBILE4’S data base. EPArecognizes that additional tests of such vehicles are particularly important because (1) fuel injection systems are much more prevalent in recent model vehi- cles and little information on the emissions performance of these sys- tems at high mileage exists and (2) manufacturers claim to have improved the durability of pollution-control equipment over the past several years. EPAplans additional testing in these areas with the results to be incorporated into MOBILES. Other Technical Issues EPAplans several other types of emission testing in developing MOBILI% These tests will address factors in MOBILE4 that are based on relatively little test data. For example, because oxygenated fuels are being used by several cities to address carbon monoxide problems, EPAplans more tests of automobiles that run on oxygenated fuels4 Testing of oxygenated- fuel vehicles used to develop MOBILJZ~focused on 1983- to 1985-model automobiles equipped with multiport fuel-injection systems; EPAintends 4 Oxygenated fuels, such as gasohol, are a blend of gasoline with other components such as ethanol. Page 4 GAO/RCED-99-139 Reliability of EPA’s Mobile Source Emission Model B-239214 to test newer models with these systems, as well as models introduced since 1983 that use oxygenated fuels in carbureted and throttle-body fuel-injection systems. EPA has sharply decreased the number of vehicles undergoing emission Funding Reduced tests in the last decade. Specifically, EPA’S contract funds for such test- EPA’s Emissions Tests ing decreased from an estimated $2.9 million in 1977 to $807,000 in 1988, while the number of vehicles tested fell from a high of 2,209 vehi- cles in 1977 to a low of 198 in 1988. Partly due to a reduced sample size in recent years, ~0~1124’s estimates for later model vehicles are less reli- able than its projections for earlier model years-because of less information. According to the former Acting Director of EPA’S Mobile Source Office (the office within EPA responsible for developing mobile models), the Mobile Source Office requested increased funding for 1983 but actually received about $1 million less than in the previous year. The acting director said that the request reflected his concerns that EPA had an inadequate basis for estimating future motor vehicle emissions. None- theless, EPA reduced funding for emissions testing because it was viewed as less essential in the short term than competing programs such as the development of air quality standards and enforcement activities. However, while the number of vehicles being tested has fallen drasti- cally, the number of tests performed per vehicle has risen, covering a wider spectrum of test conditions. This increase in the number of tests performed has been possible because in-house research funds were not cut as drastically as were the funds for leasing the vehicles to be tested5, according to the Senior Project Manager for mobile models. This expanded testing has enabled EPA to gather additional data on emissions behavior at different speeds, temperatures, and fuel volatility levels, thereby improving the accuracy of the model’s estimates at conditions other than those of the standard test procedures. Past funding inadequacies are now being addressed. According to EPA’S Senior Project Manager, in fiscal year 1989 EPAwas allocated a $1.5 mil- lion supplement for contract testing, which was increased to $2.1 million in fiscal year 1990. The funds are programmed for additional running- loss tests, pilot studies of alternative emission-testing procedures, and “Each test vehicle is individually leased from its owner for the duration of the testing. Page S GAO/RCED-90.138 Reliability of EPA’s Mobile Source Emission Model 5239214 leasing additional vehicles for the emission tests needed to improve the reliability of future model versions. While the precision and reliability of MOBILE4needs to be improved, it Conclusions remains an important tool for formulating plans to reduce the atmos- pheric concentrations of ozone and carbon monoxide pollution. When the model assumptions are revised to reflect changes in the vehicle fleet and other refinements resulting from additional emissions tests, it should produce more precise estimates of motor vehicle emissions. While funding limitations hindered the development of MOBILEd,EPAappears to have begun addressing this situation by increasing funding for emissions testing in fiscal years 1989 and 1990. As agreed with your office, we gathered information on the reliability of Objectives, Scope,and the mobile-source emissions model and the effects that budget cuts may Methodology have had on that development. To obtain information on the model’s reliability, we relied on evalua- tions by the American Petroleum Institute and the Motor Vehicle Manu- facturers Association and on interviews with representatives from EPA, the Institute, and the Association. We did not independently evaluate the reliability of MOBILEd. Concerning the effects of budgetary constraints on model development, we obtained documentation on the funds available for vehicle testing and the number of vehicles tested yearly since 1976. We relied on inter- views with representatives of EPA,the Institute, the Association, and previous directors of EPA'SOffice of Mobile Sources to identify any effects. We discussed information contained in this report with EPAofficials who generally agreed with the factual information in this report, and we have included their comments where appropriate. However, as you requested, we did not obtain official agency comments on a draft of this report. We conducted our review between July 1989 and February 1990 in accordance with generally accepted government auditing standards. As arranged with your office, unless you publicly release the contents earlier, we plan no further distribution of this report until 30 days from the date of this letter. At that time, copies of the report will be sent to Page 6 GAO/RCED9@139 Reliability of EPA’s Mobile Source Emission Model B-239214 appropriate congressional committees, the Administrator of EPA,and the Director of the Office of Management and Budget. Major contributors to this report are listed in appendix I. You may con- tact me at (202) 275-5489 should you or your staff have any questions. Sincerely yours, Richard L. Hembra Director, Environmental Protection Issues Page 7 GAO/BCED-9@139 lteliabili~ of EPA’s Mobile Source Emission Model Appendix I Major Contributors to This Report Peter F. Guerrero, Associate Director, (202) 252-0600 Resources, William F. McGee, Assistant Director Community, and Economic Development Division, Washington, DC. Norfolk Regional Everett 0. Pace, Evaluator-in-Charge Office Robert R. Floren, Site Supervisor (069461) Page 8 GAO/RCED9@138 Reliability of EPA’s Mobile !3ource Emission Model United States First-Class Mail General Accounting Office Postage fk Fees Paid Washington, D.C. 20548 GAO Permit No. GlOO ofticial Business Penalty for Private Use $300 Requests for copies of GAO reports should be sent to: U.S. General Accounting Office Post Office Box 6016 Gaithersburg, Maryland 20877 Telephone 202-276-6241 The first five copies of each report are free. Additional copies are $2.00 each. There is a 26% discount on orders for 100 or more copies mailed to a singleaddress. Orders must be prepaid by cash or by check or money order made out to the Superintendent of Documents.
Air Pollution: Reliability of EPA's Mobile Source Emission Model Could Be Improved
Published by the Government Accountability Office on 1990-05-14.
Below is a raw (and likely hideous) rendition of the original report. (PDF)