United States General Accounting Office Washington, DC 20548 January 16, 2003 The Honorable Don Young Chairman, Committee on Transportation and Infrastructure House of Representatives Subject: Highway Infrastructure: FHWA Has Acted to Disclose the Limitations of Its Environmental Review Analysis Dear Mr. Chairman: Environmental reviews of complex federally funded highway construction projects may take years.1 The Congress has an interest in identifying and, if necessary, addressing the reasons to expedite highway projects. To better understand these reasons, the Federal Highway Administration (FHWA) surveyed its 55 division offices to determine why the environmental review of certain highway projects took more than 5 years. In September 2000, FHWA made the results of its analysis of 89 projects it identified available to the Congress, and in December 2000, it posted the results on its Web site. As the reauthorization of the Transportation Equity Act for the 21st Century approaches, transportation stakeholders have used the results of this analysis in discussions about whether legislative reforms are needed for environmental reviews of federally funded highway projects. You requested that we determine (1) the reasonableness of FHWA’s survey methodology and offer suggestions for improvement, if appropriate, and (2) the feasibility of resurveying the same projects to gain an additional understanding about why environmental reviews took more than 5 years. To carry out this work, we interviewed FHWA officials about how they selected projects, surveyed their division offices, and analyzed the responses they obtained. In addition, we applied the lessons learned from FHWA’s original approach to assess the feasibility of resurveying the same projects. 1 Under the National Environmental Policy Act of 1969, the consequences of proposed transportation projects and alternative choices on the environment and on historic properties, if any, must be identified and assessed. Federally funded highway project construction cannot begin until environmental and historic preservation issues are addressed, if needed. GAO-03-338R Federal Highway Environmental Analysis Results in Brief Aspects of FHWA’s methodology, such as relying on its division offices for information rather than on a larger set of stakeholders, were reasonable, given the agency’s desire for a quick exploration of the subject. However, several other aspects of FHWA’s methodology lead us to question the usefulness of the results. FHWA’s reliance on narrative responses (rather than multiple choice questions, for example) produced results that are not particularly useful because the answers are general and typically do not delineate the underlying reasons why environmental reviews took more than 5 years. In addition, when multiple reasons were cited, many responses did not specify which reason or reasons were primary. In these instances, the FHWA staff member chose only one reason as the main reason, basing her selection, in large part, on which reason was listed first. FHWA’s results would have been more useful if it had asked respondents to provide enough information so that underlying reasons could be understood and to indicate relative importance of reasons when more than one reason was cited. Finally, FHWA did not provide enough detail about how it conducted its work to allow potential users to assess its reliability. Resurveying the 89 projects is feasible and could provide more useful and reliable information on why environmental reviews took more than 5 years. However, several issues would have to be addressed before undertaking any such effort. These issues include (1) verifying that the 14 FHWA division offices (25 percent) that did not respond to the agency’s original survey did not have projects in environmental review for 5 years or more and (2) determining whether a newer data set (e.g., projects with environmental reviews lasting more than 5 years at the time of the resurvey) might provide more current and reliable information. The costs and benefits of redoing the survey would also have to be weighed against the expected results of several current FHWA initiatives that are intended to shed light on how environmental reviews of federally funded highway projects are completed. For example, FHWA has hired a contractor to review about 250 projects from the 1990s to identify variables that correlate with time these projects spent in environmental reviews. FHWA is also conducting a Gallup poll that surveys stakeholders’ views on the environmental review process. The results for FHWA’s efforts are expected by spring 2003. The Department of Transportation offered no specific comments on a draft of this report. However, in response to our findings, FHWA revised its Web site in a manner that adequately discloses the issues discussed in this report. Background FHWA provides financial assistance to states to build and improve highways and roads and provides transportation engineering services (such as planning and design) for highway and bridges that provide access to federally owned lands. FHWA relies on its 52 division offices nationwide and 3 Federal Lands Highway Division offices to carry out this mission. For fiscal year 2003, FHWA expects to fund about $20 billion in highway infrastructure improvements and congestion mitigations. The responsibility for designing, planning, and awarding contracts for federally funded Page 2 GAO-03-338R Federal Highway Environmental Analysis highway projects generally rests with state departments of transportation and local planning organizations. Before a federally funded highway project can be constructed, it must comply with the requirements of the National Environmental Policy Act of 1969, among other things. Under the act, the consequences, if any, of proposed transportation projects and alternative choices (such as alternative routings) on the natural and human (e.g., health) environment and on historic properties must be identified and assessed. For federally funded highway projects that will have a significant impact on the environment, the state department of transportation prepares an environmental impact statement, which FHWA must approve before the project can be built. The environmental impact statement must describe the project, characterize the surrounding environment, analyze the environmental effects of a range of reasonable project alternatives, and indicate plans for complying with environmental laws and mitigating environmental damage, if any. Other federal agencies (called resource agencies), such as the Environmental Protection Agency, Army Corps of Engineers, and Fish and Wildlife Service, often participate in the preparation and review of the environmental impact statements for highway projects because of their responsibilities under federal laws, such as the Clean Air Act, Clean Water Act, and Endangered Species Act. Projects requiring an environmental impact statement are usually complex and affect sensitive populations. According to FHWA, only about 3 percent of all federally funded highway projects have a significant enough impact on the environment to require preparation of an environmental impact statement. In 2001, FHWA reported that the average time to prepare an environmental impact statement had increased from 2.2 years in the 1970s to 5.0 years in the 1990s because requirements for environmental impact statements have become more extensive and more complex over time. Environmental review now takes as much as 25 percent of the total time needed to plan, design, gain approval for, and construct a complex federally funded highway project.2 Following a March 8, 2000, hearing, the Subcommittee on Ground Transportation, House Committee on Transportation and Infrastructure, requested that FHWA provide information about active highway projects requiring an environmental impact statement whose federal environmental review had not been completed within 5 years.3 Specifically, the subcommittee requested that FHWA identify the location, scope, duration of the federal environmental review, and specific steps that FHWA had taken with the resource agencies and states to streamline the approval of these particular projects. In response, FHWA surveyed its 52 division offices and its 3 Federal Land Highways Division offices to obtain this information. In addition, 2 Federal Highway Administration, Evaluating the Performance of Environmental Streamlining: Development of a NEPA Baseline for Measuring Continuous Performance (Washington, D.C.: May 8, 2001) and U.S. General Accounting Office, Highway Infrastructure: Preliminary Information on the Timely Completion of Highway Construction Projects, GAO-02-1067T (Washington, D.C.: Sept. 19, 2002). Highway projects with no significant impact on the environment do not require preparation of an environmental impact statement. 3 This subcommittee is now the Subcommittee on Highways and Transit. Page 3 GAO-03-338R Federal Highway Environmental Analysis FHWA asked its division staff to provide a brief narrative on reasons why the projects 4 had been in the environmental review phase for more than 5 years. The survey identified 89 active highway projects that had been undergoing federal environmental review for more than 5 years. According to an FHWA official, the survey and subsequent analysis were not intended to be scientifically rigorous; rather, they were intended to provide a quick exploration of the subject so that it could begin to think about this issue and help begin to inform the public debate. FHWA provided the preliminary results of its analysis to the Congress in September 2000 and the final results 3 months later. FHWA posted its final results on its Web page.5 FHWA concluded that the top three reasons why highway projects were in the environmental review phase for more than 5 years were (1) lack of funding or low priority (33 percent), (2) local controversy (16 percent), and (3) complexity of the projects (13 percent). The analytical results presented by FHWA were primarily in the form of a pie chart, using the characterizations shown below. (See table 1.) The analysis has been widely used by FHWA and by groups in commenting on whether legislative changes are needed to environmental review requirements when the st Transportation Equity Act for the 21 Century is reauthorized. Table 1: FHWA’s Characterization of Why Environmental Reviews of 89 Active Highway Projects Took More Than 5 Years Reason cited Percent of total projects Lack of funding 18 Local controversy 16 Low priority 15 Complex project 13 Resource agency review 8 Change in scope 8 Fish and Wildlife Service/Endangered Species Act 7 Section 106 of the National Historic Preservation Act of 1966 6 Wetlands 4 Law suits 3 Hazardous materials 2 Total 100 Source: FHWA. A More Refined and Transparent Methodology Would Have Been Likely to Produce More Useful Results FHWA’s reliance on its divisions for information was reasonable given the agency’s desire for a quick exploration of the subject. However, its methodology did not yield particularly useful or necessarily reliable results. FHWA did not provide enough 4 According to FHWA, in some cases, division offices consulted with state departments of transportation. 5 The results of the analysis can be found at www.fhwa.dot.gov/environment/strmlng/eisdelay.htm. Page 4 GAO-03-338R Federal Highway Environmental Analysis detail on its methodology to allow potential users to assess the extent that they should rely on FHWA’s results. Use of Multiple-choice Questions Would Have Been Likely to Produce More Useful Results FHWA asked its division offices for a brief narrative on why the identified projects were in environmental review for more than 5 years but did not provide any guidance for preparing the narrative. As a result, (1) the responses were general and did not provide enough information for FHWA to report on the underlying causes and (2) when multiple responses were given, the FHWA staff member who conducted the effort had to decide which reason was the primary reason. The difficulty in determining the underlying causes of why environmental review lasted more than 5 years is apparent in the response categories that FHWA ultimately reported. For example, FHWA reported that local controversy was the second most prevalent reason given (16 percent of respondents). However, the response category is not specific enough to determine the reason for the controversy. The local controversy could have existed either for environmental or nonenvironmental reasons. For example, some in local communities may have believed the project was not needed, or they may have believed the project was needed but objected to its scope or its impact on wetlands or historic properties. Other response categories were similarly uninformative. For example, the resource agency review, Fish and Wildlife Service/Endangered Species Act, wetlands, and the section 106 of the historic preservation act categories are not specific enough to determine the underlying nature of the problem (e.g., availability of staff to conduct environmental and historic property reviews, issues involving working relationships among stakeholders, differences in interpretations of requirements). Because FHWA provided no guidance to its division offices on differentiating primary reasons from other reasons, it had no means of determining which reason or reasons were important and which were not when the respondent cited more than one reason and did not indicate their relative importance. As a result, FHWA cannot assure that respondents would have agreed with FHWA’s judgment as to which reasons were important. We found that many of the narrative responses contained more than one reason for the time spent in environmental review and the main reason was unambiguous in only a few responses. When the narrative responses included multiple reasons, the FHWA staff member chose only one reason as the main reason, basing her selection, in large part, on which reason was listed first. For example, one narrative response cited “[c]ontroversy, lawsuits (Consent Decree), funding/cost, and replenishment housing.” The FHWA staff member recalled that she chose to categorize “controversy” as this response’s main reason because it came first in the 6 list, even though it was not clear that this was the primary reason. 6 The FHWA staff member said that, for several projects with which she was familiar, she used her knowledge to choose which was the main reason of several reasons given. The staff member kept no records of the category to which she assigned each response. Page 5 GAO-03-338R Federal Highway Environmental Analysis The use of multiple-choice questions would probably have made the results of FHWA’s survey more useful. We recognize that developing clear multiple-choice questions with informative answers that cover most anticipated responses can be time consuming. In addition, FHWA officials told us that it did not have enough knowledge at the outset of its efforts to prepare multiple-choice questions. However, this approach could have provided clearer, more informative response categories that would not have required subjective determinations when results were compiled and it could have allowed respondents to indicate the relative importance of multiple reasons why environmental review took more than 5 years. A reasonable alternative to multiple choice questions, within the framework of a quick exploration of the topic, would have been for FHWA to ask its division offices to provide more information so that underlying reasons and their relative importance could be understood. Effects of Possible Nonresponse Error and the Lack of Verification of How Responses Were Categorized Are Unknown To identify active projects whose environmental reviews lasted more than 5 years, FHWA e-mailed a list of 70 projects that it had identified as being in environmental review for more than 5 years to each of its 55 division and Federal Land Highways Division offices and asked them to add to the list as warranted. It followed up with an e-mail reminder for divisions that did not respond. Ultimately, 41 of these offices 7 (75 percent) responded. The FHWA staff member who conducted this effort said she assumed that the nonresponding division offices had no projects meeting the criteria. While 75 percent was a substantial response rate, it is possible that the lack of data from nonrespondents (called “nonresponse error”) may have negatively affected the accuracy of FHWA’s analyses in two ways. First, FHWA may not have identified all (or nearly all) active highway projects whose environmental reviews lasted more than 5 years. Second, to the extent that such highway projects existed in nonresponding jurisdictions, the reasons that the nonresponding division offices might have cited could have been different from the results that FHWA reported. A higher response rate would have reduced the likelihood of either or both of these possible nonresponse errors. However, because we did not contact the nonrespondents to determine whether their jurisdiction included projects that met the survey criteria, we cannot determine what effect, if any, a higher response rate might have had. Achieving a very high response rate is often difficult in survey research. However, the authority that FHWA headquarters exercises over its division offices might be expected to produce a response rate nearing 100 percent. According to FHWA officials, they did not perform quality control to ensure that the FHWA staff member reasonably and accurately assigned survey responses to the individual categories. According to the FHWA staff member who performed this 7 FHWA might have elected to survey other transportation stakeholders in order to obtain additional perspectives. However, doing so would have substantially increased the effort needed. We believe that FHWA’s decision to limit the scope of its survey to its division offices was reasonable given its desire for a quick exploration of the subject. Page 6 GAO-03-338R Federal Highway Environmental Analysis analysis, no one either independently categorized the narrative responses or checked her work. We did not attempt to independently categorize responses—in part because we would have had no basis for selecting primary reasons when respondents cited multiple reasons and did not identify the primary reasons--and therefore cannot say whether a similar effort by FHWA at the time it was compiling results would have affected the results it reported. FHWA Could Have Better Disclosed How It Conducted Its Survey and Subsequent Analysis On its Web page, FHWA reported that “[d]uring the spring and summer of 2000…[it] surveyed the Division offices and the Regional Federal Land Highways Divisions…to gather information to respond to a question from [a House subcommittee].” It also listed the information it sought from its divisions. However, FHWA did not provide any further information on the quality of its data, how it resolved data issues, and 8 how it made judgments about the data it collected. Given the significant weaknesses in this effort, such disclosure would have helped potential users assess the rigor of the analysis and the extent to which they should rely on it. In response to our findings, FHWA revised its Web site in a manner that adequately discloses the issues discussed in this report. Issues to Address if the 89 Projects Are Resurveyed It would be feasible to resurvey the 89 projects using the lessons learned from our review to attempt to obtain more useful information on why the environmental review for each lasted more than 5 years. However, other approaches, such as reviewing a newer set of projects, might be more useful. The costs and benefits of a resurvey would have to be weighed against the expected results of several FHWA initiatives that are intended to shed light on how environmental reviews of federally funded highway projects are completed. Projects Could Be Resurveyed but Other Approaches and Issues Would Have to Be Addressed It would certainly be feasible for FHWA to resurvey the 89 projects that it analyzed in 2000. Even within the context of a quick exploration, FHWA might be able to obtain more useful information if it employed a methodology that allowed it to better identify the underlying causes—and their relative importance--than it did in its original analysis. However, assessing the desirability of such an endeavor would involve several considerations: 8 In addition, FHWA did not indicate whether the data obtained were based on recollection of respondents or from the respondents’ reviews of documentation. Although documentary evidence is generally preferable to recall, sometimes documentary evidence is either not available or time- consuming to review. The use of recall may be appropriate in these instances, but disclosure of its use is important. Page 7 GAO-03-338R Federal Highway Environmental Analysis • The 14 division offices that did not respond to the original survey would have to be contacted to verify that, in 2000, no active highway projects under their jurisdiction had been in environmental review for more than 5 years. • To the extent that FHWA division staff based their original responses on recall, the accuracy of their responses to a resurvey may be reduced because 2 years have elapsed since the 2000 survey. In addition, some staff with the best knowledge of events in 2000 might have moved on and documentation, if used before, might be more difficult to locate. • A newer data set--for example, highway projects that were in environmental review for more than 5 years as of the date of any resurvey--could provide better information about current conditions. • Surveying projects whose environmental review took less than 5 years could provide additional perspective about FHWA’s findings in its original analysis. For example, environmental issues (e.g., resource agency review, endangered species, historic preservation) would likely have been present in projects where environmental review lasted less than 5 years by the very fact that an environmental impact statement was required. In addition this approach might provide some insights into actions by transportation stakeholders that have reduced the time taken for environmental review. • Obtaining the views of other transportation stakeholders--such as state departments of transportation, federal and state resource agencies, community action agencies, and others--would better ensure that the reasons cited represented all viewpoints. However, because many stakeholders are involved in complex projects, identifying them and obtaining their views would add substantially to the complexity of the effort. FHWA Initiatives May Provide Information about Reasons for Time Spent in Environmental Review FHWA has several initiatives under way aimed at improving its understanding of the reasons for the time projects spent in environmental review. In particular, FHWA has two studies that follow up on a study completed in May 2001 looking for variables that correlate with time spent in environmental review.9 In the first study, a contractor is reviewing about 250 projects from the 1990s whose environmental reviews have been completed. Like the May 2001 study, this study is looking for variables that correlate with time spent in environmental review, such as whether controversy existed regarding wetlands or endangered species. FHWA expects that the results of this study will be available around March 2003. In the second initiative, FHWA is reviewing 8 highway projects whose environmental reviews took less than 5 9 Federal Highway Administration, Evaluating the Performance of Environmental Streamlining: Development of a NEPA Baseline for Measuring Continuous Performance (Washington, D.C.: May 8, 2001). Page 8 GAO-03-338R Federal Highway Environmental Analysis years and 8 highway projects whose environmental reviews took more than 5 years to complete, to identify factors that contribute to time spent in environmental review. FHWA expects the results of its analysis of projects taking less than 5 years to complete environmental review to be issued by the end of January 2003. FHWA does not yet have an estimate for when the remaining portion of the initiative will be complete. Finally, FHWA has commissioned a Gallup poll of federal and a limited number of non-federal stakeholders to survey their perceptions of the environmental review process and of how well stakeholders work together. FHWA expects the results of this poll to be completed by spring 2003. We did not assess whether these initiatives are likely to provide useful results. As you know, we are assessing for this committee the reasons that environmental review of federally funded highway projects take years. We expect to report our results in spring 2003. Conclusions By relying on often general narrative responses without directing its division offices to indicate the relative importance of responses, FHWA limited its ability to provide useful insights on why environmental reviews of certain federally funded highway projects took more than 5 years. Furthermore, by publishing its survey results and its subsequent analysis in reports and on its Web site without describing its methodology, FHWA limited the ability of the Congress and the public to evaluate the strengths and weaknesses of its work in order to understand its reliability. While some weaknesses, such as vague categories used to categorize reasons why environmental reviews took more than 5 years, are evident without explanation, other weaknesses, such as FHWA’s inability to separate primary from minor reasons, are not. If the Congress and the public are not aware of these weaknesses, they may place undue confidence in FHWA’s results to the extent that they use these results to evaluate the need for legislative reforms. FHWA’s actions to revise its Web site in response to our concerns address these disclose concerns. Agency Comments and Our Evaluation The Department of Transportation did not offer specific comments on a draft of this report. Our draft report contained a proposed recommendation that the Secretary of Transportation direct the FHWA Administrator to describe how its survey and analysis were conducted in such a way that their strengths and limitations can be understood and that this description should be included in any public use of the survey and analysis, including their posting on FHWA’s Web site. As discussed earlier, FHWA revised its Web site to address our concerns. As a result, we deleted the recommendation from this report. Page 9 GAO-03-338R Federal Highway Environmental Analysis Scope and Methodology To analyze FHWA’s methodology, we interviewed officials in FHWA’s Office of National Environmental Policy Act Facilitation to determine how they carried out and reported the results of their analysis and assessed their approach using basic analytical approaches. Specifically, we asked them to explain how they selected respondents, identified projects, provided instructions on how to complete the survey, categorized narrative responses, and performed quality control. We then reviewed FHWA’s methodology to determine how measurement error and bias were 10 minimized. To assess potential sources of measurement error, we analyzed FHWA’s methodology for determining and categorizing the main reason for time in environmental review from the narrative responses it received. This included reviewing whether categories were clear and whether coding of the main response was determined through use of consistent rules and verified by a second coder. To assess potential sources of bias, we reviewed FHWA’s methodology for identifying projects to be included in the survey, its identification of the appropriate respondents to the survey, efforts to minimize nonresponse from among those surveyed, and the sources of data used by respondents when providing narrative responses. Although we did review the data that FHWA obtained, we did not attempt to determine whether FHWA categorized all narrative responses correctly. Furthermore, we did not attempt to determine the validity of the answers provided by contacting the FHWA officials who completed the survey or through other independent verification. To assess whether it was feasible to resurvey the same projects about why environmental reviews took more than 5 years, we considered the age of the data and its sources, as well as the limitations that we observed in FHWA’s survey and subsequent analysis. Finally, we identified initiatives under way at FHWA to provide information on environmental reviews of federally funded highway projects. We did not assess whether these initiatives are likely to provide useful results. We conducted our work from October 2002 through January 2003 in accordance with generally accepted government auditing standards. _ _ _ _ _ As arranged with your office, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days after the date of this letter. At that time, we will send copies of this report to congressional committees with responsibilities for highway and environmental issues; the Secretary of Transportation; the Administrator, Federal Highway Administration; and the Director, Office of Management and Budget. We will also make copies available to others upon request. This report will be available at no charge on our home page at http://www.gao.gov. 10 Measurement error results in inaccuracy in quantifying a variable. Bias results in a misrepresentation of what is being measured. Page 10 GAO-03-338R Federal Highway Environmental Analysis If you or your staff have any questions about this report, please contact either James Ratzenberger at email@example.com or me at firstname.lastname@example.org. Alternatively, we may be reached at (202) 512-2834. Key contributors to this report were Gail Marnik, Kristen Massey, SaraAnn Moessbauer, and James Ratzenberger. Sincerely yours, Katherine Siggerud Acting Director, Physical Infrastructure Issues (542015) Page 11 GAO-03-338R Federal Highway Environmental Analysis
Highway Infrastructure: FHWA Has Acted to Disclose the Limitations of Its Environmental Review Analysis
Published by the Government Accountability Office on 2003-01-16.
Below is a raw (and likely hideous) rendition of the original report. (PDF)