oversight

NextGen Benefits Have Not Kept Pace With Initial Projections, but Opportunities Remain To Improve Future Modernization Efforts

Published by the Department of Transportation, Office of Inspector General on 2021-03-30.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

            NextGen Benefits Have Not Kept Pace
          With Initial Projections, but Opportunities
          Remain To Improve Future Modernization
                             Efforts




Report No. AV2021023
March 30, 2021
NextGen Benefits Have Not Kept Pace With Initial Projections,
but Opportunities Remain To Improve Future Modernization
Efforts
Mandated by the FAA Reauthorization Act of 2018

Federal Aviation Administration | AV2021023 | March 30, 2021




What We Looked At
The Federal Aviation Administration’s (FAA) Next Generation Air Transportation System (NextGen) is a
multibillion dollar infrastructure project aimed at modernizing our Nation’s aging air traffic system to provide
safer and more efficient air traffic management. Since 2006, our office and others have identified a number of
challenges to implementing NextGen programs and capabilities, which have led to program delays and lower
usage of new capabilities. Given these concerns, the FAA Reauthorization Act of 2018 mandated that the Office
of Inspector General (OIG) study the potential impacts of a significantly delayed, diminished, or completely
failed delivery of NextGen. Our audit objectives were to (1) compare the current expected benefits of NextGen
with the initial projections and identify the reasons for revising those projections and (2) identify lessons learned
from developing and implementing significant air traffic modernization programs.

What We Found
NextGen’s actual and projected benefits have not kept pace with initial projections due to implementation
challenges, optimistic assumptions, and other factors. FAA’s most recent business case projects total NextGen
benefits to be over $100 billion less than the Joint Planning and Development Office’s original estimate, and
benefits actually achieved to date have been minimal and difficult to measure. FAA’s projections were
optimistic about traffic growth and did not account for risk factors. We also found that significant declines in
air traffic due to COVID-19 have further extended the timeframe for realizing expected NextGen benefits. In
addition, prior OIG NextGen-related work has identified lessons that FAA could use to improve NextGen
delivery. For example, while FAA has collaborated with industry to prioritize, implement, and measure benefits
of NextGen programs, there are still opportunities for improving transparency, which will be critical to secure
industry’s long-term investment. Further advancing NextGen will depend on resolving complex implementation
challenges, including effectively prioritizing programs, integrating interdependent capabilities, and harnessing
controller automation tools to achieve benefits.

Our Recommendations
FAA concurred with our three recommendations to improve NextGen delivery and other future National
Airspace System modernization efforts, and provided appropriate actions and completion dates. Accordingly,
we consider all recommendations resolved but open pending completion of the planned actions.




All OIG audit reports are available on our website at www.oig.dot.gov.
For inquiries about this report, please contact our Office of Government and Public Affairs at (202) 366-8751.
Contents
     Memorandum                                                               1

     Results in Brief                                                         3

     Background                                                               4

     NextGen Projected and Actual Benefits Are Lower Than Expected Due to
        Implementation Challenges, Optimistic Assumptions, and Other
        Factors                                                               6

     Prior OIG Work Has Identified Lessons for Potentially Improving FAA’s
         Delivery of NextGen and Other Future Modernization Efforts          16

     Conclusion                                                              20

     Recommendations                                                         20

     Agency Comments and OIG Response                                        21

     Actions Required                                                        21

     Exhibit A. Scope and Methodology                                        22

     Exhibit B. Organizations Visited or Contacted                           24

     Exhibit C. List of Acronyms                                             25

     Exhibit D. Major Contributors to This Report                            26

     Appendix. Agency Comments                                               27




AV2021023
           U.S. DEPARTMENT OF TRANSPORTATION
           OFFICE OF INSPECTOR GENERAL




Memorandum
Date:            March 30, 2021

Subject:         ACTION: NextGen Benefits Have Not Kept Pace With Initial Projections, but
                 Opportunities Remain To Improve Future Modernization Efforts
                 Report No. AV2021023

From:            Matthew E. Hampton
                 Assistant Inspector General for Aviation Audits

To:              Federal Aviation Administrator


                 The Federal Aviation Administration’s (FAA) Next Generation Air Transportation
                 System (NextGen) 1 is a multibillion dollar infrastructure project aimed at
                 modernizing our Nation’s aging air traffic system to provide safer and more
                 efficient air traffic management. NextGen’s goal is to provide new capabilities
                 such as precision satellite navigation, digital data link communications for air
                 traffic controllers and pilots, and an integrated weather system.

                 Since 2006, our office and others have identified a number of challenges to
                 implementing NextGen programs and capabilities, which have led to program
                 delays and lower usage of new capabilities. As a result, FAA has revised its
                 projected benefits for NextGen multiple times. While NextGen is expected to
                 significantly enhance aircraft operations and foster industry growth, these
                 benefits depend on the successful delivery of the new programs and capabilities.

                 Given these concerns, the FAA Reauthorization Act of 2018 mandated the Office
                 of Inspector General (OIG) study the potential impacts of a significantly delayed,
                 diminished, or completely failed delivery of NextGen. Accordingly, we initiated an
                 audit to (1) compare the current expected benefits of NextGen with the initial
                 projections and identify the reasons for revising those projections and (2) identify
                 lessons learned from developing and implementing significant air traffic
                 modernization programs.

                 We conducted this audit in accordance with generally accepted Government
                 auditing standards. Exhibit A details our scope and methodology. Exhibit B lists



1NextGen is a collection of new programs and capabilities, including new air traffic management technologies and
procedures; airport infrastructure improvements; and environmental, safety, and security-related enhancements.


AV2021023                                                                                                          1
            the organizations we visited or contacted, and exhibit C lists the acronyms used
            in this report.

            We appreciate the courtesies and cooperation of Department of Transportation
            (DOT) representatives during this audit. If you have any questions concerning this
            report, please call me at (202) 366-0500 or Jay Borwankar, Program Director, at
            (202) 493-0970.


            cc:    The Secretary
                   DOT Audit Liaison, M-1
                   FAA Audit Liaison, AAE-100




AV2021023                                                                                      2
Results in Brief
                  NextGen benefits are lower than expected due to implementation
                  challenges, optimistic assumptions, and other factors.

                  Over the past decade, FAA has updated its projections for NextGen benefits
                  multiple times. In its most recent estimate from 2017, FAA’s overall benefits
                  projection is $113 billion lower than the Joint Planning and Development Office’s
                  (JPDO) initial projection. 2 Moreover, benefits actually achieved to date are limited.
                  According to FAA, NextGen generated $6 billion in total benefits from 2010 to
                  2018 split between the Agency, airlines, and the traveling public. NextGen’s
                  benefits have not kept pace with expectations for a number of reasons, including
                  implementation challenges—such as the 2-year delay in implementing Time-
                  Based Flow Management (TBFM) 3 capabilities for air traffic—that contributed to
                  the Agency reducing its most recent benefit projection by nearly $66 billion.
                  Furthermore, FAA and aviation stakeholders 4 told us that accrued NextGen
                  benefits are difficult to measure, making it challenging to assess NextGen’s
                  impact on the National Airspace System’s (NAS) performance. For example,
                  analyses of NextGen benefits related to reduced flight delays must also consider
                  other factors, such as the impact of weather and airline business decisions.
                  Moreover, FAA based its NextGen benefit projections on optimistic assumptions
                  about air traffic growth, and the Agency did not adjust its projections or produce
                  multiple scenarios to account for risk. For example, the initial benefit projections
                  did not factor in the impact of potential economic downturns on the aviation
                  industry or the effect of delays on one program impacting benefits from other
                  programs. Finally, FAA has acknowledged that significant declines in air traffic
                  and passengers due to the unprecedented Coronavirus Disease 2019 (COVID-19)
                  pandemic will further delay the expected timeline for achieving NextGen benefits.

                 Prior OIG work has identified lessons for improving FAA’s delivery of
                 NextGen and other future modernization efforts.

                 FAA’s experience and our prior work has shown that successful NextGen
                 implementation requires support and coordination both between FAA and
                 industry stakeholders and within multiple FAA lines of business. For example, key
                 NextGen technologies depend on airlines’ willingness to install costly avionics on
                 their aircraft. Demonstrating NextGen’s tangible benefits and prioritizing its most
                 beneficial capabilities is key to securing industry’s investment. Beginning in 2013,


2 In 2007, JPDO projected that NextGen benefits would total more than $213 billion. JPDO was established by
Congress in 2003 as a multi-agency organization managed by FAA to develop a plan for implementing NextGen.
3 TBFM is an automation decision support tool to help controllers sequence and space aircraft and enable the use of

more efficient approach procedures to airport runways.
4 Aviation stakeholders include Airlines for America and MITRE Corp.




AV2021023                                                                                                             3
                 FAA successfully worked with industry through the NextGen Advisory Committee
                 (NAC) 5 to set top implementation priorities based on benefits and
                 implementation readiness. However, industry representatives have expressed
                 concerns about the continued effectiveness of the NAC following its transition
                 from the Radio Technical Commission for Aeronautics (RTCA) to FAA in 2018. 6
                 Industry has also requested enhanced transparency about the current state of
                 NextGen benefits; however, FAA last published its NextGen business case in 2016
                 and does not currently plan to publish additional updates. Along with the need
                 for industry commitment, there are lessons learned related to FAA’s ongoing
                 challenges with implementing and integrating NextGen. For example, our work
                 has shown that FAA has struggled to integrate key NextGen technologies and
                 capabilities 7 due to extended program delays that caused ripple effect delays
                 with other programs. In addition, the Agency has not fully made use of its own
                 internal NextGen benefits analyses to help prioritize future implementation
                 decisions. Finally, critical controller automation tools are not yet in use, which
                 hinders FAA’s ability to test and evaluate the full impact of new technologies.
                 Leveraging these lessons learned will be critical for deploying NextGen’s
                 advanced capabilities, achieving benefits, and modernizing the NAS in a timely
                 manner.

                 We are making recommendations to improve FAA’s analysis and reporting of
                 NextGen benefits, along with the Agency’s delivery of NextGen and other future
                 modernization efforts.




Background
                 In 2003, Congress mandated that FAA establish JPDO to develop a plan for
                 implementing NextGen by 2025 and coordinate research efforts with other
                 Federal agencies. 8 NextGen encompasses multiple programs, procedures, and
                 systems at different levels of maturity. Implementing NextGen is a complex
                 undertaking and requires joint investments from FAA (new ground systems for
                 controllers) and airspace users (new avionics and displays for pilots) to realize
                 expected benefits. As part of its efforts, in 2007, JPDO developed a business case
                 for NextGen that presented an overview of the projected benefits and costs of
                 implementing NextGen for FAA and the airlines. JPDO predicted NextGen would


5 The NAC was established in 2010 to develop recommendations for NextGen and includes operators, manufacturers,
air traffic management, aviation safety, airports, and environmental experts.
6 The NAC’s original charter expired in 2018 and subsequently moved from the RTCA to FAA.
7 NextGen technologies and capabilities include Data Communications (DataComm), En Route Automation

Modernization (ERAM), and TBFM.
8 FAA’s partner agencies for NextGen are the National Aeronautics and Space Administration, Department of Defense,

Department of Homeland Security, and the Department of Commerce/National Weather Service.


AV2021023                                                                                                       4
                  require significant investments from airlines—an estimated $14 to $20 billion by
                  2025—to equip their fleets with new avionics to enable NextGen capabilities
                  while promising incredible returns on that investment. It also predicted the need
                  for a complete transformation of our Nation’s air transportation system to
                  facilitate the expected doubling or even tripling of demand for air travel by 2025.
                  The legacy air traffic control systems, while safe, were built on technology
                  platforms from the 1940s and 1950s that have largely exceeded their original life
                  expectancy and have since reached the limits of their ability to handle increases
                  in traffic. In 2007, JPDO reported that congestion was costing $9.4 billion per year
                  in passenger delays and predicted it could grow to $20 billion per year by 2025
                  without substantial increases in the NAS’ capacity.

                  JPDO was disbanded in 2014, but FAA continued to produce business cases for
                  NextGen. To estimate the benefits of planned NextGen improvements, FAA
                  primarily uses its simulation model—the System Wide Analysis Capability
                  (SWAC). 9 JPDO used a similar model and approach in its 2007 business case, but
                  FAA has continued to improve the modeling process and has incorporated
                  updated plans and performance of NextGen and the NAS into its modeling. In
                  addition, for programs that the NAS Systems Engineering and Integration Office 10
                  does not model, such as airborne rerouting, FAA relies on supplemental benefit
                  estimates from FAA program offices.

                  FAA’s business case also attempts to quantify benefits of improvements that the
                  Agency has already implemented. FAA does this by updating the benefits data
                  using mostly post implementation operational assessments. FAA works with
                  MITRE 11 and the Joint Analysis Team (JAT) 12 to conduct these analyses. FAA then
                  projects future benefits for these improvements and adds them to the overall
                  benefits projection.

                  FAA published its most recent and last planned business case in July 2016. 13
                  However, FAA updated its benefits estimates internally in 2017 in response to our
                  letter reviewing the 2016 business case. Our work identified several issues that
                  led to the estimates being optimistic, including using outdated program




9 The SWAC is a fast-time simulation model that estimates expected operational benefits of NextGen improvements
for the NAS.
10 FAA’s NAS Systems Engineering and Integration Office helps to design, analyze, plan, and protect the NAS by

managing the NAS Enterprise Architecture, defining NAS requirements, performing enterprise safety analyses,
maintaining systems engineering guidance and information, defining security standards, and analyzing NAS
performance using computational models and data analytics.
11 MITRE Corporation functions as FAA’s federally funded research and development center.
12 JAT is a joint FAA and industry group of experts that work to develop mutually agreed upon measurements of

NextGen programs at key sites. JAT is a subgroup under the NAC.
13 According to FAA officials, FAA does not plan to develop another full business case; instead, the Agency is focused

on working with industry on joint program benefit analyses.


AV2021023                                                                                                            5
                  schedules, aggressive assumptions about future implementations and traffic
                  growth, and a lack of risk adjustment or alternative scenarios. 14




NextGen Projected and Actual Benefits Are Lower
Than Expected Due to Implementation Challenges,
Optimistic Assumptions, and Other Factors
                  NextGen’s benefits have not kept pace with initial expectations. FAA’s most
                  recent business case projects total NextGen benefits to be over $100 billion less
                  than JPDO’s original estimate, and benefits actually achieved to date have been
                  minimal. Moreover, these benefits are difficult to measure, and FAA has faced key
                  implementation challenges that have limited its success. Additionally, NextGen’s
                  benefit projections were optimistic and did not account for risk factors. Finally,
                  significant declines in air traffic and passengers due to COVID-19 have extended
                  the timeframe for realizing expected NextGen benefits.



             NextGen Benefits Are Significantly Lower
             Than Initially Projected
                  In 2007, JPDO’s initial business case projected that NextGen would generate
                  $213 billion 15 in benefits by 2025 from capacity increases. Subsequently, in 2012,
                  FAA published its own benefit projections for NextGen totaling $118 billion
                  through 2030 16 split between FAA, airlines, and the traveling public. FAA has since
                  updated its benefit projections multiple times, with a high of $199 billion in 2013,
                  down to $100 billion in 2017 (see figure 1). FAA’s 2017 estimate was $113 billion
                  less than what JPDO originally projected and extended over a longer time
                  horizon—2030 vs. 2025. Although the more recent estimates for NextGen
                  benefits are much lower than JPDO’s initial 2007 estimate, NextGen was
                  projected to generate $100 billion in benefits, which is still significantly higher




14 Letter to Chairman Bill Shuster and Chairman Frank L. LoBiondo Regarding FAA’s July 2016 NextGen Business Case,
(OIG Correspondence No. CC2017015), August 15, 2017. OIG products are available on our website at
http://www.oig.dot.gov/.
15 The $213 billion is based on forecasted capacity increases. The analysis of other sources of benefits, including

environmental, safety, and weather, were not mature enough to project a total for 2025. All business case benefit and
cost estimates are provided in fiscal year 2018 dollars.
16 FAA’s benefit estimates are more mature and include additional sources of benefits, such as safety, and reduced

operating costs to FAA. However, reduced delays are the majority source of benefits from each of FAA’s business
cases.


AV2021023                                                                                                           6
                           than its projected total cost of $36 billion, giving it a positive return on
                           investment.

Figure 1. NextGen’s Projected Benefits and Costs (in Billions)
                $250
                         $213
                                                                 $199
                $200
                                                                                                                        $166
                                                                                                      $156
                $150                                                                $143
  In Billions




                                             $118
                                                                                                                                          $100
                $100


                                $43                 $41                 $43                                                    $38
                $50                                                                        $32               $33                                 $36


                  $0
                       2007 JPDO NextGen   2012 FAA NextGen    2013 FAA NextGen   2014 FAA NextGen   2015 FAA Model   2016 FAA NextGen   2017 FAA Model
                         Business Case*      Business Case       Business Case      Business Case        Update         Business Case        Update


                                                              Benefit Estimate           Cost Estimate


*The 2007 JPDO NextGen Business Case includes benefits through 2025. All FAA estimates are through 2030.
Additionally, JPDO’s benefit estimate is from capacity improvements only, whereas FAA’s business cases include
additional sources of benefits, such as those arising from improvement in safety and reduced operating costs.

Source: OIG analysis of JPDO and FAA data, converted into fiscal year 2018 dollars for comparison.

                           FAA has objected to using the JPDO benefit estimates as a baseline for
                           comparison because those estimates were not produced solely by FAA, and
                           officials claim they were never supported by the Agency. However, FAA was a
                           partner agency to JPDO, and the Agency had engaged the contractor that
                           developed JPDO’s estimate. Even disregarding the JPDO estimate, FAA’s 2017
                           update is the lowest estimate the Agency has produced by billions of dollars.

                           FAA’s Systems Analysis & Modeling Division asserted that NextGen benefits have
                           not been lost, but simply delayed. The 2030 end date for calculating benefits
                           does not provide a full 20-year lifecycle for many individual NextGen programs
                           due to delays in implementation. The Systems Analysis & Modeling Division
                           contended that if the 2017 estimate was extended through 2035, the total
                           benefits would exceed $160 billion.

                           Thus far, FAA claims NextGen has generated $6 billion in total benefits from 2010
                           to 2018 split between FAA, airlines, and the traveling public. These benefits
                           represent a small percentage of FAA’s overall $100 billion total projected benefits
                           and are lower than the $9 billion that FAA has already invested in implementing



AV2021023                                                                                                                                                 7
                  NextGen. 17 Airlines have also made investments to upgrade their aircraft avionics
                  for NextGen. However, the $6 billion in benefits is in keeping with FAA’s 2017
                  model update that predicted benefits would equal costs in 2024.



             Impacts of NextGen Programs Are Not
             Easily Measured
                  NextGen benefits are difficult to measure according to FAA officials, MITRE, and
                  industry experts. MITRE, JAT, and FAA use sophisticated models and carefully
                  designed pre/post implementation analyses to determine the benefits achieved
                  from NextGen capabilities at specific sites. However, factors outside FAA’s
                  control—such as weather and airline business decisions, including changes to
                  their schedules and aircraft fleet composition—can offset any improvements in
                  operations. As such, these factors can impact the projected improvements in
                  flight delay numbers and fuel burn efficiency. Furthermore, site- and technology-
                  specific studies are limited in their ability to fully capture NextGen-related
                  benefits due to the cost and difficulty of conducting them. As a result, FAA is
                  forced to extrapolate the results from some sites to others when incorporating
                  these results into the Agency NextGen benefits model, which could lead to
                  unreliable benefit projections.

                  In a separate effort, FAA is developing a set of NextGen performance metrics that
                  the Agency plans to report on a daily basis using actual operations while
                  accounting for factors outside FAA’s control, such as weather. Specifically, these
                  metrics would measure the impact of Initial-Trajectory Based Operations (iTBO). 18
                  iTBO is a step towards shifting air traffic controllers to separate traffic using time-
                  based metering, which relies on new controller tools and onboard avionics to
                  improve efficiency. iTBO integrates the key NextGen technologies and capabilities
                  that FAA’s model predicts will generate future NextGen benefits, including
                  Performance-Based Navigation (PBN), 19 controller metering automation tools,
                  and Data Communications (DataComm). 20 However, FAA could not provide a




17 The $9 billion includes NextGen funding from 2003 to 2018.
18 iTBO is the first phase in moving towards full Trajectory Based Operations (TBO), an air traffic management method
for strategically planning, managing, and optimizing flights by using time-based management (TBM), information
exchange between air and ground systems, and the aircraft’s ability to fly precise paths. TBO is a shift in the way
controllers separate air traffic from using set distance, miles-in-trail, to using time.
19 PBN delivers new flight routes that primarily use satellite-based navigation aids and on-board aircraft equipment to

navigate with greater precision and accuracy. As such, PBN can provide significant benefits, such as more direct flight
paths, improved on-time airport arrival rates, greater fuel savings, and reduced emissions and noise.
20 DataComm enables air to ground digital communications services between FAA facilities, controllers, aircraft, and

flight crews that result in reduced delays, increased throughput and efficiency, enhanced safety, and a reduction in
environmental impacts.


AV2021023                                                                                                            8
                  date for when these metrics would be available and used to assess NextGen’s
                  performance.

                  Due to FAA’s lack of system-wide measures for NextGen performance, we
                  analyzed several performance metrics from the Department of Transportation’s
                  Bureau of Transportation Statistics (BTS). Specifically, we looked at taxi times and
                  delays from 2008 to 2019 across the NAS for the 24 largest domestic airlines,
                  including American Airlines, Delta Air Lines, and United Airlines. We were not able
                  to identify any clear improvement to NAS operations overall. Instead, overall
                  operations appeared to become less efficient, with average taxi time, departure
                  delays, and arrival delays all increasing over this period. Airlines for America (A4A)
                  also conducted analysis showing increased block times. 21 However, the impact of
                  a variety of airline business decisions, weather, and other non-NextGen related
                  factors can outweigh the impact of NextGen programs. As such, this analysis is
                  not sufficient to evaluate NextGen’s impact on operations. Specifically, FAA’s
                  analysis shows that the increases in delays can be attributed to increases in traffic
                  at some large airports. The concentration of additional traffic at these airports has
                  led to significant increases in delays throughout the system.



             Implementation Challenges Have Delayed
             and Diminished Programs, Lowering
             Expected Benefits
                  Our office, the Government Accountability Office, and others 22 have identified
                  challenges to implementing NextGen programs and capabilities dating back to
                  2006. These challenges include delays and reduction of the number of planned
                  sites for new controller tools to manage air traffic and new advanced flight
                  procedures that are key drivers for NextGen benefits.

                  While FAA has deployed much of the infrastructure for NextGen, new capabilities
                  that are expected to generate benefits are not widely used or have yet to be
                  implemented, including advanced PBN procedures and reduced separation
                  standards from Automatic Dependent Surveillance-Broadcast (ADS-B). 23
                  Additionally, key controller tools such as Terminal Sequencing and Spacing
                  (TSAS), 24 which should facilitate increased use of PBN procedures when demand




21 Block time is time from departure gate to arrival gate.
22 As part of our audit, we reviewed documents from the National Research Council, NAC, and Airlines for America.
23 ADS-B uses the satellite-based Global Positioning System and is intended to allow FAA to transition from ground-

based radar to a satellite-based system for improving surveillance and management of air traffic.
24 TSAS is a new tool to help air traffic controllers merge and sequence aircraft using time-based metering in the

airspace closest to airports.


AV2021023                                                                                                             9
                   is high at airports, and Terminal Flight Data Manager (TFDM), 25 which aims to
                   improve efficiency of surface operations, have not yet been deployed.

                   Some programs—even after they have been deemed fully operational—are not
                   producing their expected benefits, and their new capabilities are not being
                   deployed as widely as originally planned. For example, the first seven completed
                   Metroplex 26 sites—a program that primarily implements new PBN procedures at
                   the largest airports—are producing about half of the benefits originally expected.
                   Additionally, in August 2017 we highlighted several concerns about FAA’s 2016
                   NextGen Business Case model, including the concern that the Agency was using
                   out-of-date program schedules to calculate projected NextGen benefits that did
                   not reflect delays to key programs. For example, each Metroplex site has taken
                   longer to implement than originally expected—4 to 5 years instead of 2 to
                   3 years—due to a range of factors including technical issues with the En Route
                   Automation Modernization (ERAM) 27 system’s installation, increased community
                   outreach due to environmental concerns, and issues with procedure design. In
                   addition to the delays, FAA’s 2016 model assumed that a full Metroplex
                   implementation would occur at 31 major airports by 2022, rather than FAA’s
                   subsequently scaled-down plan to implement these improvements at only
                   19 airports.

                   In response to our 2017 letter, FAA’s Systems Analysis & Modeling Division staff
                   addressed many of our concerns while updating the NextGen benefits model.
                   Their update resulted in reducing the benefit projection by 40 percent—from
                   $166 billion to $100 billion. FAA estimates that approximately $18 billion
                   (27 percent) of the $66 billion decrease is attributable to delays and reductions in
                   deployment sites of key NextGen programs and capabilities. Specifically, FAA
                   updated its benefits model to reflect 2-year delays to new TBFM capabilities and
                   the TSAS tool for terminal controllers; a delay and reduction in the number of
                   Metroplex sites; and an up to 8-year delay in relative spacing using Interval
                   Management, another key component of time-based metering.

                   Of the $100 billion in benefits projected in 2017, only 18 percent is projected to
                   be generated by programs that have been implemented, with the rest coming
                   from future planned programs and capabilities, such as new controller
                   automation tools and reduced separation standards. However, based on FAA’s




25 TFDM is a new decision support tool for airport surface management, including creating virtual departure queues,
data sharing with flight operators, and electronic flight strips to help enable time-based metering.
26 Metroplex is an initiative intended to improve the efficiency of airspace that affects multiple airports near large

metropolitan areas by implementing high value PBN procedures and airspace changes.
27 ERAM is a multibillion-dollar system for processing flight data at facilities that manage high-altitude traffic typically

above 10,000 feet, where aircraft reach their cruising altitudes.


AV2021023                                                                                                                10
                  past performance, as documented in our prior audit work, these programs could
                  experience delays or may not fully deliver their expected benefits.



              NextGen Benefit Projections Were Based
              on Optimistic Air Traffic Growth and
              Lacked Risk Adjustments
                  The original JPDO model as well as FAA’s later models for NextGen benefits were
                  based on optimistic assumptions about projected air traffic levels. The models
                  relied on the Terminal Area Forecast (TAF), 28 which has consistently predicted
                  significant traffic growth. However, traffic has remained largely flat since NextGen
                  began, and recently traffic has decreased significantly due to the COVID-19
                  pandemic.

                  The majority of FAA’s projected NextGen benefits are from the flying public
                  experiencing less delay per flight than they would have experienced without
                  NextGen. These benefits, known as Passenger Value of Time (PVT), make up
                  64 percent of the total benefits in FAA’s 2017 projection. The heavy reliance on
                  PVT makes NextGen benefits very sensitive to changes in air traffic and number
                  of passengers. In 2013, FAA updated the dollar value for PVT in its business case
                  model in accordance with DOT guidance 29 from $28.60 per hour to $43.50 per
                  hour (a 52 percent increase), with a 1.6 percent annual increase. This contributed
                  to a dramatic increase in the projected benefits for NextGen overall, from
                  $118 billion to $199 billion, with PVT accounting for well over half of the total
                  benefits. However, given that FAA’s models have consistently overestimated
                  traffic growth, the resulting projected benefits derived from PVT could be greatly
                  inflated.

                  There is also no consensus on how to calculate PVT. According to DOT guidance,
                  FAA counts every minute equally to add to the total reduction in delays to
                  passengers. As a result, DOT’s guidance for calculating PVT leads FAA to higher
                  benefit calculations as compared to some other countries we reviewed. For
                  example, Canada does not include increments of less than 5 minutes, and
                  Germany discounts smaller increments of time compared to larger ones when
                  calculating PVT. Meanwhile, the European Union recommends counting all time




28 The TAF is an annual report that projects air traffic at all U.S. airports. FAA has recognized actual traffic has
consistently been lower than the last several TAF projections and the shortfall is apparent even in the first projected
year, but FAA continues to use it in NextGen benefits forecasts.
29 Revised Departmental Guidance on Valuation of Travel Time in Economic Analysis (9/28/2011),

https://www.transportation.gov/resources/2011-revised-value-of-travel-time-guidance.


AV2021023                                                                                                             11
            equally but reporting small increments separately, providing greater
            transparency.

            Furthermore, traffic has not grown as expected since NextGen began. In 2007,
            JPDO predicted NextGen would need to triple capacity to accommodate traffic
            growth. This contributed significantly to JPDO’s initial benefit estimate being
            much higher than later FAA projections. When traffic levels return and potentially
            grow as FAA’s models predict, delays are expected to continue to increase even
            with NextGen programs and capabilities fully deployed. For example, FAA’s 2016
            NextGen business case predicted that without NextGen, flights would be delayed
            on average by over 20 minutes. However, with NextGen improvements, delays are
            still expected to increase by almost 15 minutes per flight.

            Finally, FAA’s NextGen benefit projections did not factor in risks—such as the risk
            that air traffic growth would not increase as expected. While FAA does provide a
            risk adjustment when estimating the benefits of individual programs as part of
            the Acquisition Management System process, it did not perform a risk adjustment
            in its business cases when projecting the overall benefits of NextGen. Staff from
            FAA’s Systems Analysis & Modeling Division explained that until recently, they
            have lacked official ranges for the TAF to incorporate in their model to generate
            different forecasts or risk adjustments. However, there are additional factors that
            may impact the overall risk of implementing NextGen, such as program
            interdependencies that are not fully captured by assessing risks for the individual
            programs. Furthermore, the projections did not factor in other significant events,
            such as economic fluctuations that occur cyclically, which can also have a
            measurable impact on traffic levels. For example, the economic recession of
            2007–2009 reduced air traffic significantly (see figure 2).




AV2021023                                                                                   12
Figure 2. Passenger Throughput Before and After 2007–2009 Economic
Recession
                                              220,000,000


                                              215,000,000
     Total Number of Passengers Per Quarter




                                              210,000,000


                                              205,000,000


                                              200,000,000          Official Start of
                                                                     Recession

                                              195,000,000


                                              190,000,000
                                                                                                     Official End of
                                                                                                       Recession
                                              185,000,000
                                                            2006   2007       2008     2009   2010       2011          2012   2013   2014


Source: OIG analysis of BTS passenger throughput data. Data have been adjusted for seasonal variation.

                                                        Factoring in the risk of such cyclical events could improve the accuracy of FAA’s
                                                        benefit estimates. This is especially the case given that FAA’s benefits model is
                                                        sensitive to the growth of air traffic, and FAA uses air traffic projections that have
                                                        historically overestimated growth. Additionally, Office of Management and
                                                        Budget (OMB) guidance recommends that multiple scenarios and probability
                                                        distributions be provided when possible in benefit-cost estimates, especially
                                                        when past experience shows initial estimates were optimistic. 30 FAA’s lack of risk
                                                        adjustments and its decision to provide only a single scenario for NextGen
                                                        benefits led to overly optimistic expectations about NextGen’s benefits. As a
                                                        result, FAA provides no level of assurance to decision makers or stakeholders on
                                                        whether potential billion-dollar NextGen investments are low, medium, or high
                                                        risk.




30   Guidelines and Discount Rates for Benefit-Cost Analysis of Federal Programs (OMB Circular A-94).


AV2021023                                                                                                                                   13
              COVID-19 Will Further Extend the
              Timeframe for Capturing NextGen
              Benefits
                    FAA acknowledges that NextGen’s benefits will be further delayed by the
                    unprecedented COVID-19 pandemic. Specifically, the COVID-19 pandemic has
                    resulted in a tremendous drop in air traffic and passengers, both of which are
                    major drivers for NextGen benefits. Since early 2020, when the pandemic began
                    in the United States, there have been days in which air traffic decreased over
                    70 percent and passengers over 90 percent (see figures 3 and 4). Additionally,
                    FAA stated that COVID-19 has led to delays in key NextGen programs, including
                    TFDM and DataComm, which will further affect NextGen’s benefit recovery
                    timeframe.

Figure 3. Percentage Change in Daily Flights 31 Compared to Prior 3-Year
Average




Source: FAA data.




31Chart depicts daily number of Instrument Flight Rules (IFR) flights, which is a set of rules governing flights relying
on the aircraft’s instruments and navigation aids. IFR permit aircraft to fly in certain limited visibility and cloud
conditions. Virtually any commercial operation—including airlines and business jets—are IFR flights.


AV2021023                                                                                                              14
Figure 4. Passenger Throughput Data, 2019 and 2020

                         3,000,000


                         2,500,000
  Number of Passengers




                         2,000,000


                         1,500,000


                         1,000,000


                          500,000


                                -
                                     1-Mar           1-Apr           1-May              1-Jun            1-Jul

                                         2020 Passenger Throughput           2019 Passenger Throughput


Source: OIG analysis of Transportation Security Administration data.

                                      It is unknown how long the impacts of the pandemic will last and the extent of
                                      the decrease in air traffic. However, according to FAA officials, there will be very
                                      little benefit from NextGen during this time, and many systems, such as TBFM
                                      and TSAS, could go completely unused until traffic levels return. In addition,
                                      capabilities such as PBN will provide much lower benefit because of the reduced
                                      traffic.

                                      There will still be some benefits from NextGen, such as consolidating systems and
                                      modernizing infrastructure, but these cost savings only made up 2 percent of the
                                      2017 benefit estimate. Furthermore, as previously mentioned, the majority of the
                                      already accumulated $6 billion in benefits are derived from reduced expected
                                      delay to passengers ($3.4 billion), or PVT. Given the impact of COVID-19 on
                                      passenger travel, PVT is unlikely to continue to be a source of significant benefits
                                      in the near-term.




AV2021023                                                                                                               15
Prior OIG Work Has Identified Lessons for
Potentially Improving FAA’s Delivery of NextGen
and Other Future Modernization Efforts
            Our longstanding body of work related to NextGen has identified lessons that
            FAA could use to improve NextGen delivery. For example, while FAA has
            collaborated with industry to prioritize, implement, and measure benefits of
            NextGen programs, there are still opportunities for improving transparency,
            which will be critical to secure industry’s long-term investment. In addition,
            further advancing NextGen will depend on resolving complex implementation
            challenges, including effectively prioritizing programs, integrating interdependent
            capabilities, and harnessing controller automation tools to achieve benefits.



        FAA Has Collaborated With Industry To
        Advance NextGen but Can Improve
        Transparency and Internal Coordination
            While FAA and industry have partnered to prioritize and plan programs, securing
            industry investment remains key to NextGen’s long-term success. Central to this
            effort is FAA being transparent with stakeholders about NextGen benefits.
            Overall, successful NextGen implementation also requires support and
            coordination both between FAA and industry stakeholders and within multiple
            FAA lines of business.

            FAA Has Partnered With Industry To Prioritize and Plan Programs but
            Will Require Additional Industry Investment

            NextGen is a large joint investment between FAA ($21 billion) and industry
            ($15 billion). As such, both FAA and industry play a role in advancing and
            measuring NextGen’s success. For example, airline business decisions can have a
            significant impact on the operational efficiency of the NAS, such as scheduling of
            flights and fleet types.

            Additionally, in order for airlines to fully benefit from NextGen, they need to
            equip their aircraft with new avionics and other technologies. While airlines have
            made significant investments in new equipment, they were reluctant to make
            such investments until they were convinced that NextGen’s benefits would
            outweigh their initial investment and that they could trust FAA would implement
            them as planned. In the past, FAA worked successfully with industry to implement
            the first segments of DataComm, a program requiring new onboard avionics



AV2021023                                                                                   16
                    investments by airlines and ground infrastructure by FAA. The collaboration was
                    successful because airlines saw clear and immediate benefits to the investment.

                    FAA and industry have also already successfully worked together to establish
                    NextGen’s top priorities. Under the RTCA–a chartered Federal Advisory
                    Committee—FAA and industry formed the NAC, which provided a forum and
                    structure to work together. In 2013, FAA asked the NAC to prioritize the NextGen
                    programs based on benefits, technological maturity, and implementation
                    readiness. As FAA has implemented these programs, the Agency has used JAT
                    analysis to influence the scope and identify additional sites for several of these
                    NextGen initiatives. For example, JAT found benefits from both Wake
                    Recategorization (Wake Recat) 32 and DataComm Tower Services, which brought
                    greater industry support and helped increase the number of sites for
                    implementation. Alternatively, the JAT analysis of the North Texas Metroplex
                    found that while there were benefits, they were lower than originally expected.
                    This helped limit the implementation of Metroplex elsewhere.

                    While the NAC was able to prioritize NextGen programs and develop an
                    implementation plan, its charter expired on May 29, 2018, and transitioned from
                    the RTCA to FAA. Industry officials we spoke with are concerned about the
                    continued effectiveness of the NAC, which could influence their continued
                    willingness to invest in NextGen related technologies. Given that a significant
                    amount of industry investment will still be required to advance NextGen, gaining
                    industry partnership and support to continually identify priorities and assess
                    benefits will remain critical.

                    Stakeholders Would Like More Transparency on NextGen Benefits

                    A key aspect of securing industry investment in NextGen is communicating
                    NextGen’s benefits. Airline officials stated that they would like more transparency
                    from FAA on actual benefits of NextGen programs. For example, JAT’s group of
                    experts work to analyze and develop mutually agreed upon results of NextGen
                    programs, such as the impact of Metroplex on fuel burn, at key sites. While
                    airlines support this effort, its scope is limited by the difficulty and cost of
                    conducting such analyses.

                    According to the airline representatives we spoke with, industry would like to see
                    regular reporting on the benefits of NextGen with clear explanations of where the
                    data are coming from. However, FAA has not published a new business case for
                    NextGen since 2016, and the Agency has no plans to publish an updated
                    business case. According to FAA officials, the Agency does not plan to develop
                    another full business case; instead, it is focused on working with industry on joint
                    program benefit analyses. Yet, given the recent changes to the airline industry in


32   Wake Recat safely reduces the separation between aircraft on arrivals and departures.


AV2021023                                                                                            17
            the wake of COVID-19, further transparency is critical to report on the effective
            status and benefits of NextGen and inform airlines’ planning and investments in
            new avionics.

            Prior and Current OIG Work Has Raised Issues About Effective
            Coordination Across FAA Lines of Business

            NextGen programs require FAA to work effectively across diverse agency lines of
            business. However, OIG has found that FAA has had issues coordinating NextGen
            programs across offices and air traffic facilities. For example, implementing
            Metroplex requires multiple offices to work together to make policy changes,
            develop new flight procedures, ensure adequate training for controllers on new
            procedures, and develop new automation tools to allow controllers to manage
            traffic effectively. Additionally, FAA, industry, and our own work found that TBFM
            requires air traffic control facilities to work together to facilitate time-based
            metering. However, lack of a national vision, standardized operating policies and
            procedures, and intra-facility agreements led to fragmented usage of the tool.
            While FAA has worked to address these issues, NextGen will continue to present
            challenges in coordinating across multiple lines of business and air traffic
            facilities.



        Other Lessons Provide Additional
        Insights Into Complex Implementation
        Challenges That May Impact NextGen
        Delivery
            Significant work and challenges remain to implement and integrate NextGen’s
            advanced capabilities. While FAA has formulated its own analysis of NextGen
            benefits and impact, the Agency does not make use of these analyses to help
            prioritize future implementation decisions. In addition, critical automation tools
            are not yet in use, which hinders FAA’s ability to evaluate and test new
            technologies and advance NextGen benefits.

            FAA Does Not Fully Leverage Its Own Analysis of NextGen Benefits To
            Help Prioritize Implementation

            Until recently, FAA did not leverage the benefits work of its own Systems Analysis
            & Modeling Division to help prioritize NextGen funding or deployments. FAA
            approves and plans implementations of new technologies and capabilities based
            on cost-benefit analyses performed as part of its Acquisition Management
            System, which relies on engineering estimates and results of demonstration
            projects. However, FAA usually does not use the updated projections and post-
            implementation analysis it already conducts to help reprioritize NextGen


AV2021023                                                                                    18
            programs and deployment locations to maximize benefits to users and the
            traveling public. Instead, according to FAA officials, the Agency prioritizes
            programs based on the Enterprise Architecture and funding. FAA was able to
            provide one example of leveraging such analyses to help prioritize site selections
            for iTBO; however, other NextGen programs have not leveraged the Systems
            Analysis & Modeling Division’s work.

            FAA has relied on similar work from JAT to reduce the number of Metroplex
            locations where benefits were lower than expected and to expand the number of
            locations using Wake Recat due to the program’s high benefits. However, other
            than the recent iTBO work, FAA could not provide examples where the Agency
            had leveraged its own work to make more efficient implementation decisions.
            According to the FAA officials we spoke with, the Agency considers a large
            number of logistical and planning considerations to make deployment decisions,
            making it difficult to adjust based on actual benefits data. However, some officials
            we spoke with also agreed that using benefits data would help the Agency make
            better decisions regarding how and where to invest in future NextGen
            capabilities.

            Integrating NextGen Technologies To Enable Advanced Capabilities
            Remains a Challenge

            NextGen requires FAA to implement multiple new and highly interdependent
            technologies. Our recent and longstanding body of work has shown that new
            systems, such as DataComm, ERAM, and TBFM, faced delays and budget
            overruns. Due to the interdependencies between these systems, delays in one
            system caused a ripple effect and therefore delayed other systems. While FAA has
            largely deployed the new infrastructure as described above, FAA continues to
            face challenges in implementing additional segments of these systems or
            capabilities. FAA also has yet to implement other key systems that drive NextGen
            benefits, including TSAS and TFDM.

            The Agency has struggled with the shift from planning to actually using the new
            capabilities to generate benefits for all stakeholders. For example, new PBN
            procedures have not been as widely implemented as planned due to mixed levels
            of avionics on aircraft operating in the NAS, lack of controller tools, delays to
            ERAM, and community legal challenges due to noise concerns. FAA’s continued
            focus on addressing these barriers is needed to achieve the benefits from
            NextGen.

            Controllers Are Not Yet Consistently Using Critical Automation Tools,
            Limiting Potential Benefits

            FAA’s experience with NextGen so far has shown that controller automation tools,
            such as TSAS and TBFM, are critical to increasing the efficiency of operations in
            the NAS. These new tools can reduce separation of aircraft and facilitate the use


AV2021023                                                                                    19
            of more advanced flight procedures with shorter and more fuel efficient flight
            paths. However, in previous audits we found that FAA lacked a national program
            vision, clear guidance, standard operating procedures, training, and metrics to
            measure success for TBFM. This resulted in fragmented and inconsistent use by
            controllers, limiting benefits. MITRE is currently working with FAA to help
            implement these new technologies, flight procedures, and operating paradigms
            at several airports. While these changes will provide little to no benefit while air
            traffic remains low due to the COVID-19 pandemic, according to MITRE, they will
            provide an environment to more easily test TBO and transition controllers and
            pilots to using the new systems and capabilities.




Conclusion
            Over the last decade, FAA and industry have invested significantly in NextGen
            and achieved a relatively small percentage of the expected benefits from new
            capabilities. Overall NextGen benefits will fall short of the predictions from JPDO
            and FAA due to delays to key programs, optimistic assumptions about traffic and
            passenger growth, and factors outside of FAA’s control, including business
            decisions by airlines and declines in air traffic caused by the COVID-19 pandemic.
            There are a number of key lessons learned from over a decade of NextGen
            planning, development, and implementation that FAA can leverage further to
            benefit future modernization efforts. Leveraging these lessons can help FAA set
            stakeholder expectations, secure additional industry investment, and continue to
            make progress in improving the efficiency of the NAS.




Recommendations
            To improve the delivery of NextGen and other future NAS modernization efforts,
            we recommend that the Federal Aviation Administrator:

               1. Publish metrics that measure performance of NextGen improvements
                  across the NAS.

               2. Develop and implement a process that incorporates interim adjusted
                  benefit projections and interim implementation analyses to support
                  prioritization of NextGen programs and deployment locations.

               3. Update and provide stakeholders a risk adjusted NextGen benefit
                  projection.




AV2021023                                                                                     20
Agency Comments and OIG Response
            We provided FAA with our draft report on February 5, 2021, and received its
            response on March 9, 2021, which is included as an appendix to this report. FAA
            concurred with all three of our recommendations and proposed appropriate
            actions and completion dates. Accordingly, we consider all recommendations as
            resolved but open pending completion of the planned actions.




Actions Required
            We consider all three recommendations to be resolved but open pending
            completion of FAA’s planned actions.




AV2021023                                                                                 21
Exhibit A. Scope and Methodology
            We conducted this audit between October 2019 and February 2021 in
            accordance with generally accepted Government auditing standards as
            prescribed by the Comptroller General of the United States. Those standards
            require that we plan and perform the audit to obtain sufficient, appropriate
            evidence to provide a reasonable basis for our findings and conclusions based on
            our audit objectives. We believe that the evidence obtained provides a
            reasonable basis for our findings and conclusions based on our audit objectives.
            Our audit objectives were to (1) compare the current expected benefits of
            NextGen with the original projections and identify the reasons for revising those
            projections, and (2) identify lessons learned from developing and implementing
            significant air traffic modernization programs.

            To determine and compare the current expected benefits of NextGen with the
            original benefit projections, we compiled prior NextGen business cases and
            documents produced by FAA and JPDO. In order to compare early benefits from
            2007 to the most recent benefit estimates of 2017, we consulted with an OIG
            economist and converted the benefit estimates into fiscal year 2018 dollars using
            the fiscal year average consumer price index. We also compared the original and
            current estimated costs using the same process.

            To identify the causes for revising projections, we compiled and examined
            program documents produced by FAA, JPDO, and prior OIG reports. We
            compared business cases produced by FAA and JPDO to identify changes in
            capabilities and how benefits were modeled. We reviewed prior OIG work to
            identify previous findings that contributed to revising projections. We also
            interviewed program officials and other stakeholders, including FAA’s Systems
            Analysis & Modeling Division and other NextGen officials, MITRE, Airlines for
            America, former JPDO members, and National Aeronautics and Space
            Administration officials.

            We worked with OIG statisticians to obtain and analyze performance metrics
            data. The statisticians downloaded, unzipped, and imported 144 Airline Service
            Quality Performance data files from DOT’s BTS TranStats website for the years
            2008 to 2019. Some of the data elements included are departure, arrival, and
            elapsed flight times as shown by the Official Airline Guide. The statistician
            standardized the variables length in those 144 files and then merged them into
            one file which had over 76 million flight records reported by 24 U.S. certificated
            air carriers for 8,925 city pairs. Carriers within 1 percent or more of the total
            domestic scheduled service passenger revenues are required to report data for
            flights involving any airport in the 48 contiguous states accounting for 1 percent
            or more of the domestic scheduled service passenger enplanements. The
            regulation also provides for the voluntary reporting of a carrier's entire domestic


Exhibit A. Scope and Methodology                                                             22
            system and voluntary reporting by other carriers. In addition, the statistician
            summarized some of these data by year and month and computed metrics that
            focused on data pertaining to this audit such as arrival delays, departures delays,
            and taxi times.

            The statisticians also downloaded from the TranStats website and imported files
            titled “T-100 Domestic Market Data – U.S. Carriers Traffic and Capacity” for the
            years 2008 to November 2019. The data fields contain information on passengers
            and seats. The variable lengths were standardized and merged into one T-100D
            file that had over 4 million flight records for over 108 million flights, reported by
            191 air carriers for 76,288 city pairs. These flights had over 8 billion passengers or
            80 percent of the over 10 billion available seats. The statisticians summarized
            these data by year and month and computed metrics that focused on capacity
            growth.

            We also downloaded passenger throughput data from the BTS and
            Transportation Security Administration website and imported data into Excel files
            for the following years: 2000–2003, 2006–2013, and 2019–2020. The data fields
            contain information on the total amount of travelers that passed through our air
            traffic system. To adjust for seasonal trends within passenger travel, we consulted
            with an OIG economist to eliminate the seasonal component. The data were
            adjusted using the Pindyck and Rubinfeld seasonal adjustment method, which
            utilizes a 12-month centered moving average. These data were used to
            determine the impact of external events on the air traffic system.

            To identify lessons learned from developing and implementing significant air
            traffic modernization programs, we reviewed prior OIG reports to determine key
            challenges and lessons learned during the development and implementation of
            NextGen. We interviewed program officials as well as external stakeholders such
            as FAA’s Office of the Chief Scientist, NAC, JAT, and MITRE to gain their
            perspective and discuss the lessons learned they identified during the
            development and implementation of NextGen.




Exhibit A. Scope and Methodology                                                               23
Exhibit B. Organizations Visited or Contacted

          FAA Offices
             Air Traffic Organization

                    Office of Performance Analysis

                    Systems Operations Services

             Office of NextGen

                    NAS Systems Engineering and Integration Office

                    Office of the Chief Scientist

                    Portfolio Management & Technology Development

                    Systems Analysis & Modeling



          Other Organizations
             Airlines For America

             DOT Bureau of Transportation Statistics

             Joint Analysis Team

             The MITRE Corporation

             NextGen Advisory Committee

             National Aeronautics and Space Administration




Exhibit B. Organizations Visited or Contacted                        24
Exhibit C. List of Acronyms
             ADS-B            Automatic Dependent Surveillance-Broadcast
             BTS              Bureau of Transportation Statistics
             DataComm         Data Communications
             DOT              Department of Transportation
             ERAM             En Route Automation Modernization
             FAA              Federal Aviation Administration
             iTBO             Initial-Trajectory Based Operations
             JAT              Joint Analysis Team
             JPDO             Joint Planning and Advisory Committee
             NAC              NextGen Advisory Committee
             NAS              National Airspace System
             OIG              Office of Inspector General
             OMB              Office of Management and Budget
             PBN              Performance-Based Navigation
             PVT              Passenger Value of Time
             RTCA             Radio Technical Commission Aeronautics
             SWAC             System Wide Analysis Capability
             TAF              Terminal Area Forecast
             TBFM             Time-Based Flow Management
             TBM              Time-based management
             TFDM             Terminal Flight Data Manager
             TSAS             Terminal Sequencing and Spacing




Exhibit C. List of Acronyms                                                25
Exhibit D. Major Contributors to This Report
             NATHAN CUSTER                     PROGRAM DIRECTOR
             JAYDEEP BORWANKAR                 PROGRAM DIRECTOR
             JAMES OVELMEN                     PROJECT MANAGER
             MELISSA PYRON                     SENIOR AUDITOR
             TAMARIA KELLY                     SENIOR ANALYST
             ANDREW WEAKLEY                    AUDITOR
             GENESIS TURMAN                    ANALYST
             AUDRE AZUOLAS                     SENIOR TECHNICAL WRITER
             ALLISON DUKAVAS                   WRITER-EDITOR
             AMY BERKS                         DEPUTY CHIEF COUNSEL
             PETRA SWARTZLANDER                SENIOR STATISTICIAN
             MAKESI ORMOND                     STATISTICIAN
             JERROD SHARPE                     SENIOR ECONOMIST
             EVAN ROGERS                       ECONOMIST




Exhibit D. Major Contributors to This Report                             26
Appendix. Agency Comments

                        Federal Aviation
                        Administration

    Memorandum
Date:        March 9, 2021
To:          Matthew E. Hampton, Assistant Inspector General for Aviation Audits

From:        H. Clayton Foushee, Director, Office of Audit and Evaluation, AAE-1
Subject:     Response of the Federal Aviation Administration (FAA) to the Office of Inspector
             General (OIG) Draft Report: NextGen Benefits Have Not Kept Pace With Initial
             Projections, but Opportunities Remain to Improve Future Modernization Efforts


The FAA-led modernization of our nation's air transportation system is one of the most
ambitious infrastructure projects in U.S. history. Working with industry and various aviation
stakeholders, we have developed long-term National Airspace System (NAS) plans and continue
to develop and assess potential new entrants to the NAS.

FAA has reviewed the OIG draft report and has the following comments regarding our approach
to benefits. The $100 billion benefit estimate for NextGen through 2030 represented a detailed
modeling analysis conducted in 2017. The result bounds the overall timeframe for realizing the
$100 billion between 2030 and 2035 (pre-COVID-19 pandemic). The FAA estimates of
NextGen benefits are segmented into the following three parts:

      • Realized Benefits from Implemented NextGen Capabilities
      Since December 2015, FAA and industry have collaborated, through the Joint Analysis Team
      (JAT), to achieve consensus on benefits estimates for key NextGen programs1. Leveraging
      JAT methodologies, FAA has completed post-operational analyses of more than 20
      capabilities at nearly 200 sites. FAA estimates that NextGen implementations from 2010
      through 2019 have accrued $7.3 billion in benefits to industry and society as follows2:
             •   $1.3 billion in fuel savings
             •   $1.5 billion in other aircraft operating cost savings

1
  Key NextGen programs include Optimized Profile Descents (OPDs), Wake Re-categorization, Metroplex,
Established on RNP (EOR), and Tower Controller Pilot Data Link Communications (CPDLC).
2
  Parts may not sum to total due to rounding.

Appendix. Agency Comments                                                                              27
            •    $4.2 billion in passenger travel time savings3
            •    $0.4 billion in safety.

    • Future Benefits from Implemented NextGen Capabilities
    Implemented NextGen capabilities will continue to contribute to NextGen benefits in future
    years. To estimate these future benefits in 2017, we extrapolated the detailed post-
    implementation analyses using standard economic and traffic growth factors. This resulted
    in a (pre-COVID-19 pandemic) $21 billion of total benefits from implemented capabilities
    expected by 2030.

    • Future NextGen Benefits from New Sites and Capabilities
    FAA conducted its first enterprise-level cost-benefit assessment of NextGen in 2012 to
    confirm that the program provided value to taxpayers and operators and to justify the
    investment to stakeholders. The analysis, which aimed to capture all expected NextGen costs
    and benefits, was updated in 2013, 2014, 2016, and finally 2017. By analyzing all NextGen
    capabilities simultaneously, it ensured proper accounting for program synergies and
    overlaps4.

In early 2020, FAA updated the Future NextGen Benefits estimates by applying a risk factor for
slower traffic growth and implementations delays resulting in realization of an expected $100
billion in the early 2030s. FAA continues to work with industry to build consensus on data,
methodologies, and the value of NextGen improvements. The achieved benefits estimated thus
far represent a small portion of expected future benefits. As programs continue to capitalize on
the deployed NextGen infrastructure, benefits are expected to grow.

Upon review of OIG’s draft report, FAA concurs with the recommendations to improve the
delivery of NextGen and other future NAS modernization efforts, as written. For
recommendations 1 and 3, we plan to complete actions by April 30, 2021. We will implement
recommendation 2 by October 31, 2021.

We appreciate this opportunity to offer additional perspective on the OIG draft report. Please
contact H. Clayton Foushee at Clay.Foushee@faa.gov if you have any questions or require
additional information about these comments.




3
  Per Department of Transportation (DOT) guidance, FAA values passenger travel time savings when conducting
cost-benefit analyses for investment, regulatory, and grant-making purposes.
4
  The latest NextGen business case includes improvements described in The Future of the NAS and in the 2017
NAS Segment Implementation Plan. No new implementations were considered after 2026.

Appendix. Agency Comments                                                                                     28
             Our Mission
 OIG conducts audits and investigations on
behalf of the American public to improve the
performance and integrity of DOT’s programs
   to ensure a safe, efficient, and effective
       national transportation system.