oversight

Major Acquisitions: Significant Changes Underway in DOD's Earned Value Management Process

Published by the Government Accountability Office on 1997-05-05.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                   United States General Accounting Office

GAO                Report to the Subcommittee on
                   Acquisition and Technology, Committee
                   on Armed Services, U.S. Senate


May 1997
                   MAJOR
                   ACQUISITIONS
                   Significant Changes
                   Underway in DOD’s
                   Earned Value
                   Management Process




GAO/NSIAD-97-108
             United States
GAO          General Accounting Office
             Washington, D.C. 20548

             National Security and
             International Affairs Division

             B-276406

             May 5, 1997

             The Honorable Rick Santorum
             Chairman
             The Honorable Joseph I. Lieberman
             Ranking Minority Member
             Subcommittee on Acquisition and Technology
             Committee on Armed Services
             United States Senate

             Despite the regularity with which defense acquisition programs have
             experienced cost overruns and schedule delays, the Department of
             Defense (DOD) does have an extensive system intended to provide program
             managers and others with early warnings of cost and schedule problems.
             In 1967, DOD issued a set of cost/schedule control system (CS2) criteria that
             it required defense contractors to meet. However, it has become widely
             accepted by DOD and the defense industry alike that this process is in need
             of reform. This report addresses the problems facing the CS2 process, the
             progress DOD has made with reforms, and the challenges DOD faces in
             fostering and managing potentially significant changes.


             DOD’s CS2was established in 1967 as a tool to measure the value of work
Background   performed as compared to the actual costs, a concept referred to as
             earned value. Earned value goes beyond the two-dimensional approach of
             comparing budgeted costs to actuals. It attempts to compare the value of
             work accomplished during a given period with the work scheduled for that
             period. By using the value of work done as a basis for estimating the cost
             and time to complete, the earned value concept should alert program
             managers to potential problems sooner than expenditures alone can.

             To illustrate, assume a contract calls for 4 miles of railroad track to be laid
             in 4 weeks at a cost of $4 million. After 3 weeks of work, only $2 million
             has been spent. By analyzing planned versus actual expenditures, it
             appears the project is underrunning the estimated costs. However, an
             earned value analysis reveals that the project is in trouble because even
             though only $2 million has been spent, only 1 mile of track has been laid;
             thus, the contract is only 25 percent complete. Based on the value of work
             done, the project will cost $8 million ($2 million to complete each mile of
             track) and the 4 miles of track will take a total of 12 weeks (3 weeks for
             each mile of track) to complete instead of the originally estimated
             4 weeks.




             Page 1                                 GAO/NSIAD-97-108 Earned Value Management
B-276406




The communities that have a vested interest in earned value generally and
CS2 specifically are the (1) program managers, who are charged with
overall management responsibility for acquisition programs;
(2) contractors, who are responsible for successful execution of the
contract; and (3) overseers, such as acquisition executives, financial
managers, contract surveillance officials, and cost estimators who are
tasked with tracking and estimating program costs. For earned value to be
effective, it must serve the basic needs of all these users. An engineer
might consider its most useful output to be technical status information on
particular components. An accountant may view its most important
product to be the cost versus budget information it provides. A program
manager may share both views or may value the scheduling data the most.
Thus, regardless of the system or process used to record information,
earned value should provide insightful information to all three
communities.

In its 1967 financial management regulations, DOD issued 35 CS2 criteria that
were to be applied to most major weapon acquisitions (see app. I for a
listing of the criteria). The criteria are not an accounting system per se, but
rather general management or internal control guidelines to be used on
flexibly priced1 contracts. The criteria require that the contractor’s
management control system provide data that (1) relate time-phased
budgets to specific contract tasks; (2) indicate work progress; (3) properly
relate cost, schedule, and technical accomplishment; (4) are valid, timely,
and auditable; (5) supply managers with summary level information; and
(6) are derived from the same internal management control systems used
by the contractor.

Over the years, the basic criteria were supplemented by additional
guidance and procedures for contractor system reviews and reporting,
which have become “de facto” requirements. For example, the government
conducted a series of implementation and surveillance reviews of
contractors’ management control systems to ensure they complied with
the criteria. In 1972, DOD developed a Joint Implementation Guide to
standardize CS2 implementation procedures. The guide contained a
checklist of about 160 specific questions, which were referred to as
subcriteria, to be covered with contractor employees during the
implementation reviews. These requirements converge with other
requirements in determining how a contractor designs its management

1
 Flexibly priced contracts include all types of cost reimbursable contracts and those fixed-price
contracts with incentive fee arrangements. Contracts not having such arrangements are called firm
fixed price. The CS2 requirement must be imposed on flexibly priced contracts with a value of
$70 million in research, development, test, and evaluation and $300 million in procurement.



Page 2                                            GAO/NSIAD-97-108 Earned Value Management
                   B-276406




                   control system. For example, DOD independent cost estimators have
                   required a different breakout of contractor cost information than the CS2
                   did and have specified that cost data be collected in uniform or consistent
                   categories. The contractor’s system must be elaborate enough to satisfy all
                   of these data requirements, regardless of their source or purpose.


                   The core concept of the CS2 process—earned value—is recognized as a
Results in Brief   sound way to measure progress on major acquisition programs. Over the
                   years, however, the process has evolved to where the needs of some of its
                   key users are being satisfied, while others are not. Specifically, DOD
                   program managers are not satisfied with the timeliness of the CS2 reports.
                   Because the data contained in the reports are typically up to 2 months old,
                   the reports do not function as an early warning system needed by program
                   managers. Moreover, the process has not fully integrated cost, schedule,
                   and technical data as intended. The want of such information can invite
                   subjective and potentially optimistic judgments to fill the void.
                   Contractors maintain that accommodating extensive government
                   certification reviews, collecting and arraying data in prescribed categories,
                   and preparing detailed reports requires significant effort and cost to the
                   government and draws some of their engineering resources away from
                   program execution. Commercial firms that use earned value systems
                   produce reports more frequently, more quickly, and in less detail than the
                   CS2 process. Users outside the program offices—such as financial
                   managers and cost estimators—find that the data generally meets their
                   needs. These users generally place more value on consistency among cost
                   categories and less value on timeliness than program managers.

                   DOD  has acknowledged the problems with CS2 for a decade, but reforms
                   have proceeded slowly mainly because responsibility for the process has
                   resided with the oversight organizations that have been its architects. DOD
                   attempted to effect change in 1989 by transferring top-level responsibility
                   for the system from the comptroller staff to the acquisition staff. Despite
                   this transfer, little progress was made because execution of CS2 at the field
                   level remained within the comptroller community. Nonetheless, DOD has
                   embarked on several reforms that could dramatically change the CS2
                   process. Recently, DOD accepted industry’s earned value management
                   criteria as a replacement for the government’s long-standing CS2 criteria.
                   DOD has also transferred responsibility and control over the process from
                   the services to the Defense Contract Management Command (DCMC),
                   which currently provides the on-site interface between the government
                   program office and the contractor. Another reform underway involves



                   Page 3                                 GAO/NSIAD-97-108 Earned Value Management
                         B-276406




                         giving DOD program managers latitude to tailor their contract data to the
                         specific needs of their program—such as the categories and the level of
                         detail. In many cases, program managers are using direct (on-line) access
                         to obtain data from the contractors’ internal management information
                         systems.

                         These recent steps to reform the CS2 process have potent implications. For
                         example, adopting the industry criteria could result in less burdensome
                         and more useful contractor management information systems, but could
                         also lessen the government’s ability to oversee defense programs. In light
                         of both the day-to-day demands of managing the process on individual
                         contracts and implementing recent reforms, DCMC faces a significant
                         challenge as it takes over stewardship of the process. Service officials are
                         concerned about how quickly it can meet these demands, given its decline
                         in staffing over the last several years. Ultimately, DCMC will have to ensure
                         that the process meets the basic needs of all its key users—program
                         managers, contractors, and oversight personnel.


                         An earned value system faces stiff and somewhat competing demands
The CS2 System Has       from its users: (1) providing the right analyses in time for program
Served Oversight         managers to use; (2) enabling adequate oversight and analysis of multiple
Needs Better Than        programs beyond the program office level; and (3) minimizing the effort
                         required of the contractor to provide the necessary systems, data, and
Program Management       analysis. DOD policy states that earned value is an integrated program
Needs                    management tool, but because it has historically been a comptroller
                         function, CS2 has been viewed by other users as a compulsory and
                         burdensome financial reporting system. Moreover, it has not fully satisfied
                         the needs of many program managers for up-to-date and integrated
                         information on cost, schedule, and technical progress.


Program Managers:        Government program managers are responsible for managing the cost,
Timeliness of CS2 Data   schedule, and technical performance of their weapon acquisition
Limits Its Utility       programs. The CS2 process was designed to integrate these three
                         parameters into an effective early warning tool. However, program
                         managers have historically expressed concerns that CS2 data, as reported,
                         are too late and too voluminous to be useful for day-to-day program
                         management. The primary CS2 report, the cost performance report, can be
                         up to 60 days old and can contain over 100 pages of detailed information.
                         We confirmed that many program managers still see data timeliness as an
                         issue. According to some managers, using CS2 data is like “managing by



                         Page 4                                GAO/NSIAD-97-108 Earned Value Management
B-276406




looking through a rear view mirror.” DOD acknowledged some of these
issues in the 1987 preface to the Joint Implementation Guide.

We contacted managers of Acquisition Category 1D2 programs in
engineering and manufacturing development to evaluate whether the CS2
data met their needs. Their comments disclosed that while they strongly
support the concept of earned value, the timeliness of the data is still a
significant concern. Ten of the 15 program managers who responded
believed that the data were delivered too late to be an effective real-time
management tool. Twelve of these managers responded that the cost
performance reports did not contain problems they were not previously
aware of. One manager stated that because the reports are between
30 and 90 days old, he usually winds up addressing long resolved issues,
which creates unnecessary work for him. Another expressed the concern
that by the time the report reaches him, the problem is usually worse. Part
of the problem is that the CS2 system is driven by users in an oversight, not
in a program execution role, another stated.

When the CS2 reporting requirements were initially generated in 1967, the
data were as timely as commonly available technological tools would
allow. The work was broken out into short, discrete work packages and
grouped into cost accounts. Engineers in the contractor’s plant were often
made responsible for managing and reporting on these cost accounts each
month. This information was accumulated, consolidated at the
contractor’s plant, and mailed to a variety of offices within DOD. Financial
analysts within DOD then reviewed and entered this data into software
applications for further analysis. By this time, the data were at least
2 months old.

Technological advances have eliminated the need for government analysts
to manually enter the data; nevertheless, many program offices still rely on
the mail to obtain the report in hard copy or on a diskette. Program
managers are not satisfied with the timeliness of these reports. With the
advent of readily accessible real-time communication tools like personal
computers, electronic mail, and the Internet, however, program managers
are unwilling to wait 2 months for program status information. Because of
this time delay, program managers generally use other means to satisfy
their needs for information, like integrated product teams. With

2
 An acquisition category 1 program is defined as a major defense acquisition program with estimated
expenditures of over $355 million in research, development, test, and evaluation, or over $2.135 billion
in procurement (in fiscal year 1996 dollars). A category 1D program is monitored by the defense
acquisition executive, not a service executive. At the time of our review, there were 16 category 1D
programs in engineering and manufacturing development.



Page 5                                              GAO/NSIAD-97-108 Earned Value Management
B-276406




technological advances in communication, program managers can directly
access contractor data on a real-time basis. This type of access allows the
government program manager to analyze and react to the same vintage
information the contractor is using to manage.

In addition to timeliness problems, the CS2 reports emphasize the cost and,
to some extent, the schedule data without fully integrating technical
information. Assessment of technical risks is left to subjective, often
optimistic judgments of program personnel. One of the managers we
surveyed pointed out that CS2 does not show if the technical performance
is being achieved or if the work on the critical path is being done on time
or within cost. In other words, even though a discrete work segment may
be completed, CS2 data do not directly inform a manager of the success or
quality of the work. Instead, the standard cost performance report
attempts to assign a cost value to schedule data. Thus, if a particular
design task has fallen behind schedule, the standard report would assign a
cost to the delay and present that cost. While this format may be ideal for a
cost analyst, it cannot by itself facilitate the timely management of a
technical problem, nor can it highlight the potential impact on critical path
schedules.

Another limitation is that the detailed data collection categories may not
correspond to individual program structures. Although the need for overall
reporting consistency is critical to the oversight community, program
managers are willing to accept progress information presented in the same
categories and format that the contractor uses to manage the program.
Most program managers we surveyed were interested in obtaining
real-time data directly from the contractor in whatever format the
contractor used, as long as the format remained consistent and the data
could be verified. According to DOD officials, program managers have had
the flexibility to modify reporting requirements for many years, but the
needs of the oversight community for consistent reporting formats and
categories have taken precedence. Many of the program managers we
contacted are now taking advantage of this flexibility to tailor their
reporting requirements. DOD officials stated that these limitations reflect
more on how the CS2 criteria and cost estimating requirements have been
implemented than on the criteria themselves. They noted that the CS2
criteria allow such flexibility but that subsequent reporting and
certification procedures have led to rigid implementation practices.




Page 6                                GAO/NSIAD-97-108 Earned Value Management
                                        B-276406




Contractors: CS2 Validation             Although contractors generally recognize the basic CS2 criteria as sound
and Reporting Are                       management principles, they believe DOD’s implementation process
Burdensome                              contains many burdensome requirements. Independent studies have found
                                        that CS2 reporting was too detailed, repetitive, and voluminous to be used
                                        effectively as a management tool either by the government or by industry.
                                        Further, they found that the requirement may actually undermine program
                                        performance by diverting the time and attention of the company program
                                        manager.

                                        The formal reports represent only the end product of what is required in
                                        CS2.Perhaps more significant are the cost and management control
                                        systems the contractors must have in place to record the required
                                        information and the government reviews of those systems to certify their
                                        acceptability. To facilitate cost estimating, DOD guidance specifies a
                                        structure for breaking the work down on developing a major weapon into
                                        uniform categories or segments. At the most basic level, CS2 guidance
                                        requires that the work is broken down into discrete short-span work
                                        packages, which are consolidated into a cost account. A cost account is a
                                        management control point at which actual costs can be accumulated for
                                        an element of work. Table 1 illustrates an excerpt of a work breakdown
                                        structure for an ongoing aircraft program.

Table 1: Work Breakdown Structure for
an Aircraft Program
                                        Level 1: Aircraft
                                          Level 2: Air vehicle
                                               Level 3: Avionics
                                                 Level 4: Communication/navigation/identification
                                                   Level 5: Navigation software
                                                       Level 6: Individual cost accounts
                                        Source: DOD.



                                        Cost accounts on complex weapon programs can number over 1,000.
                                        While these costs have to be recorded and tracked by the contractor at
                                        some level, cost-estimating requirements in conjunction with CS2
                                        implementation practices have dictated how the accounts are defined and
                                        at what level of detail. For example, if a contractor’s internal management
                                        system differs from the cost-estimating structure, the contractor may have
                                        to track costs one way to satisfy cost-estimating requirements and another
                                        way to match how it actually manages. This requires more engineering




                                        Page 7                                         GAO/NSIAD-97-108 Earned Value Management
B-276406




effort by the contractor because it is the technical staff who typically
manage the cost accounts.

For example, in aircraft programs, DOD requires contractors to record and
report earned value information for the aircraft sections. However, they
could be managing by aircraft ribs (structural components that strengthen
larger airframe sections). To satisfy DOD’s requirements, these contractors
would have to artificially segregate costs for ribs going to the fuselage
from those going into the wings and tail sections. They would also have to
prepare lengthy CS2 variance reports, which takes them away from their
engineering duties and provides information that does not directly relate
to how the program is being managed.

The standardization that work breakdown categories provide may
facilitate government analysis and oversight above the program level, but
may not aid—and in fact could inhibit—program management. DOD policy
has given program managers the flexibility to modify the level of detail in
CS2 reports and more recently, the prescribed work breakdown categories.
Program managers have used this latitude in modifying their CS2 reporting
requirements and are tailoring their reporting requirements. However, the
contractors still track costs according to the standardized work
breakdown structure to satisfy the cost estimators’ needs for data
consistency. Recently, the Cost Analysis Improvement Group, an
organization that provides independent program cost estimates to the
Secretary of Defense, has taken steps to clarify the needs of the
cost-estimating community with respect to those of CS2. The group, which
relies on actual cost data from contractor accounting systems, has
reemphasized the need for consistent cost reporting while reducing the
burden on contractors. Accordingly, the group is in the process of
improving its reporting systems to allow contractors to report in their own
formats and appropriate level of detail, using a common data format that
in turn enables DOD to convert the data into the format needed by the
cost-estimating group.

The requirement to have management control systems that comply with
CS2 criteria is contained in the basic weapons system contract. However,
the detailed requirements for these systems have been established through
an extensive certification process. During the certification reviews, the
government used a checklist of 158 specific questions to assess
compliance at all levels of the contractor organization. The various
reviews conducted on the contractor system during the life of a contract




Page 8                                 GAO/NSIAD-97-108 Earned Value Management
                                      B-276406




                                      are summarized in table 2 below, along with estimates of the resources
                                      involved.

Table 2: Traditional CS2 Compliance
Reviews                               Review title                    Purpose of review                           Days    Staff
                                                                                                          2
                                      Implementation visit            Contractor plans to implement CS            2-3     4-5
                                      Readiness assessment            CS2 implementation progress                 5       5-15
                                      Demonstration review            Compliance of contractor management         15-20   10-25
                                                                      control systems
                                      Extended subsequent             Revisions to management control             10-15   10-25
                                      application review              system
                                      Subsequent application          New contract application of CS2             5       6-10
                                      review                          requirements
                                      Baseline review                 Proper implementation of contract           3       4-6
                                                                      baseline
                                      Source: Cost/Schedule Control System Criteria, The Management Guide to C/SCSC;
                                      Quentin W. Fleming; 1992.



                                      The time and staff required for these reviews can vary, depending upon the
                                      complexity of each contract. In addition to the government personnel
                                      required to perform these reviews, an even larger number of contractor
                                      personnel may be required to support the reviews, which adds to the cost
                                      of a contract. For example, cost account managers are interviewed during
                                      the demonstration review, using the checklist of 158 questions. Because
                                      the CS2 implementation guidance dictates that work be segregated into
                                      small, short work packages, this could entail numerous contractor staff. In
                                      addition, once a contractor’s system has been validated by the
                                      government, it is then subject to periodic surveillance reviews that are
                                      performed throughout the life of the contract. Contractors ultimately
                                      viewed the government-approved management system as one that the
                                      government had imposed upon them. They could not redesign or modify
                                      such a system without first getting government approval. If a change were
                                      approved, the government would initiate another system review to
                                      revalidate the revised system. Contractors would thus shun system
                                      improvements to avoid the additional reviews.


Financial Management: CS2             Because the financial management community has presided over the
Is Satisfactory                       evolution of CS2 for most of the past 30 years, the system has been
                                      optimized to meet its needs. The initial regulation governing CS2 policy, DOD
                                      Instruction 7000.2, was issued by the DOD Comptroller as part of the




                                      Page 9                                        GAO/NSIAD-97-108 Earned Value Management
B-276406




financial management regulations.3 It was implemented and administered
by an infrastructure of DOD financial managers. This infrastructure
permeates all levels of DOD. At the program office level, analysts review the
reports, participate in contractor system reviews, and monitor contract
progress. At the buying command and at service headquarters, financial
managers were in charge of the various contractor system reviews,
analyzing the reports, and projecting trends and estimates to complete. At
the Office of the Secretary of Defense, cost analysts and program oversight
personnel still review this data in support of major milestone decisions
and through periodic oversight reports like the Defense Acquisition
Executive Summary. At the contractor’s plant, DCMC representatives
participate in the system reviews, provide on-site surveillance and review
the reports for accuracy. Until December 1995, the Performance
Measurement Joint Executive Group, comprised of financial management
experts, was the chief decision-making body for CS2 issues.

The role played by the financial management community is somewhat
unique to the federal government. Unlike the commercial world, where the
company developing a new product funds the development with its and/or
investors’ money, weapon system development is funded by the
government—the customer. The government acts as an agent for the
public trust and therefore has a responsibility to oversee the expenditure
of those funds. The cost performance report, as one of the main reports to
assist the government in that responsibility, provides auditable data from
which an analyst can generate independent estimates at completion and
may also project trends based on contract performance. Historical cost
data from the contractor’s systems also enable estimators to develop
parametric models from which future weapon systems’ cost can be
estimated. Consistent with these needs, the financial management
community, in its stewardship over the CS2 process, has placed a premium
on reliable and consistent cost data.

The cost analyses performed on weapon acquisition programs are an
important internal control that can highlight performance problems that
program managers, as advocates, may overlook. The controversy and
subsequent cancellation of the Navy’s A-12 aircraft illustrates the value of
CS2 data being available to organizations outside the program manager’s
office. According to the 1990 administrative inquiry conducted for the
Secretary of the Navy, the cost performance data from the A-12
contractors clearly indicated significant cost and schedule problems. The

3
 In 1991, DOD Instruction 7000.2 was canceled and its requirements included in DOD Instruction
5000.2, Defense Acquisition Management Policies and Procedures. This regulation was superseded in
1996 by DOD Regulation 5000.2-R, Mandatory Procedures for Major Defense Acquisitions.



Page 10                                          GAO/NSIAD-97-108 Earned Value Management
                       B-276406




                       results of an oversight review of the cost performance reports disclosed
                       that the A-12 contract would probably exceed its ceiling by $1 billion.
                       However, neither the contractors nor the Navy program manager relied
                       upon this data; instead, they used overly optimistic recovery plans and
                       schedule assumptions. The inquiry concluded that the government and
                       contractor program managers lacked the objectivity to assess the situation
                       and they disregarded financial analysts who surfaced the problems.


                       Commercial firms are increasingly adopting the concept of earned value
Commercial Firms       for development projects but are applying it in a more streamlined manner
Use Streamlined        than DOD has historically. Company managers are getting real-time
Earned Value Process   progress information in formats consistent with how they manage. Major
                       defense contractors are also overwhelmingly emphasizing up-front
                       technical planning and scheduling as opposed to traditional cost and
                       schedule monitoring. United Defense Limited Partnership, a major defense
                       contractor, has inserted earned value concepts throughout its entire
                       management structure. A senior official from that company stated that a
                       successful program hinges on good technical planning and scheduling, and
                       for earned value to succeed, it must be useful to everyone, not just a
                       requirement imposed onto one functional group by another. It must also
                       be used to manage internally or it will not be taken seriously by those
                       operating it.

                       Motorola is also convinced that technical planning and scheduling is
                       paramount to a successful project. It is using earned value to manage its
                       multi-billion dollar satellite communication system, IRIDIUM, with
                       streamlined data accumulation, reporting, and oversight mechanisms. For
                       example, technical and schedule data are monitored at very detailed levels
                       whereas costs are not included until much higher levels of reporting are
                       reached. Similarly, Lockheed-Martin Missiles and Space has announced it
                       is adopting earned value for all its contracts, regardless of whether the
                       customer is military or commercial. Lockheed-Martin has benchmarked its
                       processes and identified the best practices in program management. It
                       estimates that by adopting these practices for each contract, it could
                       reduce its non-value added activities by almost two-thirds.

                       When a firm like Motorola makes large investments in a major
                       development like IRIDIUM, detailed information requirements are
                       essential, and in many ways, similar to DOD’s. According to the firms we
                       spoke with who are funding major product developments and are using an
                       earned value system, they produce internal status reports more frequently,



                       Page 11                              GAO/NSIAD-97-108 Earned Value Management
                                      B-276406




                                      with less detail, that are more current than what the traditional CS2 process
                                      has provided to DOD. They also place emphasis on technical and schedule
                                      planning. Moreover, the manner in which the data is organized and
                                      reported is typically aligned with the way the companies manufacture.
                                      Government program managers we contacted indicated that this is
                                      precisely what they want—real-time information presented in a manner
                                      that mirrors how the contractor manufactures.

                                      The major differences between commercial earned value practices and
                                      traditional defense practices are shown in table 3.

Table 3: Comparison of Commercial
Earned Value to Traditional Defense   Characteristic                Commercial program          Traditional DOD program
CS2 Implementation Practices          Frequency of status reports   Weekly/bi-weekly            Monthly
                                      Age of information            Real-time/weekly            Up to 60 days after
                                                                                                reporting period ends
                                      Method of data dissemination Direct access to database    Mailing of reports on paper
                                                                                                or disk
                                      Work breakdown structure      Level 3                     Level 3-7
                                      Variance analysis reporting   Critical path items         All elements
                                      Management focus              Technical and schedule      Costs

                                      As indicated, commercial firms obtain data much more frequently than
                                      DOD programs typically do. Program status information was available to
                                      commercial managers on either a real-time basis or as close as the
                                      companies’ computer capabilities would allow. Another significant
                                      difference is that variance analysis reporting is much less detailed in
                                      commercial firms. Companies set tolerance limits within appropriate
                                      manufacturing processes and report on breaches of those limits as
                                      opposed to reporting on all variances regardless of the element. If the
                                      element is not critical to meeting technical progress, schedule, or cost, it
                                      may be monitored but not necessarily reported.

                                      Management emphasis differs between the two groups, as well. We found
                                      that on commercial programs, the manufacturers tended to focus on
                                      adherence to schedule as opposed to costs. They do not forsake the other
                                      benefits of earned value but focus on technical and schedule goals before
                                      costs. Their philosophy is that focusing on key technical accomplishments
                                      per the planned schedule will cause planned costs to fall in line. For
                                      example, in the IRIDIUM program, Motorola did not add costs to the
                                      program status reporting until almost halfway up the reporting channel, a
                                      much higher level of aggregation than a typical DOD program. Company




                                      Page 12                                     GAO/NSIAD-97-108 Earned Value Management
                     B-276406




                     management reasoned that because engineers and program managers on
                     the manufacturing floors are concerned with meeting technical and
                     schedule accomplishment, recording earned value information at that level
                     is more informative in terms of labor hours than in terms of dollars. DOD
                     officials believe that the CS2 criteria and basic guidance allow defense
                     programs the flexibility to manage earned value information in a similar
                     manner, but in practice, this flexibility has been limited by over
                     implementation of the guidance.


                     Despite DOD’s acknowledgement of CS2 implementation problems, until
Some Early Reforms   recently, little progress has been made in resolving them. Fundamentally,
Have Made Limited    this is because the CS2 data was responsive to the financial management
Impact               community that managed how the CS2 process was implemented. Most of
                     the reforms that have been undertaken have made improvements, but have
                     not alleviated the more significant problems with CS2. The integrated
                     baseline review, however, may prove to be an exception because it has
                     resulted in a marked reduction in traditional CS2 oversight reviews thus far.

                     The 1987 Joint Implementation Guide clarified the objectives of CS2 as “For
                     contractors to use effective internal cost and schedule management
                     control systems, and for the government to be able to rely on timely and
                     auditable data produced by those systems for determining
                     product-oriented contract status.” It went on to stress that improved
                     communication between the government and contractors could reduce
                     improper implementation of the CS2 criteria. To stimulate reform, DOD
                     transferred the responsibility for CS2 policy from the DOD Comptroller to
                     the Under Secretary of Defense for Acquisition in August 1989. This
                     top-level organizational change was not mirrored in the three services,
                     however. As a result, CS2 implementation was fragmented, with top-level
                     policy being managed by the Office of the Secretary of Defense acquisition
                     community and day-to-day implementation being managed by the services’
                     financial management community. The Performance Measurement Joint
                     Executive Group, comprised of financial management personnel,
                     maintained central oversight of the CS2 process.

                     In October 1993, the Under Secretary of Defense for Acquisition &
                     Technology undertook an initiative to return earned value to its original
                     purpose: a tool to integrate cost, schedule, and technical performance
                     management. This initiative attempted to reduce the review burden and
                     limit reporting requirements. In addition, to change the emphasis from
                     government oversight to contractor responsibility, it encouraged industry



                     Page 13                               GAO/NSIAD-97-108 Earned Value Management
B-276406




to develop its own management standards. About a year later, after limited
progress, the Office of the Secretary of Defense requested that the Service
Acquisition Executives personally revitalize CS2 reform under the
Integrated Program Management Initiative. A top-level Executive Steering
Group was formed to reengineer earned value implementation among the
services. This group noted that while there were no exceptions to the CS2
requirement, there was broad latitude to change implementation practices.
DOD and recent service policy memoranda have consistently stressed the
need to streamline traditional CS2 reporting requirements by reducing the
detailed information required, such as minute categories for reporting and
tailoring other report formats. In December 1993, DOD initiated a major
effort to revise the Joint Implementation Guide. After multiple iterations
and coordination difficulties, it was finally reissued in December 1996.

To provide more timely information to program managers, DOD is requiring
all new contracts to use electronic data interchange (EDI) as a data transfer
method. Under the EDI concept, the contractors will make the cost
performance reports, along with other reports, electronically accessible to
program managers, financial analysts, and other users like DCMC staff. To
ensure that the cost performance report data can be transmitted and
received in common data formats, DOD has developed a standardized data
set for contractors to use when transmitting their reports.

One of the benefits of using EDI for cost performance reporting is the time
saved by electronically transmitting data to customers instead of
physically mailing written reports. The time savings has been estimated at
about 1 to 2 weeks, which includes time associated with re-keying
information into the customers databases so they can perform their
analyses. Notwithstanding some of the problems that are being
encountered with electronic transfers of data, we believe the time savings
associated with this method of communication is not likely to make the
reports more useful. At best, the reports will still be received by the
program offices and other interested users about 30 to 45 days after the
end of a reporting period.

One of the more successful initiatives to date is the integrated baseline
review (IBR), which DOD implemented in 1994. The IBR focuses on the
development of a detailed and achievable technical, cost, and schedule
baseline for a program. The objectives of the IBR are twofold: to improve
the use of cost performance data by contractor and government program
managers and to reduce the number of CS2 reviews. As part of the overall
risk assessment process, the IBR is intended to integrate the technical



Page 14                               GAO/NSIAD-97-108 Earned Value Management
                           B-276406




                           content of the work with cost and schedule parameters. It is planned and
                           executed by a multi-functional government and contractor team composed
                           of engineering, logistics, manufacturing, contracting, and financial
                           personnel. Unlike traditional CS2 baseline reviews, the IBR is led by the
                           program manager, not a financial manager. By involving the program
                           manager directly in this review, the process highlights the merits of using
                           earned value to track progress. It has also reduced the number of CS2
                           compliance reviews. For example, CS2 reviews have decreased from 56 in
                           1993 to 5 in 1995 while the number of IBRs have increased from 3 in 1993 to
                           29 in 1995. The program managers that we surveyed strongly supported
                           the IBR as a valuable program management tool.


                           DOD  has recently taken a three-pronged approach to reform the CS2
Recent Reforms Have        process. This approach includes an internal organizational realignment to
Major Implications for     override cultural resistance to change, the recognition of commercial
CS2 Process                industry criteria to return the management of the system to the contractor,
                           and direct electronic access by program managers to contractor
                           information to improve the timeliness of CS2 data. These changes may have
                           significant impact on the extent, type, and number of CS2 system reviews
                           that occur; the interpretation of how the criteria/guidelines will be
                           implemented; and the extent and timeliness of data received by
                           government managers.


Organizational             In December 1995, DOD disbanded the Performance Measurement Joint
Restructuring Shifts CS2   Executive Group and made DCMC the executive agent for CS2 issues. Policy
Stewardship Away From      matters are now handled by the Executive Steering Group, which includes
                           representatives from the Service Acquisition Executives and the DCMC
Financial Management       Commander, to provide a program management orientation. In
Community                  October 1996, the Under Secretary of Defense for Acquisition and
                           Technology formally transferred CS2 compliance responsibility from the
                           services to DCMC. This latest decision completes the transfer of virtually all
                           CS2 policy, reform, and compliance decisions away from the financial
                           management community. Nonetheless, members of the various
                           communities will continue to be involved in the day-to-day operations of
                           CS2. How their roles, practices, and interaction with one another will
                           change under the aegis of DCMC remains to be seen.

                           This transfer has significant repercussions for DCMC. Service
                           representatives have expressed concern that because of its recent
                           downsizing, DCMC may not have the resources to adequately perform its



                           Page 15                                GAO/NSIAD-97-108 Earned Value Management
                           B-276406




                           increased responsibilities. In addition, about 85 percent of the CS2 field
                           staff dedicate only part of their time to cost performance monitoring. DCMC
                           officials are aware of these concerns and are working on a strategy to
                           accomplish its newly expanded mission. Since it is likely that DCMC will not
                           receive more than 6 additional billets to supplement its current level of
                           approximately 150 cost performance monitors, DCMC is reengineering its
                           approach to CS2 implementation along the lines of statistical process
                           control measurement techniques. This would represent a
                           management-by-exception approach to CS2 reviews. This may be a less
                           costly and burdensome approach that could have potential for reducing
                           the resources and infrastructure associated with traditional CS2
                           implementation practices, especially when coupled with the streamlined
                           industry earned value criteria discussed below. How well such an
                           approach meets the basic needs of all the communities is yet unknown.

                           DCMC has begun to work with contractors to change the surveillance focus
                           from the traditional oversight role to one based on what DOD refers to as
                           insight—the ability to identify problems through process-based indicators
                           such as the number of retroactive changes to cost accounts or the
                           frequency of replanning actions. Each major defense contractor will work
                           with DCMC to identify appropriate process and control objectives.
                           Ultimately, once each contractor designates its unique control limits, DCMC
                           staff would only be required to review those processes that are out of
                           tolerance. In addition, DCMC plans to work with industry to encourage
                           firms to assume responsibility for their management systems and
                           processes.


Industry Standard          In August 1996, five industry associations published an industry standard,
Developed by Contractors   Industry Standard Guidelines for Earned Value Management Systems
                           (EVMS), as a replacement to the current government CS2 criteria. DOD
                           formally recognized the guidelines in the industry standard in
                           December 1996. The Director, Defense Procurement, has issued an interim
                           rule to enable contractors to begin using the EVMS without having to wait
                           for the formal regulatory change process. The long-range plan is for the
                           standard to be approved by the American National Standards Institute4




                           4
                            The American National Standards Institute serves as the administrator and coordinator of the U.S.
                           private sector voluntary standardization system. It promotes and facilitates voluntary consensus
                           standards.



                           Page 16                                            GAO/NSIAD-97-108 Earned Value Management
B-276406




and the International Standards Organization5. This would formally move
earned value into common usage worldwide. In addition, Australia,
Canada, and the United States have signed a memorandum of
understanding concerning common cost and schedule management for
their acquisitions.

The EVMS contains 32 criteria, which are similar to the DOD criteria in
principle. For example, both sets of criteria are divided into five broad
categories: organization; planning, scheduling, and budgeting; accounting;
analysis and management reports; and revisions and data maintenance.
Each category describes the internal controls that should exist to facilitate
proper program management. They also contain provisions for breaking
out work into discrete work packages, documenting changes to the
performance baseline, and measuring cost, schedule, and technical
accomplishment. The industry standard gives the contractor the flexibility
to revise the system as needed to reflect work consistent with internal
management structures and to track costs at a higher organizational level
than the level typically tracked in DOD programs. (See app. I for a more
detailed comparison.)

Despite their general similarities, the two criteria have significant
differences that could affect government oversight. One such difference is
that the government may no longer have the same review and approval
authority over contractor management systems that it had in the past. DOD
does not accept the self-certification provisions of the EVMS standard.
Instead, it would like to find some middle ground between
self-certification and traditional government certification. DOD’s goal is to
encourage contractors to conduct self-evaluations with the government
acting as an observer. However, DOD’s recently revised implementation
guidance does not preclude a government review, when warranted, or a
third-party certification arrangement. The third-party certification concept
represents the standard industry practice for quality assurance systems.
Although not required by the International Standards Organization—the
industry organization responsible for the quality assurance standards—it is
a generally accepted practice to get a third-party certification in order to
meet its quality standards. This approach may mitigate some potential
risks of the self-evaluation process and prove to be less burdensome than
DOD’s traditional review and certification process.




5
 The International Standards Organization is a nongovernmental entity whose mission is to promote
the development of standardization and related activities in the world. Its focus is to facilitate the
international exchange of goods and services and to develop intellectual, scientific, technological, and
economic cooperation.



Page 17                                             GAO/NSIAD-97-108 Earned Value Management
                          B-276406




                          Other differences associated with the EVMS standard include the absence of
                          specific requirements for access to data by the customer (in this case, DOD)
                          for oversight purposes and the fact that program baseline and contractor
                          management systems may be changed as needed, with notification of, but
                          not approval by, the customer. Use of this standard on future
                          procurements should give the contractor greater ability to manage and
                          improve its processes. However, it could impede government oversight
                          unless DOD is able to obtain accurate and timely information on contract
                          performance. Therefore, acquisition program managers will have to ensure
                          that access to data and reporting provisions are included on individual
                          contracts. Likewise, the increased flexibility to revise the program
                          baseline may make it more difficult to track divergences from original
                          program goals.


Technology Advances Can   Current technology offers improvements that could enhance the timeliness
Improve Data Timeliness   of the CS2 data as well as integrate cost and schedule performance with
and Quality               technical performance. For several years, DOD has recognized the need to
                          improve data timeliness and has focused on developing EDI capability. We
                          found that several program managers have experimented with getting
                          direct access via personal computers to contractor databases. This permits
                          them to see at the same time the same data the contractor uses to manage
                          the program. The government program managers who have used these
                          systems are very pleased with the quality and timeliness of the
                          information. We spoke to managers in several program offices who believe
                          that having direct access to the same data the contractor uses to manage is
                          critical.

                          For example, the Joint Stand-Off Weapon system program has a direct
                          access system that was custom built for the program. It contains cost,
                          schedule, manufacturing, test, and engineering/technical information using
                          a variety of displays. The system uses commercially available software and
                          the data is updated twice a month. Cost data is available only to the
                          program manager and two others to protect the contractor’s proprietary
                          rate information. Similarly, the Sea-Launched Attack Missile Extended
                          Range program is using a direct access system that enables its technical
                          staff to pull up weekly status data by component or subsystem. Each
                          integrated product team also reports progress each week. Information is
                          displayed in terms of labor hours, not cost. Cost data is consolidated and
                          presented at a high level. Improvements in the direct-access processes are
                          being made to allow subcontractor data to be included and also to include
                          such items as indirect charges to the contract and overhead.



                          Page 18                               GAO/NSIAD-97-108 Earned Value Management
B-276406




These practices represent a departure from the traditional implementation
of CS2, but technically have always been permissible under the basic 35
criteria. Nonetheless, using the streamlined direct access process has not
been universally accepted. Some DOD officials have expressed concerns
about the potential for inaccurate data in these systems that a monthly
cost performance report review and audit would pick up. In addition, the
direct access may only provide detailed data on the prime contractor
status, and summary level information on subcontractor status. Another
concern is that the cost information may not include full overhead
charges. To implement this approach, the government may need to install
special high-speed transmission or “trunk lines” that could be expensive
and prohibit small programs from using it. Because of these concerns, DOD
has not endorsed this approach, but is focusing on EDI implementation.

Managers of these two programs believe that direct access systems are
very beneficial and have found work-arounds for some of these concerns.
According to the author of the EDI cost performance report data set, the
ultimate goal is to standardize the data format in reports. The author
believes that the optimal solution would be to standardize to a common
data format so that DOD could reach into a contractor’s database to access,
download, and analyze information. This would allow a program office to
monitor the progress of a contractor’s performance, using the currently
accepted reporting formats, on a near real-time basis. This standardization
would also allow for the almost instantaneous analysis by the users of this
information no matter what software or hardware the contractor or the
government use. This capability would eliminate some of the concerns to
unique direct access systems while still giving the program manager the
flexibility to design a system that meets the needs of his particular
program.

Another promising effort is a technical risk assessment process being
developed and tested by the Navy’s Program Executive Office for Air
Anti-Submarine Warfare, Assault, and Special Mission Programs. This
effort is intended to work in tandem with the CS2 information to enable the
manager to integrate technical progress with cost and schedule data. In an
early test, it was able to highlight and quantify technical risks on one
program much earlier than the CS2 process did. The process requires that
the critical technical performance and schedule drivers be identified at the
start of the program. By developing specific risk curves and progress plans
for each parameter, the program manager can obtain insights not just into
work progress, but also technical success. This provides an early warning
of technical problems, permitting the manager to mitigate cost and



Page 19                               GAO/NSIAD-97-108 Earned Value Management
                  B-276406




                  schedule impacts. For example, in a retrospective analysis of an aircraft
                  cockpit program, the software predicted problems more than a year before
                  the CS2 process did because the CS2 process, as applied, emphasized cost
                  and schedule integration without adequate regard for technical
                  performance. This software is currently being tested by the Federal
                  Aviation Administration and is scheduled to be used on the H-1 Helicopter
                  Upgrade program.


                  DOD has taken major steps to create an organizational environment that
Conclusions and   can facilitate making needed changes to the CS2 process. DCMC faces a
Recommendations   number of challenges as it begins its stewardship of the process. These
                  include implementing initiatives to improve its utility to program managers
                  and to streamline certification and reporting requirements, determining
                  how best to manage the certification review process, and deciding on a
                  number of proposed improvements. To be successful, DCMC will need to
                  understand the resource implications of these challenges.

                  Another challenge DCMC faces, perhaps more subtle than the above
                  reforms, is to better balance the needs of the different communities that
                  depend on earned value information. A common understanding of these
                  basic needs will be instrumental to DCMC’s ability to protect each user’s
                  basic needs from others’ secondary desires. Specifically, as DCMC
                  endeavors to make CS2 more responsive to program managers, it will have
                  to guard against basic oversight needs going unmet. We recommend that
                  the Secretary of Defense (1) promulgate the basic needs of the
                  organizations that depend on earned value information in some manner,
                  such as in the implementing guidance for EVMS and (2) take steps to ensure
                  that the “wants” of one organization do not encroach upon the basic needs
                  of other organizations that depend on earned value information as the
                  management of the CS2 process transitions to DCMC and as DCMC makes
                  decisions on reforms in the future.

                  In addition, several initiatives relevant to the CS2 process are ongoing,
                  including EDI, IBRs, direct access, technical risk assessment,
                  self-certification, and the application of statistical process control
                  techniques to the surveillance process. Some of these are being
                  demonstrated in varying degrees on different programs, while others are
                  still in the policy stage. It is important that DOD ensures that as these
                  initiatives are tested on individual programs, data is captured in a
                  disciplined enough manner to support decisions on what to implement and
                  how. Such data will not only help get the most out of each initiative, but



                  Page 20                              GAO/NSIAD-97-108 Earned Value Management
                  B-276406




                  will also make it easier to recognize possible interactions among
                  initiatives.


                  DOD generally concurred with our conclusions and recommendations. It
Agency Comments   agreed with the need to make sure that the “wants” of one organization do
                  not outweigh the needs of another. It cited the establishment of two
                  groups that will help ensure that the needs of all organizations that use
                  earned value information are met. These are (1) the Integrated Program
                  Management Initiative Executive Steering Group, which includes
                  representatives from the Office of the Secretary of Defense, the services,
                  DCMC, and other organizations; and (2) the Performance Management
                  Advisory Council under DCMC, which will have service representatives
                  from the earned value community and from the project management or
                  acquisition communities. According to DOD, the Council will also
                  participate in a contractor cost data reporting initiative, along with the
                  cost-estimating community. DOD agreed that the relationships among the
                  organizations as well as their information needs should be made clear in
                  its EVMS guidance.

                  In the draft of this report, we recommended that DOD rebaseline the basic
                  needs of the organizations that depend on earned value information. DOD
                  believed that it did not need to formally rebaseline these needs because it
                  had developed a good understanding of them in arriving at the significant
                  reforms it is currently making to CS2 and in identifying the earned value
                  skills needed by business, cost-estimating, and financial management
                  specialists. We agreed to delete the recommendation from the final report
                  with the proviso that DOD would reassess its progress on earned value
                  reform and on meeting the needs of the key organizations. Accordingly,
                  DOD stated that the Integrated Program Management Initiative Executive
                  Steering Group would assess such progress. DOD also stated that the group
                  will coordinate all earned value management improvement efforts with
                  other initiatives, such as electronic access and risk management.


                  To develop information for this report, we contacted, interviewed, and
Scope and         obtained documents from officials of the Office of the Secretary of
Methodology       Defense and DCMC because of the policy-making responsibilities and
                  reform initiatives occurring at these levels. In addition, we obtained
                  information from officials at the Office of Management and Budget; the
                  Federal Aviation Administration; the Office of the Secretary of Defense
                  Cost Analysis Improvement Group; service headquarters; Army Materiel



                  Page 21                              GAO/NSIAD-97-108 Earned Value Management
B-276406




Command; Navy Air Systems Command; Program Executive Office for Air
Anti-Submarine Warfare, Assault, and Special Mission Programs; Air Force
Materiel Command; Aeronautical Systems Center; and Defense Plant
Representative Office personnel from General Electric, Lynn, Mass;
Boeing, Seattle, Wash; Lockheed Martin, Sunnyvale, Ca; and McDonnell
Douglas, St. Louis, Mo. We contacted all program managers responsible
for Acquisition Category 1D programs in engineering and manufacturing
development. We received responses from all but 1 of these 16 programs.
We also contacted officials from the governments of Australia and Sweden
to obtain information on their requirements for cost schedule control
systems.

To obtain industry’s perspective on policy issues and implementation
practices, we contacted representatives from the following industry or
professional associations: National Security Industrial Association,
Aerospace Industries Association, Performance Management Association,
and Project Management Institute. We also discussed these issues with
representatives from the following commercial and military contractors:
Boeing, General Electric, Lockheed Martin, Loral, Magnavox, McDonnell
Douglas, Motorola, Northrop/Grumman, Raytheon, Supply Tech, Inc.,
Texas Instruments, Textron, and United Defense Limited Partnership.

To address historical information and obtain independent views on how
current policy initiatives could affect future implementation practices, we
contacted representatives from the Massachusetts Institute of Technology
and Wright State University and the following consulting groups: Coopers
& Lybrand LLP, KPMG Peat Marwick LLP, Humphreys & Associates,
Fleming Management Consultancy, and Write Concepts. We also obtained
historical information from the Management Systems Deputy, Air Force
Assistant Secretary for Financial Management and Comptroller.

We performed our review from June 1996 to March 1997 in accordance
with generally accepted government auditing standards.




Page 22                              GAO/NSIAD-97-108 Earned Value Management
B-276406




We are sending copies of this report to other interested congressional
committees; the Secretary of Defense; the Commander, DCMC; and the
Director of the Office of Management and Budget. We will also make
copies available to others upon request.

Please contact me at (202) 512-4383 if you or your staff have any questions
concerning this report. The major contributors to this report were Paul
Francis, Rae Ann Sapp, and Jeff Hunter.




Katherine V. Schinasi
Associate Director
Defense Acquisitions Issues




Page 23                              GAO/NSIAD-97-108 Earned Value Management
Appendix I

Comparison of DOD and Industry Criteria



              DOD’s Criteria                                 Industry’s Criteria
              Organization                                   Organization
              1. Define all authorized work and related      Define the authorized work elements for
              resources to meet the requirements of the      the program. A work breakdown structure,
              contract, using the framework of the           tailored for effective internal management
              contract work breakdown structure.             control, is commonly used in this process.
              2. Identify the internal organizational        Identify the program organizational
              elements and the major subcontractors          structure, including the major
              responsible for accomplishing the              subcontractors responsible for
              authorized work.                               accomplishing the authorized work, and
                                                             define the organizational elements in which
                                                             work will be planned and controlled.
              3. Provide for the integration of the          Provide for the integration of the
              contractor’s planning, scheduling,             company’s planning, scheduling,
              budgeting, work authorization, and cost        budgeting, work authorization, and cost
              accumulation systems with each other, the      accumulation processes with each other,
              contract work breakdown structure, and the     and as appropriate, the program work
              organizational structure.                      breakdown structure and the program
                                                             organizational structure.
              4. Identify the managerial positions           Identify the company organization or
              responsible for controlling overhead           function responsible for controlling
              (indirect costs).                              overhead (indirect costs).
              5. Provide for integration of the contract     Provide for integration of the program work
              work breakdown structure with the              breakdown structure and the program
              contractor’s functional organizational         organizational structure in a manner that
              structure in a manner that permits cost and    permits cost and schedule performance
              schedule performance measurement for           measurement by elements of either or both
              contract work breakdown structure and          structures as needed.
              organizational elements.
              Planning & budgeting                           Planning, scheduling, & budgeting
              6. Schedule the authorized work in a manner . . . . interdependencies required to meet
              that describes the sequence of work and     the requirements of the program.
              identifies the significant task
              interdependencies required to meet the
              development, production, and delivery
              requirements of the contract.
              7. Identify physical products, milestones,     Identify physical products, milestones,
              technical performance goals, or other          technical performance goals, or other
              indicators that will be used to measure        indicators that will be used to measure
              output.                                        progress.
                                                                                             (continued)




              Page 24                                       GAO/NSIAD-97-108 Earned Value Management
Appendix I
Comparison of DOD and Industry Criteria




DOD’s Criteria                                       Industry’s Criteria
8. Establish and maintain a time-phased              Establish and maintain a time-phased
budget baseline at the cost account level            budget baseline, at the control account
against which contract performance can be            level, against which program performance
measured. Initial budgets established for            can be measured. Budget for far-term
this purpose will be based on the negotiated         efforts may be held in higher level
target cost. Any other amount used for               accounts until an appropriate time for
performance measurement purposes must                allocation at the control account level.
be formally recognized by both the                   Initial budgets established for performance
contractor and the government.                       measurement will be based on either
                                                     internal management goals or the external
                                                     customer negotiated target cost including
                                                     estimates for authorized but undefinitized
                                                     work. On government contracts, if an over
                                                     target baseline is used for performance
                                                     measurement reporting purposes, prior
                                                     notification must be provided to the
                                                     customer.
9. Establish budgets for all authorized work         Establish budgets for authorized work, with
with separate identification of cost elements        identification of significant cost elements
(labor, material, etc.).                             (labor, material, etc.) as needed for
                                                     internal management and for control of
                                                     subcontractors.
10. To the extent the authorized work can be         To the extent it is practical to identify the
identified in discrete, short-span work              authorized work in discrete work
packages, establish budgets for this work in         packages, establish budgets for this work
terms of dollars, hours, or other measurable         in terms of dollars, hours, or other
units. Where the entire cost account cannot          measurable units. Where the entire control
be subdivided into detailed work packages,           account is not subdivided into work
identify the far-term effort in larger planning      packages, identify the far-term effort in
packages for budget and scheduling                   larger planning packages for budget and
purposes.                                            scheduling purposes.
11. Provide that the sum of all work package         Provide that the sum of all work package
budgets plus planning packages within a              budgets plus planning package budgets
cost account equals the cost account                 within a control account equals the control
budget.                                              account budget.
12. Identify relationships of budgets or             No comparable provision.
standards in underlying work authorization
systems to budgets for work packages.

13. Identify and control level of effort activity    . . . . Only that effort which is
by time-phased budgets established for this          unmeasurable or for which measurement is
purpose. Only that effort which cannot be            impractical may be classified as level of
identified as discrete or as apportioned             effort.
effort will be classed as level of effort.
                                                                                      (continued)




Page 25                                             GAO/NSIAD-97-108 Earned Value Management
Appendix I
Comparison of DOD and Industry Criteria




DOD’s Criteria                                     Industry’s Criteria
14. Establish overhead budgets for the total       Establish overhead budgets for each
costs of each significant organizational           significant organizational component of the
component whose expenses will become               company for expenses that will become
indirect costs. Reflect in the contract            indirect costs. Reflect in the program
budgets, at the appropriate level, the             budgets, at the appropriate level, the
amounts in overhead pools that will be             amounts in overhead pools that are
allocated to the contract as indirect costs.       planned to be allocated to the program as
                                                   indirect costs.
15. Identify management reserves and               Same.
undistributed budget.
16. Provide that the contract target cost plus     Provide that the program target cost goal
the estimated cost of authorized but               is reconciled with the sum of all internal
unpriced work is reconciled with the sum of        program budgets and management
all internal contract budgets and                  reserves.
management reserves.
Accounting                                         Accounting considerations
17. Record direct costs on an applied or           Record direct costs in a manner consistent
other acceptable basis in a formal system          with the budgets in a formal system
that is controlled by the general books of         controlled by the general books of account.
account.
18. Summarize direct costs from cost               When a work breakdown structure is used,
accounts into the work breakdown structure         summarize direct costs from control
without allocation of a single cost account to     accounts into the work breakdown
two or more work breakdown structure               structure without allocation of a single
elements.                                          control account to two or more work
                                                   breakdown structure elements.
19. Summarize direct costs from the cost           Summarize direct costs from the control
accounts into the contractor’s functional          accounts into the contractor’s
organizational elements without allocation of      organizational elements without allocation
a single cost account to two or more               of a single control account to two or more
organizational elements.                           organizational elements.
20. Record all indirect costs that will be         Same.
allocated to the contract.
21. Identify the basis for allocating the cost     No comparable provision.
of apportioned effort.
22. Identify unit costs, equivalent unit costs,    . . . or lot costs when needed.
or lot costs, as applicable.
23. The contractor’s material accounting           For earned value management systems,
system will provide for:                           the material accounting system will provide
  a. Accurate cost accumulation and                for:
assignment of costs to cost accounts in a          Accurate cost accumulation and
manner consistent with the budgets using           assignment of costs to control accounts in
recognized, acceptable costing techniques.         a manner consistent with the budgets
                                                   using recognized, acceptable, costing
                                                   techniques.
  b. Determination of price variances by           No comparable provision.
comparing planned versus actual
commitments.
                                                                                     (continued)




Page 26                                           GAO/NSIAD-97-108 Earned Value Management
Appendix I
Comparison of DOD and Industry Criteria




DOD’s Criteria                                   Industry’s Criteria
   c. Cost performance measurement at the        . . . but no earlier than the time of progress
point in time most suitable for the category     payments or actual receipt of material.
of material involved, but no earlier than the
time of actual receipt of material.
   d. Determination of cost variances            No comparable provision.
attributable to the excess usage of material.
  e. Determination of unit or lot costs when     No comparable provision.
applicable.
  f. Full accountability for all material        Full accountability for all material
purchased for the contract, including the        purchased for the program, including the
residual inventory.                              residual inventory.
Analysis                                         Analysis and management reports
24. Identify at the cost account level on a      At least on a monthly basis, generate the
monthly basis using data from, or                following information at the control account
reconcilable with, the accounting system:        and other levels as necessary for
  a. Budgeted cost of work scheduled and         management control using actual cost
budgeted cost of work performed.                 data from, or reconcilable with, the
  b. Budgeted cost of work performed and         accounting system:
applied (actual where appropriate) direct        1. Comparison of the amount of planned
costs for the same work.                         budget and the amount of budget earned
  c. Variances resulting from the above          for work accomplished. This comparison
comparisons classified in terms of labor,        provides the schedule variance.
material, or other appropriate elements          2. Comparison of the amount of the budget
together with the reasons for significant        earned with the actual (applied where
variances.                                       appropriate) direct costs for the same
                                                 work. This comparison provides the cost
                                                 variance.
25. Identify on a monthly basis, in detail       Identify budgeted and applied (or actual)
needed by management for effective               indirect costs at the level and frequency
control, budgeted indirect costs, actual         needed by management for effective
indirect costs, and variances along with the     control, along with the reasons for any
reasons.                                         significant variances.
26. Summarize the data elements and              Summarize the data elements and
associated variances listed in [24] and [25]     associated variances through the program
above, through the contractor organization       organization and/or work breakdown
and work breakdown structure to the              structure to support management needs
reporting level specified in the contract.       and any customer reporting specified in
                                                 the contract.
27. Identify significant differences on a        Identify, at least monthly, the significant
monthly basis between planned and actual         differences between both planned and
schedule accomplishment and the reasons.         actual schedule performance and planned
                                                 and actual cost performance, and provide
                                                 the reasons for the variances in the detail
                                                 needed by program management.
28. Identify managerial actions taken as a       Implement managerial actions taken as the
result of criteria items [24] through [27]       result of earned value information.
above.
                                                                                   (continued)




Page 27                                         GAO/NSIAD-97-108 Earned Value Management
Appendix I
Comparison of DOD and Industry Criteria




DOD’s Criteria                                  Industry’s Criteria
29. Based on performance to date, on            Develop revised estimates of cost at
commitment values for material, and on          completion based on performance to date,
estimates of future conditions, develop         commitment values for material, and
revised estimates of cost at completion for     estimates of future conditions. Compare
work breakdown structure elements               this information with the performance
identified in the contract and compare these    measurement baseline to identify
with the contract budget base and the latest    variances at completion important to
statement of funds requirements reported to     company management and any applicable
the government.                                 customer reporting requirements,
                                                including statements of funding
                                                requirements.
Revisions and access to data                    Revisions and data maintenance
30. Incorporate contractual changes in a        Incorporate authorized changes in a timely
timely manner, recording the effects of such    manner, recording the effects of such
changes in budgets and schedules. In the        changes in budgets and schedules. In the
directed effort before negotiation of a         directed effort prior to negotiation of a
change, base such revisions on the amount       change, base such revisions on the
estimated and budgeted to the functional        amount estimated and budgeted to the
organization.                                   program organizations.
31. Reconcile original budgets for those        Reconcile current budgets to prior
elements of the work breakdown structure        budgets in terms of changes to the
identified as priced line items in the          authorized work and internal replanning in
contract, and for those elements at the         the detail needed by management for
lowest level of the DOD program work            effective control.
breakdown structure, with current
performance measurement budgets in terms
of (a) changes to the authorized work and
(b) internal replanning in the detail needed
by management for effective control.
32. Prohibit retroactive changes to records     Control retroactive changes to records
pertaining to work performed that will          pertaining to work performed that would
change previously reported amounts for          change previously reported amounts for
direct costs, indirect costs, or budgets,       actual costs, earned value, or budgets.
except for correction of errors and routine     Adjustments should be made only for
accounting adjustments.                         correction of errors, routine accounting
                                                adjustments, effects of customer or
                                                management directed changes, or to
                                                improve the baseline integrity and
                                                accuracy of performance measurement
                                                data.
33. Prevent revisions to the contract budget    Prevent revisions to the program budget
base except for government-directed             except for authorized changes.
changes to contractual effort.
34. Document internally, changes to the         Document changes to the performance
performance measurement baseline and            measurement baseline.
notify the procuring activity expeditiously
through prescribed procedures.
35. Provide the contracting officer and the     No comparable provision.
contracting officer’s authorized
representatives with access to the
information and supporting documents
necessary to demonstrate compliance with
the cost/schedule control system criteria.

Page 28                                        GAO/NSIAD-97-108 Earned Value Management
Appendix II

Comments From the Department of Defense




              Page 29      GAO/NSIAD-97-108 Earned Value Management
             Appendix II
             Comments From the Department of Defense




See p. 21.




             Page 30                                   GAO/NSIAD-97-108 Earned Value Management
             Appendix II
             Comments From the Department of Defense




See p. 21.




See p. 21.




(707153)     Page 31                                   GAO/NSIAD-97-108 Earned Value Management
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order
made out to the Superintendent of Documents, when
necessary. VISA and MasterCard credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address
are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office
P.O. Box 6015
Gaithersburg, MD 20884-6015

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (301) 258-4066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any
list from the past 30 days, please call (202) 512-6000 using a
touchtone phone. A recorded menu will provide information on
how to obtain these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov




PRINTED ON    RECYCLED PAPER
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. G100
Official Business
Penalty for Private Use $300

Address Correction Requested