United States General Accounting Office GAO Report to the Chairman, Subcommittee on Technology and Procurement Policy, Committee on Government Reform, House of Representatives January 2003 ACQUISITION MANAGEMENT Agencies Can Improve Training on New Initiatives GAO-03-281 January 2003 ACQUISITION MANAGEMENT Agencies Can Improve Training on New Highlights of GAO-03-281, a report to the Chairman, Subcommittee on Technology Initiatives and Procurement Policy, House Committee on Government Reform The federal government is Industry and government experts alike recognize that training is a critical dramatically changing the way it tool in successfully implementing change. To deliver training effectively, purchases goods and services by leading organizations typically prioritize initiatives that are most important relying more on judgment and to them, identify those needing training and set requirements, and ensure initiative versus rigid rules for that their training reaches the right people. Top leadership supports these making purchasing decisions. Congress has enacted a series of efforts by working to overcome resistance, marshalling resources, and reforms to help the government building commitment to new ways of doing business. This approach, which adapt to this environment. incorporates the six elements that have been identified as key to training success, helps to ensure that training is well planned rather than left to GAO was asked to assess strategies chance. agencies are using to ensure that their acquisition workforces are This approach was not consistently evident at the agencies GAO reviewed. receiving the training needed to While agencies had efforts underway to make training available and raise operate in a changing business awareness of major acquisition initiatives, they often did not have an environment. In doing so, GAO identifiable process for assuring that training reached all those who played a looked at the General Services role in successful implementation. For example, DOD and the Army Administration (GSA), the National Aeronautics and Space employed most elements of the approach in implementing training on one Administration (NASA), and the acquisition initiative—performance-based contracting. In particular, they set Department of Defense (DOD). training as a high priority and defined who would be targeted for training. GAO also looked at the Federal But their use of the elements was not evident on another initiative GAO Aviation Administration (FAA) examined. Similarly, the approach taken by GSA and NASA did not fully because it is exempt from federal incorporate the key elements GAO identified. acquisition laws, giving it greater flexibility and discretion. The approach taken by FAA was somewhat different. It created an organizational focal point to define a process and facilitate the management of its acquisition workforce, and it employed many of the key elements in its approach. For example, it created a special council of agency executives to GAO is recommending that the establish priorities. Office of Federal Procurement Policy develop a policy that calls on agencies to adopt the elements Overall, GAO found that certain conditions tended to facilitate or hinder of the approach used by leading implementation of the key elements. For example, having a focal point that organizations; establish a focal could reach beyond the contracting community helped to ensure that point that can reach beyond the training was delivered to the right staff. Civilian agencies have not been contracting community to set supported by an agency that coordinates training on governmentwide training requirements; and initiatives. Procurement executives expressed the view that the Federal integrate training into planning for Acquisition Institute should fulfill this role for civilian agencies. policy implementation. Key Elements for Acquisition Initiative Training In response to a draft of this report, Prioritize initiatives so that the agency is Tailor training so that staff get the training that the Office of Federal Procurement focused on training that is most relevant to its is most appropriate to them Policy said that it would assess the mission current policy framework. Demonstrate top level commitment and provide Track training to ensure it reaches the right resources people at the right time www.gao.gov/cgi-bin/getrpt?GAO-03-281. Identify those needing training and set Measure the effectiveness of training so that requirements so that training reaches the right further improvements can be made To view the full report, including the scope staff and methodology, click on the link above. For more information, contact Dave Cooper at Source: GAO. (202) 512-4841 or email@example.com. Contents Letter 1 Results in Brief 2 Background 4 Critical Elements for Acquisition Initiative Training 5 Inconsistent Use of Key Elements by Agencies for Training on Initiatives 8 Conditions That Facilitate the Use of Key Elements for Acquisition Initiative Training 22 Conclusions 24 Recommendations for Executive Action 24 Agency Comments and Our Evaluation 25 Appendix I Scope and Methodology 27 Appendix II Comments from the Department of Defense 29 Appendix III Comments from the National Aeronautics and Space Administration 30 Appendix IV GAO Contacts and Staff Acknowledgments 33 Related GAO Products 34 Table Table 1. Key Elements for Acquisition Training and Why They Are Important 6 Figures Figure 1: Assessment of DOD’s Acquisition Training Approach 9 Figure 2: Assessment of GSA’s Acquisition Training Approach 15 Figure 3: Assessment of NASA’s Acquisition Training Approach 17 Page i GAO-03-281 Acquisition Workforce Training Figure 4: Organizational Structure of the Associate Administrator for Research and Acquisitions and Participants in the Intellectual Capital Investment Plan Council 20 Figure 5: Assessment of FAA’s Acquisition Training Approach 21 Abbreviations DOD Department of Defense FAA Federal Aviation Administration GSA General Services Administration NASA National Aeronautics and Space Administration OMB Office of Management and Budget Page ii GAO-03-281 Acquisition Workforce Training United States General Accounting Office Washington, DC 20548 January 15, 2003 The Honorable Tom Davis Chairman, Subcommittee on Technology and Procurement Policy Committee on Government Reform House of Representatives Dear Mr. Chairman: The federal government is dramatically changing the way it purchases goods and services. As it strives to maximize the value of the $200 billion it spends annually, it is increasingly emulating the practices of commercial industry. As a result, rigid rules have given way to practices that rely more on the judgment and initiative of the individuals that make up the acquisition workforce. To help the government adapt to this changing environment, Congress enacted a series of acquisition reform initiatives in the 1990s. To take full advantage of these and subsequent initiatives and to spend money wisely, agencies need to train their workforces to ensure they have the skills necessary to operate in a changing business environment. Our recent work on purchase cards highlighted what can happen when training is ill-planned. The use of purchase cards was greatly expanded in order to simplify small procurements, and many more people were provided with the authority to make purchases on the government’s behalf. Yet, at some Department of Defense (DOD) locations we reviewed, the expanded authority was not accompanied by appropriate training which, when coupled with poor internal controls and inadequate guidance, left DOD vulnerable to fraudulent, wasteful, or abusive purchases.1 Because training is a critical element in achieving change, you asked us to assess the strategies agencies use to ensure that their workforces are receiving the training necessary to implement acquisition initiatives. To do this, we (1) identified elements of an approach to training that are critical to training on acquisition initiatives, (2) assessed the extent that agencies’ 1 U.S. General Accounting Office, Control Weaknesses Leave Army Vulnerable to Potential Fraud and Abuse, GAO-02-863T (Washington, D.C.: July 17, 2002) and U.S. General Accounting Office, Control Weaknesses Leave Army Vulnerable to Fraud, Waste, and Abuse, GAO-02-844T (Washington, D.C.: July 17, 2002). Further purchase card testimony and report titles are at the back of this report. Page 1 GAO-03-281 Acquisition Workforce Training strategies used these elements, and (3) identified conditions that facilitate or hinder the use of the key elements in their approach to training on acquisition initiatives. Our review focused on the General Services Administration (GSA), the National Aeronautics and Space Administration (NASA), and DOD. Together, these agencies represented 76 percent2 of total contract dollars obligated in fiscal year 2001. We examined whether agencies’ strategies for translating acquisition initiatives into training incorporated the key elements. To gain an understanding of the process, we examined how DOD, GSA, and NASA applied their approaches to implementing a specific initiative, performance-based service contracting.3 In addition, at DOD we looked at another initiative, the use of commercial and nondevelopmental items, because this initiative directly affects how DOD acquires weapon systems. We also looked at the Federal Aviation Administration (FAA) approach to training its workforce because its acquisition management system is exempt from all federal acquisition regulation and laws, giving it greater flexibility and discretion. We did not assess the effectiveness of the training provided by the agencies we reviewed. Further details on our objectives, scope, and methodology can be found in appendix I. Training is recognized by industry and government experts alike as a Results in Brief critical tool in successfully introducing, implementing, and reacting to change. The critical elements important to acquisition initiative training include (1) prioritizing the acquisition initiatives most important to the agency, (2) securing top-level commitment and resources, (3) identifying those who need training on specific initiatives, (4) tailoring training to meet the needs of the workforce, (5) tracking training to ensure it reaches the right people, and (6) measuring the effectiveness of training. Agencies that do not focus their attention on these critical elements risk having an acquisition workforce that is ill equipped to implement new processes. The probability of success is higher if training is well planned rather than left to chance. 2 As reported in the Federal Procurement Data System for fiscal year 2001. Excludes construction dollars. 3 Under performance-based service contracting, the agency identifies the results it wants, and the contractor decides on the best means to achieve the agency’s objective. See U.S. General Accounting Office, Guidance Needed for Using Performance-Based Service Contracting, GAO-02-1049 (Washington, D.C.: September 2002). Page 2 GAO-03-281 Acquisition Workforce Training Our assessment of the strategies used by DOD, GSA, and NASA to train their acquisition workforces on initiatives found that the agencies generally lacked a well-defined process that fully incorporates all six elements. Agencies had efforts underway to make training available and raise awareness of major acquisition initiatives, but too often they did not have an identifiable process for ensuring that training on significant acquisition initiatives reached those who played a role in successful implementation of the initiative. DOD did not have an institutionalized approach that was applied consistently. DOD and the Army employed most elements in implementing one acquisition initiative—performance-based service contracting. For example, it set training as a high priority in its policy implementation directive and targeted training to the appropriate program, technical, financial, and other personnel. However, the use of the elements was not evident on another initiative we examined. Over the past year, DOD has moved toward a process that gives the Defense Acquisition University a greater role in training on new initiatives. GSA and NASA identified performance-based contracting as an important initiative and made training available through classroom and online learning opportunities. However, many of the key elements were absent or not fully addressed in their approach to training on new initiatives. FAA’s approach was somewhat different than the other agencies we reviewed. The Associate Administrator for Research and Acquisition created an organizational focal point to define a process and facilitate the management of their acquisition workforce. FAA employed many of the key elements in its approach, principally by creating a special council of agency executives within its acquisition and research organization to establish priorities and target resources to acquisition initiatives it assessed as a high priority. Certain conditions tended to facilitate or hinder use of an approach that incorporates the key elements. First, the presence of an organizational focal point that could reach beyond the contracting community allowed training to be targeted to staff in a range of career fields that are integral to the success of an initiative, such as program, technical, and financial personnel. Second, integrating training into the planning for policy implementation supported an approach incorporating the key elements. Finally, civilian agencies, unlike DOD, are not supported by a training organization that develops or coordinates training resources on governmentwide initiatives. The Federal Acquisition Institute, which is Page 3 GAO-03-281 Acquisition Workforce Training charged with supporting the civilian acquisition workforce, has not been engaged in training on acquisition initiatives. We make recommendations on specific steps that the Office of Federal Procurement Policy can take to facilitate a sound management approach to training on acquisition initiatives. For decades the federal government has been struggling with ways to Background make the acquisition process more efficient. During the 1990s, Congress enacted two key pieces of acquisition legislation that affected training: the Defense Acquisition Workforce Improvement Act in 1990 for DOD and the Clinger-Cohen Act of 1996 for civilian agencies. Both were enacted to improve the management of the acquisition workforce. In July 2002 GAO reported4 on agency efforts to define and train their workforces to meet the requirements of the Defense Acquisition Workforce Improvement Act and the Clinger-Cohen Act. Our report addressed the training requirements in these acts, that is, the training requirements employees must meet to qualify for specific workforce positions. Such training normally occurs during the first few years of an employee’s career. This report addresses another important element in successfully moving towards a changing business environment: training the relevant members of the workforce who have an integral role in the successful implementation of specific acquisition initiatives. The relevant workforce can include those in the contracting community, such as contracting officers and contracting officer technical representatives, as well as those outside the contracting community, such as program and financial managers. The Defense Acquisition Workforce Improvement Act recognized acquisition as a multidisciplinary career field comprised of 11 functional areas, such as program management; engineering; procurement, including contracting issues; and logistics. In response to the act’s requirements, DOD set education, training, and experience requirements for the functional areas and established the Defense Acquisition University to provide its acquisition workforce with the professional development and 4 U.S. General Accounting Office, Agencies Need to Better Define and Track the Training of their Employees, GAO-02-737 (Washington, D.C.: July 2002). Page 4 GAO-03-281 Acquisition Workforce Training training required to meet the standards for certification in specific acquisition career fields. Civilian agencies, under the Clinger-Cohen Act, are required, in consultation with the Office of Federal Procurement Policy, to establish education, training, and experience requirements for their acquisition workforces. In implementing the provisions of the Clinger-Cohen act, the Office of Federal Procurement Policy issued policy letter 97-01, which defined the acquisition workforce to include contracting and purchasing specialists, contracting officers, contracting officer representatives, and contracting officer technical representatives, as well as other positions “in which significant acquisition-related functions are performed.” The act creating the Office Of Federal Procurement Policy5 was amended to establish the Federal Acquisition Institute, which, under the direction of the Office Of Federal Procurement Policy, was to promote the development and training of the acquisition workforce. The Federal Acquisition Institute was charged with developing the core curriculum needed to train the acquisition workforces of civilian agencies. The Procurement Executives Council, an interagency body of procurement executives, chartered a working group to provide advice and guidance to the acquisition institute in developing its educational and career management programs. Critical Elements for Acquisition Initiative Training Leading private and public organizations realize that their people largely determine their capacity for success. Our past reviews show that the training methods applied by leading commercial firms on new practices are the result of a focused, institutionally driven approach. This approach recognizes that workforces are the key to successfully implementing change and that training is a critical element in the process. 5 The Office of Federal Procurement Policy Act, P.L. 93-400, codified in 41U.S.C.§401 et seq. The act created the Office Of Federal Procurement Policy within the Office of Management and Budget to provide governmentwide leadership to agencies in procurement matters. Page 5 GAO-03-281 Acquisition Workforce Training In 1999,6 we reported on how leading commercial organizations train their acquisition workforces on changing practices and how DOD would benefit from employing this approach to commercial best practices. We have also reported on how high performing organizations develop and manage their workforces. (A list of these GAO products is at the back of this report.) Based on this information, we identified and developed some key elements of an approach to training the relevant workforce on acquisition initiatives that we believe are crucial to successful implementation of acquisition initiatives. These elements and their importance are summarized in table 1. Table 1. Key Elements for Acquisition Training and Why They Are Important Key Element Importance Prioritize initiatives most important to an Focuses on those acquisition initiatives that are most relevant to accomplishing the agency agency’s mission Demonstrate top-level commitment and Emphasizes to managers, trainers, and implementers the importance of the initiative provide resources and the necessary support to sustain reform efforts Identify those needing training and set Targets training to those who are integral to the success of an initiative training requirements Tailor training to meet the needs of the Recognizes that acquisition staff with different functions or at different levels may need workforce customized training Track training to ensure it reaches the right Increases the chances of having the right people with the right skills available when people at the right time needed to implement acquisition initiatives Measure the effectiveness of training Links training to agency results, demonstrates improved individual and organization performance, and provides feedback for adjusting or redefining acquisition initiative training Source: GAO. We list prioritizing initiatives first because it sets the stage for employing the other elements. Prioritizing initiatives signals an agency’s top-level commitment and allows it to concentrate its resources on initiatives deemed important to meeting its goals and missions and encourages it to better define the target population that needs training on a specific initiative. In conjunction with setting priorities, one of the most important elements is the demonstrated commitment of leaders to change. Top leadership involvement in making improvements is critical to overcoming an organization’s natural resistance to change, marshalling the resources needed, and building an agencywide commitment to new ways of doing business. Our 1999 report found, in general, that leading commercial firms committed to and adopted seven or fewer key practices at any given time. 6 U.S. General Accounting Office, DOD Training Can Do More to Help Weapon System Programs Implement Best Practice, GAO/NSIAD-99-206 (Washington, D.C.: August 1999). Page 6 GAO-03-281 Acquisition Workforce Training One firm adopted only one or two, which enabled it to concentrate and target its resources to those employees most in need of training. Identifying those who need in-depth training on a specific initiative is important because not all members of the workforce need training on every initiative and providing that level of training would be an inefficient use of resources. While awareness training (i.e., letting the workforce know of impending change) may be appropriate for the workforce in general, agencies need to identify those members of the workforce who are relevant to the success of an initiative for in-depth training. Training requirements need to be set and appropriately tailored to target the various groups involved in implementing change. As we noted in our 1999 report, the commercial companies with whom we spoke did not leave it to chance that those needing training will avail themselves of the opportunity. In July 2002 we reported on the progress agencies were making in tracking the career education and continuous learning requirements of their acquisition employees. We reported that DOD and the military services use a centralized information system that is automatically updated with training and personnel data. Civilian agencies currently use less sophisticated programs to collect and maintain information on education, training, and continuing education, commonly relying on spreadsheets for tracking training. As of November 2002, a Web-based management information system to track training was being piloted by several federal agencies. A system to track the training received by those integral to the success of an acquisition initiative is important to ensure that the right people are getting the right training. Because training strategies interact with other strategies and factors in attempting to change people and organizations, it is difficult to isolate the performance improvements resulting from training and development efforts. High-performing organizations, however, recognize the importance of assessing the results achieved from their training investments to determine whether they improved organizational and individual performance. It is important for agencies to have some way of measuring the results of acquisition training and the amounts of resources expended. As part of a balanced approach, agency assessments of acquisition training efforts would consider feedback from customers, employees, and organizational results. Officials of the Procurement Executives Council, the Office of Federal Procurement Policy, the Director of the Federal Acquisition Institute, and Page 7 GAO-03-281 Acquisition Workforce Training others agreed that the elements we focused on embody sound, fundamental management principles. Officials believe implementing these elements by blending them into initiative training efforts, rather than leaving training to chance, can help agencies ensure that their relevant workforces have the skills to contribute to the success of acquisition initiatives. DOD, GSA, and NASA generally lack an approach for training on Inconsistent Use of acquisition initiatives that fully incorporates the six key elements. The Key Elements by agencies we reviewed varied in the extent they used the critical elements for training on acquisition initiatives, and none had fully implemented all Agencies for Training six. Most of the elements were evident in DOD’s implementation of one on Initiatives initiative—performance-based services contracting—but the approach was not consistently applied. DOD has modified its approach over the past year by expanding the role of the Defense Acquisition University, and the revised process incorporates more of the elements. GSA and NASA relied on making training available to staff without a formal system for clearly defining priorities, identifying which staff need training, or easily tracking who has been trained. The FAA created the Intellectual Capital Investment Plan Council to meet the development needs of staff in its research and acquisition organization. FAA’s approach differs significantly from that used by other agencies, and its method employs most of the key elements we identified. While there was variance among the four agencies on most of the key elements, agencies used similar approaches to evaluating the effectiveness of training. Agencies generally relied on post-course student-generated evaluations that measure the extent to which the training met learners’ expectations, was relevant to their work, and would help them do a better job in the future. For the most part, they did not obtain feedback from customers or assess organizational results. DOD’s Approach to In DOD, the Under Secretary of Defense for Acquisition, Technology and Acquisition Workforce Logistics has control over all aspects of the acquisition workforce.7 From Training this position, the Under Secretary can prioritize initiatives and target training to staff in a wide range of career fields and organizations. Within 7 10 U.S.C.§1702 Page 8 GAO-03-281 Acquisition Workforce Training the Under Secretary’s office, the Acquisition Initiatives organization plays a critical role in policy development.8 While the organizational framework is conducive to using the key elements, we found that results were inconsistent, demonstrating that the approach has not been institutionalized. DOD’s implementation of one initiative we reviewed— performance-based service acquisition—largely addressed the key elements. However, for another initiative—use of commercial and nondevelopmental items—most elements were not fully addressed. Our assessment of DOD’s approach is shown in figure 1. Figure 1: Assessment of DOD’s Acquisition Training Approach 8 This office is being merged with another organization and reorganized. As of December 2002, the structure and functional responsibilities of this office were not available. Page 9 GAO-03-281 Acquisition Workforce Training Key elements present in implementing performance-based service contracting DOD as well as the Office Of Federal Procurement Policy recognized that performance-based service contracting offered an approach for improving the government’s purchases of services. However, performance-based contracting represents a fundamental change in how the government acquires services. Before such contracting, the government commonly specified the tasks it wanted performed. With performance-based contracting, the government identifies the results it wants and the contractor decides upon the best means to achieve the agency’s objectives. Performance-based service acquisition requires that program, technical, contracting, and other staff work together to carefully define the desired results. Recognizing this, the Under Secretary issued a policy memorandum setting goals for the use of performance-based service contracting.9 The Under Secretary directed that within one year organizations were to train the relevant workforce in performance-based contracting. Thus, the Under Secretary linked training to the policy implementation process. The policy memorandum noted that training resources were available in the form of Web-based and on-site training courses specifically developed for DOD by commercial firms and that the DOD Change Management Center would, as requested, present real-time “how to” training sessions on writing performance-based service contract statements of work. 9 The Office of Management and Budget (OMB) established greater use of performance- based contracts as one of several governmentwide reforms to be highlighted by the President in the fiscal year 2002 budget. OMB created a goal of awarding not less than 20 percent of contracts over $25,000 using performance-based methods for fiscal year 2002. See OMB Memorandum M-01-15. Page 10 GAO-03-281 Acquisition Workforce Training The Army mirrored DOD’s process. The Army Deputy Assistant Secretary for Acquisition, Technology and Logistics issued memoranda linking policy and training and sending a signal to trainers and implementers about the importance of the initiative. The letters to major commands defined the core workforce as contracting officers, contracting officer representatives, program managers, and others. In response to this top-level direction, Army components undertook efforts to get relevant staff trained on performance-based contracting. At the Army Communication–Electronics Command, for example, the commanding general instructed each component to identify and train the relevant workforce on performance-based service contracting and to report progress quarterly. All contracting officers in the command’s acquisition organization were required to become current in performance- based service contracting because, according to officials, the organization provides contracting services based on a team concept to many different groups. According to command officials, the command also recognized that the success of performance-based contracting depended to a large extent on training relevant staff in various functional areas outside the contracting function, such as program management and logistics. Other command components, such as the Software Engineering Center, were required to identify those important to developing and monitoring a performance-based contract, such as engineers, requirements setters, and program managers. Training of these staff was mandatory and training progress reported on a quarterly basis to the command. In training staff, the command relied heavily on the on-line Web-based course developed by the National Association of Purchasing Managers/National Contract Management Association. The command also made onsite training courses available and employed a number of other training techniques such as Web-based knowledge centers, e-mails, and brief presentations by subject matter experts. Page 11 GAO-03-281 Acquisition Workforce Training Some key elements not evident in DOD’s training on commercial and non- developmental items Expanding the use of commercial and nondevelopmental items is a complex issue. It requires that program and technical staff be knowledgeable about the standards used in the commercial market and have the skill to conduct market research on what is available in the commercial market place. Contracting officers need an understanding of commercial pricing practices. The absence of a well thought out and carefully targeted approach to training on this complex initiative leaves it too much to chance that the right people are benefiting from the training offered on this issue. DOD has repeatedly recognized the importance of acquiring commercial items to leverage the massive technology investment of the private sector and exploit the potential for reduced development times, faster insertion of new technologies, and lower life cycle costs. For example, in a June 2000 memorandum, the Under Secretary for Acquisition, Technology and Logistics emphasized the importance of this initiative and suggested that training acquisition staff in various functional areas would be necessary for its successful implementation. The Under Secretary’s memorandum suggested training program managers in market research and training contracting and financial management personnel in commercial buying practices. However, in making these suggestions, the document does not provide a listing of what training courses are available nor does it set training requirements. Other policy documents on commercial and nondevelopmental items do not set training requirements. DOD and the Services make available numerous training aids on commercials items acquisitions developed by DOD and commercial firms, Page 12 GAO-03-281 Acquisition Workforce Training including guidebooks, Web-based knowledge centers, and distance learning courses on market research, and have incorporated information into the curriculum for career development (i.e., certification training). However, beyond career development training, acquisition staff must seek out this training. For example, Army Communication and Electronics Command officials said that while they have a commercial items knowledge center, training is not targeted to specific communities, such as contracting or program management staff, and is not mandatory. Unlike the approach for performance-based service contracting, the command does not specifically identify who needs training on commercial and nondevelopmental items. The Defense Acquisition DOD officials told us that they adopted GAO recommendations contained University is taking a more in our 1999 report in the plans for restructuring the Defense Acquisition active role in training on University. To improve its ability to train the work force on best practices, initiatives DOD revised its continuous learning program, which provides training opportunities to staff who have completed career management training.10 Officials from the Acquisition Initiatives office (recently merged into a new organization named the Defense Procurement and Acquisition Policy office) believe that the continuous learning approach will be an effective way to provide training on reform initiatives. They pointed out that many of the career and continuous learning courses on reform initiatives are interchangeable and can be used for career-level training as well as continuous learning. DOD also told us that it now has an outreach and communication template to aid in identifying the audience that needs to be aware of an initiative and recently brought in experts to help determine specific methods of delivery (e.g., e-mail message, handbook, hands-on training, Web-based training) to those audiences. According to acquisition office officials, DOD is also developing a process to leverage continuous learning modules created by the Services, industry, the Defense Procurement and Acquisition Policy office, and the Defense Acquisition University to ensure that there is no duplication of effort and to look for cost-sharing opportunities. Under the restructuring, the University will receive specific requirements to develop continuous learning modules through input from the Curriculum Development Support Center and the advisors for each of 10 Staff who have completed career management training are required to take a certain number of hours of training each year. Page 13 GAO-03-281 Acquisition Workforce Training the acquisition career fields. According to DOD, these requirements will be prioritized and sent to a decision board called the Career Management Overarching Integrated Process Team led by the Director, Acquisition Education, Training and Career Development. Implementation of this process began in 2002. This process more clearly links initiatives to training and sets priorities. However, other elements are not fully addressed. In particular, the process does not address whether the continuous learning modules would include the kind of in-depth training associated with introducing significant changes or how those needing in-depth training, as opposed to awareness training, would be identified. The process also does not identify what organizations would identify who needs this type of training and set training requirements for specific initiatives. GSA’s Approach to GSA has made efforts to improve awareness of acquisition initiatives. It Acquisition Workforce established the Office of Acquisition Workforce Transformation in 2001 to Training develop new training options and assess the skills of GSA’s acquisition workforce. It also expanded the use of Web-based tools to make training more accessible. However, as shown in figure 2, many of the elements we identified are not evident in GSA’s approach to training on acquisition initiatives. Instead, GSA has relied on making training available to staff without a system for clearly defining priorities, identifying which staff need training, or easily tracking who has been trained. Page 14 GAO-03-281 Acquisition Workforce Training Figure 2: Assessment of GSA’s Acquisition Training Approach GSA’s acquisition organization does not have a centralized process to systematically prioritize acquisition initiatives. Officials in GSA’s Federal Technology Service stated that training based in regional offices is prioritized by the individual GSA services, relying on their own interpretations of acquisition regulations and administration policy. Officials said they emphasized performance-based service contracting because of the goals established by the administration. Implementation of these policies was left to GSA’s major components. Presently, GSA does not have a process to identify professionals who need training on specific initiatives. Moreover, the headquarters acquisition organization has authority only over those included in GSA’s definition of its acquisition workforce: those professionals who hold warrants authorizing them to purchase goods and services, contracting officers, contracting officer technical representatives, property disposal professionals, and purchasing and procurement personnel. Technical, financial, or other professionals who may also be relevant to the successful implementation of acquisition initiatives are not included. Page 15 GAO-03-281 Acquisition Workforce Training In the case of performance-based service contracting, Federal Technology Service officials said they relied on supervisors to ensure that staff involved with performance-based service contracting received training. However, the Federal Technology Service encouraged contracting and other technical staff to take training. In some cases they offered tailored training and were aware of customized training needs for information technology staff. The training was not made mandatory for staff in particular fields or roles. GSA, however, does not have a system to track who has received training on acquisition initiatives. Regional officials said that an in-depth personnel file review would have to be conducted to acquire data on acquisition initiative training. The agency is currently participating in a governmentwide system pilot for the Acquisition Career Management Information System, which will monitor standardized training and certification information on the GSA acquisition workforce as well as be available to support other agencies. While the system will track the GSA acquisition workforce and its accomplishments, plans do not call for tracking relevant professionals outside GSA’s currently defined acquisition workforce. GSA is currently altering the way in which it will evaluate and educate its acquisition workforce. The Office of Acquisition Workforce Transformation plans to pilot a new assessment and training program beginning in January 2003. While the program is intended to help GSA take a more focused approach to acquisition workforce training, particularly career development training, it is too early to assess the new program or to tell whether it will help GSA improve the way in which it approaches training on acquisition initiatives or measures the effectiveness of training. NASA’s Approach to NASA also has made efforts to educate staff about initiatives such as Acquisition Workforce performance-based contracting. Additionally, officials believed that top- Training level commitment was evident and added that there was sufficient funding for acquisition initiative training. However, a well developed approach to Page 16 GAO-03-281 Acquisition Workforce Training acquisition workforce training, incorporating the elements identified by GAO, is lacking, as shown in figure 3. Figure 3: Assessment of NASA’s Acquisition Training Approach Although NASA does not have a formal system for prioritizing acquisition initiatives, the agency does take action, such as through Web-based communications and training, to inform and educate the acquisition workforce about acquisition initiatives it perceives as important to the agency’s mission. For example, to raise employee awareness and agency implementation of performance-based service contracting, NASA headquarters offered an initial orientation to employees at headquarters, Goddard Space Center, and other centers. It embedded performance- based service contracting in existing courses and formed a training committee to identify the center’s training needs. Moreover, the desire for improving implementation of performance-based contracting led NASA management to initiate an internal review led by an Office of Procurement team in 1999. The review recommended changes in the performance-based service contracting training system such as identifying training that can be customized. Page 17 GAO-03-281 Acquisition Workforce Training NASA does not use a defined process to identify personnel who should receive training on acquisition initiatives and does not mandate training beyond certification requirements. Instead, NASA relies on its centers to identify staff who need training. At Goddard, officials said that career development officers identify procurement professionals who should be trained, although no one is responsible for identifying other professionals who may benefit from training on acquisition initiatives. NASA defines its acquisition workforce as certified procurement professionals and procurement clerks and does not include in the definition other technical or program professionals who may be relevant to the implementation of an acquisition initiative. Officials stated, however, that performance-based service contracting training is made available to professionals outside the procurement field; for example, program managers and engineers. Although NASA received assistance in developing some initial training on performance-based service contracting from the Navy and the Defense Acquisition University, it tailored the training to NASA personnel. Goddard also offered a 5-day performance-based service contracting course tailored to specific personnel such as administrators and specialists, project and program managers, and contracting officer technical representatives. Officials also said that other training is usually customized to be center- specific. NASA does not have a centralized system for tracking who has been trained. Each center is responsible for tracking its own training. At Goddard, officials use employees’ individual development plans and center-specific databases to track staff training. Short of a comprehensive review of personnel records, this system does not allow management officials to identify relevant, noncertified professionals who have received training. In the future, the system being piloted by GSA will be available to NASA to support tracking of training. NASA makes some effort to go beyond course assessments to evaluate the effectiveness of training. It reports surveying its program managers to assess the quality of service provided. Page 18 GAO-03-281 Acquisition Workforce Training FAA’s Research and Acquisitions Organization Approach to Acquisition Workforce Training The FAA’s Associate Administrator for Research and Acquisitions has a unique approach for managing its acquisition workforce, one that provides a framework for implementing many of the six key elements. In October 1997, the Intellectual Capital Investment Plan Council was created to address the organization’s workforce development needs. Creation of the Council followed enactment of legislation11 that exempted FAA’s new acquisition management system from all federal acquisition regulation and laws. Responsibility for developing and managing this system was vested in the Office of Research and Acquisitions, headed by an associate administrator, as shown in figure 4. 11 The Department of Transportation and Related Agencies Appropriations Act of 1996, section 348 of P.L. 104-50. Page 19 GAO-03-281 Acquisition Workforce Training Figure 4: Organizational Structure of the Associate Administrator for Research and Acquisitions and Participants in the Intellectual Capital Investment Plan Council The Director of the Office of Business Management heads the council, which is composed of directors and deputy directors from each of the offices as well as the Chief Scientist for Human Factors. Each year the council prepares an investment plan that prioritizes initiatives and allocates funding for the associate administrator’s workforce planning and development. The council also provides a focal point to facilitate the management of workforce development. In recent years, the council identified and prioritized initiatives that it saw as important to its research and acquisitions organization, such as an emphasis on integration of human factors into system design. It did not prioritize performance-based service contracting as a priority. Page 20 GAO-03-281 Acquisition Workforce Training The figure below shows how closely the associate administrator office’s training approach incorporates the elements critical to acquisition initiative training that we identified. Figure 5: Assessment of FAA’s Acquisition Training Approach One council goal is to “establish investment priorities to support the required workforce changes.”12 The council gives the research and acquisitions organization a structure for comprehensively reviewing and funding acquisition workforce training initiatives. The acquisition organization targets training to specific groups of professionals. These groups encompass an array of disciplines, including scientists, engineers, business managers, financial analysts, and contracting officers, as well as other critical roles identified by the council. However, while the organization strongly encourages training, it does not mandate training on initiatives. 12 1998 Intellectual Capital Investment Plan. Page 21 GAO-03-281 Acquisition Workforce Training Training is often tailored to staff in different roles or disciplines. For example, when the FAA introduced its new acquisition system, it offered an overview course and later developed courses of different lengths to accommodate different professional needs. Additionally, the organization has developed its own tracking system that allows training coordinators to query it and identify who has received training on specific initiatives. Our discussions with agency officials and the input we received from the Conditions That Procurement Executives Council highlighted certain conditions that Facilitate the Use of facilitate using the elements in an approach to training on acquisition initiatives. The absence of these conditions may limit the capability of Key Elements for agencies to implement acquisition initiatives for the relevant workforce. Acquisition Initiative These conditions include: Training • establishing an organizational focal point with authority over the wide range of personnel who are involved in the acquisition process, • integrating training into the planning for policy implementation, and • using the Federal Acquisition Institute to coordinate and facilitate training on governmentwide initiatives. First, having an organizational focal point with the authority to reach beyond the contracting community to other disciplines, such as program managers and requirements setters, facilitates using the six key elements in an approach to training on acquisition initiatives. Implementing some acquisition initiatives draws on staff from a range of functions and career fields. For example, conducting a successful performance-based service acquisition requires careful evaluation of real needs and requirements so that a statement of objectives can be developed that identifies the results or outcomes the agency is trying to realize from a particular acquisition. Participation by those developing the requirement is central to this effort. Thus, successful implementation of performance-based service contracting requires participation by users as well as other program, financial, legal, and related staff. Both FAA and DOD have such a focal point. FAA established the Intellectual Capital Investment Plan Council, which is made up of directors and deputy directors of its acquisition and research programs. Within DOD, the Office of the Under Secretary for Acquisition, Technology and Logistics has authority for DOD acquisition. Both organizations have authority over the range of career fields that are engaged in the acquisition process. Page 22 GAO-03-281 Acquisition Workforce Training In contrast, civilian agencies commonly define the acquisition workforce in terms of contracting personnel. This is true in the case of NASA and GSA as well as Health and Human Services, an agency we discussed in our July 2002 report. While this approach complies with some of the Clinger-Cohen Act’s requirements, which state that the acquisition workforce should include contracting and procurement specialists, agencies have not expanded their definitions to include all positions in which “significant acquisition-related functions are performed,” as required by the act. Failing to expand the definition limits the ability of acquisition officials to target training to personnel outside the contracting function. Our July 2002 report recommended that the Office of Federal Procurement Policy work with agencies to further refine the definition of the acquisition workforce. Second, integrating training into implementation plans also facilitates this approach. Among the cases we examined, DOD’s and the Army’s implementation of performance-based service contracting included many of the six key elements. A central feature of this initiative was the recognition that performance-based service contracting represented a significant change in the way services were acquired and that training the relevant workforce would be necessary to define service requirements effectively. The policy memorandum of the Under Secretary of Defense for Acquisition, Technology and Logistics, which set a goal of increasing performance-based service contracting, established a requirement that the relevant workforce be trained within a year. This contrasted with the approach used on expanding the use of commercial and nondevelopmental items. Although the use of commercial and nondevelopmental items was emphasized and communicated in memorandums, an implementation strategy was not formalized and training requirements were not set. Finally, civilian agencies have not been supported by an organization that coordinates training on governmentwide initiatives. The Federal Acquisition Institute supports civilian agency training of the acquisition workforce. However, over the last several years the institute has focused on career management issues. Although involved in awareness training, such as conferences and lunchtime seminars, officials told us the institute has not been funded to develop training resources to support the implementation of new initiatives. Members of the Procurement Executives Council, in commenting on the key elements we developed, noted that adequate resources were needed and stated that the Federal Page 23 GAO-03-281 Acquisition Workforce Training Acquisition Institute “should be sufficiently funded to lead governmentwide training efforts on procurement reforms.” While the Office of Federal Procurement Policy provides policy direction to the Federal Acquisition Institute, the institute is located in and receives support from GSA. Recently, GSA began a process of outsourcing the functions of the Federal Acquisition Institute. This restructuring is intended to revitalize and refocus acquisition training within the federal government and to improve the institute's effectiveness in acquisition workforce development and management. The outsourcing process is expected to give the Procurement Executives Council a stronger role in setting priorities for the institute, although no plan or agreement has been developed to define and formalize this role. Leading organizations employ many key elements that provide assurance Conclusions that the right people will have the right skills to implement change. Such an approach is not consistently evident at the agencies we reviewed. We believe that it should be. As the government continues to undertake reforms aimed at making itself a more commercial-like buyer, it cannot afford to leave it to chance that people are getting the necessary training to make this transformation. The Office of Federal Procurement Policy, which provides government leadership for agencies on procurement matters, is in a good position to take a proactive role in promoting the key elements across federal agencies and to ensure that civilian agencies have a strong role in setting priorities for the Federal Acquisition Institute. Such actions would be consistent with the emphasis the President’s management agenda places on human capital. We recommend that the Administrator, Office of Federal Procurement Recommendations for Policy, develop a policy that calls on agencies to Executive Action • establish an approach to training on new acquisition initiatives that includes (1) prioritizing the initiatives most important to the agency, (2) demonstrating top-level commitment and providing necessary resources, (3) identifying those who need training on specific initiatives, (4) tailoring training to meet the needs of the workforce, (5) tracking training to ensure it reaches the right people, and (6) measuring the effectiveness of training; • establish a focal point that sets training requirements for staff integral to the success of acquisition initiatives, including those outside the contracting community; and Page 24 GAO-03-281 Acquisition Workforce Training • integrate training into the planning for policy implementation. We also recommend that the Administrator, Office of Federal Procurement Policy, define the role of agency procurement executives in setting priorities for the restructured Federal Acquisition Institute. The Office of Federal Procurement Policy provided official oral comments Agency Comments on a draft of this report. DOD and NASA provided written comments that and Our Evaluation are included in appendixes II and III. Neither FAA nor GSA provided official written or oral comments. The Office of Federal Procurement Policy stated, in response to the report’s first recommendation, that it will review the current policy framework to see whether additional guidance is needed to insure that agencies provide appropriate training to the acquisition workforce. In response to the second recommendation, the Office of Federal Procurement Policy stated that it is closely following the restructuring of the Federal Acquisition Institute and will work with the procurement community to be sure that the Institute will meet the training needs of the acquisition workforce. Given the gaps we identified in agencies’ training approaches, we believe that changes are needed in the current policy framework to incorporate best commercial practices and prepare the workforce for new challenges. DOD agreed that the report’s assessment of its training efforts on the two initiatives is valid. DOD disagreed with the statement that its outreach and communication template does not address how those needing in-depth training, as opposed to awareness training, would be identified. While DOD’s statement may reflect what is intended with the template, the template itself only asks for the identification of the intended audience for outreach efforts without specifically discussing the audience needing different types of training. NASA commented that it was pleased that the report recognized efforts to educate staff about initiatives such as performance based contracting and stated that the articulation for the first time of the “key element” standards will assist federal agencies’ future development of more effective training programs in the vitally important area of acquisition. NASA also stated that it believed that the agency was closer to meeting the high process development standard in the other elements than was recognized in the report and highlighted four areas of activity in support of that statement. Page 25 GAO-03-281 Acquisition Workforce Training We reviewed NASA’s comments, but we do not believe a change in our assessment is appropriate since our evaluation indicates that, while selected initiatives have followed a process that incorporates many of the key elements, NASA does not have a well-defined, identifiable process that fully addresses these elements. As agreed with your office, unless you publicly announce the contents of this report earlier, we plan no further distribution of it until 30 days from the date of this letter. At that time, we will send copies to other interested congressional committees, the secretaries of Defense and Transportation, and the administrators of General Services Administration, the National Aeronautics and Space Administration, and the Office of Federal Procurement Policy. We will also make copies available to others upon request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov. If you or your staff have questions about this report, please contact me at (202) 512-4125. Key contributors to this report are listed in appendix IV. Sincerely yours, David E. Cooper Director Acquisition and Sourcing Management Page 26 GAO-03-281 Acquisition Workforce Training Appendix I: Scope and Methodology Appendix I: Scope and Methodology To identify and develop the critical elements important to acquisition workforce training that are cited in this report, we conducted an extensive review and analysis of past GAO human capital and best practices reports. Appendix II provides a comprehensive list of relevant GAO reports we evaluated. We also reviewed private-sector studies and commercial training organization and federal agency Web sites for information on training approaches. Additionally, we consulted with other organizations that are engaged in acquisition initiative training issues. We solicited comments on the six key elements from the Director, Defense Procurement; the Director, Federal Acquisition Institute; the Chairman of the Procurement Executives Council’s Acquisition Workforce Committee; officials of the Office of Federal Procurement Policy; and GSA’s Deputy Associate Administrator for Acquisition Policy. In their comments, these officials generally agreed that the six key elements we identified represented fundamentally sound management principles. To assess whether the agencies in this review addressed the key elements we identified, we analyzed the degree to which each agency has developed a process that uses the elements important to acquisition workforce training. Within DOD we conducted a detailed examination of the Army, and at the Army, GSA, and NASA we followed the process through at a subordinate organization, contacting the Federal Technology Service at GSA, Goddard Space Center at NASA, and the Army’s Communications– Electronics Command in Ft. Monmouth, New Jersey. We assessed FAA’s Research and Acquisitions organization because it is exempt from federal acquisition regulation and laws and uses a different approach to managing its acquisition workforce. Additionally, we reviewed and analyzed a wide range of documents, including guidance and policy memoranda, reports, fact sheets, course attendance rosters, workforce plans, and human capital and workforce studies. We also obtained agency and training information from federal and commercial Web sites. We also consulted with numerous officials representing the following organizations: • Department of Defense • Department of the Army • Department of the Navy • Department of the Air Force • General Services Administration • National Aeronautics and Space Administration • Federal Aviation Administration • Office of Federal Procurement Policy • Procurement Executives Council • Federal Acquisition Institute Page 27 GAO-03-281 Acquisition Workforce Training Appendix I: Scope and Methodology • Defense Acquisition University To identify the conditions that facilitate or hamper the implementation of the six key elements we identified, we analyzed officials’ comments and input on the six elements and incorporated some of this feedback into our report. We also analyzed the results of our discussions with agency officials and compared this to acquisition workforce information obtained from earlier GAO reports and other documents. We did not evaluate the effectiveness of agency training programs. Our review was conducted from October 2001 through September 2002 in accordance with generally accepted government auditing standards. Page 28 GAO-03-281 Acquisition Workforce Training Appendix II: Comments from the Department Appendix II: Comments from the Department of Defense of Defense Page 29 GAO-03-281 Acquisition Workforce Training Appendix III: Comments from the National Aeronautics and Appendix III: Comments from the National Space Administration Aeronautics and Space Administration Note: GAO comments supplementing those in the report text appear at the end of this appendix. See comment 1. See comment 2. Page 30 GAO-03-281 Acquisition Workforce Training Appendix III: Comments from the National Aeronautics and Space Administration See comment 3. See comment 4. Page 31 GAO-03-281 Acquisition Workforce Training Appendix III: Comments from the National Aeronautics and Space Administration The following are GAO’s comments on the National Aeronautics and Space Administration letter dated December 20, 2002. 1. The instruction contains only cursory references to training. It does not GAO Comments define, for example, a process to identify and obtain training resources, set training requirements, identify those who need to be trained, track who received training, or monitor the effectiveness of training. 2. NASA’s pilot approach is fully consistent with the key elements presented in our report. At an appropriate point in the pilot implementation process, the agency needs to assess what training is suitable for particular staff involved in each initiative. 3. While NASA’s attention to performance based contracting was positive, NASA’s review did not measure the effectiveness of training but rather highlighted the frustration expressed by NASA personnel over the absence of high quality training in specific areas. 4. The existence of a well thought-out training strategy in these areas is laudable. However, NASA’s strategy does not go far enough in the development of a process that can be applied to acquisition workforce training. Our assessment focused on whether a well-defined process existed for developing a training strategy on all important initiatives. Page 32 GAO-03-281 Acquisition Workforce Training Appendix IV: GAO Contacts and Staff Appendix IV: GAO Contacts and Staff Acknowledgments Acknowledgments David Cooper, (202) 512-4125 GAO Contacts Karen Zuckerstein, (202) 512-6785 In addition to those named above, Kimberley Ebner, Ralph Roffo, Jeffrey Acknowledgments Rose, Sylvia Schatz, Paul Schearf, and Richard Silveira made key contributions to this report. Page 33 GAO-03-281 Acquisition Workforce Training Related GAO Products Related GAO Products Acquisition Workforce: Status of Agency Efforts to Address Future Needs. GAO-03-55. Washington, D.C.: December 18, 2002. Contract Management: Guidance Needed for Using Performance Based Service Contracting. GAO-02-1049. Washington, D.C.: September 23, 2002. Acquisition Workforce: Agencies Need to Better Define and Track the Training of their Employees. GAO-02-737. Washington, D.C.: July 29, 2002. Travel Cards: Control Weakness Leave Army Vulnerable to Potential Fraud and Abuse. GAO-02-863T. Washington, D.C.: July 17, 2002. Purchase Cards: Control Weakness Leave Army Vulnerable to Fraud, Waste, and Abuse. GAO-02-844T. Washington, D.C.: July 17, 2002. Purchase Cards: Control Weaknesses Leave Army Vulnerable to Fraud, Waste, and Abuse. GAO-02-732. Washington, D.C.: June 27, 2002. Purchase Cards: Continued Control Weaknesses Leave Two Navy Units Vulnerable to Fraud and Abuse. GAO-02-506T. Washington, D.C.: March 13, 2002. Purchase Cards: Control Weaknesses Leave Two Navy Units Vulnerable to Fraud and Abuse. GAO-02-32. Washington, D.C.: November 30, 2001. Human Capital: Attracting and Retaining a High Quality Information Technology Workforce. GAO-02-113T. Washington, D.C.: October 4, 2001. Human Capital: A Self-Assessment Checklist For Agency Leaders. GAO/OCG-00-14G. Washington, D.C.: September 1, 2000. Human Capital: Implementing An Effective Workforce Strategy Would Help EPA to Achieve Its Strategic Goals. GAO-01-812. Washington, D.C.: July 31, 2000. Human Capital: Design, Implementation, and Evaluation of Training at Selected Agencies. GAO/T-00-131. Washington, D.C.: May 18, 2000. Federal Acquisition: Trends, Reforms, and Challenges. GAO/T-00-7. Washington, D.C.: March 16, 2000. Page 34 GAO-03-281 Acquisition Workforce Training Related GAO Products Acquisition Reform: GSA and VA Efforts to Improve Training of Their Acquisition Workforces. GAO/GGD-00-66. Washington, D.C.: February 18, 2000. Human Capital: Key Principles From Nine Private Sector Organizations. GAO/GGD-00-28. Washington, D.C.: January 31, 2000. Best Practices: DOD Training Can Do More To Help Weapon System Programs Implement Best Practices. GAO/NSIAD-99-206. Washington, D.C.: August 16, 1999. (120107) Page 35 GAO-03-281 Acquisition Workforce Training The General Accounting Office, the investigative arm of Congress, exists to GAO’s Mission support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. The fastest and easiest way to obtain copies of GAO documents at no cost is Obtaining Copies of through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full- GAO Reports and text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents Testimony using key words and phrases. You can print these documents in their entirety, including charts and other graphics. Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to www.gao.gov and select “Subscribe to daily E-mail alert for newly released products” under the GAO Reports heading. Order by Mail or Phone The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. General Accounting Office 441 G Street NW, Room LM Washington, D.C. 20548 To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202) 512-6061 Contact: To Report Fraud, Web site: www.gao.gov/fraudnet/fraudnet.htm Waste, and Abuse in E-mail: firstname.lastname@example.org Federal Programs Automated answering system: (800) 424-5454 or (202) 512-7470 Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 Public Affairs U.S. General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C. 20548
Acquisition Management: Agencies Can Improve Training on New Initiatives
Published by the Government Accountability Office on 2003-01-15.
Below is a raw (and likely hideous) rendition of the original report. (PDF)