DOCUMENT RESUME 00588 - A0590927] Jpward nobility Using Job Restructuring. B-7C8S6(4); FPCD-77-8. January 10, 1977. 2 pp. + 3 appendices (8 pp.). Report to Secretary, Department of the Air Force; by . L. Krieger, Director, Federal Personnel and Compensation Div. Issue Area: Non-Discrimination and Equal Opportunity Programs. Employment Discrimination in the Federal Sector (1004). Contact: Federal Personnel and Compensation Div. BuAget Function: Gereral Government: Central Personnel Management (dO,). Congressional Relevance: House Committee on Armed Services; Senate Committee on Armed Services. Authority: 5 U.S.C. 41. In June 1976 the Department of the Air Force issued an Upward Eobility Training Agreement to enhance eployees' upward nobility opportunities. A review was conducted of the progress of the Upward Mobility Program at the departmental level and at three field activities. Findings/Conclusions: Field activities are reporting Upuard obility costs in an inconsistent manner, and lack of departmental guidelines has resulted in reporting inaccurate and incomplete Upward obility program costs. The Air Force Training Agreement emphasizes that participants should be selected in accordance with local merit promotion principles. The way in which ranking is used for selecting Upward Mobility participants is not effective, and individuals without the necessary knowledge f Upward Mobility career fields are serving on in erview panels. Recommendations: The Air FPcrce should: develop guidelines which insure that program costs are collected and reported completely and accurately; issue guidelines emphasizing the need to monitor supervisory appraisal scores of employees' current performance, take steps to insure that knowledgeable panel members are selected; and insure that effective ranking factors are used to select Upward obility participants. (RRS) 00 co L0 O - L¥r'VITED STA TES '! : GENERAL ACCOUNTING OFFICE Upward Mobility Using Job Restructuring Department of the Air Force Recent actions by te Air Force should improve its Upward Mobility programs. How- ever, additional improvements can be made by providing guidelines on collecting and reporting Upward Mobility costs and by Inak- ing sure that the effective ranking factors are used to select Upward Mobility participants. FPCD..77-8, J 1 9 77 UNITED STATES GENERAL ACCO'JNTING OFFICE WASHINGTON, D.C. 20548 FEDERAL PERSONNEL AND COMPENSATION DIVISION B-70896(4) The Honorable The Secretary of the Air Force Dear Mr. Secretary: We recently completed a limited review of the progress of the Department of the Air Force's Upward Mobility pro- gram. Upward Mobility program efforts were reviewed at the Department level and at three field activities. W also examined departmental and field activity policies, procedures, and guidelines issued on Upward Mobility as well as program evaluation procedures. Our Government-wide review of Upward Mobility programs in 1973 and 1974 led to issuance of an April 1975 report to the Congress entitled "Upward Mobility Programs in the Fed- eral Government Should Be Made More Effective" (FPCD-75- 84). Tis review included the Department's Upward Mobility program. On May 20, 1974, w discussed with Air Force of- ficials various problems which existed in the Department's program. At that time, the Department had not issued guide- lines on Upward Mobility. Recent Department actions should improve its Upward Mobility program. In June 1976 the Department issued an Air Force-wide Upward Mobility Training Agreement to enhance employees' Upward Mobility opportunities and Upward Mobility guidelines which requires program planning and management commitment. Issuance of these guidelines represents an at- tempt to improve the Air Force's Upward Mobility program, but implementation of the guidelines will lai 'ely determine the success of the program. We believe additional improvements can be made by -- providing guidelines on collecting and reporting Up- ward Mobility costJ and -- insuring that effective ranking factors are used to select Upward Mobility participants. B-70896(4) These matters are discussed in detail in appendi:es I and II of this letter. A third appendix is also included which contains recommendations to improve the Upward Mobility training program at the Aeronautical Systems Division, Wright-Patterson Air Force Base. In our opinion, adoption of the recommendations would strengthen the Departmnnt's Upward Mobility efforts. These suggestions were discussed with Air Force officials. We would appreciate being advised of any actions plan- ned or taken concerning our recommendations. As you know, section 236 of the Legislative Reorganization Act of 1970 requires the head of a Federal agency to submit a written response on actions taken on our rec)mmendations to the House and Senate Committees on Government Operations not later than 60 days after the date of the report and the House and Senate Committees on Appropriations with the agency's first request for appropriations made more than 60 days after the date of the report. We are sending copies of this letter to the Senate Com- mittee on Labor and Public Welfare; the House Committee on Education and Labor, Subcommittee on Equal Opportunities; and the Civil Service Commission. We appreciate te cooperation and courtesy extended to us by Department officials during our isit. If you wish to discuss the above issues or ned further information, please let me know. Sincerfly yours, H. L. Krieger Director 2 APPENDIX I APPENDIX I NEED TO IMPROVE COLLECTING AND REPORTING UPWARD MOBILITY COST DATA The Civil Service Commission (CSC) requires Federal agencies to report in their annual affirmative action plans all costs of formal and on-the-job training concerning Up- ward Mobility program efforts. Also, each year agcncy Equal Employment Opportunity (EEO) officials are required by Of- fice of Management and Budget (OMB) Circular A-11 to report the costs of internal FEO programs, including Upward Mobil- ity. The circular requires that the report include a con- cise description of the program, significant cost-effective- ness or other analytic findings, pertinent comments concern- ing reliability of the data, and actions planned to improve data collection. Air Force officials are aware of problems being en- countered in collectir, and reporting A-11 Upward Mobili' y costs. For example, a 3partment .of Defense-wide meeting was held in 1975 to discuss development of annual budget estimates for the OMB A-11 report. A-cording to the Chief of the Air Force's EEO office, participants at that meeting had varyin- definitions of which cost elements should be included as Upward Mobility costs. In addition to varying definitions on which costs should be ncluded, the costs that were reported were not always accurate. Despite these inaccuracies, Air Force officials never told CSC of the reliability problems as required. In addition, the Air Force's 1974 and 1975 A-11 submissions to CSC did not contain narrative statements regarding data reliability or data collection problems. Despite these shortcomings, guidelines have not been issued to field activities to insure that costs are col- lected and reported in a uniform manner for either affir- mative action plan or OMB A-11 reporting purposes. As a result, field activities are reporting Upward Mobility costs in an inconsistent manner. One field activity, for example, reported about $60,000 as the cost of its 1975 Upward Mobility program. A coopera- tive educational program comprised about 40 percent of this amount. A cognizant official admitted that this should not have been included as Upward Mobility program costs. On the other hand, neither on-the-job training nor administrative APPENDIX I APPENDIX I costs were reported, although such costs were recognized as being elements of Upward obility. Another field activity reported about $2.500 as the cost of its 1975 Upward Mobility program. These costs con- sisted of the estimated administrative staff salaries for supporting and administering the Upward Mcbility program. Direct costs for Upward Mbility training were not included although CSC guidelines require that such costs be included in agency submissions. Officials at two field activities told us that inac- curate reporting of Upward Mobility program costs resulted from a lack of departmental guidelines. Conclusions and recommendations Lack of efartmental guidelines has resulted in report- ing inaccurate and incomplete Upward Mobility program costs. We recommend, therefore, that the Secretary of the Air Force direct appropriate program officials to develop guide- lines which insure that Upward Mobility costs are collected and reported completely and accurately. These guidelines should also require that Air Force responses to CSC address the problems of data collection and reliability. 2 APPENDIX II APPENDIX II NEED TO INSURE EFFECTIVENESS OF RANKING FACTORS FOR SELECTING UPWARD MOBILITY PARTICIPANTS The Air Force Training Agreement emphasizes that par- ticipants should be selected in accordance with local merit promotion principles. According to the Air Force's merit promotion program guidelines, each installation must estab- lish a promotion evaluation pattern which identifies the ranking factors to be used in evaluating employees for selection. Upward Mobility programs at the three fie.d activities used a variety of ranking factors, including sLpervisory appraisals, panel interviews, and tests. A training, edu- cation, and xperience (TRAEX) score was also used as a ranking factor in certain Upward Mobility programs. Al- though these factors were in accordance with the merit promotion program, we believe the way in which these factors were used may inhibit their effectiveness in the selection process. Supervisory_3praisal scores The Air Force's merit promotion program guidelines state that supervisory appraisals of current employee per- formance are an important factor in the selection process. These appraisals are designed to evaluate employee perfor- mance, work habits, and attitudes. Since supervisory appraisal scores are an important factor in the selection process, they should accurately show an employee's performance. Supervisory appraisal scores, however, tended to favor certain groups of employees. For example, in one field activity we analyzed supervisory ap- praisal scores for employees in the secretarial and clerical appraisal patterns because many employees competing for en- try into Upward Mobility programs at this activity are rated under these patterns. In November 1974 approximately 20 percent of all secretarial employees received perfect ap- praisal scores while only 4 percent of all clerical employees received perfect scores. Since supervisory appraisal scores are a primary selection factor, it appears that secretaries could have a competitive advantage over clerical employees for selection into Upward Mobility programs. 3 APPENDIX II APPENDIX II This problem may be further compounded because there was a tendency for supervisors at this activity to gradually inflate appraisal scores, as in the fllowing: Percent with perfect appraisal scores Employees June 1976 1974 Secretarial 31 20 Clerical 8 4 Although officials recognized that the disproportionate appraisal scores caused problems in the selection process. no corrective action has been taken. Selection of panel memoers Air Force guidelines require that evaluation of candi- dates' potential include a review from one or more managers or supervisors who have knowledge of the characteristics needed to participate successfully in the training program and to perform successfully in the developmental position. Such a review can be made through the use of a panel inter- view. Although two of the field activities we visited used the panel interview as a ranking factor. only one of the field activities issued guidelines for selecting its panel members., According to its guidelines, one of the primary considerations used in appointing panel members was their knowledge of the target career fields. Panel members ,;ere supposed to be subject matter experts. which was defined as either managers or journeyman specialists in those career fields targeted for Upward Mobility participants. We reviewed position descriptions on 24 of the 124 individuals who had served on or were eligible to serve on Upward Mobility panels, because these individuals did not appear to be in career fields meeting the selection criteria for panel membership. Our review showed the following- -- Seven individuals did not appear to be subject mtter ex rts. Two individuals. although in a target Up- wrd Mobility career field, were not at the journeymnan level. The other five individuals included a physi- cist. a mathematician, an operations research analyst. 4 APPENDIX II APPENDIX II an accountant, and an aerospace engineer. Nune of their career fields were targeted for Upward Mobi- lity, and nothing in their position descriptions in-- dicated they had interaction with Uwar2 Mobility career fields. -- Twelve individuals were questionable subject matter experts. These individuals were working in such diversified career fields as electrical engineering, mathematics, mechanical engineering, metallurgy. gen- eral engineering. systems engineering. aerospace en- gineering, engineering psychology. and perations research. None of their career fields were targeted for Upward Mobility. An analysis of their position descriptions revealed that these individuals had only limited interface with career fields targeted for Up- ward Mobility. -- Five individuals, while in career fields different from those targeted for Upward Mobility, appeared to have enough interaction with Upward Mobility career fields to be eligible for panel membership. According to cognizant officials. the primary reason for not strictly adhering to field activity panel selection criteria was the inability to schedule panel interviews at times convenient to the panel members. In our opinion, additional steps should be taken to insure that panel members are selected from those occupa- tional career fields targeted for Upward Mobilitj. Other- wise, panelists may not be fully aware of the prerequisites needed for the Upward Mobility career ficid. Written tests and use of TRAEX Air Force merit promotion program guidelines state that written tests may be used as a primary ranking factor in the selection process. The Air Force-, ide Upward Mobility Training Agreement states that written tests may be used-to evaluate potential if they are not used as the sole screen- ing factor for selection. Two of the field activities we re- viewed used written tests as a ranking factor in selecting participants for certain Upward Mobility programs. 5 APPENDIX II APPENDIX II We reviewed the use of the written test for the program having ths largest number of program participants. Scores received on the Professional and Administrative Career Examination (PACE) were used as a ranking factor in deter- mining eligibility for potential career positions such as budget analysts, contract negotiators. and personnel specialists. riCE scores received by individuals were doubled and added to supervisory appraisal and panel inter- view scores. The total scores were then used to establish numerically ordered registers of eligible candidates. Field activity officials said their only rationale for dour':.nq the PACE score was to deemnhasize the importance of the score received on the panel inter'riew, The maximum score that an individual receives on the unweighted PACE is 100 points, while a maximum of 245 points is possible from the panel interview. Studies 1- ve not been conducted to validate whether the use of dc le-weighted scores was valid. Additionally, neither ne Upward Mobility promotion evaluation pattern nor the merit promotion plan indicated that scores were to be weighted. Since studies have not beer conducted to justify use of the weignting procedures for PACE cores. we reranked two of the registers by using supervisory appraisal scores, panel interview scores, and single-weighted PACE scores. Single weighting of PACE scores produced some major shifts on the registers. For example. individuals' positions on the registers changed from 5th to 14th place, 19th to 8th place, 9th to 2d place, and 18th to 31st place. Although we are not implying that a single-weighted PACE score is more valid tan a double-weighted score. such weighting should be both validated and publicized before it is used. The Air Force merit promotion program guidelines also require that TRAEX ratings be used when a test is not used as a ranking factor. When tests are used in ranking. TRAEX ratings are optional, but whenever practical, appli- cation of TRAEX as an additional measure is recommended. Although we did not evaluate TRAEX scores for the individuals discussed above, we believe that TRAEX should be considered as an additional ranking factor, even when written tests are used. Gae of the TRAEX score may be especially beneficial to those otherwise qualified for Up- ward Mobility but who do not do well on written tests. Air Force officials agreed with our observation. 6 APPENDIX II APPENDIX Ii Conclusions and recommendations The way in which anking is used for selecting Upward Mobility participants i; ot effective. Also, individuals without the necessary kno'wledge of Upward Mobility career fields were selected to serve on interview panels. We re- commend that the Secretary of the Air Force direct appro- priate program officials to: -- Issue guidelines emphasizing the need to monitor supervisory appraisal scores of employees' current performance. -- hake steps to insure selection of panel members who are knowledgeable of the characteristics necessary for successful performance in Upward Mobility train- ing positions. -- Insure that weights, where used in ranking, are valid and publicized. -- Consider requiring TRAEX as a ranking factor even if written tests are used. 7 APPENDIX III APPENDIX III IMPROVEMENTS NEEDED IN THE UPWARD MOBILITY TRAINING PROGRAM AT HEADQUARTERS, AERONAUTICAL SYSTEMS DIVISION, WRIGHT-PATTERSON AIR FORCE BASE CSC has suggested that agencies consider formal trair- ing programs offered in non-Government facilities as one method of providing training and educational opportunities to Upward Mobility program participants. Training programs in support of Upward Mobility are to be in accordance with chapter 41, title 5, U.S.C. The Aeronautical Systems Division (ASD) at Wright-Pat- terson Air Force Base emphasized formal training programs in support of Upward Mobility. Many of these courses were of- fered in non-Government facilities. While the ASD program provided participants additional advancement opportunities, improvements can be made in several areas to increase the efficiency and effectiveness of training in support of Up- ward Mobility. The areas requiring improvements are dis- cussed in the remainder of this appendix. Need to relate training to manpower needs and established target jobs According to 5 U.S.C. 41, agencies must (1) determine their manpower needs before enrolling employees in training programs, (2) insure that ftlly trained employees are ident- ified and given first consideration for these needs, and (3) identify target jobs to insure that training is related to present or potential job duties within the agency. Upward Mobility training provided at Government expense must be in- tegrated with manpower and staffing plans. Under its PACE Upward Mobility program, ASD offers Up- ward Mobility training to qualified individuals for whom specific target jobs have not been established. This train- ing is being offered even though ASD's systems to indentify marpo.wer and training needs appear weak. ASD's training needs are not being systematically identified. A recent evaluation by ASD's Major Command noted that training needs were not related to work force assessment, organizational, and mission changes or work re- quirement changes. Officials told us that annual training- needs surveys were not conducted. 8 APPENDIX III APPENDIX III In addition, ASD manpower needs are not being fully identified. Although we requested information identifying relative manpower needs by grade level and occupation for general schedule employees between 1973 and 1975, ASD officials could provide us with only gross statistics for 1975. They were unable to provide specific information on skill needs by grade level because there was no formal- ized procedure to project employment needs by specific skills and rgrades. Officials told us that they were unable to project these needs and that they considered such man- power planning to be nothing more than a paperwork exercise. While training and manpower needs have not been fully defined, ASD currently offers Upward Mobility training to qualified individuals for whom specific target jobs have not been established under its PACE Upward Mobility program. Eligible personnel under this program may have non-job-re- lated training approved if he following conditions are met: (1) they are in the top one-third of the register and (2) training is related to agency needs. This training is not aimed at enhancing t.e participant's qualifications for a potential target position. Instead, ASD officials believe the training is justi- fied on the basis that it will enhance the participant's competitiveness some time after being selected for a specific target position. In addition, a participant may be selected for a position completely different from the position he is trained for. Since there is no relationship among manpower planning, systematic identification of target jobs, and training needs, such training may be inefficient and ineffective because there is no assurance that an individual receiving training will be selected for a position within any of the target career fields. Need to im rove controls over formal tralnin- through use of career development plans A career development plan outlines the job progress for an employee including the training and experience required to qualify for a designated job. CSC guidelines suggest developing such plans for employees engaged in certain training and education activities. The plan should be based on a careful analysis uf the employee's needs and the future needs of the agency and should be developed before placing 9 APPENDIX III APPENDIX III the participant in the target field. Experiences and train- ing courses required for successful performance i- the tar- geted career field should be included in this plan. The em- ployee. supervisor. and personnel officials should all part- icipate in development of the plan. According to Air Force regulations on employee training and development, equestE for training are not to be author- ized if the requested course does not apoar in a written plan. Moreover, according to 1974 guidel.'nes issued by ASr, personnel selected for positions in the Aministrative/Tech- nical Career Upward Mobility program were to have a formal career development plan if formal training was required. We reviewed selected personnel information on 25 indi- viduals in this Upward Mobility program to determine whether they had current career development plans. Of the 25 indi- viduals, 4 did not have current career development plans and 6 had no career development plans. Of the 12 individuals who received college courses. 8 received courses which were not specifically identified in their career development plans. In each of these instances. the training authoriza- tions said the courses were part of a career development plan. Four of these individuals had no career development plans even though numerous training authorizations showed that such courses were part of the individuals career development plan. In November 1975 ASD also developed a list of college courses that were available to clerical personnel. These courses were to be approved if a valid job-related just- ification existed. The purpose of this program was to assist the clerical work force in broadening caceer oppor- tunities. Among the courses offered were psychology and sociology. We reviewed 31 training authorizations just- ifying courses in psychology and sociology. In every in- stance except one, the training authorizations stated that such training was part of a career development plan. Only 9 of the 30 individuals had career development plans which showed a training need for either psychology or sociology. Most of the remaining individuals did not have career deve- lopment plans at the time hese authorizations were ap- proved. ASD officials knew that many individualE receiving for- mal training did not have indiv'idual career plans. They told s that the basJe. problem was caused by supervisor; not fulfilling their career counseling responsibilities. 10 APPENDIX III APPENDIX III Although supervisors may not be fulfilling their counseling responsibilities. we believe that training officials should be making greater efforts to insure that formal training is needed and that each employee receiving formal training has an individual career development plan relating training to a specific need. CONCLUSIONS AND RECOMMENDATIONS While ASD offered a variety of undergraduate academic training to support Upward Mobility for many lower level Air Force employees. controls to insure that the programs are effective and efficient and are not in violation of 5 U.S.C. 41 do not exist. The Air Force's recently issued Upward Mobility guidance does not properly address these problems. We recommend. therefore. that the Secretary of the Air Force require appropriate ASD program officials to: -- Insure that ASD systematically determines appropriate target positions before specific training courses are approved. -- Develop individual career development plans for all Upward Mobility candidates detailing required formal training before commencing such training. 11
Upward Mobility Using Job Restructuring
Published by the Government Accountability Office on 1977-01-10.
Below is a raw (and likely hideous) rendition of the original report. (PDF)