oversight

Upward Mobility Using Job Restructuring

Published by the Government Accountability Office on 1977-01-10.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                          DOCUMENT RESUME
00588 -   A0590927]
Jpward nobility Using Job Restructuring. B-7C8S6(4); FPCD-77-8.
January 10, 1977. 2 pp. + 3 appendices (8 pp.).
Report to Secretary, Department of the Air Force; by . L.
Krieger, Director, Federal Personnel and Compensation Div.
Issue Area: Non-Discrimination and Equal Opportunity Programs.
    Employment Discrimination in the Federal Sector (1004).
Contact: Federal Personnel and Compensation Div.
BuAget Function: Gereral Government: Central Personnel
    Management   (dO,).
Congressional Relevance: House Committee on Armed Services;
    Senate Committee on Armed Services.
Authority: 5 U.S.C. 41.

          In June 1976 the Department of the Air Force issued an
Upward Eobility Training Agreement to enhance eployees' upward
nobility opportunities. A review was conducted of the progress
of the Upward Mobility Program at the departmental level and at
three field activities. Findings/Conclusions: Field activities
are reporting Upuard obility costs in an inconsistent manner,
and lack of departmental guidelines has resulted in reporting
inaccurate and incomplete Upward obility program costs. The Air
Force Training Agreement emphasizes that participants should be
selected in accordance with local merit promotion principles.
The way in which ranking is used for selecting Upward Mobility
participants is not effective, and individuals without the
necessary knowledge f Upward Mobility career fields are serving
on in erview panels. Recommendations: The Air FPcrce should:
develop guidelines which insure that program costs are collected
and reported completely and accurately; issue guidelines
emphasizing the need to monitor supervisory appraisal scores of
employees' current performance, take steps to insure that
knowledgeable panel members are selected; and insure that
effective ranking factors are used to select Upward obility
participants. (RRS)
00
co
L0
O




            -   L¥r'VITED STA TES
     '! :       GENERAL ACCOUNTING OFFICE




                Upward Mobility Using
                Job Restructuring

                Department of the Air Force


                Recent actions by te Air Force should
                improve its Upward Mobility programs. How-
                ever, additional improvements can be made
                by providing guidelines on collecting and
                reporting Upward Mobility costs and by Inak-
                ing sure that the effective ranking factors are
                used to select Upward Mobility participants.




                FPCD..77-8,                                       J   1   9 77
                        UNITED STATES GENERAL ACCO'JNTING OFFICE
                                 WASHINGTON, D.C.   20548


FEDERAL PERSONNEL AND
COMPENSATION DIVISION




       B-70896(4)




       The Honorable
       The Secretary of the Air Force

       Dear Mr. Secretary:

            We recently completed a limited review of the progress
       of the Department of the Air Force's Upward Mobility pro-
       gram. Upward Mobility program efforts were reviewed at the
       Department level and at three field activities. W also
       examined departmental and field activity policies, procedures,
       and guidelines issued on Upward Mobility as well as program
       evaluation procedures.

            Our Government-wide review of Upward Mobility programs
       in 1973 and 1974 led to issuance of an April 1975 report to
       the Congress entitled "Upward Mobility Programs in the Fed-
       eral Government Should Be Made More Effective"  (FPCD-75-
       84).  Tis review included the Department's Upward Mobility
       program. On May 20, 1974, w discussed with Air Force of-
       ficials various problems which existed in the Department's
       program. At that time, the Department had not issued guide-
       lines on Upward Mobility.

            Recent Department actions should improve its Upward
       Mobility program. In June 1976 the Department issued an
       Air Force-wide Upward Mobility Training Agreement to enhance
       employees' Upward Mobility opportunities and Upward Mobility
       guidelines which requires program planning and management
       commitment. Issuance of these guidelines represents an at-
       tempt to improve the Air Force's Upward Mobility program, but
       implementation of the guidelines will lai 'ely determine the
       success of the program.

               We believe additional improvements can be made by

               -- providing guidelines on collecting and reporting Up-
                  ward Mobility costJ and

               -- insuring that effective ranking factors are used to
                  select Upward Mobility participants.
B-70896(4)

These matters are discussed in detail in appendi:es I and
II of this letter. A third appendix is also included which
contains recommendations to improve the Upward Mobility
training program at the Aeronautical Systems Division,
Wright-Patterson Air Force Base. In our opinion, adoption
of the recommendations would strengthen the Departmnnt's
Upward Mobility efforts. These suggestions were discussed
with Air Force officials.

     We would appreciate being advised of any actions plan-
ned or taken concerning our recommendations. As you know,
section 236 of the Legislative Reorganization Act of 1970
requires the head of a Federal agency to submit a written
response on actions taken on our rec)mmendations to the
House and Senate Committees on Government Operations not
later than 60 days after the date of the report and the
House and Senate Committees on Appropriations with the
agency's first request for appropriations made more than 60
days after the date of the report.
     We are sending copies of this letter to the Senate Com-
mittee on Labor and Public Welfare; the House Committee on
Education and Labor, Subcommittee on Equal Opportunities;
and the Civil Service Commission.

     We appreciate te cooperation and courtesy extended to
us by Department officials during our isit. If you wish to
discuss the above issues or ned further information, please
let me know.

                                  Sincerfly yours,




                                  H. L. Krieger
                                  Director




                              2
APPENDIX I                                        APPENDIX I


                 NEED TO IMPROVE COLLECTING

                    AND REPORTING UPWARD

                     MOBILITY COST DATA


     The Civil Service Commission (CSC) requires Federal
agencies to report in their annual affirmative action plans
all costs of formal and on-the-job training concerning Up-
ward Mobility program efforts.  Also, each year agcncy Equal
Employment Opportunity (EEO) officials are required by Of-
fice of Management and Budget (OMB) Circular A-11 to report
the costs of internal FEO programs, including Upward Mobil-
ity. The circular requires that the report include a con-
cise description of the program, significant cost-effective-
ness or other analytic findings, pertinent comments concern-
ing reliability of the data, and actions planned to improve
data collection.

     Air Force officials are aware of problems being en-
countered in collectir, and reporting A-11 Upward Mobili' y
costs.  For example, a 3partment .of Defense-wide meeting
was held in 1975 to discuss development of annual budget
estimates for the OMB A-11 report.  A-cording to the
Chief of the Air Force's EEO office, participants at that
meeting had varyin- definitions of which cost elements
should be included as Upward Mobility costs.

     In addition to varying definitions on which costs
should be  ncluded, the costs that were reported were not
always accurate. Despite these inaccuracies, Air Force
officials never told CSC of the reliability problems as
required.  In addition, the Air Force's 1974 and 1975 A-11
submissions to CSC did not contain narrative statements
regarding data reliability or data collection problems.

     Despite these shortcomings, guidelines have not been
issued to field activities to insure that costs are col-
lected and reported in a uniform manner for either affir-
mative action plan or OMB A-11 reporting purposes. As a
result, field activities are reporting Upward Mobility
costs in an inconsistent manner.

     One field activity, for example, reported about $60,000
as the cost of its 1975 Upward Mobility program. A coopera-
tive educational program comprised about 40 percent of this
amount. A cognizant official admitted that this should not
have been included as Upward Mobility program costs. On the
other hand, neither on-the-job training nor administrative
APPENDIX I                                   APPENDIX I



costs were reported, although such costs were recognized as
being elements of Upward   obility.

     Another field activity reported about $2.500 as the
cost of its 1975 Upward Mobility program. These costs con-
sisted of the estimated administrative staff salaries for
supporting and administering the Upward Mcbility program.
Direct costs for Upward Mbility training were not included
although CSC guidelines require that such costs be included
in agency submissions.
     Officials at two field activities told us that inac-
curate reporting of Upward Mobility program costs resulted
from a lack of departmental guidelines.
Conclusions and recommendations
     Lack of efartmental guidelines has resulted in report-
ing inaccurate and incomplete Upward Mobility program costs.
     We recommend, therefore, that the Secretary of the Air
Force direct appropriate program officials to develop guide-
lines which insure that Upward Mobility costs are collected
and reported completely and accurately. These guidelines
should also require that Air Force responses to CSC address
the problems of data collection and reliability.




                             2
 APPENDIX II                                     APPENDIX II

               NEED TO INSURE EFFECTIVENESS OF

               RANKING FACTORS FOR SELECTING

               UPWARD MOBILITY PARTICIPANTS


     The Air Force Training Agreement emphasizes that par-
ticipants should be selected in accordance with local merit
promotion principles. According to the Air Force's merit
promotion program guidelines, each installation must estab-
lish a promotion evaluation pattern which identifies the
ranking factors to be used in evaluating employees for
selection.

     Upward Mobility programs at the three fie.d activities
used a variety of ranking factors, including sLpervisory
appraisals, panel interviews, and tests. A training, edu-
cation, and xperience (TRAEX) score was also used as a
ranking factor in certain Upward Mobility programs.  Al-
though these factors were in accordance with the merit
promotion program, we believe the way in which these factors
were used may inhibit their effectiveness in the selection
process.

Supervisory_3praisal scores

     The Air Force's merit promotion program guidelines
state that supervisory appraisals of current employee per-
formance are an important factor in the selection process.
These appraisals are designed to evaluate employee perfor-
mance, work habits, and attitudes.

      Since supervisory appraisal scores are an important
factor in the selection process, they should accurately show
an employee's performance. Supervisory appraisal scores,
however, tended to favor certain groups of employees.   For
example, in one field activity we analyzed supervisory ap-
praisal scores for employees in the secretarial and clerical
appraisal patterns because many employees competing for en-
try into Upward Mobility programs at this activity are rated
under these patterns. In November 1974 approximately 20
percent of all secretarial employees received perfect ap-
praisal scores while only 4 percent of all clerical employees
received perfect scores.   Since supervisory appraisal scores
are a primary selection factor, it appears that secretaries
could have a competitive advantage over clerical employees
for selection into Upward Mobility programs.



                              3
APPENDIX II                                      APPENDIX II



     This problem may be further compounded because there
was a tendency for supervisors at this activity to gradually
inflate appraisal scores, as in the fllowing:
                      Percent with perfect appraisal scores

       Employees             June 1976           1974
       Secretarial               31               20
       Clerical                   8                4
     Although officials recognized that the disproportionate
appraisal scores caused problems in the selection process.
no corrective action has been taken.

Selection of panel memoers

      Air Force guidelines require that evaluation of candi-
dates' potential include a review from one or more managers
or supervisors who have knowledge of the characteristics
needed to participate successfully in the training program
and to perform successfully in the developmental position.
Such a review can be made through the use of a panel inter-
view.

     Although two of the field activities we visited used
the panel interview as a ranking factor. only one of the
field activities issued guidelines for selecting its panel
members., According to its guidelines, one of the primary
considerations used in appointing panel members was their
knowledge of the target career fields. Panel members ,;ere
supposed to be subject matter experts. which was defined
as either managers or journeyman specialists in those career
fields targeted for Upward Mobility participants.
    We reviewed position descriptions on 24 of the 124
individuals who had served on or were eligible to serve on
Upward Mobility panels, because these individuals did not
appear to be in career fields meeting the selection criteria
for panel membership. Our review showed the following-
   -- Seven individuals did not appear to be subject mtter
      ex rts. Two individuals. although in a target     Up-
      wrd Mobility career field, were not at the journeymnan
      level. The other five individuals included a physi-
      cist. a mathematician, an operations research analyst.



                             4
APPENDIX II                                    APPENDIX II




      an accountant, and an aerospace engineer. Nune of
      their career fields were targeted for Upward Mobi-
      lity, and nothing in their position descriptions in--
      dicated they had interaction with Uwar2 Mobility
      career fields.

    -- Twelve individuals were questionable subject matter
       experts.  These individuals were working in such
       diversified career fields as electrical engineering,
       mathematics, mechanical engineering, metallurgy. gen-
       eral engineering. systems engineering. aerospace en-
       gineering, engineering psychology. and perations
       research. None of their career fields were targeted
       for Upward Mobility. An analysis of their position
       descriptions revealed that these individuals had only
       limited interface with career fields targeted for Up-
       ward Mobility.

    -- Five individuals, while in career fields different
       from those targeted for Upward Mobility, appeared to
       have enough interaction with Upward Mobility career
       fields to be eligible for panel membership.

    According to cognizant officials. the primary reason
for not strictly adhering to field activity panel selection
criteria was the inability to schedule panel interviews at
times convenient to the panel members.
    In our opinion, additional steps should be taken to
insure that panel members are selected from those occupa-
tional career fields targeted for Upward Mobilitj. Other-
wise, panelists may not be fully aware of the prerequisites
needed for the Upward Mobility career ficid.
Written tests and use of TRAEX
    Air Force merit promotion program guidelines state that
written tests may be used as a primary ranking factor in the
selection process. The Air Force-, ide Upward Mobility
Training Agreement states that written tests may be used-to
evaluate potential if they are not used as the sole screen-
ing factor for selection. Two of the field activities we re-
viewed used written tests as a ranking factor in selecting
participants for certain Upward Mobility programs.




                             5
APPENDIX II                                   APPENDIX II

    We reviewed the use of the written test for the program
having ths largest number of program participants. Scores
received on the Professional and Administrative Career
Examination (PACE) were used as a ranking factor in deter-
mining eligibility for potential career positions such as
budget analysts, contract negotiators. and personnel
specialists.  riCE scores received by individuals were
doubled and added to supervisory appraisal and panel inter-
view scores. The total scores were then used to establish
numerically ordered registers of eligible candidates.

      Field activity officials said their only rationale for
dour':.nq the PACE score was to deemnhasize the importance of
the score received on the panel inter'riew,  The maximum
score that an individual receives on the unweighted PACE is
100 points, while a maximum of 245 points is possible from
the panel interview. Studies 1- ve not been conducted to
validate whether the use of dc le-weighted scores was
valid.   Additionally, neither ne Upward Mobility promotion
evaluation pattern nor the merit promotion plan indicated
that scores were to be weighted.

     Since studies have not beer conducted to justify use
of the weignting procedures for PACE cores. we reranked two
of the registers by using supervisory appraisal scores,
panel interview scores, and single-weighted PACE scores.
Single weighting of PACE scores produced some major shifts
on the registers.  For example. individuals' positions on
the registers changed from 5th to 14th place, 19th to 8th
place, 9th to 2d place, and 18th to 31st place. Although
we are not implying that a single-weighted PACE score is
more valid tan a double-weighted score. such weighting
should be both validated and publicized before it is used.

     The Air Force merit promotion program guidelines also
require that TRAEX ratings be used when a test is not used
as a ranking factor.  When tests are used in ranking.
TRAEX ratings are optional, but whenever practical, appli-
cation of TRAEX as an additional measure is recommended.

     Although we did not evaluate TRAEX scores for the
individuals discussed above, we believe that TRAEX should
be considered as an additional ranking factor, even when
written tests are used.   Gae of the TRAEX score may be
especially beneficial to those otherwise qualified for Up-
ward Mobility but who do not do well on written tests.   Air
Force officials agreed with our observation.




                              6
APPENDIX II                                   APPENDIX Ii



Conclusions and recommendations

     The way in which anking is used for selecting Upward
Mobility participants i; ot effective. Also, individuals
without the necessary kno'wledge of Upward Mobility career
fields were selected to serve on interview panels. We re-
commend that the Secretary of the Air Force direct appro-
priate program officials to:

     -- Issue guidelines emphasizing the need to monitor
        supervisory appraisal scores of employees' current
        performance.

     -- hake steps to insure selection of panel members who
        are knowledgeable of the characteristics necessary
        for successful performance in Upward Mobility train-
        ing positions.

     -- Insure that weights, where used in ranking, are
        valid and publicized.

     -- Consider requiring TRAEX as a ranking factor even if
        written tests are used.




                             7
APPENDIX III                                       APPENDIX III

        IMPROVEMENTS NEEDED IN THE UPWARD MOBILITY

               TRAINING PROGRAM AT HEADQUARTERS,

               AERONAUTICAL SYSTEMS DIVISION,

               WRIGHT-PATTERSON AIR FORCE BASE


     CSC has suggested that agencies consider formal trair-
ing programs offered in non-Government facilities as one
method of providing training and educational opportunities
to Upward Mobility program participants.  Training programs
in support of Upward Mobility are to be in accordance with
chapter 41, title 5, U.S.C.

     The Aeronautical Systems Division (ASD) at Wright-Pat-
terson Air Force Base emphasized formal training programs in
support of Upward Mobility. Many of these courses were of-
fered in non-Government facilities. While the ASD program
provided participants additional advancement opportunities,
improvements can be made in several areas to increase the
efficiency and effectiveness of training in support of Up-
ward Mobility. The areas requiring improvements are dis-
cussed in the remainder of this appendix.

Need to relate training to
manpower needs and
established target jobs

     According to 5 U.S.C. 41, agencies must (1) determine
their manpower needs before enrolling employees in training
programs, (2) insure that ftlly trained employees are ident-
ified and given first consideration for these needs, and (3)
identify target jobs to insure that training is related to
present or potential job duties within the agency. Upward
Mobility training provided at Government expense must be in-
tegrated with manpower and staffing plans.
      Under its PACE Upward Mobility program, ASD offers Up-
ward Mobility training to qualified individuals for whom
specific target jobs have not been established. This train-
ing is being offered even though ASD's systems to indentify
marpo.wer and training needs appear weak.
     ASD's training needs are not being systematically
identified. A recent evaluation by ASD's Major Command
noted that training needs were not related to work force
assessment, organizational, and mission changes or work re-
quirement changes. Officials told us that annual training-
needs surveys were not conducted.

                                8
APPENDIX III                                   APPENDIX III


     In addition, ASD manpower needs are not being fully
identified. Although we requested information identifying
relative manpower needs by grade level and occupation for
general schedule employees between 1973 and 1975, ASD
officials could provide us with only gross statistics for
1975. They were unable to provide specific information
on skill needs by grade level because there was no formal-
ized procedure to project employment needs by specific
skills and rgrades. Officials told us that they were unable
to project these needs and that they considered such man-
power planning to be nothing more than a paperwork exercise.

     While training and manpower needs have not been fully
defined, ASD currently offers Upward Mobility training to
qualified individuals for whom specific target jobs have not
been established under its PACE Upward Mobility program.
Eligible personnel under this program may have non-job-re-
lated training approved if he following conditions are met:
(1) they are in the top one-third of the register and (2)
training is related to agency needs. This training is not
aimed at enhancing t.e participant's qualifications for a
potential target position.

     Instead, ASD officials believe the training is justi-
fied on the basis that it will enhance the participant's
competitiveness some time after being selected for a
specific target position. In addition, a participant may
be selected for a position completely different from the
position he is trained for.

     Since there is no relationship among manpower planning,
systematic identification of target jobs, and training
needs, such training may be inefficient and ineffective
because there is no assurance that an individual receiving
training will be selected for a position within any of the
target career fields.

Need to im rove controls over
formal tralnin- through use of
career development plans

     A career development plan outlines the job progress for
an employee including the training and experience required
to qualify for a designated job. CSC guidelines suggest
developing such plans for employees engaged in certain
training and education activities. The plan should be based
on a careful analysis uf the employee's needs and the future
needs of the agency and should be developed before placing


                             9
APPENDIX III                                  APPENDIX III



the participant in the target field. Experiences and train-
ing courses required for successful performance i- the tar-
geted career field should be included in this plan. The em-
ployee. supervisor. and personnel officials should all part-
icipate in development of the plan.

     According to Air Force regulations on employee training
and development, equestE for training are not to be author-
ized if the requested course does not apoar in a written
plan. Moreover, according to 1974 guidel.'nes issued by ASr,
personnel selected for positions in the Aministrative/Tech-
nical Career Upward Mobility program were to have a formal
career development plan if formal training was required.

     We reviewed selected personnel information on 25 indi-
viduals in this Upward Mobility program to determine whether
they had current career development plans. Of the 25 indi-
viduals, 4 did not have current career development plans
and 6 had no career development plans. Of the 12 individuals
who received college courses. 8 received courses which were
not specifically identified in their career development
plans. In each of these instances. the training authoriza-
tions said the courses were part of a career development
plan. Four of these individuals had no career development
plans even though numerous training authorizations showed
that such courses were part of the individuals career
development plan.

     In November 1975 ASD also developed a list of college
courses that were available to clerical personnel. These
courses were to be approved if a valid job-related just-
ification existed.   The purpose of this program was to
assist the clerical work force in broadening caceer oppor-
tunities. Among the courses offered were psychology and
sociology. We reviewed 31 training authorizations just-
ifying courses in psychology and sociology.  In every in-
stance except one, the training authorizations stated that
such training was part of a career development plan. Only
9 of the 30 individuals had career development plans which
showed a training need for either psychology or sociology.
Most of the remaining individuals did not have career deve-
lopment plans at the time hese authorizations were ap-
proved.

     ASD officials knew that many individualE receiving for-
mal training did not have indiv'idual career plans. They
told s that the basJe. problem was caused by supervisor; not
fulfilling their career counseling responsibilities.


                             10
APPENDIX III                                 APPENDIX III




     Although supervisors may not be fulfilling their
counseling responsibilities. we believe that training
officials should be making greater efforts to insure that
formal training is needed and that each employee receiving
formal training has an individual career development plan
relating training to a specific need.
CONCLUSIONS AND RECOMMENDATIONS

     While ASD offered a variety of undergraduate academic
training to support Upward Mobility for many lower level
Air Force employees. controls to insure that the programs
are effective and efficient and are not in violation of 5
U.S.C. 41 do not exist. The Air Force's recently issued
Upward Mobility guidance does not properly address these
problems.

     We recommend. therefore. that the Secretary of the Air
Force require appropriate ASD program officials to:

     -- Insure that ASD systematically determines appropriate
        target positions before specific training courses are
        approved.

     -- Develop individual career development plans for all
        Upward Mobility candidates detailing required formal
        training before commencing such training.




                             11