Personnel Appeals Board: Promotions of Banded Employees, 1991-1995

Published by the Government Accountability Office on 1999-09-30.

Below is a raw (and likely hideous) rendition of the original report. (PDF)


     2        Z
     0        r
     ul   .

                                                          September 30,1999

             The Honorable David M. Walker
             Comptroller General
             United States General Accounting Office
             Room 7000
             441 G Street, N;W.      -:..
             Washington, D.C. 20543                                                                                                  >

             Dear Mr. Walker:

             Pursuant to the authority granted to it under the General Accounting Office Personnel Act of
             1980,the Personnel Appeals Board has statutory responsibility to oversee equal employment
             opportunity at GAO. In exercise of that authority, the Board is issuing the attached report on.
             promotions of Banded employees at GAO.

            The Board’s report summarizes the findings of a study that examined the media time to
            promotion and rates of promotion for the five year period 1991-95. Employees in the study
            were differentiated by race, gender, national origin, age, and disability to discern whether there
            were any signZcant *parities amongthese groups in either the median time to promotion or
            rates of promotion at different levelsof the banding~system. We concluded that there were
            some disparities based on race, gender and age, but that the causes of these differences were
            not readily apparent from the statistics alone. Therefore, the Board has recommended that the
            Agency further investigate the disparities to determine whether additional steps need to be
            taken to ensure equal opportunity for its employees.


                                                                                              Michael Wolf


U. S. General Accounting Office    .     Suite560   l   Union Center Plaza II   l   ; @ashington;D.C.   20548   l   Phone   (202) 512-6137


Chapter I          Introduction
Background         I. History of the Promotion Study
                   II. Jurisdiction and Methodolo&                                  .

Chapter II                                                                               11
                   I. Bands                                                              11
The Promotion      II. Competitive Promotions (Banded Employees) at GAO                  15
Process                                                                      .-
Chapter III                                                                              18
                   I. Time-In-Band                                                       18
Findings           II. Promotion Rates, Adjusted for Composition of “Best-Qualified”     20
                                                                  I                      25
Chapter IV
Conclusions- and
Appendixes         Appendix    I: Agency Comments                                        28
                   Appendix    II: PAB Response toAgency Comments                        32
                   Appendix    IIk 6tatistics Lab, University of Maryland, Response to   35
                     Agency     Comments
                   Apperidix   TV: Personnel Appeals Board                               37

Tables             Competition 1                                                          9
                   Competition 2                                                          9
                   Combined Results                                                       9
                   Median Time to Promotion From I-D TO I-F                              19
                   Median Time to Promotion From I-F to II                               20
                   Table 1: Relative Odds of Proiotion by Gender, Race/National          22
                     Origin and Disability Controlling for Promotion Announcement
                     and Year: Employees under 40 Only
                   Table 2: Relative Odds of Promotion by Gender, Race/National          22
                     Origin and Disability Controlling for Promotion Announcement
                     and Year: Employees 40 and Older only

                   Page   2
 Contents                                 :
                                               I’ ./-

Table 3: Relative Odds of Promotioh by Age Group, Gender,             22
   Race/National Origin and Disability, Controlling for Promotion
  Announcement and Year: Re-gional Offices Only
Table 4: Relative Odds of Promotion ,by Age Group, Gender,            23
  Race/National Origin and Disability Controlling for Promotion
  Announcement and Year: Headquarters Only
Table 5: Relative Odds of Promotion by Gecder, Race/National      .., 23
  Origin and Disability Status Controlling for Promotion
  Announcement and Year: Regional Offices and Employees under ‘,‘I’
  40 only
Table 6: Relative Odds of Promotion by Gender, Race/National          23
  Origin And Disability Controlling for Promotion Announcement
  and Year: Regional Offices and Employees 40 and Older only
Table 7: Relative Odds of Promotion by Gender, Race/National          24
  Oi-igiri And Disability Status Contrdlling for Promotion
  Announcement and Year: Headquarters and Employees under 40
Table 8: Relative Odds, of Prdmotion by Gender, Race/National         24
  Origin and Disability Status ‘Controlling for Promotion
  Announcement and Year: Headquarters and Employees 40 and
  Older Only
Table 9: Relative Odds of Promotion by Age Group, Gender,             24
  Race/National Origin and Disability, Controlling for Promotion
  Announcement and Year: All Data Combined

Page   3

Chapter I

Background                                                                               :

                         The Board’s study and report on promotions at GAO focuses on the
Introductioti            majority of Banded,.employees: those holding evaluator, evaluator-related,
                         or specialist positions. Although attorneys at GAO are also Banded
                         employees, they are grouped differently than the evaluators and were not
            .‘,          included in the Board’s study.’

                          In 1987, the Personnel Appeals. Board (PAB or the Board) of the U.S.
I. History of the   ’1    General Accounting Office .(GAO or the Agency) published a report entitled
Promotiofi Study          EEO Oversight: Functional Study of GAO'S Career Ladder Promotional
                          Process, That study. reviewed and andlyzed career ladder promotions2 at
                          GAO from the beginning of fiscal year 1980 (October 1,198O) through the
                          end of fiscal year ,1985 (September 30,1985) by race, sex, and national
                          origin 3to determine whether there were significant differences
                          (1) between the rates at which members of protected groups were
                          promoted and (2) in the time members of protected groups spent in grade
                          prior to promotion. At the unit level, the 1987 report focused primarily on
                          evaluators; 4 agency-wide analyses were possible for evaluators,
                         :evaluator-related employeeq5 attorneys, writer-editors, and employees
                          grouped generally in an administrative category. Based on analysis of the
                          data, the Board reached two general conclusions: (1) there were no
                          significant differences inthe rates at which individuals in the protected
                          classes were being promoted6 and (2) black evaluators were spending

                         ‘For a description of the differences in the Bands, see the discussion in Section II of Chapter I,

                         “A career ladder refers to a job series that has one or more grade levels between the entry level and the
                         full performance level. Appointment to a career ladder position is competitive; subsequently, the
                         employee may proceed through the grades to full performance without further competition.

                         3Age and disability status were not included in the analysis.

                         “Only evaluators met the study’s criteria of a minimum of 15 promotion actions per group in the units.
                         GAO’s Oversight: Functional Study of GAO’s Career Ladder Promotional Process, p.4 (hereafter
                         cited as Career Ladder Promotions).

                         5Employees occupying evaluator-related positions provide technical assistance and support in the
                         audit function.

                         %n analysis of rates of promotion involves looking at whether members of protected groups were
                         promoted (or denied promotions) based on their membership in those groups and consistent with
                         their representation in the applicant pool. It also includes comparisons between and among members
                         of other groups.

                         Page   4
Chapter    I

“significantly~7 more time in grade than were white evaluators.
Specifically, the study concluded that during the time period of the study:

[TJhere were statistical disparities associated with race in evaluator career ladder
promotions. The disparities were particularly evident in cornpeons      between Black and
White evaluators.s
                                                                             :        .( .;
                                                                               _,     :_
Based on its analysis of the data and its conclusions, the Board made three
specific recommendations designed to correct disparities in the career
ladder promotion process. The Board recommended that the Agency:
(1) identify any artificial barriers or impediments that may be responsible
for disparities; (2) determine whether criteria used by units in making
promotions are approPriate; and, (3) consider developing a training course
on equal employment opportunity (eeo) for managers and supervisors
involved inthe.promotion process.g

In response to the draft report containing the Boards conclusions and
recommendations, the Agency reported that it was taking immediate
action to reduce the time-in-grade disparities in promotions revealed by
the Board’s study. The actions included establishing guidelines for
assessing individual performake’and     potential; setting time-in-grade
benchmarks; developing procedures for identifying and addressing the
developmental needs of emplo&s whose time-in-grade exceeded the
benchmarks; and developing an agency-wide database to monitor career
ladder promotions.1o

Prior to the issuance of the Board’s report, the Agency also created an
qffice of Affirmative Action Plans and implemented training programs on
equal employment opportunity and affirmative action responsibilities.
With the drafting of a new GAO Order on promotions, selecting officials

7A finding is stat&icaUysignificantwhenit canbe demonstrated         that the probability of obtaining that
finding purely by chance is relatively low. The generally accepted “probability threshold” is 5 percent,
i.e., the result tiould occur no more than 5 out of 100 times in a random sample with chance variations

*Ccweer Ladder Promo tiom, p. 5. The level of statistical significance for the time in grade that
Hispanic evaluators spent as opposed to white evaluators was .09 (or 91%). In this instance, the Board
reported this finding at the .09 level rather than the more commonly used .05 in order to call the
agency’s attention to a potential eeo problem. The study revealed no “significant” differences between
Asian and white evaluators.

9Zbid., p. 21.

loLetter from ha Goldstein, Assistant Comptroller General for Operations to Carl Moore, General
Counsel, PAB (August 20,1987)(hereafter cited as Goldstein Letter).

Page 5
                       Chapter    I                                                    :

                       w&e charged with furthering “GAO’S goal that minorities and women be
                       represented at the higher band levels of the work force.“”

II; Jurisdiction and

Jurisdiction           The GAO .Personnel Act of 1980 charges the Comptroller General with
                       maintainmg a personnel system that ,ensures that all appointments,
                       promotions and assignments are made solely on the bases of merit and
                       fitness. l2 That Act further directs the Board to exercise oversight authority
                       over equal employment opportunity at ~~0.1~ In furtherance of that
                       mandate and pursuant to its regulations, the Board reviews and evaluates
                       GAO’S regulations, procedures, and practices and may require GAO to make
                       changes it deems necessary.‘*

Methodology            In this study, the Board          set out to determine whether members of any
                       particular race, gender,          nation~~origin, disability15 or age group received
                       less favorable treatment           in the award of promotions at GAO from January 1,
                       1991 through December              31, 1995.16

                       lrGA0 Personnei Supplement 2335.8 SUP,‘ch. 151-4.

                       I231 USC. $732(b)(4).             e

                       13M. at 1751.

                       141d.at $732(f)(2)(A). See, applicable regulations at 4 C.F.R. $128.91 and 28.92. The original study,
                       resulting in the 1987 report EEO Oversight: Functional Study of GAO’s Career Ladder Promotional
                       Process, was conducted by the Board’s Office of General Counsel (pAa/OGC) and submitted to the
                       Board for review. It was shortly after the issuance of that report that the Board created a separate
                       Office of EEO Oversight to carry out its statutory mandate.

                       15At GAO, disability status depends entirely on self-reporting. When new employees first report for
                       duty, they are asked to complete GAO Form 164 “Self-Identification of Medical Disability.” During the
                       past 10 years, the percentage of the GAO workforce reporting a disability has hovered around rive
                       percent but this figure may not accurately reflect the actual population of persons with disabilities. ln
                       1996, GAO had 3,458 employees: 44 (1.27%) reported having a severe disability; 122 (3.53%) reported
                       having a non-severe disability. These are relatively small numbers from which to attempt to draw

                       ‘@Ihe Board chose to study promotions at GAO for the years 1991-1995 for two reasons: (1) to track
                       the earlier study 10 years later, and (2) to avoid coinciding, as much as possible, with the dates of the
                       recent freeze on promotions at GAO (May, 1995 through March, 1997).

                       Page   6
 Chapter I                                                    :

The Board is conducting this study as a follow-up to its 1987,study in
which the Board found that black evaluators waited a significantly longer
time to receive career ladder promotions than did white evaluators.

Soon after that study, the promotional scheme at GAO for evaluators,
evaluator-related employees and specialist was fundamentally changed by
the “broad banding” of pay rates.” Evaluators, specialists and most
attorneys at GAO no longer proceed by career ladder and/or competitive
promotions through the General schedule (GS).18 Rather, they are grouped
in three broad pay bands: Bsind I, e&ompassing the pay range from GS-7
through GS-12; Band II, encompassing the GS13 and 14 range; and Band
III, being equivalent to GS-15.1gWithin a pay band, employees may receive
pay increases related to performance without receiving a promotion. The
result of broad-banding is that there is now one non-competitive
promotion point (Band I-D to I-q20 and two competitive promotion points
within illl evaluator’s career (Band 1-F to Band II and Band II to Band III).

In this study, the Board examined the median time2’ to promotion at those
three points and rates of prom&ion for a five year period (1991-95) to
determine whether any statistically significant differences based on race,
gender, national origin, age, or disability can be discerned and whether the
prior racial disparities for time-in-grade persist under the new system.22

17Throughout the period of this study to the present, Banded employees have constituted
approximately 70% of the GAO workforce.
*@lie General Schedule is the pay schedule for most positions in the Federal Government. The
Schedule is divided into grades of difficulty and responsibility, and it ranges from GS-1 through GS-15.
An employee may progress up a career ladder without competition (e.g. GS-9 through GS-13) but after
reaching the top of the ladder, the next level involves a competitive promotion (e.g. to a GS-14). Within
each grade, there are ten rates of pay (steps). Step increases within grades are also awarded on a
non-competitive basis. 5 U.S.C. $5332.

‘%e banding scheme for attorneys in GAO’s Office of General Counsel differs from that of the rest of
the agency. Attorneys are generally grouped in two Bands that encompass grades 11-15. Bands I-D and
I-F are comparable to GS-11 through 14; Band II attorneys are comparable to GS-15s.

‘OEmployees at the I-D level are “certified” to I-F, after meeting certain minimum requirements, without
having to compete for the positions. See, discussion, supra p. 11.

“Mean, median, and mode are statistical ways to describe a central tendency or the point where the
population under study is centered. The mean is simply an arithmetical average of all of the values
(sum of the values divided by the number of the values); the median is the middle value; the mode is
the value that occurs most frequently within a set of variables. In other words, the median time to
promotion is the center of the range: half of the employees’ time to promotion fell above the center
number and half fell below.

sThe Board contracted with the Statistics Laboratory at the University of Maryland, Colle’ge Park, to
conduct the data analysis for this study and to prepare the tables and charts found in Parts I and II of
Chapter III.

Page 7

                         Chapter I                                    a;              _’
                         Background                                  :,

Methodology:             For this study, the Board looked atall promotions for Banded employees
Time-In-Band Analysis    during a five year period to determine the median time to promotion. The
                         effects of race, age, gender and disability status were factored in
                         separately at each promotion pointz3 The analysis of time-in-Band
                         accounted for the fact that the actual time in Band was only known for a
                         subset of employees. Because promotion histories for employees who
                         were previously at agencies other than GAO were not available, the analysis
                         of .the data did not consider promotion histories for GAO employees prior
                         to. January 1; 1991..For; those who were already in Band on January 1,
                         1991, or for those who were not promoted until after December 341995, a
                         minimum period of time in Band can be discerned. For example, an
                         employee hired on July 1,1995, into Band I-D and not yet promoted as of
                         December 341995, was in Band.for at least 6 months. 24

                         Standard techniques for this type of data where the entire promotion
                         history is not known were developed to analyze an employee’s known
                         history of promotions during a particular time period. The techniques also
                         allow for computation of median time in grade as the time when 50% of the
                         employees have already been promoted. It is also possible to test whether
                         two or more groups have comparable distributions of time to promotion.

Methodology: Promotion   The Board compared the promotion rates of males and females, by age,
Rate Analysis            and by race, national origin, and disability status, after adjusting for the
                         composition of the “Best-Qualified” (BQ) lists for each promotion
                         competition. For this part of the aualysis, all employees were pooled and
                         then separated (disaggregated) by age, by regional office versus
                         headquarters, and by an age-region/headquarters combination. Only those
                         employees who applied for promotions and made the BQ lists are part of
                         this analysis.25

                         %e two Board promotion studies differ in the methodology for calculating time-ln-grade/Band. The
                         1987 study compared time-in-grade for all promotions within each career ladder within each unit
                         Career ladders were also grouped on an agency-wide basis and the overall time-in-grade for each
                         career ladder was measured by race, by gender, and by race/gender combinations. For that study,
                         time-in-grade was standardized, i.e. the individual time-in-grade minus the mean time-in-grade of the
                         subgroup divided by the standard deviation of the subgroup produced a standardized score. This
                         allowed for the many different promotion criteria that were present due to the number of different
                         grades and career ladders involved. The earlier report contains no discussion of the methodology used
                         to .determine rates of promotion.

                         %uch data are called right-censored. The techniques used in this report for censored data were
                         developed to analyze lifetime data

                         %&, Part II of Chapter II for a discussion of how BQ lists are compiled.

                         Page 8
                                   Chapter I                                      :

                     Y”,            To analyze the’data on promotion rates, it was necessary to account for
                                   the fact that the BQ lists for various promotion competitions may have
                                   varied in their age, gender, race, national origin, or disability status mixes
                                   and promotion rates may have’ varied from one competition to another.
                                   If one aggregates the simple numbers on all of the BQ lists, without taking
                                   into account the different race, gender, age, national origin, and disability
                                   status of those who compose each BQ list, the result would be promotion
                                   rates that ‘do not reflect the true rates of promotion for the various groups.
                                   For this reason, comparisons of promotion rates were adjusted or
                                   controlled for the varying compositions of the BQ lists. This “adjustment”
                                   was based on a standard statistical technique that allows for sampling
                                   error, called the ManteliHaenszel? statistic. For example, a given BQ list
                                   could be comprised mostly of males or have twice as many persons 40 and
                                  ‘over as compared to,‘Fersonsunder the age of 40. Using the technique in
                                  this study, the ‘analysts were able to compare the odds of promotion
                                  independent of the variances or different numbers within groups in the
                                  composition of each BQ list.
                                  The following hypothetical data illustrate the problem for which the
                                  technique adjusts:

Competition 1 (30 Vacancies)
                                                                                        Male              Female
                                  Promoted                                                10                  20
                                  Not Promoted                                            90                 180
                                  Total                                                  1ocJ               200

Competition 2 (15 Vacancies)
                                                                                        Male              Female
                                  Promoted                                                10                   5
                                  Not Promoted                                           190                  95
                                  Total                                                  200                 100

Combined Results (45 Vacancies)
                                                                                        Male              Female
                                  Promoted                                               20                  25
                                  Not Promoted                                           280                275
                                  Total                                                  300                300

                                  Page 9
     Chapter    I


     The first competition has a 10% overall promotion rate; the second
      competition is the more difficult with a 5.0% overall promotion rate. In the
     frrst competition, there were 30 vacancies: 10 men out of 100 were
     promoted, as.were 20 women out of 200. In the second competition, there
     were only 15 vacancies: 10 men were promoted out of 200 who applied,
     but there were only 100 women in the pool, 5 of whom were promoted.
     The combined numbers, however, show that, overall, 20 men of 300 were
     promoted (6.667% promotion rate) and 25.women of 300 were promoted
     (8.333% promotion rate). The combined numbers, standing alone,
     incorrectly suggest discrimination in favor of females.26 The numbers do
     not account for the fact that there were more males in the second and
     harder competition in which the same number of people (300) was
     competing for half as many promotions (15 versus 30). A correct analysis
     using the Mantel-Haenszel technique accounts or adjusts for the
     differences in both the promotion rates and the differing male-female mix
     in the preceding hypothetical.

     ‘GThe corresponding relative odds are 0.786: 20 males promoted/280 males not promoted versus 25
     females promoted/ 275 females not promoted.

     Page   10
Chapter Il
The Promotion Process

                 Generally, new evaluators and evaluator-related employees at GAO are
I.Bands      ”   hired into Band I and assigned to the developmental level (Band I-D). At
                 that level, employees are expected to become familiar with the policies
                 and procedures associated with the evaluative work component of GAO.
                 Typically, they are assigned to gather and analyze data, conduct research
                 and interviews, and write segments of audit plans and GAO reports.27

                 Employees are “certified” to the full performance level of Band I (Band
                 I-F). Unlike promotions to Band II and Band III, certification to full
                 performance (I-D to&F) does not occur at a specified time of the year, but
                 rather may take place whenever the unit head concludes that it is merited
                 by the employee’s performance and the employee meets certain minimum
                 requirements.28 Certification will normally result from the
                 recommendation of the progress review group, which includes the
                 Director for Operations, or Deputy Regional Manager, the Human
                 Resources Manager, the supervisor and/or Assistant Director most
                 knowledgeable about the employee’s recent performance.

                 At the I-F level, employees are expected to perform the full range of
                 evaluator functions. These include developing job plans, taking the lead in
                 data collection efforts, selecting and applying the analytical method
                 appropriate to a given situation, drafting chapters of GAO reports, and
                 leading meetings with GAO officials to communicate the results of the
                 work. Staff at this level are expected to perform all tasks with decreasing
                 levels of supervision.2g

                 Promotions from Band I-F to Band II and from Band II to Band III are
                 competitive. Band II evaluators are expected to develop, evaluate, and
                 review data collection efforts; to review and revise written products and
                 consolidate them into reports; to be involved inthe planning function; and
                 to ensure the completion of report processing. Their work products are
                 presumed to be technically complete and are reviewed only for
                 conformance to GAO policy. Band III evaluators initiate project proposals

                 “‘Performance Appraisal System for Band I, II, and III Employees, Appendix VIII, p. 95
                 (October 1997) (hereafter cited as Appendix VIII).

                 %These requirements include that the employee receive a six month progress review and that the
                 employee serve at least 12 months in an evaluator or evaluator-related position. Prior service at GAO,
                 at another federal agency or outside the federal government is creditable toward the 12-month
                 requirement under certain circumstances. &, GAO Order 2540.1, Ch. 3, $1(b)(3).

                 &Appendix VIII at 9697.

                 Page 11
Chapter II                                                   :
The Promotion     Process

and-direct their implementation; manage and supervise employees; and
work under. very general guidance from a superior.3o

As discussed in Section II of this chapter, promotions to the Band II or
Band III levels normally occur as part of an annual assessment cycle, with
all applications, selections and promotions being made at approximately
the same time, agency-wide.

For informational purposes, the following charts show the profile, by
gender, race, and national origin, of evaluators, evaluator-related
employees and specialists in Bands at GAO in 1994.31

30Appendix VIII at 97-100.

31At the same time, employees 40 and over constituted 31% of Band I; 72.5% of Band II; and 90.8% of
Band III. Employees claiming a disability constituted 6.4% of Band I; 4.5% of Band Q and, 4.1% of Band

Page 12
 Chapter II                                                      :
 The Promotion        Process

 100   Banded Employees          by Gender   (percentages)











100    Band I (by rac+aiional,origi,n)       percentage?     ,

 90       ..





Page 13

                      Chapter     11                                              :
                      The-Promotion    Process

II*   competi&e      The promotion process begins with an annual “needs determination” in
                     which each office and division submits a proposal to the Assistant
Promotions (Banded   Comptroller General for Operations (ACG/OPS) stating the number of
Employees) at GAO    evaluator, evaluator-related, and specialist positions that it would like to
                     fiu at each Band level and includes a brief justification of the need for
                     these positions.32

                     The Needs Determination Committee, consisting of senior management
                     officials working under the aegis of the ACG/OPS, considers the proposals
                     and may also identify positions that may be filled by reassignment rather
                     than promotion.33 After a decision has been made on the positions to be
                     filled, information about eligibility for promotions, paperwork
                     requirements, application procedures and deadlines is set out in a special
                     supplement to the GAO Management News. A second supplement is then
                     published containing ComIjrehensive job opportunity announcements
                     listing the numbers, levels, locations, and types of positions to be filled.
                     Some vacancies are only open to employees within the division or unit
                     where the vacancy occurs; others are announced GAO-wide at
                     management’s discretion.

                     An employee wishing to be considered for one of the Band II or Band III
                     vacancies must tile an application. All applicants must have at least 52
                     weeks in Band at their current level by the effective date of the promotion.
                     Applicants for evaluator-related-positions  must also meet selective
                     placement factors ;and applicants for specialist positions must meet
                     government-wide requirements for those positions, in addition to selective
                     placement factors.    ’

                     To apply for a promotion, an employee must submit an application for
                     consideration; an employee profile which demonstrates that the employee
                     has the requisite knowledge, skills, and ability (KSA) to perform at the

                     ?his section describes the current promotion process. The process has changed substantively very
                     little since 1991,the first year that the Board is studying.
                     33Currently,the Committee consists of the Assistant Comptroller General for Operations, the Assistant
                     Comptroller General for Planning and Reporting, and the Deputy Assistant Comptroller General for
                     Human Resources.

                     Page   15
    Chapter II
    The Promotion    Process


    higher Band34 a-statement of contributions and accomplishments; 35and
    performance appraisals for the current and preceding,two.years.36

    Once the applications ‘are received for a promotion in a particular unit, a
    promotion pane1,37selected by the unit head, is typically convened to
    review the applications and,prepare a ranked list of applicants. (A
    promotion panel is not required if fewer than 10 employees request
    assessment for promotion in that unit.) The panel must include three unit
    employees, all of whom are at least ‘one Band higher than the employees
    who are being assessed. It may not include the selecting official.
    The promotion process at GAO is a relative ranking system. Candidates are
    compared to others in their group, and not against established
    benchmarks. Comparisons are based on performance, experience, and to a
    lesser extent, education, training, awards, and professional development
    that demonstrate important knowledge, skills and abilities at the next
    Band level.

    After the panel has prepared a ranked list of applicants, the chair of the
    panel decides how many employees to refer as “Best-Qualified” (BQ). The
    panel chair must follow the rank order established by the panel, but he or
    she has discretion as to how many. candidates to. refer and where to draw
    the cut-off ‘line. When drawing a cut-off, panel chairs are cautioned to
    consider factors such as natural breaks in scoring, as well as the number
    of opportunities available within the unit. During the time period of this
    study, the chair was permitted toconsider affirmative action goals when
    deciding how many names to forward. The agency’s current affirmative       ’
    action program, which covers hiring, promotions, separations, and
    training, places much of the responsibility for the success of the program
    on the unit managers. Specifically, they are charged with:

    (1) when requestedby ACOOPS, conducting appropriate barrier analysesregardinghiring,
    promotions, training, and separations,to determinewhy disparitiesexist in the unit and if

    %valuator KSAs are found in Appendix 2 to GAO Order 2335.8. They are listed for the following areas:
    planning; data gathering and documentation; data analysis; written communication; oral
    communication; workiug relationships, teamwork, and equal opportunity; and, supervision, appraisal,
    and counseling.

    %prior to 1994, employees submitted a Contribution   Statement It is no longer a requirement but still
    may be submitted.

    “Band I employees applying for Band II positions submit their Band I-F appraisals only; I-D appraisals
    are not considered. GAO Order 2335.8 SUP, Appendix 1, I-l[a-51.
    3TFormerly known as a management review panel.
    Page 16
      Chapter II
      The Promotion      Process

      such dispariti& cannot be ekplainedon the basisof merit factors, developinga plan and
      taki$ stepsto correct any identified problems. . .%

      Unit managers are also held responsible for evaluating promotions on an
      on-going basis “to ensure that all employees are treated in an equitable
      manner.” The performance,of unit managers is evaluated on the basis of
      their equal employment opportunity efforts and results.39

      All employees designated “Best-Qualified” are automatictiy considered
      ‘for any vacancies that occur in the same occupational series in their home
      unit. Any BQ candidate may also apply for any other vacancies where the
      area of consideration is “GAO-Wide”, i.e. to ah qualified employees of GAO.                                         7
      However; even employees who are not designated BQ in their home unit
      may apply for GAowide vacancies for specialist positions if they meet the
      The selecting official is presented with the BQ list containing the names
      listed in alphabetical order. Banks are not indicated on the BQ list. The
;-.   selecting official may select any ‘candidate on the BQ list, or may make no
      selection at all. The selecting official may interview candidates prior to
      selection, but must interview all BQ candidates if any are interviewed.

      If the’selecting official does determine that interviews are necessary but
      the number of internal candidates on the BQ list is too large to allow for
      interviewing, he or she may convene a panel to winnow the list.4o
      Winnowing panels use the same process as is used to develop the original
      BQ list. Again, if the panel conducts interviews, everyone on the BQ list
      must be interviewed; Employees may request feedback about the
      promotion process. They may learn their BQ status, as well as their
      ranking, total score and distance between their score and the bottom
      score among the B&s. Merit selection files containing documentation of
      the qualilication, evaluation, and selection portions of the process must be
      maintained by the unit for three years.41

      38Affiumative Action Program, U.S. General Accounting Office (1998), p.5. For purposes of analyzing
      promotion data to determine the existence of disparities in rates, the agency’s benchmarks are based
      on appropriate civilian labor force data as well as data on the current population of employees eligible
      for promotion.

      3%id. ACGQS    conducts statistical analysis on an agency-wide basis by race/ethnicity or gender to
      determine whether there are statistically significant disparities. If statistical disparities exist, ACG/O~S
      will work with unit management to correct any problems that are not merit based.

      4”The Director of Operations and the Director of Planning and Reporting in each unit constitute the
      panel for winnowing purposes.

      4’GAO Order 2335.8, ch. 3(l).

      Page 17
    Chapter III


                                 As noted previously;the Board’s 1987 study of promotions at GAO found
    I. ‘Time-In-Band             that black evaluators spent significantly more time in grade than did white
                                 evaluators. Hispanic evaluators also spent more ,time in grade than white
                                 evaluators but not at a level determined to be statistically significant. The
                                ,study found no significant differences based on gender alone (male versus
                                 female) but when comparing race and gender simultaneously, the same
                                 time-in-grade patterns prevailed (i.e., black females spent more time in
          I                      grade than white females).

                                In this analysis, the distributions of time-in-Band for Bands I-D, I-F, II, III,
                                were examined to determine the median time to promotion. The effects of
                                race/national origin, age, gender and disability status were factored in
                                separately at each of the three promotion points.

    From Band I-D to Band I-F   At this non-competitive point, where promotion rests completely on the
                                unit head’s determination that an employee has moved from the
                                developmental level to full performance, the median time of promotion,
                                overall, for white employees was faster than that of black, Asian, or
~                               Hispanic employees. White employees spent a median time of 490 days in
                                Band I-D prior to promotion; black employees spent a median of 546 days;
                                Asian employees spent a median time of 560 days; and Hispanic employees
                                spent a median of 574 days. There was no difference by gender.
                                Employees without disabilities spent a median time of 518 days in Band
                                I-D; employees with disabilities spent a median time of 504 days. The
                                largest gap was by age, with employees under 40 spending a median of 518
                                days in Band I-D compared with a median of 420 days for employees 40
                                and over.

                                          Chapter     III

Median Time (in Days)30     Promotion
From I-D to I-F
                                         By Race/National     Origin
                                         Black                                                                       546
                                         White                                                                       490
                                         Asian                                                                       560
                                         Hispanic                                                                    574
                                         By Gender
                                         Female                                                                      518
                                         Male                                                                        518
                                         Bv he
                                         -.     -I-

                                        Under 40                                                                     518
                                        40 and pver                                                                  420
                                        BvI Disabilitv . Status
                                        Yes                                                   :                      504
                                         .-                                                                          518

From Band I-F to Band II                The ‘first part of this analysis shows the median time to promotion for
                                        employees who were under the age of 40. The figures for employees 40
                                        and over are not shown because they exceeded five years in all
                                        demographic groups-beyond          the five year period encompassed by the
                                        Board’s study. The second part of the analysis shows the median time to
                                        promotion for all Banded employees regardless of age.

                                        Of the employees under 40, white, Asian, and Hispanic employees spent a
                                        median number of 1,526 days in Band I-F; black employees spent a median
                                        of more than five years. Employees under 40 with disabilities spent a
                                        median of 1,806 days in Band 1-F; employees under 40 iyithout disabilities
                                        spent a median of 1,526 days.

                                        Males and females under 40 spent the same median time in Band I-F prior
                                        to promotion. However, when all age groups were combined, females
                                        fared noticeably better than males; the median time to promotion was
                                        approximately four and a half years for females and more than five years
                                        for males.

                                        The median time for all white, Asian, and Hispanic employees regardless
                                        of age was under five years; the median time to promotion for all black
                                        employees at this promotion point was more than five years.

                                        Page    19
                                          Chapter     III

Median Time (in Days) to Promotion   -*                                                                                      .A
From I-F to II                                                                          Under 40                     .AII   !
                                          By Race/National Origin
                                          White                                            1,526                1,806
                                          Black                                         >5 years             X5 years
                                          A&an                                             1,526                1,582       ~
                                          Hispanic                                         1’,526               1,806
                                          By Gender
                                          Female                                           1,526                1,638       i
                                          Male                                             1,526             >5 years
                                          By Disability Status
                                          Yes                                              1,806             >5 years
                                          No                                               1,526                1,806

From Band II to Band III              No disparities based on race, national origin, gender, age, or disability in
                                      time-in-Band were discerned at this promotion point. Due to the small
                                      number of promotion opportunities available, most of the Band II
                                      population never received any promotions during the course of,the
                                      Board’s study. On the average, the Band II population constitutes about
                                      two-thirds of Banded employees (1,746 after three promotion cycles);
                                      Band III’s, were 16 percent (436) of the Banded employees at the same
                                      point.                ,..,

                                      The other prong of the 1987 Board study focused on rates of promotion.
II. Promotion Rates,                  The 1987 study found nonsignificant differences based on race, sex or
Adjusted for.                         national origin in the rates at which employees in those groups were
Composition of                        promoted.
“Best-Qualified” Lists                In this analysis, the promotion rates of males and females, under 40 and 40
                                      and over, by race, national origin, and disability, after adjusting for the
                                      varying compositions of the BQ lists for all of the competitive promotions,
                                      were compared.‘This analysis was performed from several perspectives:
                                      separating by age group (under 40 and 40 and over), by regional office
                                      versus headquarters, by an age-region/headquarters combination and by
                                      pooling all categories of employees.

                                      Tables 1 through 9 provide the “relative odds” for promotion by age, by
                                      regional office versus headquarters, by an age-region/headquarters
                                      combination and by pooling all categories of employees. Relative odds

                                      Page       20
                            Chapter    III


                            reflect the likelihood of being promoted between two groups under
                            comparison, i.e. the relative success rate. In each odds ratio in each table,
                            there are two groups being compared to each other (e.g., male/female;
                            under 40/40 and over). Table II, for example, shows that the odds of
                            promotion for men 40 and over is only 51 percent as likely as that for
                            females 40 and over.

                            Application of the previously discussed Mantel-Haenszel statistical
                            technique also allows for the production of confidence limits. When two or
                            more concepts are believed to be related, the relationship is confirmed
                            with a “degree of confidence;” In the following tables, the confidence
                            limits are reported at a 95 percent rate, i.e., the analyst is certain that the
                            limits constructed will bracket the finding within 5 percent or, in some
                            instances, a 99 percent rate, i.e. the limits will bracket the finding within
                             1 percent. In other words, the 95 and 99 percent confidence hmiti
                            expressed in these tables represent the upper and lower boundaries or
                            range of values. The analyst is 95 or 99 percent confident that v&Gn the
                            interval (the range from lower to upper limit) lies the true mean of the

                            A single asterisk in the Tables indicates a significant difference at the 0.05
                            level. This odds ratio. is statistically significant at the level of 95 percent
                            confidence, limits. A doubled asterisk indicates a significant difference at
                            the 0.01 level. This odds ratio is statistically significant at the level of
                            99 percent confidence limits. For the purposes of this report, any odds
                            ratio without an asterisk is not considered statistically significant.

Tables 1 and 2: Employees   Tables 1 and 2 are based on separate analyses, disaggregating the
Under 40 & 40 and Over      promotion candidates on the basis of age. Among the younger employees,
                            there are no differences due to gender, race/nation origin or disability. By
                            contrast, among the older employees, the odds ofpromotion are only half
                            as good for males as for females. There are no statistically significant
                            differences due to race/national origin or disability.
                                        Chapter     III

by Gender, Race/National Origin                                                                       95% Confidence
(White v. Minoriiy)@‘and Disability,    Group                                         Odds Ratio                Limits
Controlling for Promotion               Gender                  Mate                       0.904                 0.734
Announcement      and Year: Employees                           v. Female                                        1.112
Under40 Only            ‘.              Race/Nat’1 origin       White                      0.812                 0.640
                                                                v. Minority                                      1.029
                                        Disability Status       Yes                         1.076                0.541
                                                                v. No                                            2.138
                                                                     : .
                                        4oFor this report, the term “minority” includes black, Hispanic, and Asian

Table 2: Relative Odds of Promotion
by Gender, Race/National Origin                                                                       95% Confidence
(White v. Minority) and Disability      Grou,p                                        Odds Ratio                Limits
Controlling for Promotion               Gender                  Male                       0.508**               0.378
Announcement and Year: Employees                                v. Female                                        0.682
40 and Older Only                       Race/Nat’1 Origin       White                      0.982                 0.655
                                                                v. Minority                                      1.473
                                        Disability status       Yes                         1.629                0.797
                                                                v. No                                            3.328

Tables 3 and 4: Regions v.              Tables 3 and 4 are based on separate analyses, disaggregating promotions
Headquarters                            in regional offices and in Headquarters. In both sets of promotion
                                        competitions, there was a-disparity in favor of yormger employees when
                                        compared to older employees and in favor of females when compared to
                                        males. Moreover, there is evidence that whites were less likely to be
                                        promoted than minority employees in the regional offices. Disability status
                                        had no effect on promotions.

Table 3: Relative Odds of.Promotion
by Age Group, Gender, Race/National       ..                                                          95% Confidence
Origin (White v. Minority) and          Group                                         Odds Ratio                Limits
Disability, Controlling for Promotion   Age                     40 and Older               0.647**               0.483
Announcement      and Year: Regional                            v. Under 40                                      0.866
Offices Only                            Gender                  Male                       0.682**               0.517
                                                                v. Female                                        0.899
                                        Race/Nat’1 Origin       White                      0.706*                0.505
                                                                v. Minority                                      0.987
                                        Disability Status       Yes                        0.681                 0.272
                                                                v. No                                            1.700

                                        Page   22
                                         Chapter    III

Table 4: l?elative,Cdds of Promotion.
by Age Group, Gender, Race/National                                                                    95% Confidence
Origin (White v. Minority) and           Group                                       Odds Ration                 Limits
Disability Controlling for Promotion     Age                    40 and Older               0.552**                0.449
Announcement and Year:                                          v. Under 40                                       0;678
Headquarters Only                        Gender                 Male                        0.660**               0.539
                                                                v. Female                                         0.807
                                         Race/Nat’1 Oriain      White                       0.828                 0.651
                                                                v. Minority                                       1.054
                                         Disability status      Yes                         1.509                 0.873
                                                                v.No:.      I.                                    2:607

Tables 5 Through 8: Age                 Tables 5 through 8 present separate analyses, disaggregating both on the
Plus Region/Headguarters                basis of Age and re&onIheadq~arters. The re+xlts reveal .no statistically
                                        $#ficant differences due to rake/national origin or &i&b&       status in any
                                        of the four subsets of candidates. There were no gender differences among
                                        younger candidates nor among candidates in regional offices. Among older
                                        candidates in headquarters, males had a smaller statistically signilkant
                                        chance of promotion than females.

Table 5: Relative Odds of Promotion
by Gender, Race/National Origin                                                                       95% Confidence
(White v. Minority) and Disability      Group                                         Odds Ratio                Limits
Status Controlling for Promotion        Gender                  Male                       0.871                 0.615
Announcement      and Year: Regional                            v. Female                                        1.235
Offices and Employees Under 40 Only     Race/Nat’1 Origin       White                      0.708                 0.478
                                                                v. Minority                                      1.049
                                        Disability status       Yes                        0.439                 0.060
                                                                v. No                                            3.230

Table 6: Relative Odds of Promotion
by Gender, Race/National Origin                                                                       95% Confidence
(White v. Minority) and Disability      Group                                         Odds Ratio                Limits
Controlling for Promotion               Gender                 Male                        0.549                 0.292
Announcement      and Year: Regional                           v. Female                                         1.030
Offices and Employees 40 and Older      Race/Nat’1 Oriain      White                       1.360                 0.545
Only                                                           v. Minority                                       3.396
                                        Disability Status      Yes                         1.152                 0.338
                                                               v. No                                             3.930

                                        Page 23
                                        Chapter    III

by Gender, Race/National Origin                                                                        95% Confidence
(White v. Minority) and Disability      Group                                          Odds Ratio                 Limit
Status Controlling for Promotion        Gender                  Male                        0.922,                0.712
Announcement and Year:                                          v. Female                                         1.194
Headquarters and Employees Under 40                                                                                 I.
                                        Race/Nat’1 Origin       White                       0.878      ”          0.652
Only                                                            v. Minority                                       1.181
                                        Disability Status       Yes                         1.289                 0.619
                                                                v. No                                             2.685

Table 8: Relative Odds of Promotion
by Gender, Race/National Origin                                                                        95% Confidence
(White v. Minority) and Disability      Group                                         Odds Ratio                 Limits
Status Controlling for Promotion        Gender                  Male                       0.495**                0.355
Announcement and Year:                                          v. Female                                         0.691
Headquarters and Employees 40 and       Race/Nat’1 Origin       White                       0.893                 0.568
Older Only                                                      v. Minority                                       1.403
                                        Disability Status   j   Yes                         2.040                 0.845
                                                                v. No                                             4.923

Table 9 - AU Data                       Table 9 compares the promotion rates of various groups, after adjusting
Corhbined-                              for variations in the composition of the BQ list and for variations in the
                                        overall promotion rates in each of the competitions.

                                        The table reflects a disparity in favor of younger employees compared to
                                        older employees; a disparity in favor of females over males; and a disparity
                                        in favor of minority employees versus white employees. There are no
                                        si,@ificant differences in promotion rates between persons with
                                        disabilities and persons without disabilities.

Table 9: Relative Odds of Promotion
by Age Group, Gender, Race/National                                                                    95% Confidence
Origin (White v. Minority) and          Group                                         Odds Ratio                 Limits
Disability, Controlling for Promotion   Age                     40 and Older               O-582*”                0.492
Announcement and Year: All Data                                 v. Under 40                                       0.688
Combined                                Gender                  Male                        0.667**               0.567
                                                                v. Female                                         0.785
                                        Race/Nat? Origin        White                       0.784*                0.644
                                                                v. Minority                                       0.953
                                        Disability Status       Yes                         1.175                 0.736
                                                                v. No                                             1.875

                                        Page 24
Chapter IV

Conclusions and Recommendations

                   .‘&      The Board’s 1987 study analyzed noncompetitive promotions that
                            occurred at GAO during the 1980-1985 fiscal years. The current study
                            analyzes promotions that took place between January 1,199l and
                         ‘. December 31,1995. During the time between the two studies, GAO
                            revamped its pay and grade. structure for evaluators, for those holding
                            evaluator-related positions, and for most attorneys. Qne of the Board’s
                         y goals for the current study was to determine whether the disljarity in
                            time-in-grade between white and black evaluators, revealed in the
                            previous analysis of non-competitive promotions, persisted in the new
                            system.       .’
                          Between 1991 and 1995, white employees were promoted from Band I-D to
                          Band I-F (a non-competitive promotion) at a median time of 56 days faster
                          than black employees. White employees were also promoted a median of
                          76 days faster than Asian employees and a median of 84 days faster than
                          Hispanic employees.

                          From Band I-F to Band II (a competitive promotion), white, Asian and
                          Hispanic employees under 40 spent a median of 1,526 days in Band prior
                          to promotion compared to a median of more than five years for black
                          employees under 40. The median time to promotion for all black
                          employees at the Band I-F promotion point was also more than five years.
                          When all age groups were combined, females spent a median of 1,638 days
                          in Band; males spent more than five years. The median time to promotion
                          for all employees 40 and over was more than five years.

                          No differences in time-in-Band were discerned at the second competitive
                          promotion point (Band II to Band Ill).

                          It appears that the time-in-grade disparity revealed between black and
                          white evaluators persisted into the Band system through the 1995
                          promotion cycle. This disparity was more pronounced at the
                          non-competitive promotion point (Band I-D to Band I-F) and was also
                          evident at the first competitive promotion point (Band I-F to Band II).
                          Males were also promoted more slowly than females at the second
                          promotion point.

                          The second prong of the Board’s study concerned rates of promotions. In
                          the earlier study, the Board found no disparities based on race, sex, or
                          national origin in rates of promotion at GAO from 1980 through 1985. The
                          most recent analysis, however, reveals disparities in rates of promotion at
                          GAO during the five years studied. At headquarters, employees under 40, in

                          Page   25
! ‘:
Appendix I

Agency Comments


                                   Assistant Comptroller General
                                   of the United States
                                   Washington,   D.C. 20648

                                   June l&l999
                                   Ms.Gall Gerebenics
                                   8201’ Street,N.E.,Suite560
                                   Washington,DC. 20548
                                   Thisis in responseto your March12,1999letter submittinga draft report from the
                                   PersonnelAppealsBoard(PAB) on Promotionsof BandedEmployeesfrom 1991-
                                   1995.Our observationsandgeneralcommentsare asfollows. Attachment 1contains
                                   detailedcommentsandsuggestions  to improvethe accuracyand clarity of the report
                                   ThePAB draft fast summsrlzes   the findings of a 1987PAB report on careerladder
                                   promotionsfrom 1980through1985.It then examinesall promotions for banded
                                   employeesfor the timeperiod1991-1995      to determinethe mediantime for promotion
                                   andthe promotionrates of males andfemalesby age,race, national origin, and
                                   disabilitystatus.Webelievethat the methodof analysesand conclusionsin the
                                   report with respectto time-in-bandandpromotionratesare fiawed in several

                                   With regardto mediantimefor promotion,the PAB iirst looks at the time for
                                   certificationfrom BandsID to Il?. It reportsthat white employeeshada shorter
                                   mediantime than did African Americans,Asian,or Hispanicemployees;employees
                                   with disabilitieshad‘ashortermediantime than non-disabledemployees;and
                                   employeesover age40hada shortermediantimethan employeesunderage40.
                                   PromBandlF’to BandII, the PAB finds that African American employeeshad a
                                   longermedianthee for promotionthan did white,Asian or Hispanicemployees.It
                                   alsoreachesa similarconclusionwhenageis factored into the race/nationalorigin
                                   analysis,with respectto African Americanemployeesunder40. According to the
                                   analysisthere wereno disparitieswith respectto the mediantime for promotionfrom
                                   BandsII to lII.

                                   In performingthe analysisof time-in-bandfrom BandID to IF, the PAB appearsto
                                   havegroupedall Bar&ID employeestogether,potentially biasingthe results. Band
                                   ID employeesarehired at different pay ratesbasedon their qualifications,which
                                   includeexperienceandeducation.Within the ID level there arethree qualification

                            Page   28
            Appendix I                                                                              :
            Agenby Comments


     pay levels,roughly equivalentto GS-7,GS9,and GS11paylevelsin the Executive
     Branch. For example,a candidatewith a master’sdegreeor equivalentwithout work
     experiencecanbehired at the GS-9paylevelwhile a candidatewith a Ph.D.andno
     work experiencecanbe luredat the GSll paylevel, Alternatively, a personwith a
     bachelor’sdegreeandno work experiencecanbe hir&at the GS-7level and a person
     with z&bachelor’s degreeandqualifyingwork experiencecanqualifyfor appointment
     at the GSll. Clearlythosestaff who start at the.higherpaylevelswithin BandID
     have a greaterlikelihoodof promotionto BandIF soonerthanstaff who start at a
     lower level. Failureto considerthis differencein hiring levelsoverlooksa slgnliicant
     factor that shouldbe includedin this analysis.
     Furthermore, althoughthe summaryof time-in-band     analysesnotesdifferences
     amongracial groupsfor BandsID to IF certiilcationsandBandsIF to II promotions,
     the report doesnot statewhetherthesedifferencesriseto the level of statistical
     significance.’Significancelevelsshouldbepresentedin orderto allowthe readerto
    judge the severity of anyreporteddifference. Indeed,it-isgenerallyrecognizedthat
    without statisticallysignificantdisparitieswhat actually happened in a decision
     making processcould reasonablybe attributedto randomvariationor chancewith
    respectto a protectedgroup. Of courseevenif statisticallysignificantdisparities
    exist, this doesnot necesarily leadto a findingof discriminationasthere may be
    merit basedexplanationsfor the results.
    The PAB notesthat its study isa follow-upto its 1987studyin whichit found that
    African Amermanevaluatorswaiteda significantlylongertimeto receivecareer
    ladder promotionsthan did white evaluators.However,the report fallsto note the
    signiBcantimprovementAfrican Americanshavemadein the lengthof time it takes
    to move from Bands ID to IF. Basedon the 1987PAB report,on average,anAfrican
    American wouldtake 155dayslongerthanwhitesto movefrom GS7to GS-12and
    195dayslongerto move from GS-9to GS-12.The currentstudy statesthat it takes
    Afi-lcan Americans56 dayslonger,on average;to movefromID to IF -the equivalent
    of promotion from GS7 to 6512. Thisrepresentsan improvementof 99days(or
    64%)in the GS7 to GS12 categoryand49days(or 4736)in the GS9to GS12 category.

    Moreover, there appearsto belittle demonstrable   differenceamongAfrican
    Americans,Hispanicsandwhites,with respectto median thnefor promotionsfrom
    Bands IF to II. This fact, how.ever,is maskedby the combinationof daysandyearsin
    the related tables. The report statesthat themediantimefor promotionfrom Bands
    IF to II for African Americansis “morethan5 years,”while the timefor whitesand
    Hispanicsis “1,806days”. However,1,806daysis over 4.9years.

     % problem cased by the lack of data on signiiicance is compounded by the wording ued in ch. 3 and again in the concksion,
    where ?he rep-art states that no stadstically SigniGtit disparities were found al the Rands II to 111level. This tends to imply that
    results in the preceding discussions regarding tin&n-grade      from Bands ID to IF, and iF to II, were significant.

             Appendix    I                                          :
             Agency   Comments


     With regard to the analyses of promotion rates, the PAB finds a disparity in
     promotion rstes in favor of younger employees compared to older employees, in
     favor of females over males, and in favor of mlnorltiesover white employees. In
     reaching this conclusion, the PAB used the Mantel-Haenssel test and aggregated the
    .promotion data for all 5 years in question, .1991-1995,‘andfor sll band levels. We
     believe this aggregation is inappropriate, and results in a misleading representation of
     the condition.

    In order to accurately reflect the selectiorrprocess when applying the Mantel-
    Haenszel test, each of the years and band levels should have been treated as
    independent decision processes and not have been aggregated. One of the
    assumptions underlying the use of thls test is that each observation, in this case
    selection from BQ list; is independent. While the existence of multiple applications by
    the same employee is not a significant problem when the data is examined year-by-
    year (as most employees do not apply multiple times in the same year), when cycles
    are combined for several years,,there clearly are multiple applications by the Same
    employee. Indeed, for the five promotion cycles in the years 1991-1995,some
    employees were on over 20 BQ lists over the course of the 5 years. One employee
    was on 28 such.lists. Multiple applications across several promotion cycles from
    many staff can distort the statistical analysis. This distortion can largely be overcome
    by analyzing each cycle separately.
    When we performed the Mantel-Haenszel analyses of promotion rates by individual
    year and by individual band level, we found no statistically sign&ant dlsparltks in
    any cycle from BQ to selection from, 1991-1995,with two exceptions. There was a
    statistically significant disparity in‘favor of women over men in 1991 at the Band ILI
    level and in 1993 at the Band II level. We note, however, that these disparities
    occurred,in only 2 of the’10 sets of data analyzed. (Each analysis consisted of a
    1 year period for the Band II or Band LKLlevelsfrom 1991-1995.) Moreover, our data
    for the most recent yesrs of 19961998 shows no statistically slgniflcant disparities in
    favor of women. Thus, there is nothing to indicate a pattern of significant statistical
    disparities ln favor of women in promotions.

    In the report’s conclusion, the PAB compares the data from the 1987 report and
    concludes that the time-in-band disparity noted ln 1987for African Americans
    persisted into the banded system through the 1995promotion cycle, and that, as to
    other protected groups, certain dlsparlties exist&din promotion rates for the
    1991-1995time perlod thatwere not evident in the 1987 study. We believe these
    comparisons are misleading. The 1987 report was based only on career ladder
    promotions, which were not competitive. In contrast, the PAP analysis of promotion
    data for 1991-1995merged non-competitive data (for Bands ID to IF’) wltb competitive
    data (for Bands IF to II). Therefore, because the comparisons involve different
    universes, the reporteris in concluding that the length of time for promotion for

    rage 3

             Page 30
Appendix II

PAB Response to Agency Commehts

              In commenting on the PAB'S study of promotions at GAO over a five year
              period (Appendix I), the Assistant Comptroller General for Operations
              made the following observations: (1) employees in Band I-D should have
              been grouped by three separate pay levels for purposes of the
              time-in-Band analysis; (2) levels of sigiziscance should be presented in
              every discussion of differences; (3) there is little difference in time-in-Band
              with respect to median times to promotion from Band I-F to Band u[;
              (4) the Board failed to note improvements that AlEcan American
              evaluators have made in time-in-Band (5) in analyzing rates of promotion,
              the Board should have reviewed the data by cycle/year rather than
              aggregating it; and, (6) the Board should not have merged non-competitive
              promotion data (Band I-D to I-F) with competitive promotion data (Band
              I-F to II and Band II to III). The Boards responses to those points follow:

              (1) The criteria for certification from the developmental level of Band I
              (I-D) to the Full-Performance (I-F) level are found in GAO Order 2540.1 The
              pertinent part provides that certification “may occur at any time after the
              first 6month progress review and the employee has completed 12 months
              in an evaluator or evaluator-related position.” Ch. 3, §&73(a). Although
              employees may be hired at different pay levels within Band I-D, all are
              equally eligible for certification once the time criteria are satisfied. The
              data provided to the Board by the Agency in the initial stages of its study
              were not separated by pay levels for any Band. The Agency did not provide
              any data to the Board to support the claim made in its comment letter that
              staff hired at a higher level of pay have a greater likelihood of earlier
              promotion to Band I-F than those hired at a lower pay level.

               (2) The Boards study sets out the time-in-Band numbers for I-D to I-F and
              from I-F to II. The numbers show that some members of some protected
              groups spent longer in Band at those points than members of other
              protected groups. The report does not attempt to measure the statistical
              significance of the numbers. At the Band II to Band IlI promotion point,
              the Board notes that it found no disparities. Again, the Board does not
              attach any statistical significance to this finding. To eliminate confusion,
              the report no longer describes the lack of disparities at the Band II to Band
              III level as statistically significant.

              (3) The analysis of time-in-Band took into account that the actual time in
              Band wti only known for some employees. For those who were already in
              Band at the beginning of the study (January 1,199l) or for those who were
              not promoted until after the end of the study (December 31,1995) only a
              minimum period of time in Band can be discerned. The techniques used in

              Page 32
 Appendix11                                    :
 PAR Response to Agenizy Comments

  the,Board’s report were developed to analyze an employee’s known
  history of promotions during a particular time period. They allow for
  computation of median time in Band as the time when 50% of the
  employees have already been promoted. The Agency’s point that there is
  “little‘demonstrable difference” ivith: respect to median time-in-Band at the
 I-F to II point is an oversimplification of the data. The numbers for
 time-in-Band at the I-F to IIpromotion point show that 50% of the white
 employees under 40 were promoted within 1,526 days (4.2 years); fewer
 than 50% of black employees under 40 were promoted from Band I to II
 during the entire five year study period. At that same promotion point, for
 all employees regardless of age, 50% of white Banded employees had been
 promoted by the time that 1,806 days had elapsed (4.9 years); again, fewer
 than 50% of black employees were promoted within the entire five year
-period encompassed by the study. Because of the five year period of the
‘study, the,Board is unable to determine the precise median for black
 employees in each of these categories; we can only say that it exceeded
 five years. :

 (4) This study was conceived as a follow-up to the 1987 report only in the
sense that the Board was revisiting timing and rates of promotions at GAO
 over a subsequent five year period.> Shortly after the Board published its
first report on career ladder promotions for evaluators and
evaluator-related employees, GAO completely revamped its pay system for
those employees, grouping them into three broad pay bands. Due to the
fundamental changes that banding caused in the promotional scheme, the
Board could not track or compare data on a category-by-category basis.
Rather, the Board examined promotions within the confines of the new
system to ascertain whether patterns that had been discerned previously
persisted under the new system. In addition, the current study added age
as well as disability analysis.

(5) GAO contends that aggregating the promotion data for the five year
period was inappropriate because multiple applications by the same
employee over several promotion cycles can distort the statistical analysis.
The Agency noted that some employees were on more than 20 BQ lists
over the five-year period. Analysis of the data shows, however, that nearly
97 percent of the employees on BQ lists appeared on five or fewer; nearly
70 percent appeared on just one or two lists. In addition, further
disaggregating the data beyond what was done in the analysis in this
report decreases the availability of comparative data which, in turn,
reduces the chance of identifying differences which may be present. In
response to the Agency’s comments, the Boards contractor, the Statistics

Page 33
     Appendix  II
     PAR Response   to Agency   Comments

     Lab at the University of Maryland, analyzed the data by year. The Lab
     noted that it found no significant differences in the odds ratios when the
     analysis was split by year.

   (6) With respect to its Time-in-Band analysis, the Board did not merge
;’ non-competitive and competitive promotion data, but presented it
   separated by promotion points both in its Findings section and its
   Conclusions section. The analysis of rates-of promotion is based solely on
   selections from BQ lists. As there are no BQ lists involved in the
   non-competitive promotions (I-D to I-F’), they were not included in the
   analysis of rates.

     Finally, in a separate communication, the Agency expressed
     dissatisfaction with the charts found at pages 13-14 in the report,
     questioning why data was presented for only one promotion cycle. The
     Agency indicated that, had the Board presented similar charts for each of
     the five years in the study, they would show increased representation of
     women and minorities in Bands II and III over the course of the study. The
     Board’s use of the charts was purely informational and not conclusory.
     The Board’s intention was to present readers with benchmarks that
     provide a general description of the composition of the GAO workforce as it
     existed midway in the Boards study.


     Page   34
Appendix III

Statistics Lab, University of Maryhnd,
Response to Agency Comments

                 The agency’s comments on page three, paragraph one oversimplify                             the
                 results of the analysis. The findings were in fact the following:

                 1. When all-employees are considered together, there is a disparity in
                promotion rates favoring younger employees over older, females over
                males and nonwhites over whites.
                2. When promotion candidates are. disaggregated on the basis of age, we
                find no disparkies among the under-40 employees. The disparities are
                concentrated in the 40-plus group.

                 The comments reveal a misunderstanding of the Mantel-Haenszel test. This
                 test does indeed combine data from promotion competitions. However,
                 the inferences areconditional on knowing the numbers of promotions
                 granted in each competition and the demographic characteristics of the
                members of the BQ group for that competition.’ II-I other words, the
                numbers of promotions and characteristics of the BQ lists are treated as
                 “fixed” in this analysis. Under >thk condition, the only characteristic which
                varies from competition to competition is the proportion of protected
                individuals who are promoted at each competition. Moreover, this
                proportion varies independently Tom competition to competition once the
                promotion rates and BQ list demographics are held fixed. The
                Mantel-Haenszel test does not require independence, but only conditional
                independence given promotion rate ,and BQ demographics of each

               The Mantel-Haenszel procedure tests the hypothesis that, holding all other
               factors fixed, the chance of a protected individual’s promotion in a given
               competition is identical to that of an unprotected individual’s promotion in
               the same competition. In other words, even though the composition of the
               BQ lists and the overall promotion rates may differ from competition to
               competition, the hypothesis says that the common odds ratio is 1.

               The GAO comments suggest that the methodology is invalid because some
               persons compete for multiple promotions. In fact, by looking only at the
               conditional odds ratio of promotion given the total numbers of promotions
               and the demographics of the BQ list, the test properly adjusts for
               employees who compete for several promotions.

               ‘Agesti, Alan “An Introduction to Categorical Data Analysis” (1996) New York: J. Wiley, p. 231.


               Page     35
Appendix III
Statistics Lab, University of lkryland,
Response to Agency Comments

The Mantel-Haenszel- test is frequently used to compare life data from two
separate groups (experimental and control) of patients with chronic
disease. In such studies, the patients at risk are compared whenever a
death occurs in order to compare’the odds of death in the experimental
and control groups. This means that the same subjects contribute to many
computations of odds ,ratios. There are certainly many more multiple
examinations of patients in this clinical setting than in the GAO application,
where very few employees were involved.in more than five competitions.

The authors of the comments chose to disaggregate the data by year and
band. They do not provide any substantive reasons for disaggregation. As
outlined above, there is also no statistical justification for disaggregation,
since the Mantel-Haenszel test accounts for differences in promotion rates
and BQ demographics. We also tested whether the odds ratios were equal
across competitions using the standard Breslow Day test.3 The results
overwhelmingly supported the hypothesis that odds ratios did not vary
significantly from competition to competition, thereby justifying our
combined Mantel-Haenszel analysis. Nevertheless, we also performed
separate analyses by band, obtaining the same findings as in the combined
analysis. We did not attempt to reproduce the erroneous analyses based
on d&aggregating by both.band and year.

Unnecessary disaggregation reduces the power of statistical tests, thereby
.making it difficult to distinguish genuine effects fiorn sampling error. In
plain words, breaking up the data into little subsets reduces the chance of
finding disparities if they do exist.

31bid.,   p. 238.

Page 36
Appendix IV

Personnel Appeals Board

Personnel Appeals   1
                          M. Gail Gerebenics, Director, EEO Oversight
Board Staff

                         *Term expired

                        Page   37