oversight

Report 2012-009-REV - U.S. Equal Employment Opportunity Commission Office of the Inspector General Review of Evaluations

Published by the Equal Employment Opportunity Commission, Office of Inspector General on 2013-04-09.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

U.S. Equal Employment Opportunity Commission
        Office of the Inspector General


            Review of Evaluations


              Contract No. GS-10F-0240U

            OIG Report Number 2012-09-REV




                  Final Report
                     April 9, 2013
                                                      TABLE OF CONTENTS

                                                                                                                                           Page
EXECUTIVE SUMMARY .....................................................................................................................ii
A.     OVERVIEW AND PURPOSE ...................................................................................................... 1
B.     METHODOLOGY ...................................................................................................................... 3
     B1.     Reviewing the Literature .................................................................................................. 3
     B2.     Interviewing EEOC Leadership ......................................................................................... 4
     B3.     Synthesizing Information ................................................................................................. 4
C.     FINDINGS ................................................................................................................................. 6
     C1.     Improving the Consistency of Intake Procedures ............................................................ 6
     C2.     Determining the Significance of a Charge ........................................................................ 7
     C3.     Addressing Systemic Discrimination ................................................................................ 9
     C4.     Improving Data Collection .............................................................................................. 10
D.     SUMMARY OF RECOMMENDATIONS ................................................................................... 13
E.     AGENCY COMMENTS AND OUR EVALUATION ..................................................................... 14
APPENDIX A: REFERENCES ........................................................................................................... 15
APPENDIX B: PRIORITY CHARGE HANDLING PROCESS (PCHP) .................................................... 18
APPENDIX C: FY 2012–2016 STRATEGIC PLAN ............................................................................. 19
APPENDIX D: EEOC LEADERSHIP AND STAFF INTERVIEW GUIDE................................................. 20
APPENDIX E: CONVERSATION WITH DISTRICT DIRECTORS PROTOCOL ....................................... 22
APPENDIX F: VALIDITY OF THE SELECTED STUDIES ...................................................................... 24
APPENDIX G: COMMENTS FROM THE OFFICE OF THE CHAIR ..................................................... 26
APPENDIX H: COMMENTS FROM THE OFFICE OF FIELD PROGRAMS .......................................... 29
APPENDIX I: COMMENTS FROM THE OFFICE OF RESEARCH, INFORMATION, AND PLANNING .. 32




IMPAQ International, LLC                                             Page i                                                       Final Report
                                      EXECUTIVE SUMMARY

The Equal Employment Opportunity Commission (EEOC) Office of the Inspector General (OIG)
contracted with IMPAQ International, LLC in September 2012 to conduct a review of
evaluations of EEOC’s private sector enforcement program. This was the first review of
evaluations contracted by the EEOC’s OIG. The purpose of the review is to provide EEOC
leadership and stakeholders with recommendations to enhance the efficiency and effectiveness
of EEOC private sector enforcement activities including mediation.1 A thorough analysis based
on research about new and useful approaches to enforcement will help inform the Commission
and its workforce about EEOC’s strengths and concerns. Study recommendations are geared
toward aiding the EEOC in its efforts to improve the effectiveness and efficiency of its private
sector enforcement activities.

Overview and Purpose. The review focused on synthesizing information from evaluations
examining EEOC’s key charge processing activities, as well as input obtained through interviews
with EEOC leadership. The following research questions guided the review:

       What factors are associated with the efficiency with which the EEOC carries out private
        charge processing activities?

       What factors are associated with the effectiveness with which the EEOC carries out
        private charge processing activities?

       What are the key information gaps related to developing strategies to more efficiently
        and effectively carry out private sector charge processing activities?

EEOC’s litigation activities were not part of the scope of the review.

Methodology. This project had three overarching tasks. The first task to review the recent
literature examining private sector enforcement activities of the EEOC was executed during the
period of September – November 2012. Numerous data sources including online law literature
(e.g., HeinOnline) and research databases (e.g., EBSCOHost, Google Scholar), as well as online
library resources at the University of Maryland and The Johns Hopkins University were used to
identify and retrieve relevant studies. The literature search included peer-reviewed journal
articles, government reports, books, integrated reviews of relevant literature, meta-analyses,
and grey literature (e.g., unpublished reports and conference proceedings). To avoid omissions
of key studies, the project team used a snowballing technique to identify publications that were
not uncovered during the initial search. This approach entailed reviewing the bibliographies and
reference lists of the publications identified during the initial search in order to identify other
relevant studies. The Contracting Officer’s Technical Representative (COTR) also assisted in
identifying and obtaining relevant documents. Studies as current as 2013 were identified. The
period of the last 10 years (since 2003) was considered the most relevant, although older
studies were included if pertinent. Studies obtained through the literature search were

1
  The review did not identify information gaps or concerns with EEOC mediation activities. Accordingly, the
recommendations do not pertain to mediation activities.


IMPAQ International, LLC                          Page ii                               Final Report
assessed for relevance to the project and validity. Relevant EEOC documents were reviewed, as
well.

The second task was interviewing EEOC leadership to obtain their insights regarding the
effectiveness and efficiency of private charge processing activities. In collaboration with the
COTR, the project team scheduled telephone interviews with key individuals. The following
individuals from the EEOC were interviewed:

       Chief Operating Officer
       Director, Office of Research, Information and Planning
       Director, Office of Field Programs
       Operations and Policy Specialist, Office of Field Programs
       Attorney Advisor, Office of Field Programs
       Three District Directors

Interview protocols developed and followed to gather information in a systematic manner are
provided as appendices. All interviews were conducted in either October 2012 or February
2013.

The third task was to synthesize this information to identify recommendations to aid the EEOC
in improving the effectiveness and efficiency of its private sector enforcement activities. This
task was carried out throughout the course of the project from October 2012 – February 2013.
The information synthesis identified four areas related to the Priority Charge Handling Process
(PCHP) in which recommendations to improve effectiveness and efficiency could be made. The
areas are (1) improving the consistency of intake procedures, (2) determining the significance of
a charge, (3) addressing systematic discrimination, and (4) improving data collection.

The COTR provided a draft of this report to the EEOC for their review and comment. The Office
of the Chair (OC), Office of Field Programs (OFP), and Office of Research, Information, and
Planning (ORIP) provided written comments, which have reprinted in appendices G, H and I,
respectively. Technical elements of these comments have been incorporated as appropriate.

Recommendations. Based on the review of the literature and the interviews conducted in this
study, the following recommendations are made to enhance the effectiveness and efficiency of
EEOC’s private sector charge processing:

    1. The EEOC should further standardize intake procedures across field offices. Consistent
       with ongoing efforts to improve the uniformity of intake materials and other efforts to
       establish greater communication with and between field offices, best practices related
       to the intake interview should be identified and widely adopted with accompanying
       training as needed. For example, field staff in some offices have adopted the practice of
       being “frontal” with charging parties. In this approach, field staff gather during the
       intake interview as much information as possible while simultaneously clearly explaining


IMPAQ International, LLC                      Page iii                          Final Report
        the law and the relief to be expected to the charging party. The intent of this approach
        is to have the decision to file a charge be made with clear expectations and to avoid
        instances in which the charging party files a charge “just to file a charge.” The Office of
        Field Programs is working toward standardizing intake procedures across field offices.
        Oversight by EEOC leadership to ensure that progress continues in this area is
        encouraged.
    2. The EEOC should document criteria for determining Category C charges. To enhance
       efficiency and effectiveness and promote the delivery of excellent and consistent service
       as sought in the Strategic Plan, the EEOC should continue to pursue ways to improve its
       triaging of charges, especially with regard to identifying Category C charges. The criteria
       used for assessing the significance of a charge vary across field staff and case
       characteristics, but it is challenging to standardize criteria. Accordingly, the EEOC should
       consider new approaches to efficiently determining Category C charges to reduce the
       inventory of pending charges and enable field staff to devote limited resources to
       effectively identifying and addressing potentially significant charges (i.e., Category A and
       B charges). The PCHP guidelines, the EEOC FY 2012–2016 Strategic Plan, and the
       Strategic Enforcement Plan define how to categorize and thus pursue a charge.
       Nevertheless, detailed guidelines are needed to aid consistency across field staff in
       determining whether a charge should be classified as Category C. Criteria for
       determining whether a charge is Category C should be distributed widely across field
       offices with appropriate training for field staff. Field office managers should be
       instructed to oversee adherence to the criteria. Additionally, to further improve
       consistency, a standardized set of parameters and keywords based on these criteria
       should be incorporated into the EEOC Integrated Mission System (IMS) database to
       allow field staff during their case research to refer to determinations of similar cases
       across field offices.
    3. The EEOC should continue efforts to develop a national approach for addressing systemic
       discrimination. The EEOC has made progress in recent years addressing system
       discrimination. The Commission has set a target that by FY 2016, 22-24 percent of the
       cases on its active litigation docket will constitute systemic cases. This is an increase
       from the 20 percent baseline of FY 2012. To attain the FY 2016 goal, we encourage the
       EEOC to remain committed to identifying systemic discrimination through a strategic,
       nationwide, and coordinated approach aided by newly adopted and future technological
       capabilities, excellent guidance from subject experts, and continued oversight by EEOC
       leadership.
    4. The EEOC should continue to review the range of information obtained during intake
       interviews and how it is stored in the IMS. Along with further standardizing intake
       procedures, the EEOC should examine the information that is currently obtained from
       charging parties and assess whether every element of information is essential and
       whether additional information is needed for a given charge or for addressing systemic
       discrimination. In addition, assessments should be made regarding how to more
       effectively store intake information in the IMS by identifying and defining standardized



IMPAQ International, LLC                      Page iv                            Final Report
        phrases, clauses, parameters, and keywords to enable field staff to input data more
        consistently and investigate charges more effectively and efficiently.
    5. The EEOC should investigate the merits of expanding the information it obtains related
       to hiring and terminations. The EEOC should investigate whether it would be
       advantageous to its mission to obtain more information. Currently, much of the data
       collected on hiring and termination is determined by law. The EEOC should consider
       both obtaining additional information from the employers, employees, and applicants
       currently providing information, and expanding the groups of employers and employees
       required to provide the EEOC with information. This is a key information gap. Should the
       EEOC determine that additional information would enhance its effectiveness and
       efficiency in addressing employment discrimination, it should assess what changes to
       procedures, rules, and laws would be necessary.




IMPAQ International, LLC                     Page v                           Final Report
                                  A. OVERVIEW AND PURPOSE

The Equal Employment Opportunity Commission (EEOC) Office of the Inspector General (OIG)
contracted with IMPAQ International, LLC in September 2012 to conduct a review of
evaluations of EEOC’s private sector enforcement program. This was the first review of
evaluations contracted by the EEOC’s OIG. The purpose of the review is to provide EEOC
leadership and stakeholders with recommendations to enhance the efficiency and effectiveness
of EEOC private sector enforcement activities including mediation.2 A thorough analysis based
on research about new and useful approaches to enforcement will help inform the Commission
and its workforce about EEOC’s strengths and concerns. Study recommendations are geared
toward aiding the EEOC in its efforts to improve the effectiveness and efficiency of its private
sector enforcement activities.

The review focused on synthesizing information from evaluations examining EEOC’s key charge
processing activities, as well as input obtained through interviews with EEOC leadership. The
following research questions guided the review:

       What factors are associated with the efficiency with which the EEOC carries out private
        charge processing activities?

       What factors are associated with the effectiveness with which the EEOC carries out
        private charge processing activities?

       What are the key information gaps related to developing strategies to more efficiently
        and effectively carry out private sector charge processing activities?

EEOC’s litigation activities were not part of the scope of the review.

The EEOC is the key Federal agency responsible for enforcing Federal laws that prohibit
employment discrimination based on a person’s age (40 or older), race, sex (including
pregnancy), color, religion, national origin, disability, or genetic information. These laws cover
both small and large employers, including those in the private sector, the public sector, and
nonprofits, and pertain to all aspects of employment including hiring, training, wages, benefits,
harassment, promotion, and termination. The EEOC has the authority to investigate charges of
systemic or individual discrimination against employers who are covered by these laws.3 The
EEOC’s private sector charge processing system registers discrimination charges based on a
variety of types of discrimination, including disability, HIV/AIDS status, cancer status, sexual
harassment, and race.4 To enhance efficiency, the EEOC adopted the Priority Charge Handling
Procedure (PCHP) in April 1995, which serves as its current system of case triage.5 For a variety

2
  The review did not identify information gaps or concerns with EEOC mediation activities. Accordingly, the
recommendations do not pertain to mediation activities.
3
  http://www.eeoc.gov/eeoc/.
4
  Hersch (2010).
5
  EEOC Strategic Enforcement Plan (2012).

IMPAQ International, LLC                          Page 1                                Final Report
of reasons, the inventory of pending charges at year-end nearly tripled from FY 2002 (29,041)6
to FY 2010 (86,338).7 More recently, the EEOC has made progress in reducing the inventory. In
the last two years, the inventory was reduced to 78,136 in FY 20118 and 70,312 in FY 2012.9 To
put the reduction in the inventory in context, in FY 2012 the EEOC received 99,412 private
sector workplace discrimination charges and resolved 111,139 charges.10

The Commission’s FY 2012–2016 Strategic Plan11 emphasizes its commitment to serve the
nation more efficiently. The Strategic Plan is based upon three values—commitment to justice,
accountability, and integrity—and departs from previous strategic plans by focusing less on
measuring output and more on achieving long-term goals. As such, the Strategic Plan requires
the Commission to be proactive and prioritizes a coordinated approach to law enforcement.

The Strategic Plan outlines three objectives, each with its own set of outcome goals:

    1. Combat employment discrimination through strategic law enforcement, with outcome
       goals of (1) having a broad impact on reducing employment discrimination at the
       national and local levels, and (2) remedying discriminatory practices and securing
       meaningful relief for victims of discrimination.
    2. Prevent employment discrimination through education and outreach, with outcome
       goals of (1) better informing the public about how to exercise their right to employment
       free of discrimination, and (2) informing employers, unions, and employment agencies
       (covered entities) about how to better address and resolve Equal Employment
       Opportunity (EEO) issues, thereby creating more inclusive workplaces.
    3. Deliver excellent and consistent service through a skilled and diverse workforce and
       effective systems, with an outcome goal of timely, informative, and high quality
       interactions with the public.

The recommendations in this review will assist the EEOC in its efforts to meet the objectives
and goals of the FY 2012-2016 Strategic Plan. Section B of this report describes the
methodology employed in the review. Section C presents the findings of the synthesis, grouping
the evidence into four broad categories:

    1. Improving the consistency of intake procedures
    2. Determining the significance of a charge
    3. Addressing systemic discrimination
    4. Improving data collection.

Section D summarizes the recommendations for ease of reference.
6
  http://www.eeoc.gov/eeoc/plan/archives/annualreports/par/2003/index.html
7
  http://www.eeoc.gov/eeoc/plan/2010par_message.cfm
8
  http://www.eeoc.gov/eeoc/plan/2011par_performance.cfm
9
  http://www.eeoc.gov/eeoc/plan/2012par.cfm
10
   Ibid.
11
   http://www.eeoc.gov/eeoc/plan/strategic_plan_12to16.cfm.


IMPAQ International, LLC                          Page 2                        Final Report
                                    B. METHODOLOGY

This project had three overarching tasks. The first task was to review the recent literature
examining private sector enforcement activities of the EEOC was executed during the period of
September – November 2012. The second was interviewing EEOC leadership to obtain their
insights regarding the effectiveness and efficiency of these activities. All interviews were
conducted in either October 2012 or February 2013. The third task was to synthesize this
information to identify recommendations for aiding the EEOC in improving the effectiveness
and efficiency of its private sector enforcement activities. This task was carried out from
October 2012 – February 2013.

B1.     Reviewing the Literature

The IMPAQ team conducted a literature search examining the private sector charge processing
activities of the EEOC. Numerous data sources were used to identify and retrieve relevant
studies. Sources included online law literature (e.g., HeinOnline) and research databases (e.g.,
EBSCOHost, Google Scholar), as well as online library resources at the University of Maryland
and The Johns Hopkins University. The systematic search began with four primary keywords: (1)
EEOC, (2) private sector employment discrimination, (3) EEOC and private sector charges, and
(4) private charge handling process. The literature search included peer-reviewed journal
articles, government reports, books, integrated reviews of the literature, meta-analyses, and
grey literature (e.g., unpublished reports and conference proceedings). The search focused on
studies published after 2003 to ensure that the studies examined the period after the EEOC
adopted the PCHP.

Although the literature search focused on the private sector charge processing activities of the
EEOC, the IMPAQ team also reviewed publications describing the inception, organization,
functioning, and responsibilities of the EEOC to understand the challenges faced in addressing
charges. To avoid omissions of key studies, the project team used a snowballing technique to
identify publications that were not uncovered during the initial search. This approach entailed
reviewing the bibliographies and reference lists of the publications identified during the initial
search in order to identify other relevant studies. In addition, if the project team was unable to
retrieve a relevant document online, a member of the team contacted the author directly and
requested a copy of the publication. The COTR also assisted the project staff in identifying and
obtaining relevant documents. Studies as current as 2013 were identified. The period of the last
10 years (since 2003) was considered the most relevant, although older studies were included if
pertinent.

Studies obtained through the literature search were assessed for relevance to the project and
overall validity as detailed in Appendix F. The IMPAQ team facilitated the assessment by
extracting key information from each study and then developing a summary of each study.
Included in each summary was information about authorship, funding, time period, and
geographical area addressed. The project team also reviewed the methodology, outcomes,
findings, and limitations of each study and added this information to the summary. The


IMPAQ International, LLC                      Page 3                             Final Report
assessment of the relevance of a study focused on the research question(s) addressed by the
study and the time period examined. Studies deemed relevant addressed a variety of research
questions and sub-questions related to the organization, functioning, responsibilities, types of
charges, and activities performed by the EEOC related to private sector charge processing. The
period of the last 10 years (since 2003) was considered the most relevant, although older
studies were included if pertinent. Appendix F describes in detail how studies were assessed
for validity. Key factors for assessing validity included the reputation of the journal/research
outlet, study methodology, and the study population/geography. Relevant EEOC documents
were reviewed, as well.

B2.        Interviewing EEOC Leadership

The IMPAQ team sought further insights into private sector enforcement activities from EEOC
leadership and staff. In collaboration with the COTR, the project team scheduled telephone
interviews with key individuals. The interviews were conducted in October 2012 and February
2013. The following individuals from the EEOC were interviewed:

          Chief Operating Officer

          Director, Office of Research, Information and Planning

          Director, Office of Field Programs

          Operations and Policy Specialist, Office of Field Programs

          Attorney Advisor, Office of Field Programs

          Three District Directors

Interview protocols were developed and followed in order to gather information in a systematic
manner. The interview protocols developed for these interviews are presented in Appendices D
and E. The semi-structured interviews began with an introduction and an explanation of the
nature of the project and the interview. At that time, the interviewer requested permission to
record the interview to facilitate note taking and ensure accuracy. The interviewee was also
informed that all responses would remain confidential and that the interviewee would not be
identified in the report. The interview questions were developed to allow interviewees to
provide their views, concerns, and recommendations regarding improving the effectiveness and
efficiency of EEOC private sector enforcement activities.

B3.        Synthesizing Information

IMPAQ utilized content analysis to analyze the data from the literature review and the
interviews. Project staff applied an inductive data coding strategy,12 in which the data drive the

12
     http://www.fmhs.auckland.ac.nz/soph/centres/hrmas/_docs/Inductive2003.pdf .



IMPAQ International, LLC                             Page 4                        Final Report
identification of themes or coding categories. For example, when analyzing data from the
interviews, the project analysts initially categorized the data at the question level. Based on the
responses to the questions, secondary categories were identified, creating a hierarchical
structure. As more qualitative data were reviewed, new categories were created. Finally, the
project analysts adjusted the categories to ensure that they were inclusive and exhaustive. This
process is shown in Exhibit 1.

The IMPAQ project analysts summarized the findings within each category and then across
categories in order to provide a clear and succinct description of the findings. The literature
review and the interviews revealed both reinforcing and conflicting viewpoints. In instances in
which viewpoints conflicted, the project analysts further examined the evidence supporting
each viewpoint and identified any gaps in current knowledge.

                                 Exhibit 1. Synthesis Methodology

                                                            Reduce overlap
                                            Label the                            Create a
  Initial read-     Identify specific                            and
                                           segments of                       framework with
through of text       segments of                            redundancy
                                         information to                    the most important
      data            information                             among the
                                        create categories                       categories
                                                              categories
Many pages of        Many segments        Initial list of    Consolidated   Categories that are
    text                of text            categories           list of       relevant to the
                                                              categories    research questions




IMPAQ International, LLC                        Page 5                           Final Report
                                             C. FINDINGS

The information synthesis identified four areas related to the PCHP in which recommendations
to improve effectiveness and efficiency could be made: (1) improving the consistency of intake
procedures, (2) identifying a significant charge, (3) addressing systemic discrimination, and (4)
improving data collection. Per the nature of this review, recommendations at times build on
previous recommendations.

C1.     Improving the Consistency of Intake Procedures

A principal objective of the EEOC FY 2012–2016 Strategic Plan is to deliver excellent and
consistent service that results in timely, informative, and high quality interactions with the
public.13 Providing consistent service starts with providing and obtaining consistent information
during the initial intake interview. The intake interview is the first step of determining the
nature of a discrimination charge and how it will be pursued. It has been found that when
charging parties do not receive consistent information at intake regarding their rights,
opportunities, and obligations, it can lead to some charging parties dropping charges that
would otherwise be found meritorious.14

Intake information offered to customers and intake procedures vary across field offices.15,16
Evidence suggests that improvements have been made in recent years in addressing
consistency at intake across field offices.17 In particular, the Office of Field Programs has
adopted uniform intake questionnaires, brochures and checklists for field offices. Training
related to intake, interviewing and the PCHP has been provided to staff, as well, to further
promote consistency at intake.18

Nevertheless, achieving consistency across field offices is challenging.19 While all field offices
are equipped with accredited staff trained to file and pursue charges, field offices differ in
terms of the population served, available resources, and volume of charges filed.20 As a result,
field offices differ in how they strike a balance between addressing charges and providing
intake services to charging parties. The most notable difference relates to the amount of time
spent conducting the intake interview with the charging party. Interviewees agree that
determining the significance of a charge is “more art than science and requires excellent
interpersonal skills.”21 Variation in the depth of the information obtained at intake likely
reduces consistency in charge processing and priority classification across field offices22 and

13
   http://www.eeoc.gov/eeoc/plan/strategic_plan_12to16.cfm.
14
   Development Services Group (2006).
15
   Interview with EEOC leadership.
16
   Development Services Group (2006).
17
   Interview with EEOC leadership.
18
   Ibid.
19
   Ibid.
20
   Ibid.
21
   Ibid.
22
   Ibid.


IMPAQ International, LLC                          Page 6                         Final Report
may be a contributing factor in the systematic differences in charge determinations by charging
party characteristics found in some studies.23 Additionally, data entered into the IMS system
will reflect any intake differences across field offices and affect the reliability of IMS data for
analysis.

Consistent with ongoing efforts to improve the uniformity
of intake materials and other efforts to establish greater           Recommendation 1: The EEOC
                                                      24           should further standardize intake
communication with and between field offices, best
practices related to the intake interview should be                 procedures across field offices.
identified and widely adopted with accompanying training
as needed. For example, field staff in some offices have adopted the practice of being “frontal”
with charging parties.25 In this approach, field staff gather during the intake interview as much
information as possible while simultaneously clearly explaining the law and the relief to be
expected to the charging party. The intent of this approach is to have the decision to file a
charge be made with clear expectations and to avoid instances in which the charging party files
a charge “just to file a charge.”26 The Office of Field Programs is working toward standardizing
intake procedures across field offices. Oversight by EEOC leadership to ensure that progress
continues in this area is encouraged.

C2.     Determining the Significance of a Charge

In order to be both effective and efficient, the EEOC instructs field office staff to focus on
charges of discrimination deemed significant.27,28,29 Under the PCHP, each incoming charge
receives a priority assessment and is assigned a priority status (Category A, B, or C). Priority
status assignment is based on information provided by the charging party (the alleged victim of
discrimination) during the initial intake interview and on the field staff’s perception of the
credibility of the charging party. Category A indicates that the charge falls within the national or
local enforcement plan, or that further investigation is likely to lead to a finding of cause.
Category B indicates that additional evidence will be required to determine whether further
investigation will reveal discrimination. Category C charges are those in which the office has
sufficient information to conclude that it is not likely that further investigation will result in a
cause finding.30 Establishing a precise definition of what constitutes a significant charge is
difficult, especially in relationship to the priority status of a charge.31,32 Some criteria for
categorizing a charge as significant are straightforward. For example, if the discrimination

23
   Conyers et al. (2005a); Conyers et al. (2005b); Hersch (2013); McKenna (2005).
24
   Interview with EEOC leadership.
25
   Ibid.
26
   Ibid.
27
   Selmi (1996).
28
   U.S. Equal Employment Opportunity Commission National Enforcement Plan;
   http://eeoc.gov/eeoc/plan/nep.cfm.
29
   National Employment Lawyers Association (2007).
30
   http://www.eeoc.gov/eeoc/litigation/manual/1-3-a_intro.html
31
   Modesitt (2010).
32
   Interview with EEOC leadership.


IMPAQ International, LLC                              Page 7                        Final Report
charge refers to egregious harm caused to an individual, then it is categorized as significant.
Charges of systemic discrimination and workplace discrimination charges which are endemic in
a particular industry also constitute significant charges.33,34 In such cases, current intake
procedures that include the uniform collection of background information through
standardized intake questionnaires and “layers of review to determine the charge category”35
are adequate. Determining whether a charge is significant based on its potential to develop the
law or influence policy making; however, is not as straightforward as it is unclear how to
determine such potential.36 Moreover, evidence suggests that prioritizing a discrimination
charge depends on the presentation of the case by the alleged victim. For example, cases
presented by a legal representative are often perceived as being more credible and more likely
to be assessed as significant.37,38,39,40 On the other hand, field staff experience suggests that
legal representatives also may guard information or provide information in “piecemeal” during
intake to prevent the use of such information during a subsequent investigation.41 Variation in
charge assessment criteria across subpopulations (e.g., individuals with HIV/AIDS, cancer) and
across investigating staff has been found in studies, with investigators burdened with a high
flow of charges being more resistant to deeming a charge significant.42,43 Compounding the
problem is the lack of consistency in intake procedures across field offices.

To enhance efficiency and effectiveness and promote the
                                                                Recommendation 2: The EEOC
delivery of excellent and consistent service as sought in
                                                                 should document criteria for
the Strategic Plan, the EEOC should continue to pursue
ways to improve its triaging of charges, especially with       determining Category C charges.
regard to identifying Category C charges. The criteria
used for assessing the significance of a charge vary across
field staff and case characteristics as described above, but it is challenging to standardize
criteria.44 Accordingly, the EEOC should consider new approaches to efficiently determining
Category C charges to reduce the inventory of pending charges and enable field staff to devote
limited resources to effectively identifying and addressing potentially significant charges (i.e.,
Category A and B charges).45 The PCHP guidelines, the EEOC FY 2012–2016 Strategic Plan, and
the Strategic Enforcement Plan define how to categorize and thus pursue a charge. Field offices
provide training to field staff on the application of PCHP guidelines to improve consistency in


33
   Modesitt (2010).
34
   Interview with EEOC leadership.
35
   Ibid.
36
   Ibid.
37
   Modesitt (2010); Moss et al. (2001).
38
   See also NELA (2007).
39
    NELA (2007).
40
   Interviews with EEOC leadership.
41
   Ibid.
42
   Conyers et al. (2005a); Conyers et al. (2005b); McKenna (2005).
43
   National Employment Lawyers Association (NELA) (2007).
44
   Interviews with EEOC leadership.
45
   Modesitt (2010)


IMPAQ International, LLC                              Page 8                     Final Report
charge categorization.46 Nevertheless, detailed guidelines are needed to aid consistency across
field staff in determining whether a charge should be classified as Category C.47 Criteria for
determining whether a charge is Category C should be distributed widely across field offices
with appropriate training for field staff. Field office managers should be instructed to oversee
adherence to the criteria. Additionally, to further improve consistency, a standardized set of
parameters and keywords based on these criteria should be incorporated into the EEOC
Integrated Mission System (IMS) database to allow field staff during their case research to refer
to determinations of similar charges in other field offices.

C3.     Addressing Systemic Discrimination

The EEOC defines systemic discrimination as “a pattern or practice, policy, or class case where
the alleged discrimination has a broad impact on an industry, profession, company or
geographic area.”48 Examples of systemic discrimination include not hiring qualified women in
traditionally male-dominated occupations, and illegal pre-employment inquiries about disability
status. An integral part of the Commission’s mission is to identify, investigate, and act upon
cases of systemic discrimination.49 The Commission’s FY 2012–2016 Strategic Plan50 and the
Strategic Enforcement Plan 2013–2016 (SEP)51 document EEOC’s commitment to pursue
systemic investigations.

Evidence indicates that although the EEOC successfully investigates, conciliates, and litigates
numerous systemic cases, it does not identify systemic discrimination in a consistent and
coordinated manner.52 This is not a new concern as reflected in the report of the 2006 Systemic
Task Force.53 Task force recommendations such as requiring District Offices to develop systemic
discrimination enforcement plans have been adopted. Through the SEP, the EEOC reaffirms the
approach and principles of the Systemic Task Force - that systemic enforcement must be
strategic, nationwide, coordinated and adequately resourced.54 The Commission is extending its
partnerships with employer groups, state and Federal agencies, advocacy groups, the plaintiffs’
bar and other organizations to identify and address discriminatory practices. These actions will
be supported by enhanced outreach efforts to inform and educate employers to encourage the
prevention of systemic discrimination.55

Currently, the Agency’s internal incentives and resources are better aligned to address systemic
cases than they have been in the past.56 Specific examples include training opportunities with

46
   Interviews with EEOC leadership.
47
   Modesitt (2010).
48
   http://www.eeoc.gov/eeoc/systemic/index.cfm.
49
   Ibid.
50
   http://www.eeoc.gov/eeoc/plan/strategic_plan_12to16.cfm.
51
   http://www.eeoc.gov/eeoc/plan/sep.cfm.
52
   Interview with EEOC leadership.
53
   Silverman et al. (2006).
54
   http://www.eeoc.gov/eeoc/plan/sep.cfm.
55
   http://www.eeoc.gov/eeoc/systemic/index.cfm.
56
   Interview with EEOC leadership.


IMPAQ International, LLC                          Page 9                        Final Report
appropriate support materials and technical tools and greater coordination and collaboration
between field offices facilitated by Headquarters personnel. Other examples include the
development of an internal EEOC systemic website and the Systemic Watch List Tool which uses
IMS data to network EEOC offices investigating systemic cases by notifying investigators and
attorneys when a related charge or inquiry is received or when existing charges meet specified
systemic criteria.57 Also, many field offices are having more senior investigators guide other
investigators and social scientists and subject matter experts on board to help with systemic
cases.58

The EEOC has made progress in recent years addressing
                                                                 Recommendation 3: The EEOC
systemic discrimination. The Commission has set a target
                                                                   should continue efforts to
that by FY 2016, 22-24 percent of the cases on its active
                                                                  develop a national approach
litigation docket will constitute systemic cases.59 This is an
increase from the 20 percent baseline of FY 2012. To attain          for addressing systemic
the FY 2016 goal, we encourage the EEOC to remain                         discrimination.
committed to identifying systemic discrimination through a
strategic, nationwide, and coordinated approach aided by newly adopted and future
technological capabilities, excellent guidance from subject experts, and continued oversight by
EEOC leadership.

C4.     Improving Data Collection

The EEOC receives charges on a variety of topics. Information gathered at intake and
throughout the investigation is recorded in the EEOC’s IMS database.60 The IMS database
contains information on charges that have been processed and investigated by the EEOC, as
well as information obtained from employers. Evidence suggests, however, that resource
constrained field offices may not always update the IMS database in a timely manner as details
of a case change (e.g., a case reassessment leads to a change in the initial charge
categorization).61 Moreover, as noted above, since uniformity in intake procedures across field
offices is a challenge, IMS data will reflect any variation in intake across field offices.
Accordingly, the reliability and completeness of IMS data for analysis could be a concern.
Indications are that the EEOC is addressing data infrastructure issues such as increasing
transmission capacity (bandwidth) to better facilitate electronically maintaining administrative
case documents.62 Such electronic documentation will greatly enhance efficiency in addressing
charges and the value of the IMS to field staff. Additionally, the IMS now includes new codes to
identify, for example, the type of relief provided to charging parties and to track lesbian, gay,
bisexual, and transgender (LGBT) cases.63

57
   http://www.eeoc.gov/eeoc/plan/upload/2012par_1.pdf
58
   Ibid.
59
   Ibid.
60
   Interview with EEOC leadership.
61
   Ibid.
62
   Ibid.
63
   Ibid.


IMPAQ International, LLC                        Page 10                         Final Report
Some information that might enhance the effectiveness and efficiency of the EEOC in areas
such as determining the significance of a charge is not collected. For example, while the IMS
compiles data about charges filed, information about the charging party’s charge history for
other charges is collected and stored for only specific charging parties and respondents.64 More
broadly, the information collected by the EEOC does not adequately reflect the full workforce.65
For example, the EEOC does not capture information from small private employers covered
under Title VII such as those with fewer than 100 employees that are not affiliated or owned by
another company.66 Also, EEOC forms require reporting the number of employees by race,
ethnicity, and gender in each job category, but other key information such as salaries by race
and gender is not collected.67 To identify early signs of employment discrimination, it would be
useful to capture differences, if any, between the demographic distribution of the applicant
pool and that of new hires, for each job position in each industry. However, providing such
applicant data is currently largely voluntary. Also lacking is information on employee
terminations.68 Moreover, temporary hiring agencies are not required to collect and provide
EEO-1 data. So, it is difficult to track any employment discrimination against those who use the
services of these agencies.69

Along with further standardizing intake procedures, the
                                                                  Recommendation 4: The EEOC
EEOC should examine the information that is currently
                                                                  should continue to review the
obtained from charging parties and assess whether every
                                                                  range of information obtained
element of information is essential and whether
additional information is needed for a given charge or for during intake interviews and how it
addressing systemic discrimination. In addition,                is stored and updated in the IMS.
assessments should be made regarding how to more
effectively store intake information in the IMS by identifying and defining standardized phrases,
clauses, parameters, and keywords to enable field staff to input data more consistently and
investigate charges more effectively and efficiently.

The EEOC should investigate whether it would be
advantageous to its mission to obtain more                  Recommendation 5: The EEOC
information. Currently, much of the data collected on      should investigate the merits of
hiring and termination is determined by law. The EEOC       expanding the information it
should consider both obtaining additional information        obtains related to hiring and
from the employers, employees, and applicants                       terminations.
currently providing information, and expanding the
groups of employers and employees required to provide the EEOC with information. Should the
EEOC determine that additional information would enhance its effectiveness and efficiency in


64
   U.S. Equal Employment Opportunity Commission Intake Questionnaire.
65
   Modesitt (2010).
66
   http://www.eeoc.gov/employers/eeo1survey/whomustfile.cfm
67
   Modesitt (2010).
68
   Ibid.
69
   Interview with EEOC leadership.


IMPAQ International, LLC                          Page 11                       Final Report
addressing employment discrimination, it should assess what changes to procedures, rules, and
laws would be necessary.




IMPAQ International, LLC                   Page 12                          Final Report
                           D. SUMMARY OF RECOMMENDATIONS

Based on the review of the literature and the interviews conducted in this study, the following
recommendations are made to enhance the effectiveness and efficiency of the EEOC’s private
sector charge processing:

    1. The EEOC should further standardize intake procedures across field offices.

    2. The EEOC should document criteria for determining Category C charges.

    3. The EEOC should continue efforts to develop a national approach for addressing
       systemic discrimination.

    4. The EEOC should continue to review the range of information obtained during intake
       interviews and how it is stored and updated in the IMS.

    5. The EEOC should investigate the merits of expanding the information it obtains related
       to hiring and terminations.




IMPAQ International, LLC                    Page 13                            Final Report
                     E. AGENCY COMMENTS AND IMPAQ’S RESPONSE

The COTR provided a draft of this report to the EEOC for their review and comment. The Office
of the Chair (OC), Office of Field Programs (OFP), and Office of Research, Information, and
Planning (ORIP) provided written comments, which have reprinted in appendices G, H and I,
respectively. Technical elements of these comments have been incorporated as appropriate.

OC indicated general agreement with the report recommendations. It noted that current
agency strategic planning and implementation efforts are consistent with the purpose and
recommendations of this report. OFP generally concurred with the report recommendations
indicating that efforts to implement the Strategic Plan for FY 2012 – 2016 and the Strategic
Enforcement Plan are consistent with observations in the draft report. Recommendations by
OFP to acknowledge recent efforts to reduce the inventory of private sector charges and
coordinate systemic discrimination activities have been incorporated in the report. ORIP
disagreed with the methodology of the evaluation. As described in the report, the study
employed a rigorous methodology to identify and select the most relevant research, then
synthesized that information along with information drawn from several stakeholder interviews
into the recommendations provided in this report. The approach suggested by ORIP in which
the analysis would involve the extraction, evaluation and regeneration of data to produce more
quantitative results was beyond the scope and resources of this study. Greater specificity has
been incorporated as appropriate.




IMPAQ International, LLC                   Page 14                           Final Report
                               APPENDIX A: REFERENCES

Burris, S., Moss, K., Ullmann, M.D., Johnsen, M. (2000). Disputing Under The Americans with
   Disabilities Act: Empirical Answers, and Some Questions. Temple Political and Civil Rights
   Law Review, 9(2), 237-252.

CFI Group and Federal Consulting Group. (2008). Equal Employment Opportunity Commission
    Customer Satisfaction and Knowledge of Law Study. Washington, DC: Equal Employment
    Opportunity Commission.

Conyers, L., Boomer, K.B., McMahon, B.T. (2005a). Workplace Discrimination and HIV/AIDS: The
   National EEOC ADA Research Project. Work, 25(1), 37-48.

Conyers, L., Unger, D., Rumrill, Jr., P.D. (2005b). A Comparison of Equal Employment
   Opportunity Commission Case Resolution Patterns of People with HIV/AIDS and Other
   Disabilities. Journal of Vocational Rehabilitation, 22(3), 171-178.

Development Services Group. (2006). Evaluation of Intake and End-of-Fiscal-Year Closures of
   the Equal Employment Opportunity Commission Private Sector Charge Process. Bethesda,
   MD: Development Services Group.

EEOC. (2002). Combined Annual Reports Fiscal Years 1999-2001 (October 1, 1998 through
   September 30, 2001). Washington, DC: U.S. Equal Employment Opportunity Commission.

EEOC. (2012). Strategic Enforcement Plan, 2013-2016. Washington, DC: U.S. Equal Employment
   Opportunity Commission.

EEOC. (2012). United States Equal Employment Opportunity Commission Strategic Plan for
   Fiscal Years 2012–2016. Washington, DC: U.S. Equal Employment Opportunity Commission.
   http://www.eeoc.gov/eeoc/plan/strategic_plan_12to16.cfm

Federal Consulting Group, U.S. Department of Interior. (2010). U.S. Equal Employment
   Opportunity Commission Evaluation of the Priority Charge Handling Procedures Report.
   Washington, DC: U.S. Equal Employment Opportunity Commission.

Green, M.Z. (2001). Proposing a New Paradigm for EEOC Enforcement After 35 Years:
   Outsourcing Charge Processing by Mandatory Mediation. Dickinson Law Review, 105, 305-
   342.

Hersch, J. (2011). Compensating Differentials for Sexual Harassment. Paper presented at
   American Economic Association Annual Meeting, Denver, CO, January 8.

Hersch, J. (2013). Characteristics of Color Discrimination Charges Filed with the EEOC. In R.E.
   Hall (Ed.), The Melanin Millennium: Skin Color as 21st Century International Discourse, 231-
   246. Dordrecht, NL: Springer.




IMPAQ International, LLC                    Page 15                           Final Report
Igasaki, P.M. (2001). Doing the Best with What We Had: Building a More Effective Equal
    Employment Opportunity Commission During the Clinton-Gore Administration. The Labor
    Lawyer, 17(2), 261-284.

Igasaki, P.M., Miller, P.S. (1998). Priority Charge Handling Task Force and Litigation Task Force
    Report. Washington, DC: U.S. Equal Employment Opportunity Commission.

Kearney, R.A. (2001). Who's "In Charge" at the EEOC? Drake Law Review, 50(1), 1-31.

McAllister, S.B., Burbano, F., Christopher, G., Hamilton, M.R., White, B. (2003). Equal
  Employment Opportunity Commission: Organizing for the Future. A Report by a Panel of
  the National Academy of Public Administration for the U.S. Equal Employment Opportunity
  Commission. Washington, DC: National Academy of Public Administration.

McDermott, E.P., Jose, A., Obar, R., Bowers, M., Polkinghorn, B. (2002). Has the EEOC Hit A
  Home Run? An Evaluation of the EEOC Mediation Program from the Participants’
  Perspective. Advances In Industrial and Labor Relations, 11, 1-40.

McDermott, E.P., Obar. R. (2004). "What's Going On" in Mediation: An Empirical Analysis of the
  Influence of a Mediator's Style on Party Satisfaction and Monetary Benefit. Harvard
  Negotiation Law Review, 9, 75-114.

McDermott, E.P., Obar, R., Jose, A., Bowers, M. (2000). An Evaluation of the Equal Employment
  Opportunity Commission Mediation Program. Washington, DC: U.S. Equal Employment
  Opportunity Commission.

McDermott, E.P., Obar, R., Jose, A., Polkinghorn, B. (2001). The EEOC Mediation Program:
  Mediators' Perspective on the Parties, Processes, and Outcomes. Washington, DC: U.S.
  Equal Employment Opportunity Commission.

McGill, S., Tracey, A.M. (2011). Building a New Bridge over Troubled Waters: Lessons Learned
  from Canadian and U.S. Arbitration of Human Rights and Discrimination Employment
  Claims. Cardozo Journal of International and Comparative Law, 20(1), 1-73.
  http://www.cjicl.com/uploads/2/9/5/9/2959791/cjicl_20.1_mcgill__tracey_article.pdf.

McKenna, M.A. (2005). Discriminatory Practices Charged Under EEOC: An Empirical Analysis of
  Investigated Complaints Filed by Those Who Have Cancer. Dissertation, University of
  Maryland College Park.

Modesitt, N.M. (2010), Reinventing the EEOC. Southern Methodist University Law Review, 63(4),
  1237-1277.

Morra, L.G. (1995). Burgeoning Workload Calls for New Approaches. Testimony Before the
  Committee on Labor and Human Resources, U.S. Senate, May 23. GAO/T-HEHS-95-170.
  Washington, DC: U.S. General Accountability Office.
   http://www.gao.gov/assets/110/106040.pdf




IMPAQ International, LLC                     Page 16                            Final Report
Moss, K., Burris, S., Ullman, M.D, Johnsen, M., Swanson, J.W. (2001). Unfunded Mandate: An
  Empirical Study of the Implementation of the Americans with Disabilities Act by the Equal
  Employment Opportunity Commission. Kansas Law Review 50(1), 1-109.

National Employment Lawyers Association. (2007). Worker's Rights in Jeopardy: EEOC’s
   Enforcement of Equal Employment Opportunity Laws Impeded by Inadequate Funding.
   Washington, DC.

Occhialino, A.N., Vail, D. (2005). Why the EEOC (Still) Matters. Hofstra Labor & Employment Law
   Journal, 22(2), 671-708.
   http://lawarchive.hofstra.edu/pdf/Academics/Journals/LaborAndEmploymentLawJournal/la
   bor_Occhialino_Vail_vol22no2.pdf.

Robertson, R.E. (2005). Equal Employment Opportunity Commission: Actions Taken, but Agency
   Restructuring Efforts Could Benefit from a More Systematic Consideration of Advisory
   Panel's Recommendations. GAO-06-10. Washington, DC: U.S. General Accountability Office.

Selmi, M. (1996). The Value of the EEOC: Reexamining the Agency's Role in Employment
   Discrimination Law. Ohio State Law Journal, 57, 1-64.

Silverman, L.E., et al. (2006). Systemic Task Force Report.       Washington, DC: U.S. Equal
    Employment Opportunity Commission.

Sternlight, J.R. (2004). In Search of the Best Procedure for Enforcing Employment Discrimination
    Laws: A Comparative Analysis. Tulane Law Review, 78, 1401-1499.

Zinc, D.L., Gutman, A. (2005). Statistical Trends in Private Sector Employment Discrimination
    Suits. In F.J. Landy (ed.), Employment Discrimination Litigation: Behavioral, Quantitative,
    and Legal Perspectives. San Francisco: Jossey Bass.




IMPAQ International, LLC                    Page 17                            Final Report
                APPENDIX B: PRIORITY CHARGE HANDLING PROCESS (PCHP)

Under the Priority Charge Handling Process (PCHP), each incoming charge is assigned a priority
status based on the information provided by the charging party (the alleged victim of
discrimination) during an initial intake interview and on the perceived credibility of the charging
party. Exhibit 1 details the steps involved in the current private sector charge processing
system. First, the EEOC conducts a priority assessment of the charge and assigns it a priority
status based on the likelihood of establishing that discrimination occurred. More specifically,
Category A indicates that the charge falls within the national or local enforcement plan, or that
further investigation is likely to lead to a cause finding. Category B indicates that additional
evidence will be required to determine whether further investigation will reveal discrimination.
Category C indicates that the charge will most likely be dismissed since the EEOC is unlikely to
prove that discrimination occurred based on the evidence presented by the charging party.
With this assessment from EEOC, the charging party must then decide whether to pursue the
charge. Accordingly, the charge is settled either through mediation or through further
investigation, which may result in conciliation or litigation.

                              Exhibit B1. Private Sector Charge Processing System


                                                                                                     Conciliation,
                                                                                                     Negotiations
                                                                                                    and Agreement
                          Priority
                                                                           Determination
                        Assessment/
                                                                           Issued by EEOC
       Intake          Charging Party    Mediation         Investigation
                                                                             Field Office
                      Decides Whether
                                                                               Director
                           to File
                                                                                                   Litigation Decision


                                         Settlement




The PCHP was instrumental in reducing the pending charge inventory in the 1990s; by 2001, the
inventory consisted of approximately 32,000 charges, and processing time was reduced to just
over 6 months.70 The inventory of pending charges, however, nearly tripled from FY 2002
(29,041)71 to FY 2010 (86,338)72 for a variety of reasons. More recently, the EEOC has made
progress in reducing the inventory. The year-end inventory was 70,312 in FY 2012 with the
EEOC receiving 99,412 private sector workplace discrimination charges and resolving 111,139
charges in FY 2012.73




70
   http://www.eeoc.gov/eeoc/statistics/enforcement/index.cfm.
71
   http://www.eeoc.gov/eeoc/plan/archives/annualreports/par/2003/index.html
72
   http://www.eeoc.gov/eeoc/plan/2010par_message.cfm
73
   http://www.eeoc.gov/eeoc/plan/2012par.cfm


IMPAQ International, LLC                              Page 18                               Final Report
                           APPENDIX C: FY 2012–2016 STRATEGIC PLAN

The EEOC is committed to serve the nation more efficiently, as emphasized in the Commission’s
FY 2012-2016 Strategic Plan.74 The FY 2012-2016 Strategic Plan reflects a new strategy for
implementing the power granted to the Commission by Congress in Title VII of the Civil Rights
Act of 1964: "to prevent any person from engaging in any unlawful employment practice." In
line with the Commission’s primary mission to “stop and remedy unlawful employment
discrimination,” with a vision for “justice and equality in the workplace,” the Strategic Plan is
founded upon three values—commitment to justice, accountability, and integrity.75 In keeping
with these values, the plan prioritizes a coordinated approach to law enforcement and
recognizes that operational silos have at times hampered the agency's efforts to prevent and
remedy unlawful discrimination. To accomplish this, it outlines three objectives, each with its
own set of outcome goals:

     1. Combat employment discrimination through strategic law enforcement, with the
        outcome goals of (1) having a broad impact on reducing employment discrimination at
        the national and local levels, and (2) remedying discriminatory practices and securing
        meaningful relief for victims of discrimination.

     2. Prevent employment discrimination through education and outreach, with the outcome
        goals of (1) better informing the public how to exercise their right to employment free
        of discrimination, and (2) informing employers, unions, and employment agencies
        (covered entities) how to better address and resolve Equal Employment Opportunity
        (EEO) issues, thereby creating more inclusive workplaces.

     3. Deliver excellent and consistent service through a skilled and diverse workforce and
        effective systems, with the outcome goal of timely, informative, and high quality
        interactions with the public.

The FY 2012–2016 Strategic Plan departs from the previous plan by focusing less on measuring
output and more on achieving long-term goals. As such, the plan requires the Commission to
be proactive. This includes embarking on a year-long effort to create a conceptual framework
that will inform, justify, and support the quantitative and qualitative performance measures
identified in the plan.




74
   See, for example, Modesitt (2010), Federal Consulting Group (2010), National Employment Lawyers Association
(2007), Morra (2005), Silverman et al. (2006).
75
   This section draws heavily on the EEOC’s documentation of the Strategic Plan;
http://www.eeoc.gov/eeoc/plan/strategic_plan_12to16.cfm.


IMPAQ International, LLC                            Page 19                                 Final Report
            APPENDIX D: EEOC LEADERSHIP AND STAFF INTERVIEW GUIDE

Thank you for agreeing to speak with us today. I am Manan Roy from IMPAQ International and
I have Dr. Steven Garasky, Vice President, Human Services with me on the phone. As you may be
aware, IMPAQ International has been hired by the EEOC to conduct a Review of Evaluations to
identify significant new and useful conclusions and recommendations regarding EEOC’s private
sector enforcement activities.

Through this Review of Evaluations project, we aim to answer a few research questions such as:

        What factors are associated with the EEOC efficiently carrying out private charge
         processing activities?

        What factors are associated with the EEOC effectively carrying out private charge
         processing activities?

        What are the key information gaps related to developing strategies to more efficiently
         and effectively carry out private charge processing activities?

The purpose of this interview is to learn from your experience about the efficiency and
effectiveness of EEOC regarding its private sector enforcement activities.

The information you provide to us will be used in combination with what we learn from others,
in addition to the studies we have reviewed, to document recommendations and useful
conclusions regarding EEOC’s private sector enforcement activities.

Your comments are confidential and you will not be identified by name in any report. Steve and I
will be taking notes while we talk. With your permission, we would also like to record this
session so that we can refer to the audio to clarify our notes later if necessary. Do we have your
permission to begin recording? Thank you.

Do you have any questions before I begin?

1. Background and Context

We’d like to start with some general questions to give us a little bit of background.

   What is your current position and how long have you been at EEOC?

   Can you describe your individual role and job responsibilities?

2. Project-Related Questions

Now, we’d like to discuss your opinion and thoughts about the private sector enforcement
activities of the EEOC

   In your current capacity and based on your experience, what according to you are the
    strengths of the EEOC’s private sector enforcement activities?



IMPAQ International, LLC                      Page 20                             Final Report
   In your current capacity and based on your experience, what are your concerns about the
    EEOC’s current private sector enforcement activities?

   What would be your practical recommendations to address these concerns?

   What would be your practical recommendations to make the private sector enforcement
    activities more efficient?

   What would be your practical recommendations to make the private sector enforcement
    activities more effective?

   Are there any other studies that you would recommend us to include in the review?




IMPAQ International, LLC                   Page 21                           Final Report
       APPENDIX E: CONVERSATION WITH DISTRICT DIRECTORS PROTOCOL

Thank you for agreeing to speak with us today. I am Dr. Steven Garasky, Vice President for
Human Services Research at IMPAQ International. Joining me is Dr. Manan Roy, also from
IMPAQ. As you are aware, our firm has been hired by the EEOC to conduct a review of
evaluations of EEOC’s private sector enforcement activities. The goal of the project is to provide
recommendations to enhance effectiveness and efficiency in this area.

The purpose of this conversation is to obtain your thoughts regarding conclusions we have
reached that will inform recommendations we are considering. The information you provide will
be used in combination with what we have learned from other sources. Please know that your
comments are confidential and you will not be identified by name in any report.

We will be taking notes while we talk, but with your permission, we would also like to record
this session so that we can refer to the audio to clarify our notes later, if necessary. The audio
files will not be shared with anyone outside of IMPAQ and will be destroyed at the completion of
the project. Do we have your permission to begin recording?

Do you have any questions before I begin?

1. Background and Context

We’d like to start with some general questions to give us a little bit of background.

   All three of you are Field Directors. Is that correct?

   Can each of you tell us how long have you been with the EEOC and describe your role and
    responsibilities?

2. Project-Related Questions

Now, we’d like to discuss the private sector enforcement activities of the EEOC. You received a
two-page document derived from our draft report from Larkin Jennings of the Office of the
Inspector General. Mr. Jennings is the Contracting Officer’s Technical Representative for this
project. The first page of the document was an attempt to give you some context for the study
in terms of our approach. We’d be happy to discuss our methodology further if that would be
beneficial, but would especially like to focus on the second page which is a broad overview of
our conclusions based on the numerous studies we reviewed and interviews we conducted with
EEOC leadership and staff.

   To begin, do you have any questions about the purpose of the project or the methodology
    we used?

Shifting to our conclusions, we focused on three elements of private sector enforcement that
offered in our review and interviews a relative consensus regarding areas in which changes
could be made to enhance efficiency and/or effectiveness. These areas are identifying a
significant charge, improving the consistency of intake procedures, and improving data


IMPAQ International, LLC                       Page 22                            Final Report
collection. We would like your thoughts on our assessments in each of these areas. Again, we
know that the materials you received provide a very broad overview and limited context for
these conclusions. Nevertheless, we believe that your feedback will be very beneficial to the
study.

We’d like this to be a free flowing conversation. Feel free to provide overall comments, identify
where you agree and disagree with our thoughts, etc.

   Let’s begin with identifying a significant charge. Again, for efficiency, rather than our
    reading the materials provided to you, please jump right in with your thoughts.

   Let’s move to intake procedures. Again, please jump in with your thoughts on this area.

   Lastly, let’s talk about data collection. Do you have any comments on that paragraph?

3. Closing Remarks

Thank you again for taking the time to speak with us. We truly appreciate your thoughtful
remarks. Please do not hesitate to reach out to me if you have further questions or additional
thoughts or feedback. I can be reached at 443-539-2087. Have a good afternoon. Good-bye.




IMPAQ International, LLC                     Page 23                            Final Report
                     APPENDIX F: VALIDITY OF THE SELECTED STUDIES

We conclude that the studies included in the review are suitable and valid in relation to the
research questions they seek to answer. Importantly, the breadth of topics addressed by the
studies spanned a wide variety of research questions and sub-questions related to the
organization, functioning, responsibilities, types of charges, and activities performed by EEOC.

A key factor in assessing validity was the reputation of the research outlet. Many studies were
articles drawn from a range of peer-reviewed publications including the following:

       Harvard Negotiation Law Review

       Tulane Law Review

       Hofstra Labor & Employment Law Journal

       Drake Law Review

       Kansas Law Review

       Dickinson Law Review

       Temple Political & Civil Rights Law Review

       Ohio State Law Journal

       Cardozo Journal of International and Comparative Law

       Southern Methodist University Law Review

       Journal of the American Economic Association

Other studies were obtained from nationally renowned research organizations, for example,
the ILR School Employment and Disability Institute at Cornell University and the National
Academy of Public Administration. Some studies were conducted by external contractors to
EEOC including Dr. E. Patrick McDermott and his co-authors; Federal Consulting Group; National
Employment Lawyers Association; and Virginia Commonwealth University. Some of the studies
were conducted by EEOC staff.

Another factor in assessing validity was study methodology. The methodologies adopted in the
reviewed studies included the following:

       Review of EEOC documents and legal documents

       Review of previous studies

       Analysis of IMS and CPS databases for EEOC strengths and weaknesses

       Surveys conducted at mediation meetings



IMPAQ International, LLC                     Page 24                           Final Report
       Focus groups conducted with participants (charging parties, respondents, and EEOC
        mediators)

       Interviews with EEOC staff and EEOC customers (charging parties and respondents).

Study geography was important to assessing validity as well. The selected studies included
some that were national in scope, others that covered most EEOC field offices, and still others
that focused on special sub-populations, such as individuals with disabilities, individuals with
HIV/AIDS, and cancer patients.

Finally, although the project team reviewed many studies dating back to the 1990s, this
evaluation synthesis report emphasizes studies published after 2003. We referred to some of
the older studies where they were deemed pertinent.




IMPAQ International, LLC                    Page 25                            Final Report
              APPENDIX G: COMMENTS FROM THE OFFICE OF THE CHAIR

                U.S. EQUAL EMPLOYMENT OPPORTUNITY COMMISSION
                             Washington, D.C. 20507




Office of the Chair

March 21, 2013



 TO:              Milton Mayo
                  Inspector General

 FROM:            Claudia A. Withers
                  Chief Operating Officer

 RE:              Response to OIG Draft Report on the Review of Evaluations


 Thank you for the opportunity to comment on the draft report on the review of
 evaluations of the EEOC’s charge processing activities. The draft states that its
 purpose is to provide EEOC leadership and stakeholders with recommendations to
 enhance the efficiency and effectiveness of EEOC private sector enforcement activities,
 including mediation. We reviewed your recommendations with great interest, and we
 are in general agreement. Indeed, the robust agency wide strategic planning and
 implementation effort in which we have been engaged since July of 2011 is very much
 consistent with the purpose and recommendations in the report. We appreciate the
 independent evaluation of the Office of Inspector General, and we are pleased that our
 efforts to date appear to be in sync with this evaluation.

 The report made the following specific recommendations to the agency:

     1. The EEOC should further standardize intake procedures across field offices.
     2. The EEOC should document criteria for determining category C charges
     3. The EEOC should develop a national approach for addressing
        systemic discrimination
     4. The EEOC should continue to review the range of information obtained
        during intake interviews and how it is stored in the IMS.
     5. The EEOC should investigate the merits of expanding the information it obtains
        related to hiring and terminations.


IMPAQ International, LLC                    Page 26                       Final Report
EEOC’ s FY 2012-2016 Strategic Plan

 The mission for the agency under the Strategic Plan is to Stop and Remedy
 Unlawful Employment Discrimination. The Vision is: Justice and Equality
 in the Workplace.

 The plan contains three strategic objectives:

     1. Combat employment discrimination through strategic law enforcement
     2. Prevent employment discrimination through education and outreach
     3. Deliver excellent service through effective systems, updated technology and
        a skilled and diverse workforce

 The plan includes a number of outcome goals, strategies and performance
 measures. Based on our review of the draft report in the context of our strategic
 plan, it is clear that the report’s recommendations are contemplated in our
 implementation of the strategic plan.

          The Strategic Plan’s Performance Measure 2 calls for the development of
         a quality control plan, which will establish criteria for quality investigations and
         conciliations. The Quality Control Plan workgroup, which includes EEOC
         employees from across the agency, is engaged in discussions which should
         include issues involving intake, the data that is included in IMS, and the issues
         of consistency of service across the agency. Accordingly, the draft report’s
         recommendations will be a part of the discussion.

          Performance Measure 4 of the Strategic Plan builds on EEOC’s 2006
         Systemic Task Force Report, and requires that the agency develop a baseline
         for systemic litigation as a percentage of the litigation filed by the agency.
         Such a baseline has been set, and the agency is working to meet the baseline
         metrics.

          Performance Measure 1 of the Strategic Plan calls for the development
         of a Strategic Enforcement Plan. Such a plan was approved by the
         Commission in December 2012. Since that time, we have been engaged in
         implementation of the Strategic Enforcement Plan.

         Among other things, the SEP outlined national priorities:

                    Eliminating barriers in recruitment and hiring;
                    Protecting immigrant, migrant, and other vulnerable workers;
                    Addressing emerging and developing issues;
                    Enforcing equal pay laws;
                    Preserving access to the legal system; and


IMPAQ International, LLC                    Page 27                           Final Report
                     Preventing harassment through systemic enforcement and targeted
                    outreach

        The SEP calls for the development of teams corresponding to the above-named
priorities, to recommend national strategies for effectively addressing each priority.
The strategies under consideration will span all of EEOC’s available tools, including
administrative and legal enforcement, outreach and communications, data collection
and research, and policy development throughout the private, public and
federal sectors.

      Finally, the SEP also calls for the development of a multi-year Research and
Data Plan in FY 2013 which will focus on the SEP national priories. Work has
begun on the development of the plan, which can include a discussion of
Recommendation 5 in the draft report.

       Thank you for the opportunity to comment on the draft report. We
appreciate the independent assessment of our work, and are gratified that the
report’s recommendations are congruent with the work we are doing to fulfill the
agency’s mission.




131 M Street, N. E., Suite 6NW08F Phone (202) 663-4001 TTY (202) 663-4141 FAX (202) 663-4110 JACQUELINE.BERRIEN@EEOC.GOV




IMPAQ International, LLC                                      Page 28                                         Final Report
         APPENDIX H: COMMENTS FROM THE OFFICE OF FIELD PROGRAMS

                 U.S. EQUAL EMPLOYMENT OPPORTUNITY COMMISSION
                                Washington, D.C. 20507




Office of
Field Programs
                                         March 21, 2013
MEMORANDUM

TO :             Milton A. Mayo, Jr.
                 Inspector General

FROM :           Nicholas M. Inzeo, Director
                 Office of Field Programs

SUBJECT : Comments on Draft Report, “Review of Evaluations”
(OIG Report Number 2012-09-REV), prepared by IMPAQ International

This is in response to your request for review of the draft report, “Review of Evaluations,”
prepared by IMPAQ International, LLC. We have completed our review and offer the following
comments.

The draft report examines studies, law review articles, and reports about the EEOC in order to
synthesize the information to identify recommendations to improve the efficiency and
effectiveness of the private sector charge process, including mediation. Most of the studies were
published before 2011. The draft report relies in particular on two consultant-prepared
evaluations of the charge process (Development Services Group, “Evaluation of Intake and End-
of- Fiscal-Year Closures of the EEOC Private Sector Charge Process,” 2006; and Federal
Consulting Group, “U.S. Equal Employment Opportunity Commission Evaluation of the Priority
Charge Handling Procedures Report,” 2010), and on the EEOC’s own 2006 Systemic Task Force
Report, which made recommendations that were adopted and have been largely implemented by
the Commission in its systemic program. We appreciate this summary and are pleased that the
draft report acknowledges the accomplishments of the EEOC’s mediation program, citing several
positive studies and noting that these studies raised no “information gaps or concerns” about
mediation.

The Commission’s adoption last year of the Strategic Plan for FY 2012 - 2016 and the Strategic
Enforcement Plan introduced a comprehensive set of new strategies and mechanisms to
strengthen the effectiveness and efficiency of the agency’s work in all areas. In addition, the
draft report contains several recommendations that we agree with and have previously
implemented. With respect to the private sector charge process, we have taken a number of


IMPAQ International, LLC                       Page 29                          Final Report
actions in the years since issuance of the reports and evaluations that the draft report cites, in
order to accomplish much of what the draft report is recommending. Currently, we are focusing
our efforts on implementation of the Strategic Plan and Strategic Enforcement Plan. These
efforts are consistent with the draft report’s observations, but we note that the situation at the
agency is different now than when the cited reports and evaluations were written several years
ago.

We have the following comments on selected portions of the draft report:

     At the top of page 2, the report states that the EEOC received 99,412 private sector
        charges in FY 2012 but that “the inventory of charges has more than doubled over the last
        decade.” This static comparison is less informative than an examination of the current
        trend in charge intake and inventory. The draft report neglects to mention the significant
        reductions in inventory that have been achieved in the last two fiscal years, for an overall
        20% reduction in inventory by the end of FY 2012. Also, the report in footnotes 7 and 69
        cites to the EEOC’s Performance and Accountability Report (PAR) Highlights for FY
        2011. We suggest that the draft report reflect the most recent PAR, FY 2012, and utilize
        as its source the full report, which is available on our web site. These reports for the last
        several fiscal years provide details of many of the steps that have been taken to
        implement the Systemic Task Force Recommendations and make improvements to
        charge processing, which address not only the issues raised in the earlier studies and
        evaluations cited in the draft report, but also the recommendations contained in this draft
        report.

     With respect to the draft report’s recommendation on improving the consistency of intake
        procedures, the report primarily relies upon the 2006 evaluation by the Development
        Services Group (DSG). The draft report does not seem to capture what was done in
        response to or after the DSG report and other evaluations. In the draft report at page 7,
        the description of some “best practices” as occurring in only some field offices is
        inaccurate; in fact, all offices follow these intake practices. We have taken many actions
        since 2006, including development of uniform questionnaires, brochures and checklists;
        intake and interviewing skills training; development and refinement of the intake
        questionnaire; and refresher training for all field enforcement staff on PCHP. Moreover,
        one of the key components of the current Strategic Plan is the development of a Quality
        Control Plan, which will establish criteria for quality investigations and conciliations.
        This work is well underway.

     In the section on addressing systemic discrimination, the draft report recognizes that the
        EEOC’s FY 2012 – 2016 Strategic Plan and Strategic Enforcement Plan signify EEOC’s
        commitment to pursuing systemic investigations. However, the balance of this section
        primarily repeats the findings of the EEOC’s 2006 Systemic Task Force Report. For
        example, while acknowledging some “laudable” recent efforts to address systemic
        discrimination, the draft report concludes that “evidence indicates” that the EEOC “does
        not identify systemic discrimination in a consistent and coordinated manner” and that
        “there is a need for establishing a national approach to combat systemic discrimination.”
        Those were the very findings of the Task Force Report. When the Task Force Report was


IMPAQ International, LLC                       Page 30                             Final Report
        issued in April 2006, the Commission simultaneously adopted its numerous
        recommendations to launch a concerted nationwide effort to identify and remedy
        systemic discrimination. The Strategic Plan and Strategic Enforcement Plan build on
        what the EEOC has accomplished since 2006, and the accomplishments reported in the
        PAR demonstrate the progress that has been made.

     With respect to systemic enforcement activities, the draft report further states that “Steps
        toward a national approach should begin with district reports to EEOC headquarters
        detailing how each district is identifying and addressing systemic discrimination,” and
        that these reports should be updated regularly. In fact, this was a recommendation of the
        2006 Systemic Task Force. It was adopted by the Commission and has been a routine part
        of the agency’s systemic program since that time. District Office systemic plans were
        submitted in the first quarter of FY 2007 and, as required by the Task Force
        recommendations, provided detailed information on how the district was identifying and
        addressing systemic discrimination, with reference to specific focus areas and industries,
        targets, potential and pending litigation, and outreach and training efforts. District Office
        reporting on systemic investigations and litigation, to the General Counsel and the Office
        of Field Programs, has continued since that time on a quarterly basis.

     On page 10, the report references limitations on data in IMS but the referenced evaluation
        which serves as the basis for this finding was published in 2001 and was based on the
        agency’s previous data system (Charge Data System—CDS) and not IMS. The IMS
        significantly expanded not only the data collection capacity for the agency on its charge
        processing activity in general but also provides a search capacity for prior charge activity
        involving specific Charging Parties and Respondents.

If you have any questions about our comments, please contact me, or Sharon Shoemaker or Sue
Murphy of my staff.

cc: Claudia Withers, Chief Operating Officer

    Deidre Flippen, Director
    Office of Research, Information and Planning




IMPAQ International, LLC                       Page 31                             Final Report
APPENDIX I: COMMENTS FROM THE OFFICE OF RESEARCH, INFORMATION, AND
                            PLANNING

Response from Deidre Flippen, Director, Office of Research, Information, and Planning, EEOC

We appreciate the opportunity to comment on the Review of Evaluations - OIG Report Number
2012-09-REV). We hope our comments are useful.

Generally, we believe this evaluation suffers from two key limitations in its methodology. First,
the report is not really research per se but a review of relevant literature that is generally used
to develop research and generally not used to make recommendations. While such “reviews”
can be helpful they are generally carefully structured as meta analyses and data is extracted,
evaluated and often regenerated to produce more quantitative results. This report lacks that
type of rigor and as a result may make recommendations that are not empirically based. The
study reports a number of measures of association (page ii). Associations are generally weaker
level relationships and might be considered inappropriate measures for program evaluations.
Instead some measure of causality is preferred before programs modify their practices in
response to an evaluation. “Systemic evaluation of program impact incorporates the elements
of causality by establishing covariation and time order and by eliminating all possible spurious
and confounding influences” (Discovering Whether Programs Work: A Guide to Statistical
Methods for Program Evaluation, Laura Irwin Langbein, p. 41.)

These measurement issues may be inherent in these types of “reviews” rather than primary
research. However, the problem is greatly exacerbated by the literature selected for review.
The articles cited do not appear to be empirically based in most instances. The sources used are
surprising over represented with law reviews rather than academic journals from the social
sciences. Our office works with a number of researchers who have used our survey and charge
data to examine a number of issues relevant to the statutes we enforce and EEOC activities but
not a single one of these researchers is used in this review.

The study lacks specificity that would be important for EEOC decision makers. For example on
page 7, the study calls for greater standardization but provides no concrete examples of
inconsistencies and how these inconsistencies are problematic.




IMPAQ International, LLC                      Page 32                              Final Report