oversight

Department of Homeland Security: Taking Further Action to Better Determine Causes of Morale Problems Would Assist in Targeting Action Plans

Published by the Government Accountability Office on 2012-09-28.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                 United States Government Accountability Office

GAO              Report to Congressional Requesters




September 2012
                 DEPARTMENT OF
                 HOMELAND
                 SECURITY
                 Taking Further Action
                 to Better Determine
                 Causes of Morale
                 Problems Would
                 Assist in Targeting
                 Action Plans




GAO-12-940
                                               September 2012

                                               DEPARTMENT OF HOMELAND SECURITY
                                               Taking Further Action to Better Determine Causes
                                               of Morale Problems Would Assist in Targeting
                                               Action Plans
Highlights of GAO-12-940, a report to
congressional requesters




Why GAO Did This Study                         What GAO Found
DHS is the third largest cabinet-level         Department of Homeland Security (DHS) employees reported having lower
department in the federal government,          average morale than the average for the rest of the federal government, but
employing more than 200,000 staff in a         morale varied across components and employee groups within the department.
broad range of jobs. Since it began            Data from the 2011 Office of Personnel Management (OPM) Federal Employee
operations in 2003, DHS employees              Viewpoint Survey (FEVS)—a tool that measures employees’ perceptions of
have reported having low job                   whether and to what extent conditions characterizing successful organizations
satisfaction. DHS employee concerns            are present in their agencies—showed that DHS employees had 4.5 percentage
about job satisfaction are one example         points lower job satisfaction and 7.0 percentage points lower engagement in their
of the challenges the department faces
                                               work overall. Engagement is the extent to which employees are immersed in their
implementing its missions. GAO has
                                               work and spending extra effort on job performance. Moreover, within most
designated the implementation and
transformation of DHS as a high risk
                                               demographic groups available for comparison, DHS employees scored lower on
area, including its management of              average satisfaction and engagement than the average for the rest of the federal
human capital, because it represents           government. For example, within most pay categories DHS employees reported
an enormous and complex undertaking            lower satisfaction and engagement than non-DHS employees in the same pay
that will require time to achieve in an        groups. Levels of satisfaction and engagement varied across components, with
effective and efficient manner. GAO            some components reporting scores above the non-DHS averages. Several
was asked to examine: (1) how DHS’s            components with lower morale, such as Transportation Security Administration
employee morale compared with that             (TSA) and Immigration and Customs Enforcement (ICE), made up a substantial
of other federal employees, and (2) the        share of FEVS respondents at DHS, and accounted for a significant portion of
extent to which DHS and selected               the overall difference between the department and other agencies. In addition,
components have determined the root            components that were created with the department or shortly thereafter tended to
causes of employee morale, and                 have lower morale than components that previously existed. Job satisfaction and
developed action plans to improve              engagement varied within components as well. For example, employees in TSA’s
morale. To address these objectives,           Federal Security Director staff reported higher satisfaction (by 13 percentage
GAO analyzed survey evaluations,               points) and engagement (by 14 percentage points) than TSA’s airport security
focus group reports, and DHS and               screeners.
component action planning documents,
and interviewed officials from DHS and         DHS has taken steps to determine the root causes of employee morale problems
four components, selected based on             and implemented corrective actions, but it could strengthen its survey analyses
workforce size, among other things.            and metrics for action plan success. To understand morale problems, DHS and
                                               selected components took steps, such as implementing an exit survey and
What GAO Recommends                            routinely analyzing FEVS results. Components GAO selected for review—ICE,
GAO recommends that DHS examine                TSA, the Coast Guard, and Customs and Border Protection—conducted varying
its root cause analysis efforts and add        levels of analyses regarding the root causes of morale to understand leading
the following, where absent:                   issues that may relate to morale. DHS and the selected components planned
comparisons of demographic groups,             actions to improve FEVS scores based on analyses of survey results, but GAO
benchmarking, and linkage of root              found that these efforts could be enhanced. Specifically, 2011 DHS-wide survey
cause findings to action plans; and            analyses did not include evaluations of demographic group differences on
establish clear and measurable metrics         morale-related issues, the Coast Guard did not perform benchmarking analyses,
of action plan success. DHS concurred          and it was not evident from documentation the extent to which DHS and its
with our recommendations.                      components used root cause analyses in their action planning. Without these
                                               elements, DHS risks not being able to address the underlying concerns of its
                                               varied employee population. In addition, GAO found that despite having broad
                                               performance metrics in place to track and assess DHS employee morale on an
                                               agency-wide level, DHS does not have specific metrics within the action plans
View GAO-12-940. For more information,         that are consistently clear and measurable. As a result, DHS’s ability to assess
contact David C. Maurer at (202) 512-9627 or   its efforts to address employee morale problems and determine if changes
maurerd@gao.gov.
                                               should be made to ensure progress toward achieving its goals is limited.
                                                                                      United States Government Accountability Office
Contents


Letter                                                                                 1
               Background                                                              5
               DHS Employees Reported Lower Morale than the Rest of the
                 Federal Government, but Morale Varied across DHS
                 Components and Employee Groups                                        7
               DHS Took Steps to Determine Root Causes of Morale Problems
                 and Implemented Corrective Actions, but Could Strengthen Its
                 Efforts                                                              17
               Conclusion                                                             34
               Recommendations for Executive Action                                   35
               Agency Comments and Our Evaluation                                     35

Appendix I     Statistical Analysis of Employee Morale at Department of Homeland
               Security and Other Agencies                                            37



Appendix II    Scope and Methodology                                                  58



Appendix III   DHS and Selected Component Steps Taken to Determine Root Causes
               of Morale Problems                                              61



Appendix IV    Selected Components’ Data Sources for Evaluating Morale, Other
               than the Federal Employee Viewpoint Survey                             68



Appendix V     Comments from the Department of Homeland Security                      71



Appendix VI    Comments from the U.S. Office of Personnel Management                  73



Appendix VII   GAO Contact and Staff Acknowledgments                                  74




               Page i                                      GAO-12-940 DHS Employee Morale
Tables
          Table 1: 2011 FEVS Job Satisfaction and Engagement Scores by
                   DHS Component (Sorted by Job Satisfaction Index Score)        11
          Table 2: DHS-wide and Component Action Plan Goals and
                   Examples of Low-Scoring FEVS Topics Addressed through
                   such Goals                                                    25
          Table 3: DHS-Wide and Selected Components’ Action Planning
                   Steps                                                         29
          Table 4: Examples of DHS-Wide and Selected Components’
                   Measures of Success                                           31
          Table 5: DHS and Non-DHS Employee Engagement Index by
                   Demographic Group for the 2011 FEVS                           41
          Table 6: Model Estimates of Employee Engagement Index at DHS
                   and Other Agencies Using the 2011 FEVS                        44
          Table 7: OPM Employee Morale Index by DHS Component and
                   Offices Using the 2011 FEVS                                   52
          Table 8: Morale at Preexisting and Recently Created Components
                   of DHS Using the 2011 FEVS                                    55
          Table 9: Model Estimates of the Difference in Engagement and Job
                   Satisfaction between Employees in DHS Components and
                   Non-DHS Agencies Based on the 2011 FEVS                       56


Figures
          Figure 1: Percentage of Satisfied DHS Employees Compared with
                   Governmentwide Averages, 2006, 2008, 2010, and 2011            8
          Figure 2: OPM Job Satisfaction and Employee Engagement Indexes
                   in the 2011 FEVS, for Selected Categories of DHS and
                   Non-DHS Employees                                             10
          Figure 3: Satisfaction and Engagement by TSA Employee Group,
                   2011                                                          13
          Figure 4: The Extent to Which OCHCO, TSA, CBP, ICE, and the
                   Coast Guard Incorporated Recommended Factors in
                   Analyzing 2011 FEVS Results                                   20
          Figure 5: OPM’s Six Steps for Action Planning to Improve FEVS
                   Scores                                                        28
          Figure 6: Engagement Index Scores by Supervisory Status and
                   Tenure, for DHS and Non-DHS Employees                         47
          Figure 7: Satisfaction Index Scores by Supervisory Status and
                   Tenure, for DHS and Non-DHS Employees                         49




          Page ii                                     GAO-12-940 DHS Employee Morale
Figure 8: DHS’s 2011 Component Comparison Based on Four
         HCAAF Indexes                                                                    63
Figure 9: DHS HCAAF Scores since 2006                                                     64




Abbreviations

AES               Annual Employee Survey
CBP               U.S. Customs and Border Protection
CIS               U.S. Citizenship and Immigration Services
DHS               Department of Homeland Security
EE                employee engagement
EEESC             Employee Engagement Executive Steering Committee
FEMA              Federal Emergency Management Agency
FEVS              Federal Employee Viewpoint Survey
FLETC             Federal Law Enforcement Training Center
FOCS              Federal Organizational Climate Survey
GS                General Schedule
HCAAF             Human Capital Assessment and Accountability Framework
I&A               Intelligence and Analysis
ICE               Immigration and Customs Enforcement
JS                job satisfaction
MGMT              Management Directorate
MVP               Most Valuable Perspective
NPPD              National Protection and Programs Directorate
OAS               Organizational Assessment Survey
OCHCO             Office of the Chief Human Capital Officer
OIG               Office of the Inspector General
OPM               Office of Personnel Management
OS                Office of the Secretary
S&T               Science and Technology
TSA               Transportation Security Administration
USSS              U.S. Secret Service


This is a work of the U.S. government and is not subject to copyright protection in the
United States. The published product may be reproduced and distributed in its entirety
without further permission from GAO. However, because this work may contain
copyrighted images or other material, permission from the copyright holder may be
necessary if you wish to reproduce this material separately.




Page iii                                                 GAO-12-940 DHS Employee Morale
United States Government Accountability Office
Washington, DC 20548




                                   September 28, 2012

                                   The Honorable Susan M. Collins
                                   Ranking Member
                                   Committee on Homeland Security and Governmental Affairs
                                   United States Senate

                                   The Honorable Michael T. McCaul
                                   Chairman
                                   The Honorable William R. Keating
                                   Ranking Member
                                   Committee on Homeland Security, Subcommittee on Oversight,
                                   Investigations, and Management
                                   House of Representatives

                                   The Department of Homeland Security (DHS) is the third largest cabinet-
                                   level department in the federal government, employing more than
                                   200,000 staff in a broad range of jobs, including aviation and border
                                   security, emergency response, cybersecurity analysis, and chemical
                                   facility inspection. The DHS workforce is situated throughout the nation,
                                   carrying out activities in support of DHS’s mission to (1) prevent terrorism
                                   and enhance security, (2) secure and manage the nation’s borders, (3)
                                   enforce and administer immigration laws, (4) safeguard and secure
                                   cyberspace, (5) ensure resilience from disasters, and (6) provide
                                   essential support to national and economic security.

                                   Since it began operations in 2003, DHS employees have reported having
                                   low job satisfaction. In 2011, for example, DHS’s scores on the Office of
                                   Personnel Management (OPM) Federal Employee Viewpoint Survey
                                   (FEVS)—a tool that measures employees’ perceptions of whether and to
                                   what extent conditions characterizing successful organizations are
                                   present in their agency—and the Partnership for Public Service’s (the




                                   Page 1                                          GAO-12-940 DHS Employee Morale
Partnership) rankings of the Best Places to Work in the federal
government, were generally low. 1

DHS employee concerns about job satisfaction are one example of the
challenges the department faces in implementing its missions. In January
2003, we designated the implementation and transformation of DHS as
high risk, including its management of human capital, because it
represented an enormous and complex undertaking that would require
time to achieve in an effective and efficient manner, and it has remained
on our high-risk list since that time. 2 Improving human capital
management is a DHS priority, reflected through several DHS-wide
strategy documents. In June 2012, DHS provided us with its updated
Integrated Strategy for High Risk Management (Integrated Strategy),
which identified activities to improve employee job satisfaction scores,
among other things. In addition, DHS has issued various other strategies
and plans for its human capital activities and functions, such as a human
capital strategic plan for fiscal years 2009 through 2013, 3 and a workforce
strategy for fiscal years 2011 through 2016, which contains the




1
 OPM, the central human resources agency for the federal government, has conducted
the FEVS every year since 2010. Prior to 2010, OPM conducted the survey during even
numbered years, beginning in 2004. The most recent survey sample of 2011 included
employees from 29 major federal agencies, as well as 54 independent federal
organizations. The survey results represent a snapshot in time of the perceptions of the
federal workforce. In 2012 the FEVS will be implemented as a census, rather than a
sample-based survey, in an effort to gather the opinions of the entire federal workforce.
According to the Partnership, the Best Places to Work ranking is based on employee
responses to the following three FEVS assessment items: (1) I recommend my
organization as a good place to work. (2) Considering everything, how satisfied are you
with your job? (3) Considering everything, how satisfied are you with your organization?
2
 In determining whether a government program is high risk, we consider whether it
involves national significance, a management function that is key to performance and
accountability, or whether there is an inherent or systematic problem, among other things.
Our prior work has identified four high-risk areas for which DHS has primary or significant
responsibilities: (1) Implementing and Transforming DHS, (2) The National Flood
Insurance Program, (3) Protecting the Federal Government’s Information Systems and the
Nation’s Critical Infrastructure, and (4) Establishing Effective Mechanisms for Sharing
Terrorism-Related Information to Protect the Homeland. GAO, Department of Homeland
Security: Progress Made in Implementation and Transformation of Management
Functions, but More Work Remains, GAO-10-911T (Washington, D.C.: Sept. 30, 2010).
3
DHS, Human Capital Strategic Plan, Fiscal Years 2009-2013 (Washington, D.C.).




Page 2                                                   GAO-12-940 DHS Employee Morale
department’s workforce goals, objectives, and performance measures for
human capital management. 4

We have previously reported that successful organizations empower and
involve their employees to gain insights about operations from a frontline
perspective, increase their understanding and acceptance of
organizational goals and objectives, and improve motivation and morale. 5
In light of the critical nature of DHS’s mission to protect the security and
economy of our nation and the importance of attracting and retaining
engaged and satisfied DHS employees to perform its work, you asked us
to assess DHS’s efforts to address employee morale. 6 Thus, this report
addresses the following questions:

•   How does DHS’s employee morale compare with that of other federal
    government employees?
•   To what extent have DHS and selected components determined the
    root causes of employee morale and developed action plans to
    improve morale?

To address these questions, we analyzed survey evaluations for the 2011
FEVS, focus group reports from 2007, and DHS and selected component
2011 action planning documents and compared the documents with OPM
and Partnership guidance. We also interviewed officials from DHS’s
Office of the Chief Human Capital Officer (OCHCO), and human capital
officials from four components—U.S. Customs and Border Protection
(CBP), U.S. Immigration and Customs Enforcement (ICE), Transportation
Security Administration (TSA), and U.S. Coast Guard (Coast Guard). We
selected these four DHS components based on their workforce size and
how their 2011 Job Satisfaction and Engagement Index scores compared




4
DHS, Workforce Strategy for Fiscal Year 2011-2016 (Washington, D.C.).
5
 GAO, High-Risk Series: Strategic Human Capital Management, GAO-03-120
(Washington, D.C.: January 2003).
6
 For the purposes of this report, we define employee morale as being characterized by job
satisfaction and employee engagement, both of which are measured in OPM’S FEVS. The
job satisfaction index, composed of seven FEVS questions such as “my work gives me a
feeling of personal accomplishment,” indicates the extent to which employees are satisfied
with their jobs and various aspects thereof. The Engagement Index, composed of 15
FEVS questions, indicates the extent to which employees are immersed in the content of
the job and energized to spend extra effort in job performance.




Page 3                                                  GAO-12-940 DHS Employee Morale
with the non-DHS average. 7 The components selected had scores both
above, below, and similar to the average. In addition, we interviewed
representatives of employee groups within the four selected components
to gather employee perspectives on drivers of morale. Details of the
selected component index scores, and their statistical significance, are
reported in appendix I.

To compare DHS’s employee morale with that of other federal
government employees, we analyzed the 2011 FEVS results and
reviewed OPM survey results issued since 2004, the first full year in
which survey data are available. During the course of our analysis, we
interviewed knowledgeable agency officials, reviewed relevant
documentation, tested data for errors, and determined that the FEVS data
are sufficiently reliable for the purposes of this report. As part of this
analysis, we compared 2011 DHS and non-DHS job satisfaction and
engagement score results by several categories of employees, such as
supervisory status, pay grade, and age. We also compared satisfaction
and engagement scores within the selected components by employee
group, where possible. For example, within TSA, we compared
satisfaction and engagement scores reported by Transportation Security
Officers, Federal Security Director staff, headquarters staff, and Federal
Air Marshals.

To determine the extent to which DHS and the selected components
identified the root causes of employee morale and developed action plans
for improvements, we reviewed agency analysis results, interviewed
agency human capital officials and representatives of employee groups,
and evaluated action plans for improving morale. We also compared DHS
and selected components’ morale root cause analyses and related action
plans with available guidance for such efforts.

We conducted this performance audit from October 2011 through
September 2012 in accordance with generally accepted government
auditing standards. Those standards require that we plan and perform the
audit to obtain sufficient, appropriate evidence to provide a reasonable
basis for our findings and conclusions based on our audit objectives. We
believe that the evidence obtained provides a reasonable basis for our



7
 Throughout this report, non-DHS refers to all federal employee FEVS responses outside
of DHS.




Page 4                                                GAO-12-940 DHS Employee Morale
             findings and conclusions based on our audit objectives. Appendix II
             contains more detailed information about our scope and methodology.


             Federal employees are routinely surveyed through OPM’s administration
Background   of the FEVS, which is administered to collect data on federal employees’
             perceptions about how effectively agencies are managing their
             workforces. The FEVS is a tool that measures employees’ perceptions of
             whether, and to what extent, conditions that characterize successful
             organizations are present in their agencies, according to OPM. 8 This
             survey was administered for the first time in 2002 and then repeated in
             2004, 2006, 2008, 2010, 2011, and April through June 2012. 9 The survey
             provides general indicators of how well the federal government is
             managing its human resources management systems. It also serves as a
             tool for OPM to assess individual agencies and their progress on strategic
             management of human capital, and gives senior managers employee
             perspectives on agency management. Specifically, the survey includes
             categories of questions asking employees for their perspectives on their
             work experience, work unit, agency, supervisor, leadership, and
             satisfaction. OPM intends for agency managers to use the findings to
             develop policies and action plans for improving agency performance. In
             2011, OPM provided a summary of FEVS findings to DHS. In that report,
             OPM summarized DHS’s survey results relative to governmentwide
             averages and provided positive and negative response levels for each
             survey question. Also included in the report was action planning guidance
             for using FEVS results to improve human capital management.

             To guide governmentwide efforts to support agency mission results with
             human capital strategies and in response to the Chief Human Capital
             Officers Act of 2002, 10 OPM created the Human Capital Assessment and
             Accountability Framework (HCAAF). Agencies are evaluated by OPM on
             their progress in meeting HCAAF standards in areas such as talent



             8
              OPM, 2011 Federal Employee Viewpoint Survey, Empowering Employees, Inspiring
             Change, Department of Homeland Security, Agency Management Report. (Washington,
             D.C.).
             9
              Data for the 2012 FEVS will not be available until November 2012 and thus are not relied
             upon for this report.
             10
               Pub. L. No. 107-295, § 1304, 116 Stat. 2315, 2289 (2002) (codified at 5 U.S.C. §
             1103(c)).




             Page 5                                                  GAO-12-940 DHS Employee Morale
management, which is focused on agencies having quality people with
the appropriate competencies in mission-critical activities. The FEVS job
satisfaction index is one of the metrics used by OPM to assess whether
agencies are effectively managing the talent management system. The
FEVS provides one source of information for evaluating success on other
HCAAF standards as well by measuring responses to groups of FEVS
questions for four indices. The four index measures are: Leadership and
Knowledge Management; Results-Oriented Performance Culture; Talent
Management; and Job Satisfaction. In addition, in 2011, OPM added an
index to measure employee engagement, which OPM defines as the
extent to which an employee is immersed in the content of the job and
energized to spend extra effort in job performance.

DHS’s OCHCO is responsible for implementing policies and programs to
recruit, hire, train and retain DHS’s workforce. As the department-wide
unit responsible for human capital issues within DHS, OCHCO provides
OPM with a DHS-wide action plan every other year, with the next plan
due in January 2013. OCHCO also provides guidance and oversight to
the DHS components related to morale issues. For example, OCHCO
provides a survey analysis and action planning tool that the components
must use in response to FEVS results to develop action plans for
improving employees’ positive scores. 11 These plans are to state
objectives and identify actions to be taken in response to survey results.
OCHCO also has provided oversight by reviewing and providing feedback
on component action plans.




11
  OCHCO’s authority for requiring components to use the action planning tool is based on
DHS’s Human Capital Line of Business Integration and Management Directive, issued in
2004.




Page 6                                                 GAO-12-940 DHS Employee Morale
                          Data from the 2011 FEVS show that DHS employees have lower average
DHS Employees             levels of job satisfaction and engagement overall and across most
Reported Lower            demographic groups available for comparison, such as pay grade, when
                          compared with the average for the rest of the federal government. Levels
Morale than the Rest      of satisfaction and engagement vary across components, with some
of the Federal            components reporting satisfaction or engagement above the average for
Government, but           the rest of the government. Similarly, these measures of morale vary
                          within components as well, with some employee groups reporting higher
Morale Varied across      morale than other groups within the same component.
DHS Components and
Employee Groups
DHS Employees as a        As shown in figure 1, DHS employees generally reported improvements
Whole Reported Lower      in job satisfaction index levels since 2006 that narrowed the gap between
Satisfaction and          DHS and the governmentwide average. 12 However, employees continue
                          to indicate less satisfaction than the governmentwide average. 13
Engagement According to
                          Partnership analysis of FEVS data also indicates consistent levels of low
Several Measures          employee satisfaction relative to other federal agencies. Similar to its
                          2011 ranking, 31st of 33 federal agencies, the Partnership ranked DHS
                          28th of 32 in 2010, 28th of 30 in 2009, and 29th of 30 in 2007 in the Best
                          Places to Work ranking on overall scores for employee satisfaction and
                          commitment. 14




                          12
                            Two thousand six is the first year in which job satisfaction index data were made
                          available and can be compared between DHS and the rest of the federal government.
                          13
                            For the purposes of our report, we list governmentwide averages in some instances,
                          which include DHS. In other instances where we were able to make statistical
                          adjustments, we report non-DHS averages, which exclude DHS.
                          14
                            Partnership for Public Service and the Institute for the Study of Public Policy
                          Implementation at the American University School of Public Affairs, The Best Places to
                          Work in the Federal Government. The Partnership for Public Service’s ranking cited here
                          is composed of large agencies, defined as agencies with more than 2,000 full-time
                          permanent employees.




                          Page 7                                                 GAO-12-940 DHS Employee Morale
Figure 1: Percentage of Satisfied DHS Employees Compared with Governmentwide
Averages, 2006, 2008, 2010, and 2011




Note: Because the FEVS was not administered each year, the job satisfaction index and DHS versus
governmentwide averages are available only for 2006, 2008, 2010, and 2011.


Our analyses of 2011 FEVS results also indicate that average DHS-wide
employee satisfaction and engagement scores were consistently lower
when compared with average non-DHS employee scores in the same
demographic groups. As shown in figure 2, comparisons of DHS with
non-DHS employees by supervisory status, pay group, and tenure
indicate that satisfaction and engagement are lower across many of the
DHS groups where statistically significant differences are evident. 15 For
example, across pay categories DHS satisfaction and engagement were
lower than the scores for the same non-DHS employee pay groups, with
the exception of senior executives, senior leaders, employees with less



15
  Because statistical significance is a function of two things—the size of the difference and
the size of the sampled groups being compared—the biggest differences are not always
the differences that are significant.




Page 8                                                      GAO-12-940 DHS Employee Morale
than 1 year of tenure, and General Schedule pay grades 1-6. 16 Similarly,
job satisfaction and engagement scores for DHS management and non-
management employees were lower than for the same non-DHS
employee groups.




16
  The differences between DHS and non-DHS senior leader, General Schedule 1-6, and
less than 1 year of tenure satisfaction and engagement were not statistically significant.




Page 9                                                   GAO-12-940 DHS Employee Morale
Figure 2: OPM Job Satisfaction and Employee Engagement Indexes in the 2011 FEVS, for Selected Categories of DHS and
Non-DHS Employees




                                       Note: Estimates of job satisfaction and employee engagement have a margin of error at the 95
                                       percent confidence level of no more than plus or minus 5.1 percentage points, except for employees
                                       in the Senior Leader pay group. Because statistical significance is a function of two things—the size
                                       of the difference and the size of the sampled groups being compared—the biggest differences are not
                                       always the differences that are significant.




                                       Page 10                                                        GAO-12-940 DHS Employee Morale
                            a
                             The Job Satisfaction index, composed of seven Federal Employee Viewpoint Survey (FEVS)
                            questions, indicates the extent to which employees are satisfied with their jobs and various aspects
                            thereof.
                            b
                             The Engagement index, composed of 15 FEVS questions, indicates the extent to which employees
                            are immersed in the content of the job and energized to spend extra effort in job performance.
                            c
                             The “Other” pay group may include employees who are not part of the General Schedule system,
                            such as those in pay-band systems.
                            d
                             Denotes statistically significant differences between DHS and non-DHS employees that are
                            distinguishable from zero at the 0.05 level.




Employee Satisfaction and   Satisfaction and engagement variation across components. As
Engagement Vary across      shown in table 1, the 2011 FEVS data indicate that satisfaction and
and within DHS              engagement levels vary across DHS components. For example, TSA is
                            11.6 percentage points below the non-DHS average on the job
Components                  satisfaction index, but other large components, such as CBP and the
                            Coast Guard, are 1 and 1.5 percentage points more satisfied than the
                            average for the rest of the government, respectively. Three other
                            components—the Inspector General, the Federal Law Enforcement
                            Training Center, and the U.S. Secret Service—also score above the non-
                            DHS engagement averages on the job satisfaction or engagement
                            indexes. Morale varies widely across all components, with the job
                            satisfaction index ranging from 56.9 to 71.6 percent and the employee
                            engagement index ranging from 53.2 to 70.6 percent.

                            Table 1: 2011 FEVS Job Satisfaction and Engagement Scores by DHS Component
                            (Sorted by Job Satisfaction Index Score)

                                                                                                 Job              Employee
                                                                                  Satisfaction index       Engagement index
                                Federal Law Enforcement Training Center                           71.6*                      66.2
                                Inspector General                                                  70.8                     70.5*
                                U.S. Coast Guard                                                   70.0                     70.6*
                                U.S. Customs and Border Protection                                 69.5                     62.9*
                                U.S. Secret Service                                                69.4                      68.1
                                Non-DHS government employees                                       68.5                      67.1
                                U.S. Citizenship and Immigration Service                          66.6*                     64.0*
                                Under Secretary for Management                                    65.7*                      65.7
                                DHS, no sub-agency                                                 65.2                     60.5*
                                DHS (overall)                                                     64.0*                     60.1*
                                Federal Emergency Management Agency                               63.2*                     58.0*
                                Office of the Secretary                                           63.0*                     63.6*
                                National Policy and Programs Directorate                          61.9*                     57.7*




                            Page 11                                                         GAO-12-940 DHS Employee Morale
                                                                   Job             Employee
                                                    Satisfaction index      Engagement index
 U.S. Immigration and Customs Enforcement                          60.6*                    58.0*
 Science and Technology Directorate                                59.7*                    55.7*
 Office of Intelligence and Analysis                               57.6*                    53.2*
 Transportation Security Administration                            56.9*                    54.4*
Source: GAO analysis of 2011 FEVS data.

Note: Estimates of job satisfaction and employee engagement have a 95 percent margin of error of
no more than plus or minus 6.3 percentage points. Asterisks indicate estimates that are
distinguishable from the non-DHS estimate at the 0.05 level.


Satisfaction and engagement variation within components. Within
the selected components reviewed—the Coast Guard, ICE, CBP, and
TSA—satisfaction and engagement varied across workgroups as well.
For example, as shown in figure 3, responses by TSA employee groups
varied widely, with the screening workforce reporting significantly lower
satisfaction and engagement scores. 17




17
  For the purposes of this report we define employee groups as the different occupational
groups that exist within DHS components, such as TSA’s Federal Air Marshals, screeners,
and federal security director staff.




Page 12                                                      GAO-12-940 DHS Employee Morale
Figure 3: Satisfaction and Engagement by TSA Employee Group, 2011




                                      Note: Estimates of job satisfaction and employee engagement have a 95 percent margin of error of
                                      no more than plus or minus 4.4 percentage points.


                                      Employee group scores showed variability in the three other selected
                                      DHS components as well. 18 For example:

                                      •    CBP. Border Patrol employees were 8 percentage points more
                                           satisfied and 12 percentage points more engaged than CBP field
                                           operations employees. 19


                                      18
                                        All differences between pairs of groups in this paragraph are distinguishable from zero
                                      at the 0.05 level.
                                      19
                                        A border patrol agent is involved in detection, prevention and apprehension of terrorists,
                                      undocumented aliens and smugglers of aliens at or near the land border. A CBP field
                                      operations officer is responsible for, among other things, determining the nationality and
                                      identity of each applicant for admission to the United States and for preventing the entry of
                                      ineligible aliens, including criminals, terrorists, and drug traffickers.




                                      Page 13                                                      GAO-12-940 DHS Employee Morale
•    Coast Guard. Satisfaction and engagement levels exceeded the non-
     DHS average for some civilian employee groups, such as those under
     the Chief of Staff for Mission Support, but some groups were
     substantially higher. These groups include Districts 13 and 14, which
     were 18.6 and 10.3 percentage points more satisfied than non-DHS
     employees, respectively. 20
•    ICE. Homeland security investigators and immigration enforcement
     employees were less satisfied and engaged than many other
     employee groups in DHS and the average for the rest of the
     government. 21 Homeland security investigators were 5.5 percentage
     points lower on satisfaction and 8.2 percentage points lower on
     engagement than the non-DHS averages. Enforcement and Removal
     employees were also less satisfied and engaged, with scores 12.7
     percentage points below on satisfaction and 14.4 percentage points
     lower on engagement than the non-DHS averages.

In addition to variation in satisfaction and engagement levels across
employee groups, representatives of DHS employee groups we
interviewed identified a range of issues that may be creating lower
satisfaction rates among employees. 22 These examples highlight the
variety of issues that can lead to morale problems and may be unique to
particular DHS components. 23 For example:

•    A TSA screener union representative described TSA’s performance
     assessment system as a key driver of morale problems among
     passenger screeners. According to the union representative, if a
     screener fails a portion of the annual examination more than three


20
 The Coast Guard is organized into two major commands that are responsible for overall
mission performance: the Pacific area that includes District 13 which covers the Pacific
Northwest area, and District 14 which covers the Hawaii and Guam region.
21
  Homeland security investigators investigate crime, human rights violations and human
smuggling, smuggling of narcotics, weapons and other types of contraband, financial
crimes, cybercrime, and export enforcement issues while enforcement removal operations
employees work to enforce the nation’s immigration laws by ensuring the removal of
aliens who pose a threat to national security or public safety.
22
   For employee groups that had union representation, we interviewed union
representatives who identified employee group perspectives on employee morale. For
those groups without union representation, we convened focus groups of employees to
discuss employee perspectives on morale.
23
  The examples provided by agency officials are the observations of individual employees
and are not representative of all employee opinions.




Page 14                                                GAO-12-940 DHS Employee Morale
    times, the screener will be terminated. The union representative
    explained that failing three times is possible, even for highly effective
    screeners who may not be effective under testing conditions because
    of anxiety about the examination, resulting in a significant burden on
    the screeners and, therefore, the performance assessment will result
    in lower morale.
•   ICE homeland security investigators who participated in a focus group
    we held cited frustrations with frequent turnover in regional leadership
    positions, which they stated negatively affects employee morale.
    Unequal resource allocations across investigative groups were also
    described as leading to lower morale among investigators.
•   A union representative for CBP’s field operations employees
    described staffing shortages at ports of entry, temporary assignments
    to the southwest border that affect work-life balance, and
    management resistance to employee telework arrangements, among
    other things, as resulting in employee morale problems.
•   Border Patrol union representatives cited uncertainties in overtime
    pay policy, living conditions at small, temporary shelters for Border
    Patrol agents deployed in an area, and inflexible employee work
    scheduling practices, among other things, as creating morale
    problems among Border Patrol agents.

Coast Guard civilian officials who participated in a focus group we held,
on the other hand, provided examples of drivers of high levels of morale
within the Coast Guard. The officials described a Coast Guard culture of
mission focus that has led to high morale among civilian Coast Guard
employees. For example, the officials stated that a sense of making a
difference in maritime security and safety through work activities such as
vessel inspections, contingency planning for natural disasters, and
training Coast Guard employees results in employees who are engaged
and satisfied with their jobs. The officials also described a Coast Guard
cultural focus on team cohesion and shared successes, among both
military and civilian Coast Guard personnel, both of which are recognized
by Coast Guard leadership, according to the officials. For example, the
officials stated that Coast Guard leadership designates a portion of
awards for team successes, rather than individual achievement, which the
officials we interviewed found more satisfying than individual awards.




Page 15                                         GAO-12-940 DHS Employee Morale
A statistical analysis of 2011 FEVS and employee demographic data we
conducted suggested several other explanations for differences in
morale 24 between DHS and non-DHS agencies:

•    Several of the DHS components with lower morale, such as TSA and
     ICE, make up a substantial share of FEVS respondents at DHS, as
     shown in appendix I, table 7. Those components have more influence
     on the agency’s overall morale score than smaller components—
     many of which have higher average morale scores. Consequently, the
     gap between DHS and the rest of the government in employee morale
     is driven primarily by the scores of a few large components.
•    DHS is not more likely than other agencies to employ the types of
     staff who tend to have lower morale across all agencies, as shown in
     appendix I, table 6. Instead, employees in the various groups we
     analyzed had lower morale at DHS than the same types of employees
     at other agencies. This suggests that the gap may be explained by
     factors unique to DHS, such as management practices and the nature
     of the agency’s work, or by differences among employees we could
     not analyze. 25
•    DHS employees who joined the department since its creation tend to
     have lower morale than employees who joined the department as part
     of preexisting components, as shown in appendix I, table 8. DHS
     employees who started working for the agency between 1 and 10
     years ago are less engaged than employees with similar tenures and
     demographics at other agencies. In addition, several of the least
     engaged components, such as the Intelligence and Analysis and
     Science and Technology divisions, as shown in appendix I, table 7,
     were created with the department or subsequently, rather than being
     added from elsewhere in the federal government.




24
 We used measures of job satisfaction and employee engagement as indicators of
morale in our analysis.
25
  We used a statistical method called “Oaxaca decomposition” to divide the overall
difference in morale between DHS and non-DHS agencies into two parts: the part
explained by employee characteristics present in the FEVS (i.e. supervisory status,
employee tenure, age and location), and the part explained by how those characteristics
affect morale. This let us assess whether the morale gap is explained by available
employee characteristics or by how those characteristics affect morale differently at DHS
because of unique characteristics of the agency such as management practices or
program goals. The method cannot identify specific unique characteristics that are
responsible, however. Appendix I describes this analysis in more detail.




Page 16                                                 GAO-12-940 DHS Employee Morale
                         The variation of factors that can result in morale problems as suggested
                         by these examples, as well as the variation in levels of satisfaction and
                         engagement among employee groups, underscores the importance of
                         looking beyond survey scores to understand where problems, such as
                         low employee satisfaction, are taking place within the organization, and to
                         identify and address the causes of these problems.

                         Appendix I provides comparisons for additional demographic groups,
                         including age, component tenure, and location, and summarizes our
                         statistical analysis that examines the relationships between each of these
                         factors and satisfaction and engagement, holding constant each of the
                         other factors. The appendix also provides a more detailed list of
                         satisfaction and engagement estimates for components and offices, in
                         some cases holding constant demographic differences among
                         employees.


                         DHS and the selected components have taken steps to understand
DHS Took Steps to        morale problems, such as holding focus groups, implementing an exit
Determine Root           survey, and routinely analyzing FEVS results. On the basis of FEVS
                         results, DHS and the selected components planned actions to improve
Causes of Morale         FEVS scores. However, we found that DHS could enhance its survey
Problems and             analysis and monitoring of action plan results. In addition, according to
Implemented              DHS’s Integrated Strategy for addressing the implementing and
                         transforming high risk area, DHS has begun implementing activities to
Corrective Actions,      address morale but has not yet improved DHS’s scores on OPM’s job
but Could Strengthen     satisfaction index or its ranking on the Partnership’s Best Places to Work
                         in the Federal Government.
Its Efforts
DHS and Selected         DHS’s OCHCO has taken several steps to understand morale problems
Components’ Have Taken   DHS-wide. Specifically, since 2007, OCHCO:
Steps to Understand
                         •   Conducted focus groups DHS-wide in 2007 to determine employee
Morale Problems              concerns related to morale, which identified employee concerns in
                             areas of leadership, communication, empowerment, and resources.
                         •   Performed statistical analysis in 2008 to identify workplace factors that
                             drove employee job satisfaction, finding that the DHS mission and
                             supervisor support, among other things, drove employee job
                             satisfaction.
                         •   Initiated an exit survey, first administered DHS-wide in 2011, to
                             understand why employees chose to leave their position. The survey




                         Page 17                                         GAO-12-940 DHS Employee Morale
     found lack of quality supervision and advancement opportunities were
     the top reasons for leaving. 26
•    Analyzed 2011 FEVS results, among other things, showing where
     lower scores on HCAAF indices were concentrated among several
     components—Intelligence and Analysis, TSA, ICE, National
     Protection and Programs Directorate, and the Federal Emergency
     Management Agency (FEMA).
•    Launched an Employee Engagement Executive Steering Committee
     (EEESC) in January 2012 that will identify action items for improving
     employee engagement by September 2012, according to OCHCO
     officials.
The selected components also evaluated FEVS results to identify morale
problems and considered additional information sources. For example:

•    TSA convened a corporate action planning team in March 2011, as
     part of its response to FEVS results, which relied on data sources
     such as the TSA-administered exit survey, employee advisory groups,
     and an online employee suggestion tool, to gain perspectives on
     systemic challenge areas and to develop plans to address morale,
     according to TSA officials. TSA’s action plan for improving morale,
     based on these sources, was completed in July 2012.
•    ICE considered results of a Federal Organizational Climate Survey
     (FOCS), last completed in March 2012, and held focus groups to
     gauge the extent to which employees view ICE as having an
     organizational culture that promotes diversity.
•    CBP launched a quarterly online employee survey in 2009 to solicit
     opinions on one specific topic per quarter, such as use of career
     development resources and how the resources contributed to
     employees’ professional growth at CBP.
•    The Coast Guard relied on an Organizational Assessment Survey
     (OAS), last administered by OPM in 2010, to understand employee
     morale. The OAS solicits opinions on a range of topics, including job
     satisfaction, leadership, training, innovation, and use of resources. It
     included civilian and military Coast Guard personnel, but is not
     administered governmentwide so comparisons between the Coast




26
  Results from DHS’ Exit Survey should be interpreted with caution. Due to the method in
which it is administered, the survey’s response rate in 2011 was quite low, close to 40
percent. It is likely that a higher response rate would have produced somewhat different
results.




Page 18                                                GAO-12-940 DHS Employee Morale
                            Guard and other federal employees are limited to organizations that
                            may use the OAS, according to Coast Guard officials.
                        Appendix III provides more detailed descriptions of DHS’s steps to
                        address morale problems and selected components’ 2011 FEVS analysis
                        methods and findings. Appendix IV provides additional information on the
                        selected components’ data sources beyond FEVS for evaluating root
                        causes of morale, including a summary of results and how the information
                        was used by the components.


DHS and Selected        For the 2011 FEVS, DHS and the selected components completed
Components Conducted    varying levels of analyses to determine the root causes of low morale.
Limited FEVS Analyses   However, DHS and the selected components conducted limited analysis
                        in several areas that is not consistent with OPM and Partnership
                        guidance that lays out useful factors for evaluating root causes of morale
                        problems through FEVS analysis, as shown in figure 4.




                        Page 19                                        GAO-12-940 DHS Employee Morale
Figure 4: The Extent to Which OCHCO, TSA, CBP, ICE, and the Coast Guard
Incorporated Recommended Factors in Analyzing 2011 FEVS Results




Note:
Demographic comparisons. In its guidance to federal agencies for improving employee job
satisfaction, the Partnership for Public Service advises, among other things, determining
demographic group differences. According to the Partnership for Public Service, demographic group
analysis shows where there may be gaps in satisfaction at an agency or subcomponent (i.e., perhaps
some demographic groups report lower satisfaction than others). For the purposes of this report,
demographic group is used to describe any common characteristic among employees, such as pay
grade, supervisory status, or work group.
Benchmarking. Benchmarking agency survey results against those of similar organizations can
provide a point of reference for improvements, according to the Partnership for Public Service. OPM’s
management report to DHS for the 2011 FEVS also suggests, as part of action planning, comparing
agency results with governmentwide results and noting which survey questions and Human Capital
Assessment and Accountability Framework indexes scored lowest relative to the governmentwide
averages.




Page 20                                                        GAO-12-940 DHS Employee Morale
Linking analysis results with action plans. Both OPM and the Partnership for Public Service action
planning guidance list analyzing survey results as a first step to developing action plans to address
employee concerns. According to OPM’s guidance, the data analysis may include reviewing FEVS
results and following up on survey findings with focus groups to clarify reasons for low scores. OPM’s
guidance then calls for agencies to translate issues uncovered through data analysis into a set of
action plan goals. Similarly, the Partnership for Public Service advises agencies to develop action
plan approaches to improve employee satisfaction, based on issues the data identify, while
considering the organization’s mission, culture, available time, and resources.
a
 CBP partially benchmarked its FEVS results because it compared results with governmentwide and
DHS averages, but not with those for similar organizations for the 2011 FEVS. CBP officials stated
that few agencies within the United States both use FEVS and have occupations similar to those of
CBP.
b
 OCHCO and the selected components partially linked root causes with action plans because low-
scoring questions were listed on action plans as the reason why actions were chosen. However,
additional root cause analysis findings, such as those listed in appendix IV, were not included in the
action plan documentation.


Usage of the three factors described in figure 4 varied across DHS-wide
and component-level 2011 FEVS analyses we reviewed. In some
instances, the factors were partially or not used. For example:

•    Demographic group comparisons. According to our reviews of
     OCHCO’s analyses, OCHCO’s DHS-wide analyses did not include
     evaluations of demographic group differences on morale-related
     issues for the 2011 FEVS. According to OCHCO officials, DHS’s
     Office of Civil Rights and Civil Liberties reviews survey results to
     identify diversity issues that may be reflected in the survey, and
     OCHCO officials considered these results when developing one of the
     current (as of August 2012) DHS action plans to create policies that
     identify barriers to diversity. In 2007 and 2009, years in which DHS
     administered the Annual Employee Survey (AES), demographic
     comparisons were made. For example, on the basis of 2009 AES
     data, DHS found no significant demographic differences other than
     supervisors’ positive responses to questions were generally higher
     than those of non-supervisors and differences among pay grade
     levels. Because OPM now administers the survey each year, DHS is
     not able to make significant demographic group comparisons because
     of the format of the data provided by OPM, according to OCHCO
     officials. However, we obtained FEVS data from OPM that allowed us
     to make demographic group comparisons. For example, we compared
     DHS and non-DHS employee satisfaction and engagement scores
     across available demographic groups and found that both satisfaction
     and engagement were generally lower for DHS employees, which is
     summarized in appendix I, table 5.
     For the DHS component analyses we reviewed, TSA and CBP
     conducted some demographic analysis. For example, TSA compared
     screeners, Federal Security Director staff, Federal Air Marshals, and



Page 21                                                          GAO-12-940 DHS Employee Morale
     headquarters staff on each FEVS dimension (e.g., work experiences,
     supervisor/leader, satisfaction, and work/life). As a result, TSA was
     able to identify screeners as having survey scores below those of
     other TSA employee groups. CBP also compared race, ethnicity,
     gender, and program office scores. CBP found that no significant
     differences were present in the positive responses to the 2011 FEVS
     core questions when comparing race, ethnicity and gender, and found
     that Border Patrol employees reported higher job satisfaction than
     field operations employees (74 versus 66 percent on the job
     satisfaction index). In contrast, the Coast Guard did not conduct
     analysis in addition to data that was provided by DHS OCHCO.
     Because OCHCO’s data did not include demographic information for
     the 2011 FEVS, Coast Guard did not make demographic group
     comparisons. 27 ICE and CBP officials stated that they did not have
     access to 2011 FEVS data files necessary to conduct more detailed
     demographic comparisons. However, as shown in appendix I, we
     were able to make various demographic comparisons based on a
     more detailed data file provided by OPM, which is similar to a file that
     OPM makes available to agencies and the public. 28

•    Benchmarking against similar organizations. TSA benchmarked
     its FEVS results against results from similar organizations, by
     comparing results with CBP, and OCHCO’s DHS-wide analysis
     highlighted Partnership rankings data, showing DHS’s position relative
     to the positions of other federal agencies as a Best Place to Work.
     Similarly, ICE benchmarked its FEVS results overall and for program
     offices, such as homeland security investigators, against other DHS
     components, including the U.S. Secret Service and CBP. For the
     2011 FEVS, CBP performed more limited benchmarking, by
     comparing FEVS results with governmentwide averages. According to
     CBP officials, when analyzing annual employee surveys prior to 2011,
     CBP benchmarked its results against agencies with high positive



27
  In the 2010 OAS report to the Coast Guard, OPM made demographic comparisons
among several employee groups within the Coast Guard, including work groups (such as
ship versus shore-based employees), race and ethnicity, and gender, and comparisons
between civilian and military ranks. However, the results of the OAS analysis are not
included in the Coast Guard’s FEVS-based action planning documentation. See appendix
IV for a description of how the OAS is used by the Coast Guard.
28
  Our file contained the same variables as the public release file, but it identified smaller
demographic groups and work units.




Page 22                                                    GAO-12-940 DHS Employee Morale
    FEVS scores, such as the Social Security Administration, the Federal
    Bureau of Investigation, the Internal Revenue Service, and the
    Nuclear Regulatory Commission. CBP is in the initial planning phase
    of a larger benchmarking project that would benchmark CBP against
    foreign immigration, customs, and agriculture inspection agencies,
    such as the Canadian Border Services Agency and the Australian
    Customs and Border Protection Service. If approved, this
    benchmarking project is expected to occur in fiscal year 2013,
    according to CBP officials. The Coast Guard did not perform FEVS
    benchmarking analysis, according to the documentation we reviewed,
    but did make OAS-based comparisons between the Coast Guard and
    other organizations that use the OAS, according to Coast Guard
    officials.

•   Linkage of root causes with action plans. For both DHS-wide and
    selected component action plans, FEVS questions with low scores
    were linked with action plan areas. For example, in the DHS-wide
    action plan, low scores on employee satisfaction with opportunities to
    get a better job in the organization were linked to action plan items for
    enhancing employee retention. However, the extent to which DHS
    and the components used root causes found through other analyses
    to inform their action plans, such as quarterly exit survey results or
    additional internal component surveys, was not evident in action plan
    documentation (see appendix IV for a description of these additional
    root cause analyses). For example,

    •     OCHCO’s DHS-wide action plan was last updated based on 2010
          FEVS data and therefore did not rely on data from the DHS 2011
          exit survey, since those results were not published until January
          2012. Similarly, the EEESC was launched in January 2012 and
          therefore its efforts are not yet documented in DHS-wide action
          planning documents. According to OCHCO officials, the 2010
          DHS-wide action plan includes consideration of results from
          OCHCO’s 2008 statistical analysis identifying key drivers of job
          satisfaction and results from the 2007 focus groups. However,
          linkage to items in the DHS-wide action plan to these results is not
          clearly identified because a new action plan template OPM
          introduced in 2010 did not provide an area to identify the linkage
          between each action and the driver, according to OCHCO
          officials. In addition, DHS’s September 2009 action plan indicates
          consideration of the 2008 key driver analysis and 2007 focus
          group effort that led to a focus on leadership effectiveness
          initiatives.




Page 23                                          GAO-12-940 DHS Employee Morale
    •     According to CBP and TSA officials, data from other root cause
          analysis efforts are not explicitly documented in action plans
          developed in response to FEVS results because DHS has not
          included linkage of other root cause analysis efforts to actions
          items in the FEVS action planning templates used by the
          components. TSA officials also stated that other root cause efforts
          (see appendix IV) were used to develop TSA’s July 2012 action
          plan update. However, the July 2012 plan did not include linkage
          of root cause findings other than FEVS results, such as exit
          survey results, to action plan items.
    •     ICE officials stated that results from other root cause efforts, such
          as its FOCS, have not yet been considered in FEVS-based action
          planning but that ICE plans to do so in future efforts to address
          morale.
    •     The Coast Guard uses information from its OAS as part of a
          process separate from FEVS-based action planning for
          addressing morale, so OAS results are not linked to FEVS-based
          action plans.

OCHCO and component human capital officials described several
reasons for the variation in root cause analysis of FEVS results. OCHCO
officials described resource constraints and leadership changes within the
OCHCO position as resulting in a lack of continuity in root cause analysis
efforts. For example, one OCHCO official stated that because of resource
constraints, OCHCO has focused more efforts on workforce planning than
on morale problem analysis since 2009. ICE human capital officials stated
that ICE’s human capital services were provided via a contract with CBP
until 2010, when the human capital function became an independently
funded part of the ICE organization. Only since moving to its current
position within ICE has the human capital office been able to devote more
resources to addressing morale issues, according to the officials. CBP
human capital officials stated that for assessing morale issues, CBP uses
both quantitative and qualitative information. However, according to the
officials, qualitative evidence is preferable over quantitative survey
analysis because focus groups and open-ended surveys, such as the
Most Valuable Perspective online survey, allow CBP to better understand
the issues affecting employees. Because of CBP human capital officials’
preference for qualitative information, CBP has not emphasized extensive
quantitative analysis of survey results, such as statistical analysis that
may determine underlying causes of morale problems.

Without a complete understanding of which issues are driving low
employee morale, DHS risks not being able to effectively address the



Page 24                                           GAO-12-940 DHS Employee Morale
                                         underlying concerns of its varied employee population. Emphasis on
                                         survey analysis that includes demographic group comparisons,
                                         benchmarking against similar organizations, and linkage of other analysis
                                         efforts outside of FEVS within action plan documentation could assist
                                         DHS in better addressing its employee morale problems.


DHS and Its Components                   DHS and the selected components routinely update their action plans to
Completed Action Plans                   address employee survey results in accordance with the Office of
                                         Management and Budget’s budget guidance; the DHS-wide plan is
                                         updated every two years, and components update their plans at least
                                         annually. 29 According to OPM’s guide for using FEVS results, action
                                         planning involves, among other things, identifying goals and actions for
                                         improving low-scoring FEVS satisfaction topics such as reviewing survey
                                         results to determine steps to be taken to improve how the agency
                                         manages its workforce. DHS-wide and component action plan goals and
                                         examples of low-scoring FEVS satisfaction topics are listed in table 2.

Table 2: DHS-wide and Component Action Plan Goals and Examples of Low-Scoring FEVS Topics Addressed through such
Goals

                                                                                                                       Example of low-scoring
                                                                                                                       FEVS satisfaction topics
DHS unit      Summary of action plan goals to address FEVS results                                                     addressed by action plan goal
DHS-wide      Enhance leadership, recruitment, employee retention, and DHS unification.                                Opportunity to get a better job in
                                                                                                                       the organization
TSA           Launch a corporate action planning team to study employee issues and develop                             Discussions with supervisors
              recommendations, enhance employee performance management, and improve                                    about performance
              TSA communication mechanisms.
ICE           Advance telework opportunities, increase communication between employees and                             Physical conditions that allow
              management, and develop an awards handbook for distribution to employees.                                employees to do their job well
CBP           Develop action plans within CBP program offices to address results, enhance        Policies and practices of senior
              communication between management and employees, create career and                  leaders
              leadership development opportunities, replace pass/fail performance appraisal with
              multi-leveled performance management system, implement training improvements,
              and maintain an existing virtual focus group to enable upward feedback to senior
              leaders.
Coast Guard   Improve communication with employees and training options.                                               Information received from
                                                                                                                       management
                                         Source: GAO analysis of DHS-wide, TSA, Coast Guard, CBP, and ICE FEVS action plans based on FEVS results.




                                         29
                                           The selected component action plans we reviewed were updated as of January 2012. Subsequent
                                         updates to the plans, due to OCHCO in July 2012, were not included in our evaluation.




                                         Page 25                                                                      GAO-12-940 DHS Employee Morale
As part of DHS’s efforts to address our high-risk designation of
implementing and transforming DHS, DHS described a plan for improving
employee morale in its Integrated Strategy for High Risk Management
(Integrated Strategy). In June 2012, DHS provided us with its updated
Integrated Strategy, which summarized the status of the department’s
activities for addressing its implementation and transformation high-risk
designation. In the Integrated Strategy, DHS identified activities to
improve employee job satisfaction scores, among other things. The status
of the activities included ongoing analysis of the 2011 FEVS results,
launch of the EEESC to address DHS scores on the HCAAF indexes,
ongoing coordination between the OCHCO and components to develop
action plans in response to the 2011 FEVS results, and launch of an
online employee survey in the first quarter of fiscal year 2013.

Within the Integrated Strategy action plan for improving job satisfaction
scores, DHS reported that three of six efforts were hindered by a lack of
resources. 30 For example, resources are a constraining factor for DHS’s
Office of the Chief Human Capital Officer to consult with components in
developing action plans in response to 2011 FEVS results. Similarly,
resources are a constraining factor to deploy online focus discussions on
job satisfaction-related issues.




30
  The six efforts are: (1) OPM releases 2011 survey results; DHS issues employee
communications and conducts analysis; (2) CHCO leads an Enterprise-wide Executive
Steering Committee to develop a deliberate way forward for addressing FEVS key indices;
(3) OCHCO works with components to develop action plans responding to 2011 FEVS
results; (4) On-line focus discussion survey deploys across Department; (5) CHCO
provides components with feedback on action plans; and (6) Updated component and
headquarter action plans are due. The three efforts that are hindered by a lack of
resources are: OCHCO working with components to develop action plans, on-line focus
discussions, and CHCO providing components with feedback.




Page 26                                               GAO-12-940 DHS Employee Morale
DHS and Selected           According to our review of the action plans created in response to the
Components Generally       FEVS and interviews with agency officials, DHS and the selected
Followed OPM’s Six Steps   components generally incorporated the six action planning steps
                           suggested by OPM, but the agency does not have effective metrics to
for Effective Action       support its efforts related to monitoring. 31 (See figure 5.)
Planning but Do Not Have
Effective Metrics for
Monitoring Efforts




                           31
                             The most recent DHS-wide action plan update was based on 2010 FEVS results.
                           According to OCHCO officials, the next update to the action plan will be released in
                           January 2013.




                           Page 27                                                  GAO-12-940 DHS Employee Morale
Figure 5: OPM’s Six Steps for Action Planning to Improve FEVS Scores




We found that, in general, DHS and its components are implementing the
six steps for action planning as demonstrated in table 3 below.




Page 28                                           GAO-12-940 DHS Employee Morale
Table 3: DHS-Wide and Selected Components’ Action Planning Steps

Action
Planning Step        DHS and Component Response                         Examples of What Has Been Done
1. Identify issues   All the components and OCHCO                       For example, CBP’s action plan identified the FEVS questions that
                     evaluated the FEVS results by, at a                pertained to the job satisfaction dimension along with the positive
                     minimum, determining high and low                  response results for each of the survey questions.
                     positive response levels to survey
                     questions.
2. Set goals         All of the components and OCHCO                    For example, TSA’s action plan included a goal of enhancing
                     developed broad goals to improve FEVS              overall communication and innovation.
                     scores.
3. Identify staff    All of the components and OCHCO                    For example, TSA’s action plan described a cross-functional team
                     identified personnel responsible for their         that included representatives from existing councils and from all
                     action plan.                                       levels and organizational units to address an action item.
4. Develop plan      All of the components and OCHCO            For example, ICE’s action plan for increasing communication
                     developed plans that identified actions to included utilizing ICE broadcast announcements for more tangible
                     be taken to accomplish their goals.        and employee-related purposes.
5. Implement plan    All of the components and OCHCO set      For example, Coast Guard’s action plan listed the redesign of the
                     target dates and in some cases indicated civilian website by December 20, 2012.
                     whether an action item was completed or
                     on-going.
6. Monitor results   All of the components and OCHCO                    For example, OCHCO’s action plan included a goal to retain an
                     established measures of success,                   engaged workforce. The three measures of success listed for this
                     providing the status of success or                 goal are (1) improve DHS ranking on the Partnership for Public
                     completion dates.                                  Service Best Places to Work, (2) reduce attrition rates, and (3)
                                                                        achieve an HCAAF index positive response average of 58 percent
                                                                        on the 2011 FEVS.
                                          Source: GAO analysis of DHS-wide, TSA, ICE, CBP, and Coast Guard action plans and U.S. Office of Personnel Management action
                                          planning guidance.



                                          Although we found that OCHCO and the four selected components are
                                          generally taking actions to execute the sixth step—monitor and evaluate
                                          the results of implementation—they have not established effective
                                          measures of the agency’s achievement of action plan goals. For example,
                                          in the DHS-wide action plan, one of the four goals is to build an effective,
                                          mission focused, diverse and inspiring cadre of leaders. A measure of
                                          success listed for this goal is that progress will be measured against the
                                          2011 HCAAF index. For this measure, it is not clear which HCAAF index
                                          will be assessed and it does not include a target for improvement—such
                                          as a percent increase in satisfaction—by which DHS can benchmark its
                                          results. The measures of success within the DHS-wide and selected
                                          component action plans could be improved by including additional
                                          attributes of successful metrics. Specifically, in our prior work, we
                                          identified attributes of successful metrics that allow agencies to better
                                          determine whether they are meeting their goals while holding agency staff




                                          Page 29                                                                       GAO-12-940 DHS Employee Morale
accountable for improving performance. 32 Three attributes relevant to the
action plans are:

•   linkage—determines whether there is a relationship between the
    performance measure and the goals;
•   clarity—determines whether the performance measures are clearly
    stated; and
•   measurable target—determines whether, performance measures
    have quantifiable, numerical targets or other measurable values,
    where appropriate.

In general, DHS and component measures satisfied the linkage attribute
but did not address the clarity and measurable targets attributes. We
compared DHS and the four components measures of success to the
three attributes and found that all 54 measures of success incorporated
the linkage attribute, 12 of the 54 measures of success did not address
the clarity attribute, and 29 of the 54 measures of success did not
address the measurable target attribute.

As shown in table 4 below, we found that these measures demonstrate
linkage because they align with the action plan goals. However, we
determined that the measures demonstrate neither clarity nor a
measurable target. Specifically, the measures do not demonstrate clarity
because they do not provide enough detail to clearly state the metric used
to measure success. They also do not demonstrate a measureable target
because they do not list quantitative goals or provide a qualitative
predictor of a desired outcome, which would allow the agency to better
determine the extent to which they were making progress toward
achieving their goals.




32
  GAO, Tax Administration: IRS Needs to Further Refine Its Tax Filing Season
Performance Measures, GAO-03-143 (Washington, D.C.: Nov. 22, 2002). Of the nine
attributes of successful metrics listed in this report, we determined that linkage, clarity and
measurable target are relevant to this evaluation. The six attributes that we did not
evaluate are: objectivity, reliability, core program activities, balance, governmentwide
priorities, and limited overlap. We did not include these six attributes because they were
not relevant to employee morale action planning efforts.




Page 30                                                    GAO-12-940 DHS Employee Morale
Table 4: Examples of DHS-Wide and Selected Components’ Measures of Success

DHS                         Measure of        Examples of Actions to
unit     Goal               Success           Achieve Goal                     GAO Assessment
DHS-     Recruit a highly   Recruitment of    •   Streamline the hiring        Clarity
wide     qualified and      a highly              process to increase          The measure lacks key information that would make it
         diverse            qualified and         applicant and manager        more clear—namely, what constitutes a highly qualified
         workforce          diverse               satisfaction                 workforce and how should “diverse workforce” be
                            workforce         •   Improve manager              interpreted.
                                                  involvement                  Measurable Target
                                              •   Hold managers                The measure does not list quantifiable or other
                                                  accountable for hiring       measurable values to help determine when the goal
                                                  process involvement          has been reached. For example, the measure does not
                                                  through performance          provide a target number of employees against which
                                                  evaluations                  DHS can benchmark its results.
                                              •   Streamline all DHS job
                                                  opportunity
                                                  announcements
TSA      Performance        Programs          •   Provide status updates to    Clarity
         Management         launched and          the employee                 The measure lacks key information that would make it
         Enhancement        systems               performance management       more clear—namely, which programs and systems
                            deployed              program                      TSA is to launch.
                                              •   Provide training and         Measurable Target
                                                  resources to ensure
                                                  managers and supervisors     The measure does not list quantifiable or other
                                                  use fair, objective, and     measurable values to help determine when the goal
                                                  consistent merit-based       has been reached. For example, the measure does not
                                                  principles during            provide a target number of programs against which
                                                  performance evaluations      TSA can benchmark its results.
CBP      Continue to        Increase          •   Initiate Division Director   Clarity
         Implement          awareness of          communiqués                  The measure lacks key information that would make it
         Communication      survey results    •   Hold a series of town hall   more clear—namely, what topics in the survey results
         Strategies for     to employees          meetings across program      are targeted, which employees should be included, and
         Disseminating                            offices                      how “increase awareness” should be interpreted.
         Information from
                                              •   Launch a virtual town hall   Measurable Target
         Management to
                                                  meeting to address           The measure does not list quantifiable or other
         Employees via
                                                  employee concerns            measurable values to help determine when the goal
         Technology
                                              •   Launch a series of town      has been reached. For example, the measure does not
                                                  hall meetings in various     provide a target number of employees or geographic
                                                  geographic locations to      locations against which CBP can benchmark its results
                                                  address issues identified    nor does it provide a target measurement for “increase
                                                  through the FEVS             awareness.”




                                             Page 31                                                 GAO-12-940 DHS Employee Morale
DHS                     Measure of        Examples of Actions to
unit    Goal            Success           Achieve Goal                                 GAO Assessment
ICE     Increasing      Survey            •  Increase engagement with                  Clarity
        Communication   employees            union representatives                     The measure lacks key information that would make it
                                          •  Development of labor                      more clear—namely, what issues are to be surveyed
                                             management forums                         and which employees should be included.
                                             within ICE                                Measurable Target
                                                                                       The measure does not list quantifiable or other
                                                                                       measurable values to help determine when the goal
                                                                                       has been reached. For example, the measure does not
                                                                                       provide a target number of employee survey responses
                                                                                       against which ICE can benchmark its results.
Coast   Training and    Develop e-        •     Enhance guidance                       Clarity
Guard   Development     learning                provided to employees                  The measure lacks key information that would make it
                        course for new          describing their role in               more clear—namely, what is the course content or the
                        employees               individual development                 specific training being provided through the e-learning
                                                plans development                      course.
                                                                                       Measurable Target
                                                                                       The measure does not list quantifiable or other
                                                                                       measurable values to help determine when the goal
                                                                                       has been reached. For example, the measure does not
                                                                                       provide a target number of new employees who will
                                                                                       receive the training against which the Coast Guard can
                                                                                       benchmark its results.
                                         Source: GAO analysis of DHS-wide, TSA, CBP, ICE and Coast Guard action plans.



                                         Officials provided several reasons why their measures of success may fall
                                         short of the attributes for successful metrics. According to OCHCO
                                         officials, OCHCO considers accomplishment of an action item step as a
                                         success and relies on the measures of success listed in its action plan as
                                         a metric for whether the action plan items were implemented. OCHCO
                                         considers whether positive responses to survey questions noted in the
                                         action plan improve over time as the outcome measure for whether action
                                         plans are effective. However, as part of its oversight and feedback on
                                         component action plans, OCHCO does not monitor or evaluate measures
                                         of success for action planning and therefore is not in a position to
                                         determine whether the measures reflect improvement. CBP officials
                                         stated that they monitor the change in FEVS results overall as the intent
                                         of the action planning is to improve their scores on the HCAAF indexes.
                                         Coast Guard officials stated that they rely on qualitative feedback from
                                         employees on action plan items, such as improved training and website
                                         updates, to measure action plan performance. TSA officials stated they
                                         assess action plan results by tracking completion dates for action items
                                         and updating OCHCO on results at least semi-annually, and ICE officials
                                         have stated they have not yet fully developed monitoring efforts to
                                         evaluate job satisfaction action planning because the human capital office


                                         Page 32                                                                         GAO-12-940 DHS Employee Morale
                       received funding in the summer of 2011 to implement human capital
                       programs.

                       We acknowledge that positive responses in survey results and positive
                       employee feedback are good indicators that action planning is working.
                       However, until DHS and its components begin to see positive results, it is
                       important for them to (1) understand whether they are successfully
                       implementing the individual steps of their action plans and (2) make any
                       necessary changes to improve on them. By not having specific metrics
                       within the action plans that are clear and measurable, it will be more
                       difficult for DHS to assess its efforts to address employee morale
                       problems, as well as determine if changes should be made to ensure
                       progress toward achieving its goals. Furthermore, effective measures are
                       key to DHS’s action plan as it is part of a process that informs the Office
                       of Management and Budget and OPM of DHS efforts to address survey
                       results. According to an OPM official responsible for federal action
                       planning to improve morale, DHS should carefully consider, for each
                       action step, what success means to the agency, such as increased
                       employee engagement targets. The official said that when success is
                       defined, it should not only be clear and measurable, but should also take
                       into account as many of the different demographic groups evaluated as
                       possible.


DHS and the Selected   DHS and the selected components have initiated efforts to determine how
Components Consulted   other entities approach employee morale issues. DHS officials stated they
Best Practices         have started to review and implement what they consider to be best
                       practices for improving employee morale, such as the following:

                       •   DHS working group—OCHCO leads a survey engagement team that
                           holds monthly meetings during which action planning efforts from
                           across the different components are shared and discussed.
                           Representatives from other federal agencies such as the National
                           Aeronautics and Space Administration and the Federal Aviation
                           Administration have also attended these meetings and presented their
                           action plans for addressing survey results.
                       •   Idea Factory—a TSA web-based tool adopted by DHS that empowers
                           employees to develop, rate, and improve innovative ideas for
                           programs, processes, and technologies. According to a DHS
                           assessment, the Under Secretary for Management plans to use this




                       Page 33                                        GAO-12-940 DHS Employee Morale
                   tool for internal DHS employee communication so as to promote
                   greater job satisfaction and enhance organization effectiveness. 33

             Component officials we interviewed also stated they have started to
             review, implement, and share what they consider to be best practices for
             improving morale. For example:

             •     ICE officials stated they consult with other agencies and DHS
                   components, such as the U.S. Marshal’s Service, when addressing
                   morale challenges and developing policies and programs. For
                   example, the U.S. Marshal’s Service has a critical incident response
                   program for employees encountering a traumatic event and ICE is
                   exploring adopting a similar program.
             •     TSA officials stated that they reached out to Marriott Corporation,
                   CBP, and the National Aeronautics and Space Administration to
                   identify actions for increasing employee rewards and employee
                   confidence in leadership.
             •     CBP officials stated they have established several ongoing working
                   groups that routinely meet and share human capital best practices
                   within the agency. One of these working groups has conducted
                   benchmarking work with high-FEVS-scoring federal agencies such as
                   the Social Security Administration, the U.S. Secret Service, the
                   Federal Bureau of Investigation, the Internal Revenue Service and the
                   Nuclear Regulatory Commission.
             •     Coast Guard officials stated they share human capital best practices
                   that may improve job satisfaction with other DHS components such as
                   (1) their performance appraisal system which was adopted, in part,
                   DHS-wide; (2) their automated cash award process with FEMA; and
                   (3) Coast Guard training to supervisors with both DHS headquarters
                   officials and FEMA.

             Given the critical nature of DHS’s mission to protect the security and
Conclusion   economy of our nation, it is important that DHS employees are satisfied
             with their jobs so that DHS can retain and attract the talent required to
             complete its work. Employee survey data indicate that when compared to
             other federal employees, many DHS employees report being dissatisfied
             and not engaged with their jobs. It is imperative that DHS understand
             what is driving employee morale problems and address those problems



             33
                 DHS, Privacy Impact Assessment for the Idea Factory, January 21, 2010.




             Page 34                                                GAO-12-940 DHS Employee Morale
                      through targeted actions that address employees’ underlying concerns.
                      DHS has made efforts to understand morale issues across the
                      department, but those efforts could be improved. Specifically, given the
                      annual employee survey data available through the FEVS, DHS and its
                      components could improve their efforts to determine root causes of
                      morale problems by comparing demographic groups, benchmarking
                      against similar organizations, and linking root cause findings to action
                      plans. Uncovering root causes of morale problems could help identify
                      appropriate actions to take in efforts to improve morale. In addition, DHS
                      has established performance measures for its action plans to improve
                      morale, but incorporating attributes such as improved clarity and
                      measurable targets could better position DHS to determine whether its
                      action plans are effective. Without doing so, DHS will have a more difficult
                      time determining whether it is achieving its goals.


                      To strengthen DHS’s evaluation and planning process for addressing
Recommendations for   employee morale, we recommend that the Secretary of Homeland
Executive Action      Security direct OCHCO and component human capital officials to take the
                      following two actions:

                      •   examine their root cause analysis efforts and, where absent, add the
                          following: comparisons of demographic groups, benchmarking against
                          similar organizations, and linkage of root cause findings to action
                          plans; and
                      •   establish metrics of success within the action plans that are clear and
                          measurable.

                      We requested comments on a draft of this report from DHS. On
Agency Comments       September 25, 2012, DHS provided written comments, which are
and Our Evaluation    reprinted in appendix V, and provided technical comments, which we
                      incorporated as appropriate. DHS concurred with our two
                      recommendations and described actions planned to address them.
                      Specifically:

                      •   DHS stated that it will ensure that department-wide and component
                          action plans are tied to root causes and that the department will
                          conduct benchmarking against other organizations. DHS also stated
                          that its ability to conduct demographic analysis is limited due to the
                          data set OPM makes available to federal agencies. However,
                          according to OPM, DHS has access to the data necessary for
                          conducting analysis similar to our comparison of demographic groups.



                      Page 35                                        GAO-12-940 DHS Employee Morale
•   DHS stated it will review action plans to ensure that each action is
    clear and measurable.
We also requested comments on a draft of this report from OPM. On
September 18, 2012, OPM provided a written response, which is
reprinted in appendix VI. OPM’s letter indicated that it reviewed the draft
report and had no comments.


As agreed with your offices, unless you publicly announce the contents of
this report earlier, we plan no further distribution until 30 days from the
report date. At that time, we will send copies to the Secretary of
Homeland Security, the U.S. Office of Personnel Management, and
interested congressional committees. The report also will be available at
no charge on GAO’s website at http://www.gao.gov.

If you or your staff have any questions about this report please contact
me at (202) 512-9627 or maurerd@gao.gov. Contact points for our
Offices of Congressional Relations and Public Affairs may be found on
the last page of this report. GAO staff who made major contributions to
this report are listed in appendix VII.




David C. Maurer
Director, Homeland Security and Justice Issues




Page 36                                         GAO-12-940 DHS Employee Morale
Appendix I: Statistical Analysis of Employee
                           Appendix I: Statistical Analysis of Employee
                           Morale at Department of Homeland Security
                           and Other Agencies


Morale at Department of Homeland Security
and Other Agencies
                           We conducted a statistical analysis of the 2011 Federal Employee
                           Viewpoint Survey (FEVS) to assess employee morale at the Department
                           of Homeland Security (DHS). Our analysis addressed two specific
                           questions. First, how does morale at DHS and its components compare
                           with morale at other agencies, holding constant demographic differences
                           among employees? Second, to what extent is the morale gap between
                           DHS and other agencies explained by differences in the demographic
                           composition of the DHS workforce versus other unique characteristics of
                           the agency or unmeasured demographic factors?

                           This appendix explains the value of statistical analysis for understanding
                           the employee morale gap, describes the data and methods we used, and
                           provides additional details about our findings, which are summarized in
                           the body of the report. In sum

                           •   DHS employees with the same demographic profiles (measured by
                               FEVS) were about 7 percentage points less engaged and 6 points
                               less satisfied than non-DHS employees.
                           •   Demographic differences (measured by FEVS) between DHS and
                               other agencies are unlikely to explain the overall morale gap. Unique
                               features of DHS (or unmeasured demographics) are more likely to be
                               responsible.
                           •   DHS middle managers and employees with 1 to 10 years of tenure at
                               their components—those hired after the department’s creation—have
                               lower morale than similar employees at other departments.
                           •   Morale varies widely across DHS components, and some have similar
                               morale as non-DHS agencies. Individual offices can strongly influence
                               the morale gap at the component level.
                           •   The morale gap is smaller for DHS components that existed before
                               the department was created.

Understanding the Morale   The morale gap between DHS and other agencies may be due to unique
Gap at DHS                 issues within DHS or common issues faced by all agencies in similar
                           circumstances. Unique issues might include developing an agency-wide
                           culture, the decisions and composition of senior leaders, and the inherent
                           uniqueness of homeland security programs. Common characteristics
                           might include having many law enforcement and front-line customer
                           service occupations, and having employees dispersed among many
                           headquarters and field offices.

                           Determining whether unique or shared issues account for the overall
                           morale gap is important for understanding the cause of the problem. If
                           morale at DHS was not uniquely low, compared with morale at agencies


                           Page 37                                        GAO-12-940 DHS Employee Morale
Appendix I: Statistical Analysis of Employee
Morale at Department of Homeland Security
and Other Agencies




with similar demographics and programs, the agency might learn from
peer agencies facing similar challenges. Alternatively, if morale was lower
at DHS for reasons unique to the agency, DHS might put more emphasis
on understanding its own particular challenges. Distinguishing among
these possible explanations can help develop a solution that is narrowly
tailored to the problem.

Our analysis focused on one group of shared circumstances that might
explain the morale gap: employee demographics. If DHS were more likely
to employ the types of workers who tend to have lower morale across all
agencies of the government, the composition of the workforce might
account for the gap to a greater extent than factors specific to DHS. In
other words, morale at DHS may be no worse than at other agencies
among demographically equivalent employees. Our analysis focused on a
limited number of demographic differences, such as location and age, but
attitudinal differences about pay, benefits, supervision, training,
mentoring, and other human capital issues could be assessed in a similar
way.

We also considered how large of a morale gap there was between
employees in various DHS components and work groups and non-DHS
employees. The gap at the department level can mask groups of
employees with higher or lower morale. Disaggregating morale into small
work groups identifies areas of DHS in which morale may be high or low,
and thus provides sufficiently detailed data for focused solutions to the
problem.

Any analysis of morale in employee surveys is limited by the fact that
associations among the variables of interest may not represent cause-
and-effect relationships. Nevertheless, a limited observational analysis
remains useful for evaluating human capital programs. Since federal
agencies cannot easily conduct high-quality randomized controlled trials
of various approaches to managing their employees, the use of
observational methods is common, often in the form of quantitative survey
analyses or qualitative interviews and focus groups. We have previously
found that a pragmatic approach to answering necessary policy
questions, using the best methods and data that are feasible, is widely
supported by academic experts and practitioners in policy analysis. 1


1
 See GAO-10-30, “Program Evaluation: A Variety of Rigorous Methods Can Help Identify
Effective Interventions,” 20-31.




Page 38                                             GAO-12-940 DHS Employee Morale
                     Appendix I: Statistical Analysis of Employee
                     Morale at Department of Homeland Security
                     and Other Agencies




                     Moreover, statistical theory has shown that observational methods can
                     estimate cause-and-effect relationships in certain conditions. 2

                     Associations between morale and demographic characteristics are useful
                     for understanding the operation of human capital programs, when
                     interpreted cautiously and in the context of all the available evidence. Our
                     analysis here describes patterns across the demographic groups
                     identified in the 2011 FEVS and determines whether the aggregate
                     differences between DHS and other agencies persists among
                     demographically similar employees. We make no causal interpretations of
                     these relationships, and our approach is only one that might be valid and
                     useful.


The 2011 Federal     The Office of Personnel Management (OPM) provided us with a version
Employee Viewpoint   of the 2011 FEVS that included more detailed demographic and
Survey               organizational data than the file it released to the public. Specifically, our
                     file contained the same variables as the public file but identified more
                     detailed groups of employees. The 2011 survey included responses from
                     266,376 full-time, permanent federal employees, working for agencies
                     that, according to OPM, constituted 97 percent of the executive branch
                     workforce. OPM sampled employees within strata formed by supervisory
                     status and organizational subgroup (e.g., component and work group). 3
                     This produced generally large sample sizes even for many small work
                     groups within components, which allowed us to analyze morale among
                     small groups of employees with an acceptable degree of precision.

                     We focused on two types of variables in the FEVS: (1) employee
                     demographics and (2) OPM’s Employee Engagement and Job
                     Satisfaction indexes. A series of questions at the end of the survey
                     collected the demographic data, rather than preexisting administrative
                     records. OPM reported independently developing and validating the
                     engagement indexes using factor-analytic procedures, which are common
                     psychometric statistical methods. The survey items that made up each




                     2
                      See, for example, Donald Rubin, “The Use of Matched Sampling and Regression
                     Adjustment to Remove Bias in Observational Studies,” Biometrics 29 (1973): 185-203,
                     and Paul Rosenbaum, Observational Studies, New York: Springer-Verlag,1995.
                     3
                      OPM refers to these organizational subgroups as “bureaus” and “offices.”




                     Page 39                                                GAO-12-940 DHS Employee Morale
                         Appendix I: Statistical Analysis of Employee
                         Morale at Department of Homeland Security
                         and Other Agencies




                         index used five-point, Likert-type scales, with “agree/disagree,”
                         “satisfied/dissatisfied,” or “good/poor” response options. 4

                         We used weights provided by OPM to calculate estimates and sampling
                         variances for all analyses. The weights were the product of the unequal
                         sampling probabilities across strata and non-response and post-
                         stratification adjustments. Because some strata had relatively small
                         population sizes—one-quarter with 18 employees or fewer—we corrected
                         for finite populations.


Morale Differences       One explanation for lower morale at DHS is that its employees could be
Between DHS Employees    members of demographic groups that typically have lower morale across
and Employees at Other   all agencies. If this is true, the cause of morale problems and their
                         solutions might focus less on factors that are unique to DHS and more on
Agencies                 approaches that apply to any agency with a similar workforce.

                         Table 5 provides basic evidence to help assess the demographic
                         explanation. The table presents the average OPM Engagement Index for
                         several demographic groups in the 2011 FEVS. If engagement problems
                         at DHS were isolated to particular subgroups of employees, the morale
                         gap should vary widely across those subgroups. In fact, engagement at
                         DHS is lower (or statistically indistinguishable from zero) than at other
                         agencies in each demographic subgroup we analyzed, and the gap
                         relative to DHS does not vary by large amounts across most subgroups.
                         However, the gap is somewhat larger among employees who were in
                         certain subgroups, such as those who had 4 to 10 years of experience
                         with their components and who worked outside of headquarters.




                         4
                          For a detailed description of the questions that made up the OPM indexes, see OPM,
                         2011 Federal Employee Viewpoint Survey, Empowering Employees, Inspiring Change,
                         Department of Homeland Security, Agency Management Report. (Washington, D.C.), 21
                         and 43.




                         Page 40                                              GAO-12-940 DHS Employee Morale
                                       Appendix I: Statistical Analysis of Employee
                                       Morale at Department of Homeland Security
                                       and Other Agencies




Table 5: DHS and Non-DHS Employee Engagement Index by Demographic Group for the 2011 FEVS

                       OPM Employee                          OPM Employee               95%       OPM Employee
                         Engagement       95% Margin           Engagement         margin of         Engagement       95% margin
                     Index - Non-DHS      of Error (+/-)        Index - DHS       error (+/-)   Index - difference   of error (+/-)


All employees                   67.1                0.3                  60.1            0.7                 -7.0*             0.7


Supervisory status
Non-management                  65.7                0.3                  58.2            0.8                 -7.5*             0.9
Management                      74.1                0.4                  67.0            1.1                 -7.2*             1.2


Pay group
Federal Wage                    61.6                1.3                  57.3            3.7                 -4.3*             3.9
System
GS 1-6                          65.6                1.0                  61.4            5.1                  -4.2             5.2
GS 7-12                         66.8                0.4                  61.3            1.1                 -5.5*             1.2
GS 13-15                        70.5                0.4                  63.0            1.2                 -7.5*             1.2
Senior Executive                82.7                1.2                  79.8            3.0                  -3.0             3.2
Service
SL                              70.5                3.7                  66.2           16.5                  -4.4            16.9
        a
Other                           65.6                1.1                  53.1            1.4                -12.5*             1.8


Agency tenure
<1 year                         76.0                1.3                  75.6            4.2                  -0.4             4.4
1-3 years                       69.7                0.6                  65.2            1.6                 -4.6*             1.7
4-5 years                       66.0                0.9                  58.4            1.8                 -7.7*             2.0
6-10 years                      65.8                0.6                  56.0            1.1                 -9.9*             1.2
11-20 years                     65.1                0.6                  61.3            1.7                 -3.8*             1.8
20+ years                       67.2                0.5                  60.3            2.3                 -6.9*             2.3


Location
Headquarters                    68.5                0.4                  64.9            1.5                 -3.5*             1.5
Field                           66.5                0.3                  59.0            0.8                 -7.5*             0.8


Age
<26                             74.6                2.0                  66.7            4.8                 -7.9*             5.2
26-29                           70.6                1.4                  58.9            3.2                -11.7*             3.5
30-39                           67.9                0.7                  60.5            1.4                 -7.4*             1.5




                                       Page 41                                                   GAO-12-940 DHS Employee Morale
                                       Appendix I: Statistical Analysis of Employee
                                       Morale at Department of Homeland Security
                                       and Other Agencies




                       OPM Employee                                 OPM Employee              95%        OPM Employee
                         Engagement        95% Margin                 Engagement        margin of          Engagement            95% margin
                     Index - Non-DHS       of Error (+/-)              Index - DHS      error (+/-)    Index - difference        of error (+/-)
40-49                           67.2                    0.5                      60.6           1.2                     -6.6*                   1.3
50-59                           66.2                    0.4                      58.9           1.3                     -7.3*                   1.3
60+                             67.8                    0.7                      58.1           2.2                     -9.8*                   2.3
Race
Hispanic                        66.8                    1.1                      63.7           1.7                     -3.1*                   2.0
American Indian /               60.1                    1.8                      52.7           7.6                      -7.3                   7.8
Alaska Native
Asian                           72.4                    1.2                      65.0           3.4                     -7.4*                   3.6
Black                           68.2                    0.7                      60.8           2.0                     -7.4*                   2.1
Hawaiian / Pacific              65.4                    4.1                      56.8           6.6                     -8.5*                   7.8
Islander
White                           67.5                    0.3                      58.5           0.9                     -9.0*                   0.9
Two or more races               62.3                    1.9                      56.3           4.0                     -6.0*                   4.4
                                       Source: GAO analysis of 2011 FEVS data.

                                       Note: Asterisks denote differences that are statistically distinguishable from zero at the 0.05 level.
                                       a
                                        Because respondents to the 2011 FEVS reported their own pay groups, the “other” category may
                                       have included workers in various groups other than the GS system. At DHS, this group may have
                                       included Transportation Security Administration airport security screeners.


Multivariate Analysis                  We developed several statistical models to further assess the
                                       demographic explanation. These models held constant the demographic
                                       profiles of DHS and non-DHS employees, in order to isolate the portion of
                                       the morale gap that was specifically due to non-demographic factors. The
                                       models allowed us to compare morale at DHS and other agencies among
                                       employees who were in the same demographic groups, as measured by
                                       the FEVS.

                                       To avoid methodological complications with modeling latent variables, we
                                       created a binary measure that identified whether a respondent was
                                       engaged or satisfied on each item in the respective scales. Our measure
                                       equaled 1 if the respondent gave positive answers (4 or 5) to each item in
                                       the index and 0 if the respondent gave neutral or negative responses
                                       (1,2, or 3) to at least one item. Collapsing the scale loses some
                                       information, since morale and satisfaction are continuous, latent
                                       variables. However, a collapsed measure provides some degree of
                                       comparability between OPM’s aggregate indices and our individual-level
                                       analysis, since the OPM’s indices also collapse the scale. The differences




                                       Page 42                                                           GAO-12-940 DHS Employee Morale
            Appendix I: Statistical Analysis of Employee
            Morale at Department of Homeland Security
            and Other Agencies




            among agencies and subgroups of employees are generally similar using
            either our measure or OPM’s. 5

            We focused on the associations between broad measures of morale and
            fixed demographic characteristics available in the 2011 FEVS. Fixed
            demographics and broad measures of satisfaction are not subject to
            artificially high correlations that a survey’s design can produce among
            attitudinal measures.

            The models took the following form:

 (                              )     (
E Morale ij | DHS j , Demog ij = Λ αDHS j +Demog ij β        )                                (1)

E (Moraleij | DHS j , Demog ij ) = Λ (αDHS j +DHS j × Demog ij β D + Demog ij β G )           (2)

            Moraleij indicates whether employee i at agency j was engaged or
            satisfied, using the binary measure we calculated from the survey items
            that make up the OPM indexes (see above). DHSj indicates whether the
            employee worked for DHS, Demogij is a vector of demographic indicators
            (listed in table 6), Λ is the logistic function, and α and β are vectors of
            coefficients that estimate how morale varied among employees in
            different demographic groups. We included all demographic factors
            measured by the FEVS that plausibly could have predicted morale and
            were clearly causally prior to morale. We excluded pay group, however,
            because of its high correlation with supervisory status. Model 2 allows
            DHS and non-DHS employees in the same demographic groups to have
                                                                        β
            different levels of morale, as described by β D and G .We estimated each
            model using cluster-robust maximum likelihood methods, with 365 agency
            clusters (e.g., Transportation Security Administration [TSA]).

            Our multivariate analysis found that DHS employees remained an
            average of 6.4 percentage points less engaged (+/- 3.2) (see table 6) and
            5.5 points less satisfied (+/- 2.2) (not shown) on our scales than
            employees at other agencies who had the same age, office location, race,
            sex, supervisory status, and tenure. This suggests that measured
            demographic differences between employees at DHS and other agencies


            5
             We discuss options for a more sophisticated analysis of engagement and satisfaction on
            latent, continuous scales at the end of this appendix.




            Page 43                                                GAO-12-940 DHS Employee Morale
                                      Appendix I: Statistical Analysis of Employee
                                      Morale at Department of Homeland Security
                                      and Other Agencies




                                      do not fully explain the morale gap. Instead, factors that are intrinsic to
                                      DHS, such as culture or management practices, or demographic factors
                                      not measured by FEVS, such as education or occupation, are likely to be
                                      responsible.

Decomposing the Morale Gap            We can further explore the roles of demographics and unique DHS
                                      characteristics by performing an Oaxaca decomposition of the results of
                                      model 2, in order to compare DHS with other agencies. 6 Oaxaca
                                      decomposition can assess whether the overall morale gap is explained by
                                      the demographic characteristics of DHS employees, or whether it is
                                      explained by lower morale among DHS employees in the same
                                      demographic groups. In other words, does DHS employ an unusually
                                      large number of workers who tend to have low morale across all
                                      agencies, or do workers with the same backgrounds have uniquely lower
                                      morale at DHS?

Table 6: Model Estimates of Employee Engagement Index at DHS and Other Agencies Using the 2011 FEVS

                                                        DHS                                         Other Agencies
                                       Percentage engaged                               Percentage engaged
                                           (GAO measure)            Standard error          (GAO measure)          Standard error
All employees                                             20.7                 1.5                        27.1                  0.5
                                                                a
Gap between DHS and other                                -6.3
agencies
Portion of gap due to the                                  0.1
demographic composition of DHS
Portion of gap due to unique                              -6.4
differences between DHS and non-
DHS employees with similar
demographic profiles

Race
Hispanic                                                  27.0                 1.1                        29.0                  0.7
American Indian / Alaska Native                           13.2                 3.7                        21.3                  0.8




                                      6
                                          Oaxaca decomposition is a method of disaggregating an average difference between two
                                      groups into (1) the part due to the differences in the values of the variables that determine
                                      the outcome of interest and (2) the part due to differences in the partial relationships
                                      between the predictor variables and the outcome. See Ronald Oaxaca, “Male–Female
                                      Wage Differentials in Urban Labor Markets,” International Economic Review 14 (1973).
                                      693-709




                                      Page 44                                                   GAO-12-940 DHS Employee Morale
                             Appendix I: Statistical Analysis of Employee
                             Morale at Department of Homeland Security
                             and Other Agencies




                                                    DHS                                                      Other Agencies
                             Percentage engaged                                                Percentage engaged
                                 (GAO measure)                     Standard error                  (GAO measure)          Standard error
Asian                                                29.0                           3.5                           34.4               0.8
Black                                                23.7                           2.0                           27.8               0.5
Hawaiian/Pacific Islander                            23.1                           2.5                           28.4               1.3
White                                                19.0                           1.5                           26.7               0.5
Two or more races                                    18.3                           3.0                           20.7               0.8

Years worked for component
<1                                                   34.6                           2.5                           38.1               0.9
1-3                                                  22.8                           2.2                           30.5               0.6
4-5                                                  20.0                           1.8                           27.3               0.7
6-10                                                 17.6                           1.6                           25.7               0.5
11-20                                                19.3                           2.2                           24.7               0.5
20+                                                  21.2                           3.0                           26.6               0.6


Sex
Male                                                 20.6                           1.6                           27.1               0.5
Female                                               20.9                           1.5                           27.2               0.5


Supervisory status
Non-supervisor                                       18.2                           1.5                           23.3               0.5
Team leader                                          17.8                           1.7                           26.3               0.6
Supervisor                                           23.3                           1.7                           31.7               0.6
Manager                                              31.7                           2.5                           40.5               0.7
Executive                                            47.3                           2.8                           54.2               1.3

Age
< 26                                                 20.0                           2.9                           28.6               1.1
26-29                                                16.7                           2.6                           24.9               0.9
30-39                                                19.3                           1.6                           26.3               0.6
40-49                                                20.8                           1.9                           27.4               0.5
50-59                                                21.0                           1.7                           26.7               0.5
60+                                                  22.8                           1.6                           29.3               0.6
Location
Headquarters                                         21.4                           1.6                           27.8               0.6
Field                                                20.2                           1.9                           26.6               0.6
                             Source GAO analysis of U.S. Office of Personnel Management 2011 FEVS data.




                             Page 45                                                                      GAO-12-940 DHS Employee Morale
Appendix I: Statistical Analysis of Employee
Morale at Department of Homeland Security
and Other Agencies




Note: We created a unique measure of whether each employee in the 2011 FEVS was engaged for
the purposes of statistical analysis and describing differences among groups of employees. Because
our measure is defined differently than the OPM Employee Engagement Index, the measures are not
comparable. Our measure counts an employee as engaged if he or she gave positive answers to
each item in the OPM index (see text).Engagement statistics in this table are the in-sample
percentage of employees who are predicted by model 2 to be engaged on GAO’s measure.
a
Gap estimates and their decomposition are estimated using Oaxaca methods applied to model 2.


As shown in table 6, the model suggests that the demographic profile of
DHS employees (measured by FEVS) tends to slightly increase their
engagement and reduce the gap compared with employees at other
agencies. The demographic characteristics we can observe in FEVS
reduce the overall gaps in the proportion engaged and satisfied on our
scales by 0.1 and 1.0 percentage points, respectively. 7 Instead, the
morale gap is better explained by unique differences in morale between
DHS and other agencies among demographically similar employees.
Such intrinsic differences increase the gaps in the proportion engaged
and satisfied by 6.4 and 5.5 percentage points, respectively. If the
demographic profile of the DHS workforce did not change, but DHS could
achieve the same levels of morale as other agencies from the same types
of employees, our model predicts that DHS employees would not have
lower morale than employees at other agencies.

DHS employees with lower-level positions and component tenure were
among those with lower morale, relative to employees in other agencies.
As shown in figures 6 and 7, our measures of engagement and
satisfaction generally increased with seniority and decreased with tenure,
among employees at DHS and other agencies. At DHS, however, morale
increased more slowly as employees gained more seniority, and it
declined more quickly as they spent more time at the agency. For
example, the average newly hired employee at DHS and similar
employees at other agencies had statistically indistinguishable levels of
engagement. By their sixth years, however, satisfaction for the DHS
employee declined to an average of 18 percentage points, whereas
satisfaction for the non-DHS employees declined to an average of only 26
percentage points. A similar pattern exists with respect to supervisory
status (see figures 6 and 7). These patterns are particularly important for
explaining the overall morale gap, because DHS had about 30 percent
more supervisors and about twice as many people with 6 to 10 years of


7
 We omitted results for job satisfaction from table 6 to conserve space, but they generally
resemble the results for engagement.




Page 46                                                      GAO-12-940 DHS Employee Morale
Appendix I: Statistical Analysis of Employee
Morale at Department of Homeland Security
and Other Agencies




component tenure (as a share of all employees), compared with people at
other agencies (according to FEVS). 8

Figure 6: Engagement Index Scores by Supervisory Status and Tenure, for DHS and
Non-DHS Employees




8
 DHS employees may have answered the 2011 FEVS question on tenure differently,
depending on whether they worked for a component that existed before the department
was created. Specifically, employees who reported more than 10 years of service may
have interpreted the question to include their service prior to the creation of DHS.




Page 47                                               GAO-12-940 DHS Employee Morale
Appendix I: Statistical Analysis of Employee
Morale at Department of Homeland Security
and Other Agencies




Page 48                                        GAO-12-940 DHS Employee Morale
Appendix I: Statistical Analysis of Employee
Morale at Department of Homeland Security
and Other Agencies




Figure 7: Satisfaction Index Scores by Supervisory Status and Tenure, for DHS and
Non-DHS Employees




Page 49                                            GAO-12-940 DHS Employee Morale
                            Appendix I: Statistical Analysis of Employee
                            Morale at Department of Homeland Security
                            and Other Agencies




Morale Differences within   Low employee morale is not a uniform problem throughout DHS. As
DHS Components and          shown in table 7, engagement varies widely across components within
Work Groups                 the department, with employees in some components not being
                            significantly different from the average employee at non-DHS agencies.
                            These components include the U.S. Coast Guard (Coast Guard), Federal
                            Law Enforcement Training Center (FLETC), Management Directorate
                            (MGMT), and U.S. Secret Service (USSS). Job satisfaction at these
                            components also matches or exceeds that found at other agencies (not
                            shown in table 7).

                            DHS has a number of components whose employees have substantially
                            lower morale than employees at other agencies and elsewhere in the
                            department. The large share of DHS employees working in these
                            components accounts for the overall morale gap between DHS and other
                            agencies.


                            Page 50                                        GAO-12-940 DHS Employee Morale
Appendix I: Statistical Analysis of Employee
Morale at Department of Homeland Security
and Other Agencies




Components with lower morale include Federal Emergency Management
Agency (FEMA), Immigration and Customs Enforcement (ICE),
Intelligence and Analysis (IA), National Protection and Programs
Directorate (NPPD), Science and Technology (ST), and the TSA. The
engagement scores of these components range from 9.1 to 13.9
percentage points lower than the average score for non-DHS agencies
(see table 7). As a group, these components make up 46 percent of the
employees interviewed for the FEVS. Consequently, the components with
substantially lower morale have a large influence on the gap relative to
the rest of the government, despite the fact that morale at many smaller
DHS components is no worse.

Morale at some of the less engaged and satisfied components is, in turn,
strongly influenced by particular employee workgroups (see table 7). For
example, the average engagement at TSA is 12.8 percentage points
(apart from rounding) lower than at non-DHS agencies. Within TSA,
however, the collectively large groups of air marshal, law enforcement,
and screening workers account for much of the overall difference. A
similar pattern applies to the enforcement, removal, and homeland
security investigation staffs at ICE, the field operations staff at CBP, and
the Federal Protective Service. Such variation within components further
suggests that the morale gap is isolated to particular areas within DHS
that account for a large proportion of its workforce.

At other components, morale is more uniformly lower across most offices.
Average engagement at all work groups within FEMA is 5.8 to 17.7
percentage points lower than the non-DHS average, with the exception of
two regional offices and the offices of the Administrator and Chief of Staff.
The components of ST and IA also have more consistently low morale
across work groups.




Page 51                                         GAO-12-940 DHS Employee Morale
                                      Appendix I: Statistical Analysis of Employee
                                      Morale at Department of Homeland Security
                                      and Other Agencies




Table 7: OPM Employee Morale Index by DHS Component and Offices Using the 2011 FEVS

                                                                                   EE Index
                                    OPM Employee       95%    EE Index       95% minus non-       95%
                                  Engagement (EE) margin of minus DHS margin of        DHS margin of
                                           Index error (+/-)   average error (+/-)  average error (+/-)                 N
DHS component
Non-DHS                                         67.1         0.3               .        .         0.0        0.0   250,870
Citizenship and Immigration
Services                                        64.0         2.0               .        .       -3.2*        2.0     1,303
CBP                                             62.9         1.3               .        .       -4.3*        1.3     3,057
Coast Guard                                     70.6         2.5               .        .        3.5*        2.5      863
FEMA                                            58.0         2.3               .        .       -9.1*        2.3      972
FLETC                                           66.2         1.9               .        .        -1.0        1.9      611
ICE                                             58.0         2.5               .        .       -9.1*        2.5     1,313
Intel and Analysis                              53.2         4.0               .        .      -13.9*        4.0      150
IG                                              70.5         3.3               .        .         3.3        3.3      307
MGMT                                            65.7         2.1               .        .        -1.4        2.2      628
NPPD                                            57.7         2.1               .        .       -9.4*        2.2      771
Science and Technology                          55.7         2.8               .        .      -11.5*        2.8      258
USSS                                            68.1         2.4               .        .         1.0        2.4      959
Office of Secretary                             63.6         2.5               .        .       -3.6*        2.6      409
TSA                                             54.4         1.1               .        .      -12.8*        1.2     3,701
DHS, no sub-agency                              60.5         6.3               .        .       -6.6*        6.3      204


TSA
Non-DHS                                         67.1         0.3            4.4       0.9         0.0        0.0   250,870
Other DHS                                       62.7         0.8            0.0       0.0       -4.4*        0.9    11,805
Headquarters staff                              61.5         4.4           -1.2       4.5       -5.6*        4.4      279
Law enforcement/air marshal                     52.9         4.4           -9.8       4.4      -14.2*        4.4      246
Federal security director staff                 64.8         2.6            2.1       2.7        -2.3        2.6      630
Screeners                                       50.9         1.4          -11.8       1.6      -16.2*        1.4     2,273


ICE
Non-DHS                                         67.1         0.3            6.7       0.7         0.0        0.0   250,870
Other DHS                                       60.4         0.7            0.0       0.0       -6.7*        0.7    14,193
Director                                        66.3        23.4            5.9      23.4        -0.9      23.4        10
Acquisitions                                    41.2        31.0          -19.2      31.0       -26.0      31.0        13
CFO                                             68.3        12.6            7.9      12.6         1.2      12.6        33




                                      Page 52                                               GAO-12-940 DHS Employee Morale
                                         Appendix I: Statistical Analysis of Employee
                                         Morale at Department of Homeland Security
                                         and Other Agencies




                                                                                      EE Index
                                       OPM Employee       95%    EE Index       95% minus non-       95%
                                     Engagement (EE) margin of minus DHS margin of        DHS margin of
                                              Index error (+/-)   average error (+/-)  average error (+/-)                 N
CIO                                                69.8        14.1            9.4      14.1         2.6      14.1        29
Enforcement and removal operations                 52.8         3.8           -7.6       3.9      -14.4*        3.8      491
Human capital                                      51.2        24.3           -9.2      24.3       -16.0      24.3        11
Intelligence                                       59.1        16.7           -1.3      16.7        -8.1      16.7        27
International affairs                              57.5        11.0           -2.9      11.0        -9.6      11.0        21
Homeland security investigation                    59.0         4.1           -1.4       4.2       -8.2*        4.1      433
Principle legal advisor                            69.4         7.7            9.0       7.8         2.2        7.7       62
Professional responsibility                        66.6        12.4            6.2      12.4        -0.6      12.4        40
Other                                              77.2        20.1           16.8      20.1        10.1      20.1        13


Coast Guard
Non-DHS                                            67.1         0.3            7.6       0.7         0.0        0.0   250,870
Other DHS                                          59.6         0.7            0.0       0.0       -7.6*        0.7    14,643
Vice commandant                                    71.9        18.1           12.3      18.1         4.7      18.1        12
Chief of staff, mission support                    77.8         8.9           18.2       9.0       10.6*        8.9       66
Deputy commandant for operations                   89.1         5.1           29.6       5.2       22.0*        5.1       15
Force readiness command                            67.4        17.8            7.9      17.8         0.3      17.8        13
Marine safety security stewardship                 68.8        14.5            9.2      14.5         1.6      14.5        21
Engineering and logistics                          69.9         9.8           10.3       9.9         2.8        9.8       20
Command, control, communications,
computers, and IT                                  79.3        18.7           19.8      18.7        12.2      18.7        13
Atlantic                                           72.6        10.0           13.0      10.1         5.4      10.0        44
Pacific                                            61.0        14.5            1.4      14.5        -6.1      14.5        19
Districts 1                                        76.7         8.7           17.1       8.8        9.6*        8.7       46
Districts 5                                        69.5         8.8            9.9       8.9         2.3        8.8       69
Districts 7                                        66.8        12.2            7.2      12.2        -0.3      12.2        41
Districts 8                                        71.8         8.7           12.2       8.7         4.7        8.7       45
Districts 9                                        71.8         9.9           12.2      10.0         4.7        9.9       19
Districts 11                                       64.3        14.2            4.7      14.2        -2.9      14.2        35
Districts 13                                       78.5        17.4           18.9      17.4        11.4      17.4        18
Districts 14                                       80.2        19.4           20.6      19.4        13.0      19.4        12
Districts 17                                       68.7        14.5            9.1      14.5         1.6      14.5        22
Other                                              69.2         4.6            9.6       4.6         2.0        4.6      270




                                         Page 53                                               GAO-12-940 DHS Employee Morale
                                        Appendix I: Statistical Analysis of Employee
                                        Morale at Department of Homeland Security
                                        and Other Agencies




                                                                                     EE Index
                                      OPM Employee       95%    EE Index       95% minus non-       95%
                                    Engagement (EE) margin of minus DHS margin of        DHS margin of
                                             Index error (+/-)   average error (+/-)  average error (+/-)                                         N
CBP
Non-DHS                                             67.1               0.3                 8.5              0.8             0.0        0.0   250,870
Other DHS                                           58.6               0.8                 0.0              0.0           -8.5*        0.8    12,449
Office of the commissioner                          73.2             18.4                14.5             18.5              6.0      18.4        15
Chief Counsel                                       58.3             15.9                 -0.3            15.9             -8.8      15.9        29
Human resources management                          73.3             11.5                14.7             11.6              6.2      11.5        36
Border Patrol                                       69.2               2.2               10.6               2.3             2.1        2.2      861
International trade                                 66.1               7.4                 7.5              7.4            -1.1        7.4       80
Internal affairs                                    73.0             13.0                14.4             13.0              5.9      13.0        36
Field operations                                    57.5               1.9                -1.1              2.0           -9.6*        1.9     1,326
Administration and public affairs                   61.5               8.4                 2.9              8.4            -5.6        8.4       69
Information and Technology                          66.9               6.8                 8.3              6.8            -0.3        6.8       98
Training and development                            59.4               9.1                 0.8              9.1            -7.7        9.1       47
Internal Affairs                                    59.2             19.6                  0.6            19.6             -7.9      19.6        16
Air and marine                                      58.2               6.6                -0.5              6.7           -9.0*        6.6      103
Intelligence and operations
coordination                                        60.9             20.3                  2.3            20.4             -6.2      20.3        16
Technology, innovation, and
acquisition                                         80.1             12.4                21.5             12.4            13.0*      12.4        23
                                        Source: GAO analysis of U.S. Office of Personnel Management 2011 FEVS data.

                                        Note: Asterisks denote differences that are distinguishable from zero at the 0.05 level. Offices listed
                                        within the components are described as they are identified in OPM’s 2011 FEVS data files provided
                                        to GAO.


Comparison of Morale between            One explanation for why morale varies across components focuses on
Employees in Preexisting                the length of time each organization has existed. Components that
Components and Components               existed prior to the creation of DHS may have had more time to develop
Created with DHS                        successful cultures and management practices than components that
                                        policymakers created with the department in 2003. As a result, the
                                        preexisting components may have better morale today than components
                                        with less mature cultures and practices.

                                        To assess this explanation, we analyzed morale among two groups of
                                        components, divided according to whether the component was
                                        established with the creation of DHS or existed previously (see table 8).
                                        We considered three components to be preexisting—FLETC, USSS, and
                                        the Coast Guard—and the rest to be newly created. Because TSA was




                                        Page 54                                                                       GAO-12-940 DHS Employee Morale
                        Appendix I: Statistical Analysis of Employee
                        Morale at Department of Homeland Security
                        and Other Agencies




                        created about 2 years before DHS, we included it with components that
                        were created with DHS.

                        Our analysis shows that employees at the more recently created
                        components were less engaged and satisfied on average than employees
                        at the preexisting components and at non-DHS agencies. For the
                        preexisting components, engagement was about 2.2 percentage points
                        higher than at the rest of the government, and the difference in
                        satisfaction was small (less than 1.4 percentage points). In contrast,
                        engagement and satisfaction at the more recently created components
                        were about 8 and 5.1 percentage points lower than at the rest of the
                        government, respectively.

                        Table 8: Morale at Preexisting and Recently Created Components of DHS Using the
                        2011 FEVS

                                                                                                          OPM
                                                                        OPM Job                      Employee
                                                                      Satisfaction   Margin of     Engagement    Margin of
                                                                             Index   error (+/-)         Index   error (+/-)
                         Non-DHS                                             68.5           0.2           67.1          0.3
                         Preexisting components                              69.9           1.4           69.3          1.6
                         Components created with
                         DHS                                                 63.4           0.6           59.2          0.7
                        Source: GAO analysis of OPM 2011 FEVS data.

                        Note: Preexisting components include FLETC, USSS, and Coast Guard, with all others classified as
                        being created with DHS.


Multivariate Analysis   We developed a statistical model to confirm whether the differences
                        among components persist, holding constant demographic differences
                        among their employees. In an alternative version of model 1 above, we
                        replaced DHSj with a vector of variables indicating whether the employee
                        worked for DHS components or at an agency other than DHS. All other
                        parts of the model were identical.

                        The model estimates generally confirmed the differences in engagement
                        between non-DHS and DHS component employees in the raw data (see
                        table 9), with two exceptions. The model estimated that, holding constant
                        demographic differences, employees in the Management Directorate and
                        Office of the Secretary were 6.9 and 7.7 percentage points less engaged
                        on average than employees in non-DHS agencies. This suggests that the
                        engagement gap for employees in these offices is more similar to the gap
                        at other offices, holding constant the demographic differences among



                        Page 55                                                             GAO-12-940 DHS Employee Morale
                                             Appendix I: Statistical Analysis of Employee
                                             Morale at Department of Homeland Security
                                             and Other Agencies




                                             offices measured by FEVS. The model estimated that differences in
                                             satisfaction between the components and non-DHS agencies were
                                             generally similar to such differences in engagement (see table 9).

                                             The fact that differences among components remained, even among
                                             demographically equivalent employees, suggests that either unmeasured
                                             demographic variables or intrinsic characteristics of the components are
                                             responsible for the differences in morale.

Table 9: Model Estimates of the Difference in Engagement and Job Satisfaction between Employees in DHS Components and
Non-DHS Agencies Based on the 2011 FEVS

                                                   Difference in engagement Standard                 Difference in satisfaction Standard
DHS component                                             (GAO measure, %)      error                       (GAO measure, %)        error
CIS                                                                                    -5.6   0.5                             -3.5         0.5
CBP                                                                                    -6.1   0.5                             -2.5         0.6
Coast Guard                                                                            2.6    0.5                             -3.3         0.5
FEMA                                                                                   -9.9   0.5                             -5.7         0.4
FLETC                                                                                  0.5    0.6                             6.8          0.5
ICE                                                                                    -9.8   0.6                             -8.4         0.5
I&A                                                                               -13.9       0.5                           -10.6          0.5
IG                                                                                     6.4    0.9                             4.2          0.8
MGMT                                                                                   -6.9   0.6                             -3.0         0.6
NPPD                                                                              -11.3       0.5                             -9.0         0.5
S&T                                                                               -15.7       0.6                             -9.6         0.8
USSS                                                                                   3.6    0.6                             0.1          0.5
Office of Secretary                                                                    -7.7   0.6                             -8.5         0.7
TSA                                                                               -12.4       0.5                           -12.2          0.4
DHS, no component                                                                      7.5    0.3                             3.3          1.4
 Components existing prior to DHS creation                                             2.5    0.8                             0.5          2.0
Components created with DHS (plus TSA)                                                 -8.8   1.4                             -6.8         1.9
                                             Source: GAO analysis of 2011 FEVS data.

                                             Note: Engagement statistics are the in-sample proportion of employees who are predicted by model 1
                                             to be engaged or satisfied on GAO’s measure.




Opportunities for                            Our analysis discussed in this appendix has a narrow scope: assessing
Additional Analysis                          whether demographic differences among employees explain the morale
                                             differences across DHS and non-DHS employees. Consequently, DHS or
                                             others could expand and improve upon our findings.




                                             Page 56                                                      GAO-12-940 DHS Employee Morale
Appendix I: Statistical Analysis of Employee
Morale at Department of Homeland Security
and Other Agencies




Future work could examine whether attitudinal differences among
employees at DHS and other agencies explain the overall morale gap, in
addition to demographic differences. The 2011 FEVS measures
employee attitudes about pay, benefits, health and safety hazards,
training, supervisors, and other issues that could vary meaningfully
between employees at DHS and other agencies and, therefore, explain
why DHS has lower morale. One might include these factors in a
decomposition similar to the one we performed in this appendix. This
could further assess how factors unique to DHS and factors that are
common across all agencies explain the overall morale gap.

A broader attitudinal analysis likely would require the use of more
sophisticated statistical methods for estimating the values of and
relationships among latent variables. The broad measures of morale we
analyze in this appendix, such as the OPM Employee Engagement index,
are made up of responses to questions on smaller dimensions, such as
leadership and supervision. To avoid simply replicating the correlations
that were used to create the indexes, latent variable models could be
useful to examine the relationships among these concepts and compare
morale on latent scales between DHS and non-DHS agencies. This was
beyond the scope of our work.




Page 57                                        GAO-12-940 DHS Employee Morale
Appendix II: Scope and Methodology
              Appendix II: Scope and Methodology




              The objectives for this report were to evaluate (1) how DHS employee
              morale compares with that of other federal government employees and
              (2) to what extent DHS and its selected components determined the root
              causes of employee morale and developed action plans to improve
              morale.

              To address our objectives, we evaluated both DHS-wide efforts and
              efforts at four selected components to address employee morale—CBP,
              ICE, TSA, and the Coast Guard. We selected the four DHS components
              based on their workforce size and how their 2011 job satisfaction and
              engagement index scores 1 compare with the non-DHS average. 2 The
              components selected had scores both above, below, and similar to the
              average: TSA—below average on both indexes, constituting 25 percent of
              the DHS workforce; ICE—below average on both indexes, accounting for
              9 percent of the DHS workforce; CBP—at the non-DHS average for
              satisfaction and below on engagement, representing 27 percent of the
              DHS workforce; and the civilian portion of the Coast Guard—at the non-
              DHS average for satisfaction and above on engagement, composing 4
              percent of the DHS workforce. 3 Together these components represent 65
              percent of DHS’s workforce.

              To evaluate how DHS’s employee morale compares with that of other
              federal government employees, we analyzed employee responses to the
              2011 FEVS. We determined that the 2011 FEVS data were reliable for
              the purposes of our report, based on interviews with OPM staff, review
              and analysis of technical documentation of its design and administration,
              and electronic testing. We used two measures created by OPM—the
              employee job satisfaction and engagement indexes—to describe morale


              1
               The job satisfaction index, composed of seven FEVS questions such as “my work gives
              me a feeling of personal accomplishment,” indicates the extent to which employees are
              satisfied with their jobs and various aspects thereof. The Engagement Index, composed of
              15 FEVS questions, indicates the extent to which employees are immersed in the content
              of the job and energized to spend extra effort in job performance.
              2
               Throughout this report, non-DHS refers to all federal employee FEVS responses outside
              of DHS.
              3
               The FEVS does not survey military personnel. Therefore, for the purposes of our review,
              our focus at the Coast Guard was on morale-related concerns for the Coast Guard’s
              civilian workforce, which included 8,342 employees in 2011. The Coast Guard’s civilian
              workforce is responsible for supporting the Coast Guard mission through over 200
              different types of professional and trade fields, such as engineering, information
              technology, administration, and electrical work.




              Page 58                                                GAO-12-940 DHS Employee Morale
Appendix II: Scope and Methodology




across the federal government and within DHS. We calculated these
measures for various demographic groups, DHS components, and work
groups, in order to compare morale at DHS and other agencies among
employees who were demographically similar, in part using statistical
models. Appendix I describes our methods and findings in more detail. In
addition, we interviewed employee groups about morale to identify
examples of what issues may drive high and low morale within DHS. We
selected the employee groups based on the size of the employee group
within each selected component, ensuring we met with employees from
employee groups that composed significant proportions of FEVS
respondents, such as screeners from TSA (61 percent of TSA
respondents) and homeland security investigators from ICE (33 percent
of ICE respondents). The comments received from these interviews are
not generalizable to entire groups of component employees, but provide
insights into the differing issues that can drive morale.

To determine the extent to which DHS and the selected components
identified the root causes of employee morale and developed action plans
for improvements, we reviewed analysis results, interviewed agency
human capital officials and representatives of employee groups, and
evaluated action plans for improving morale. To identify criteria for
determining effective root cause analysis using survey data, we reviewed
both OPM and Partnership for Public Service guidance for action planning
based on annual employee survey results. On the basis of these
guidance documents, we identified factors that should be considered in
employee survey analysis that attempts to understand morale problems,
such as use of demographic group comparisons, benchmarking results
against results at similar organizations, and the linking results of root
cause analyses to action planning efforts. We evaluated documents
summarizing DHS-wide and selected component root cause analyses of
the 2011 FEVS to determine whether the factors we identified were
included in the analyses. In addition, we interviewed DHS officials who
conducted the analyses in order to fully understand root cause analysis
efforts. To identify criteria for determining agency action plans we
reviewed OPM guidance for using FEVS results and previous GAO work
indicating agencies’ success in measuring performance. On the basis of
these guidance documents, we identified OPM’s six steps that should be
considered in developing action plans and identified three attributes that
were relevant for measuring action plan performance—linkage, clarity,
and measurable target. We compared the action plans with these criteria
to determine whether these items were included in the action plans. In
addition, we interviewed DHS and component officials to identify efforts to
leverage best practices for improving morale.


Page 59                                       GAO-12-940 DHS Employee Morale
Appendix II: Scope and Methodology




We conducted this performance audit from October 2011 through
September 2012 in accordance with generally accepted government
auditing standards. Those standards require that we plan and perform the
audit to obtain sufficient, appropriate evidence to provide a reasonable
basis for our findings and conclusions based on our audit objectives. We
believe that the evidence obtained provides a reasonable basis for our
findings and conclusions based on our audit objectives.




Page 60                                      GAO-12-940 DHS Employee Morale
Appendix III: DHS and Selected Component
                           Appendix III: DHS and Selected Component
                           Steps Taken to Determine Root Causes of
                           Morale Problems


Steps Taken to Determine Root Causes of Morale
Problems
DHS Efforts to Determine   Since 2007 DHS’s Office of the Chief Human Capital Officer (OCHCO)
Root Causes of Morale      has completed several efforts to determine root causes of morale DHS-
Problems                   wide.

                           Focus groups. In 2007 OCHCO conducted focus groups to determine
                           employee concerns related to employee morale. DHS’s focus group effort
                           probed for insights into four areas—(1) leadership, (2) communication, (3)
                           empowerment, and (4) resources—and highlighted concerns raised by
                           focus group participants in each of those areas. For example, within the
                           leadership area, OCHCO’s focus group analysis found that the Customs
                           and Immigration reorganization was a topic discussed by many of the
                           U.S. Customs and Border Protection (CBP), U.S. Immigration and
                           Customs Enforcement (ICE), and Citizenship and Immigration Services
                           (CIS) personnel, especially what they felt was a lack of mission
                           understanding on the part of their managers. According to the analysis,
                           non-supervisory participants expressed dissatisfaction with the
                           combination of three types of inspection functions to present “one face at
                           the border.”

                             One Face at the Border

                             For operations at ports of entry, in September 2003 CBP issued its plan for
                             consolidating the inspection functions formerly performed by separate inspectors from
                             the three legacy agencies—customs inspectors from U.S. Customs, immigration
                             inspectors and Border Patrol from the former Immigration and Naturalization Service,
                             and the agriculture border inspectors from the Department of Agriculture’s Animal and
                             Plant Health Inspection Service. The plan, referred to as “One Face at the Border,”
                             called for unifying and integrating the legacy inspectors into two new positions—a CBP
                             officer and a CBP agricultural specialist. The new CBP officer would serve as the
                             frontline officer responsible for carrying out the priority anti-terrorism mission as well as
                             the traditional customs and immigration inspection functions while also identifying and
                             referring goods in need of a more extensive agricultural inspection to the agricultural
                             specialist. CBP anticipated that having a well-trained and well-integrated workforce that
                             could carry out the complete range of inspection functions involving the processing of
                             individuals and goods would allow it to utilize its inspection resources more effectively
                             and enable it to better target potentially high-risk travelers. Together, CBP envisioned
                             the result to be more effective inspections and enhanced security at ports of entry while
                             also accelerating the processing of legitimate trade and travel.



                           Focus group results were distributed to DHS components for
                           consideration in action planning efforts, according to OCHCO officials.
                           CBP, CIS, TSA, the Federal Emergency Management Agency (FEMA),
                           and the Federal Law Enforcement Training Center each addressed at
                           least one of the focus group results relating to leadership, communication,
                           empowerment, or resources in subsequent action plans, according to
                           OCHCO officials.


                           Page 61                                                    GAO-12-940 DHS Employee Morale
Appendix III: DHS and Selected Component
Steps Taken to Determine Root Causes of
Morale Problems




Statistical analysis. In 2008 OCHCO performed statistical analysis of
Federal Employee Viewpoint Survey (FEVS) data, beyond examining
high- and low-scoring questions, in an effort to determine what workplace
factors drove employee job satisfaction. Specifically, the analysis involved
isolating which sets of FEVS questions most affect employee job
satisfaction. The analysis found that five work areas identified in FEVS
questions drive employee job satisfaction: (1) performance and rewards,
(2) supervisor support, (3) physical conditions and safety, (4) senior
leadership effectiveness, and (5) the DHS mission. According to OCHCO
officials, DHS components were encouraged to conduct follow-up
discussions at the lowest possible organizational level based on
component survey scores in each of the five work areas. However,
OCHCO officials stated that they are not aware of any results of this effort
because OCHCO did not track or follow-up with the components on the
effect of key driver discussions that may have occurred. In addition,
increased emphasis on supervisor performance management training
was also implemented as a result of the analysis, according to OCHCO
officials.

Exit survey. In 2011, DHS began administering an exit survey to
understand why employees choose to leave their DHS positions.
Specifically, according to OCHCO officials, the DHS exit survey was
designed to determine where departing employees were moving both
inside and outside of DHS, to identify barriers related to diversity, to
identify reasons that veterans may be leaving DHS, and to capture
feedback from interns. The 2011 exit survey found, among other things,
that 27 percent of departing employees who responded to the exit survey
were staying within DHS or moving to a different position, and an
additional 12 percent of respondents were retiring. Lack of quality
supervision and advancement opportunities were the top reasons
responding employees indicated for leaving their positions. 1 Exit survey
results are shared with DHS components on a quarterly and annual basis.

2011 FEVS analysis. For the 2011 FEVS, DHS’s OCHCO evaluated the
results by comparing Human Capital Assessment and Accountability
Framework (HCAAF) index results by component. The analysis showed


1
 Results from DHS’ exit survey should be interpreted with caution. Because of the
method in which the survey is administered, its response rate in 2011 was quite low, close
to 40 percent. It is likely that a higher response rate would have produced somewhat
different results.




Page 62                                                 GAO-12-940 DHS Employee Morale
                                     Appendix III: DHS and Selected Component
                                     Steps Taken to Determine Root Causes of
                                     Morale Problems




                                     where the lowest index scores were concentrated. As shown in figure 8,
                                     lower scores across the indexes were concentrated among several
                                     components, including Intelligence and Analysis, Transportation and
                                     Security Administration (TSA), ICE, National Protection and Programs
                                     Directorate, and FEMA.

Figure 8: DHS’s 2011 Component Comparison Based on Four HCAAF Indexes




                                     The analysis also determined how DHS’s scores on the four indexes
                                     trended over time and compared with governmentwide averages. As
                                     shown in figure 9, DHS-wide scores have generally trended upward over




                                     Page 63                                      GAO-12-940 DHS Employee Morale
                                        Appendix III: DHS and Selected Component
                                        Steps Taken to Determine Root Causes of
                                        Morale Problems




                                        time, but continue to lag behind governmentwide averages for each
                                        index.

Figure 9: DHS HCAAF Scores since 2006




                                        Employee Engagement Executive Steering Committee (EEESC). In
                                        January 2012 the DHS Secretary directed all component heads to take
                                        steps to improve employee engagement through launch of the EEESC.
                                        According to OCHCO officials, the EEESC was launched in response to
                                        congressional concerns about DHS employee morale and the Partnership
                                        for Public Service results showing DHS’s low placement on the list of Best
                                        Places to Work. The EEESC is charged with serving as the DHS
                                        corporate body responsible for identifying DHS-wide initiatives to improve
                                        employee engagement, oversee the efforts of each DHS component to
                                        address employee engagement, and provide periodic reports to the
                                        Under Secretary for Management, Deputy Secretary, and Secretary on
                                        DHS-wide efforts to improve employee morale and engagement.
                                        Specifically, the Secretary made the following directives to component
                                        heads:


                                        Page 64                                       GAO-12-940 DHS Employee Morale
                       Appendix III: DHS and Selected Component
                       Steps Taken to Determine Root Causes of
                       Morale Problems




                       •   develop and assume responsibility for employee engagement
                           improvement plans,
                       •   identify and assign specific responsibilities for improved employee
                           engagement to component senior executive performance objectives,
                       •   identify and assign a senior accountable official to serve on the
                           EEESC,
                       •   conduct town hall meetings with employees,
                       •   attend a Labor-Management Forum meeting, and
                       •   provide monthly reports on actions planned and progress made to the
                           Office of the Chief Human Capital Officer.

                       As of August 2012, each of the Secretary’s directives were completed,
                       with the exception of assigning responsibilities for improved employee
                       engagement to Senior Executive performance objectives, which DHS
                       plans to implement in October 2012 as part of the next senior executive
                       performance period. The EEESC met in February 2012, and component
                       representatives shared their latest action plans and discussed issues of
                       joint concern. In preparation for the 2012 FEVS, the EEESC released a
                       memorandum from the Secretary describing the responsibilities of the
                       EEESC, highlighting department actions, and encouraging employee
                       participation in the FEVS, which began in April 2012. The EEESC also
                       agreed that a corresponding message should be released from
                       component heads outlining specific component actions taken in response
                       to past survey results and encouraging participation in the next survey. In
                       an April 2012 EEESC meeting, the Partnership for Public Service
                       provided a briefing describing the Best Places to Work in the Federal
                       Government rankings and best practices across the government for
                       improving morale scores. The EEESC members also discussed methods
                       for improving the response rates for the upcoming survey and engaged in
                       an action planning exercise designed to help identify actions for
                       department-wide deployment, according to OCHCO officials. As of
                       August 2012, EEESC action items were in development and had not
                       been finalized. According to OCHCO officials, the EEESC plans to decide
                       on action items by September 2012, but a projected date for full
                       implementation has yet to be established because the actions have not
                       been decided upon.


Components Have Also   In addition to the DHS-wide efforts, the components we selected for
Conducted Some Root    review—ICE, TSA, the U.S. Coast Guard (Coast Guard), and CBP—
Cause Analyses Using   conducted varying levels of analyses regarding the root causes of morale
                       issues to inform agency action planning efforts. The selected components
FEVS Results           each analyzed FEVS data to understand leading issues that may relate to



                       Page 65                                        GAO-12-940 DHS Employee Morale
Appendix III: DHS and Selected Component
Steps Taken to Determine Root Causes of
Morale Problems




morale, but the results indicated where job satisfaction problem areas
may exist and do not identify the causes of dissatisfaction within
employee groups. A discussion of the four selected components’ 2011
FEVS analysis and results are described below.

TSA. In its analysis of the 2011 FEVS, TSA focused on areas of concern
across groups, such as pay and performance appraisal concerns, and
also looked for insight on which employee groups within TSA may be
more dissatisfied with their jobs than others by comparing employee
group scores on satisfaction-related questions. TSA compared its results
with CBP results, as well as against DHS and governmentwide results.
When comparing CBP and TSA scores, TSA found that the greatest
differences in scores were on questions related to satisfaction with pay
and whether performance appraisals were a fair reflection of
performance. TSA scored 40 percentage points lower on pay satisfaction
and 25 percentage points lower on performance appraisal satisfaction. In
comparing TSA results with DHS and governmentwide results, TSA found
that TSA was below the averages for all FEVS dimensions. 2 TSA also
evaluated FEVS results across employee groups by comparing
dimension scores for headquarters staff, the Federal Air Marshals,
Federal Security Director staff, and the screening workforce. TSA found
that the screening workforce scored at or below scores for all other
groups across all of the dimensions.

ICE. In its analysis of the 2011 FEVS, ICE analyzed the results by
identifying ICE’s FEVS questions with the top positive and negative
responses. ICE found that its top strength was employees’ willingness to
put in the extra effort to get a job done. ICE’s top negative result was
employees’ perceptions that pay raises did not depend on how well
employees perform their jobs. ICE also sorted the primary low-scoring
results into action planning themes, such as leadership, empowerment,
and work-life balance. ICE found, among other things, that employee
views on the fairness of its performance appraisals were above DHS’s
average but that views on employee preparation for potential security
threats were lower. When comparing ICE’s results with average
governmentwide figures, ICE found, among other things, that ICE was
lower on all of the HCAAF indexes, including job satisfaction. According



2
 The FEVS includes questions grouped into the following dimensions: work experiences,
supervisor/team leader, agency, work unit, leadership, satisfaction, and work/life.




Page 66                                               GAO-12-940 DHS Employee Morale
Appendix III: DHS and Selected Component
Steps Taken to Determine Root Causes of
Morale Problems




to ICE human capital officials, future root cause analysis plans for the
2012 FEVS are to benchmark FEVS scores with those of similar law
enforcement agencies such as the Drug Enforcement Agency; Federal
Bureau of Investigation; Federal Law Enforcement Training Center;
United States Secret Service; Alcohol, Tobacco and Firearms, and the
U.S. Marshals.

CBP. In its analysis of the 2011 FEVS, CBP focused its analysis on
trends since 2006. For example, the analysis showed that CBP increased
its scores by 5 or more percentage points for 36 of the 39 core FEVS
questions. CBP highlighted its greatest increases in HCAAF areas, such
as results-oriented performance, which showed a 21 percent
improvement over 2006 responses to the question—my performance
appraisal is a fair reflection of my performance. The analysis also
identified areas in greatest need of improvement, which showed progress
since 2006 but continued low scores, such as questions on dealing with
poor performers who cannot or will not improve (28 percent positive),
promotions based on merit (28 percent positive) and differences in
performance are recognized (34 percent positive). 3

Coast Guard. In its review of high and low 2011 FEVS responses, the
Coast Guard identified employee responses to two questions that
warranted action planning items—(1) How satisfied are you with the
information you receive from management on what’s going on in your
organization (53 percent positive) and (2) My training needs are assessed
(51 percent positive). 4 The Coast Guard officials did not provide any
additional FEVS analyses that were used to inform action planning.




3
 CBP noted its three FEVS scores as low, but CBP’s scores are not substantially lower
than the governmentwide scores. In the 2011 FEVS, the governmentwide average for the
three questions were: dealing with poor performers who cannot or will not improve (31
percent positive), promotions based on merit (36 percent positive) and differences in
performance are recognized (36 percent positive).
4
 The Coast Guard’s scores on these questions are not substantially different from the
governmentwide averages (51 percent positive on information satisfaction with information
received and 54 percent positive on training needs). However, according to an OCHCO
official who monitored the Coast Guard’s action planning in previous years, these
questions were addressed in the Coast Guard’s action plan because they have an impact
on other low-scoring items and were important for employee satisfaction.




Page 67                                                GAO-12-940 DHS Employee Morale
Appendix IV: Selected Components’ Data
                                          Appendix IV: Selected Components’ Data
                                          Sources for Evaluating Morale, Other than the
                                          Federal Employee Viewpoint Survey


Sources for Evaluating Morale, Other than
the Federal Employee Viewpoint Survey

Component           Data source                     Purpose                                   Summary of results and how used
U.S. Immigration    DHS exit survey                 Identify why employees leave the          The number of exit survey respondents
and Customs                                         agency and where they are going.          from ICE was too low to identify any
Enforcement                                                                                   results and have not been used to
                                                                                              address morale as of June 2012,
                                                                                              according to ICE officials.
                    Federal Organizational          Last conducted in March 2012, the         The survey showed low employee
                    Climate Survey (FOCS) and       FOCS is a data-gathering tool for         perceptions of ICE as an organization
                    focus groups                    addressing the extent to which            where people trust and care for each
                                                    employees perceive their                  other, relative to the federal average,
                                                    organizational culture as one that        according to ICE officials. The results
                                                    incorporates mutual respect,              from the FOCS and feedback from the
                                                    acceptance, teamwork, and                 focus groups and individual one-on-one
                                                    productivity among individuals who are    interview sessions are provided to ICE
                                                    diverse in the dimensions of human        program offices with recommended
                                                    differences. Additionally, ICE conducts   strategies to improve the program
                                                    focus groups and individual one-on-one    office’s organizational climate.
                                                    interview sessions to obtain clarifying
                                                    information pertaining to the FOCS
                                                    results and written comments.
U.S. Customs and    Focus groups                    Conducted in 2007, focus groups were      The focus groups identified employees’
Border Protection                                   launched in response to the 2006          perceived problems in specific work
                                                    annual employee survey results, which     environment areas, such as leaders
                                                    showed CBP below DHS and                  lacking supervisory or communication
                                                    governmentwide averages.                  skills.
                                                                                              Among other things, the issues
                                                                                              identified by focus group participants
                                                                                              allowed CBP to develop action plans
                                                                                              that addressed these issues, according
                                                                                              to CBP officials.
                    Most Valuable Perspective       Launched in 2009, this survey was         In the July 2012 MVP, which solicited
                    online survey (MVP)             implemented to solicit employee           employee preferences for future CBP
                                                    opinions on one topic per quarter as a    webcasts to employees, employees
                                                    mechanism for gathering further           suggested retirement planning and
                                                    insights on FEVS results. The MVP         financial management as their top two
                                                    was implemented as a continuation of      preferences. CBP’s action plan
                                                    the CBP focus groups completed in         planning process in response to FEVS
                                                    2007.                                     results includes consideration of MVP
                                                                                              results, according to CBP officials.




                                          Page 68                                                   GAO-12-940 DHS Employee Morale
                                       Appendix IV: Selected Components’ Data
                                       Sources for Evaluating Morale, Other than the
                                       Federal Employee Viewpoint Survey




Component        Data source                     Purpose                                    Summary of results and how used
Coast Guard      U.S. Office of Personnel        Beginning in 2002, in order to provide     OPM’s report to the Coast Guard on
                 Management Organizational       the granularity, detail, and reliability   the 2010 OAS results identified seven
                 Assessment Survey (OAS)         needed to ensure the best                  strong organizational areas (diversity,
                                                 organizational value, the Coast Guard      teamwork, work environment,
                                                 adopted the OAS as its primary             leadership and quality, communication,
                                                 personnel attitude survey, according to    employee involvement and supervision)
                                                 Coast Guard officials. The OAS is          and three areas for improvement
                                                 administered to military (active and       (innovation, use of resources, and
                                                 reserve) and civilian personnel            rewards/recognition).
                                                 biennially.                                Coast Guard unit commanders and
                                                                                            headquarters program managers use
                                                                                            the OAS to support overall Coast
                                                                                            Guard improvement. This improvement
                                                                                            is achieved by feeding results of the
                                                                                            OAS to Coast Guard Unit Commanders
                                                                                            and Program Managers who then use
                                                                                            OAS results in conjunction with other
                                                                                            information as part of routine unit and
                                                                                            program leadership and management.
Transportation   TSA exit survey                 Identify why employees leave the           Top reasons for leaving overall were
Security                                         agency, launched in 2005.                  personal reasons, career
Administration                                                                              advancement, management, schedule,
                                                                                            and pay. Each quarterly report includes
                                                                                            actions managers should take to
                                                                                            reduce turnover. A real-time reporting
                                                                                            system is also available for each airport
                                                                                            and office within TSA so managers can
                                                                                            gain access to their results and use
                                                                                            them to reduce turnover and make
                                                                                            improvements, according to DHS
                                                                                            officials.
                                                                                            Results from the exit survey were also
                                                                                            used by TSA officials in updating TSA’s
                                                                                            action plan, according to TSA officials.
                                                                                            However, the July 2012 action plan did
                                                                                            not link exit survey findings to action
                                                                                            items.
                 Idea Factory                    An online tool for gathering employee      Results were not available for our
                                                 suggestions for agency improvement.        evaluation.
                                                 Each week, approximately 4,000 TSA
                                                 employees log on to rate, comment, or
                                                 search, or to submit ideas of their own.
                                                 The Idea Factory team reviews all
                                                 submissions and uses Idea Factory
                                                 challenges to implement solutions to
                                                 issues.
                 National Advisory Council       Two-year detail for TSA field              Results were not available for our
                                                 representatives to collect and address     evaluation.
                                                 workplace environment issues




                                       Page 69                                                    GAO-12-940 DHS Employee Morale
                                Appendix IV: Selected Components’ Data
                                Sources for Evaluating Morale, Other than the
                                Federal Employee Viewpoint Survey




Component   Data source                     Purpose                                                    Summary of results and how used
            DHS Ombudsman                   Provides informal problem resolution                       Results were not available for our
                                            services with the mission of promoting                     evaluation.
                                            fair and equitable treatment in matters
                                            involving TSA, according to TSA
                                            officials. The Ombudsman assists
                                            customers by identifying options,
                                            making referrals, explaining policies
                                            and procedures, coaching individuals
                                            on how to constructively deal with
                                            problems, facilitating dialogue, and
                                            mediating disputes.
            Employee Advisory               Each airport and TSA headquarters   Results were not available for our
            Committee                       has an employee advisory council    evaluation.
                                            made up of elected members who work
                                            on understanding and addressing a
                                            variety of workplace issues.
                                Source: GAO analysis of interviews with agency officials and documents provided by DHS.




                                Page 70                                                                          GAO-12-940 DHS Employee Morale
Appendix V: Comments from the Department
             Appendix V: Comments from the Department
             of Homeland Security



of Homeland Security




             Page 71                                    GAO-12-940 DHS Employee Morale
Appendix V: Comments from the Department
of Homeland Security




Page 72                                    GAO-12-940 DHS Employee Morale
Appendix VI: Comments from the U.S. Office
              Appendix VI: Comments from the U.S. Office of
              Personnel Management



of Personnel Management




              Page 73                                         GAO-12-940 DHS Employee Morale
Appendix VII: GAO Contact and Staff
                  Appendix VII: GAO Contact and Staff
                  Acknowledgments



Acknowledgments

                  David C. Maurer, (202) 512-9627 or maurerd@gao.gov
GAO Contact
                  In addition to the contact named above, Dawn Locke (Assistant Director),
Staff             Sandra Burrell (Assistant Director), Lydia Araya, Ben Atwater, Tracey
Acknowledgments   King, Kirsten Lauber, Jean Orland, Jessica Orr, and Jeff Tessin made key
                  contributions to this report.




(441016)
                  Page 74                                      GAO-12-940 DHS Employee Morale
GAO’s Mission         The Government Accountability Office, the audit, evaluation, and
                      investigative arm of Congress, exists to support Congress in meeting its
                      constitutional responsibilities and to help improve the performance and
                      accountability of the federal government for the American people. GAO
                      examines the use of public funds; evaluates federal programs and
                      policies; and provides analyses, recommendations, and other assistance
                      to help Congress make informed oversight, policy, and funding decisions.
                      GAO’s commitment to good government is reflected in its core values of
                      accountability, integrity, and reliability.

                      The fastest and easiest way to obtain copies of GAO documents at no
Obtaining Copies of   cost is through GAO’s website (http://www.gao.gov). Each weekday
GAO Reports and       afternoon, GAO posts on its website newly released reports, testimony,
                      and correspondence. To have GAO e-mail you a list of newly posted
Testimony             products, go to http://www.gao.gov and select “E-mail Updates.”

Order by Phone        The price of each GAO publication reflects GAO’s actual cost of
                      production and distribution and depends on the number of pages in the
                      publication and whether the publication is printed in color or black and
                      white. Pricing and ordering information is posted on GAO’s website,
                      http://www.gao.gov/ordering.htm.
                      Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
                      TDD (202) 512-2537.
                      Orders may be paid for using American Express, Discover Card,
                      MasterCard, Visa, check, or money order. Call for additional information.
                      Connect with GAO on Facebook, Flickr, Twitter, and YouTube.
Connect with GAO      Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.
                      Visit GAO on the web at www.gao.gov.
                      Contact:
To Report Fraud,
Waste, and Abuse in   Website: http://www.gao.gov/fraudnet/fraudnet.htm
                      E-mail: fraudnet@gao.gov
Federal Programs      Automated answering system: (800) 424-5454 or (202) 512-7470

                      Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-
Congressional         4400, U.S. Government Accountability Office, 441 G Street NW, Room
Relations             7125, Washington, DC 20548

                      Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800
Public Affairs        U.S. Government Accountability Office, 441 G Street NW, Room 7149
                      Washington, DC 20548




                        Please Print on Recycled Paper.