oversight

Measuring Performance: The Corporation for National and Community Service Faces Challenges Demonstrating Outcomes

Published by the Government Accountability Office on 2012-02-17.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                United States Government Accountability Office

GAO             Report to Congressional Committees




February 2012
                MEASURING
                PERFORMANCE
                The Corporation for
                National and
                Community Service
                Faces Challenges
                Demonstrating
                Outcomes




GAO-12-310
Contents


Letter                                                                                           1

Appendix I             Briefing Slides                                                           7



Appendix II            List of CNCS Grantees Interviewed                                       33



Appendix III           Reported Legal Constraints on Holding Senior Corps Grantees
                       Accountable                                                             34



Appendix IV            Comments from the Corporation for National and
                       Community Service                                                       35



Appendix V             GAO Contacts and Staff Acknowledgments                                  36



Related GAO Products                                                                           37




                       Abbreviations
                       CNCS      Corporation for National and Community Service
                       NCCC      National Civilian Community Corps
                       VISTA     AmeriCorps Volunteers in Service to America




                       Page i                                    GAO-12-310 CNCS Strategic Planning
This is a work of the U.S. government and is not subject to copyright protection in the
United States. The published product may be reproduced and distributed in its entirety
without further permission from GAO. However, because this work may contain
copyrighted images or other material, permission from the copyright holder may be
necessary if you wish to reproduce this material separately.




Page ii                                               GAO-12-310 CNCS Strategic Planning
United States Government Accountability Office
Washington, DC 20548




                                   February 17, 2012

                                   The Honorable Tom Harkin
                                   Chairman
                                   The Honorable Michael B. Enzi
                                   Ranking Member
                                   Committee on Health, Education, Labor & Pensions
                                   United States Senate

                                   The Honorable John Kline
                                   Chairman
                                   The Honorable George Miller
                                   Ranking Member
                                   Committee on Education and the Workforce
                                   House of Representatives

                                   It is important for federal agencies—and their grantees—to demonstrate
                                   the impact of their work and ensure the efficient use of their resources.
                                   This becomes even more important during times of fiscal stress and tight
                                   budget constraints as Congress makes difficult funding choices. In fiscal
                                   year 2011, Congress appropriated approximately $1 billion for the
                                   Corporation for National and Community Service (CNCS)—the federal
                                   agency responsible for administering the country’s national service
                                   programs—to carry out a variety of federally funded programs, such as
                                   AmeriCorps. Through these programs, CNCS provides grants to more
                                   than 70,000 volunteer organizations to address a range of community
                                   challenges through service activities such as tutoring children, responding
                                   to natural disasters, or helping seniors to live independently. In 2009, the
                                   Serve America Act made substantial changes to CNCS’s mission,
                                   including giving CNCS new areas of emphasis, such as working with
                                   veterans, and directing it to focus more on performance evaluation and
                                   cost-effectiveness. 1 The Serve America Act also mandated that we
                                   perform a series of reviews relating to program effectiveness with respect
                                   to CNCS’s performance measures. 2




                                   1
                                    Pub. L. No. 111-13, §§ 1101(6), 1302, and 1607, 123 Stat. 1460, 1463, 1485, 1532.
                                   2
                                       Id. at § 1712, 123 Stat. 1460, 1551.




                                   Page 1                                              GAO-12-310 CNCS Strategic Planning
In 2010, we reported that the performance measures from CNCS’s
previous strategic plan were poorly aligned with its strategic goals and did
not demonstrate the results of its work. 3 Given that CNCS is in the
process of implementing its new strategic plan and performance
measures, we examined the following key questions: (1) To what extent
does CNCS fund service activities that are covered by its performance
measures? (2) What key challenges, if any, affect the ability of CNCS and
its grantees to measure performance?

To answer the first question, we examined agency strategic planning and
grant-making documents and budget data to better understand how
CNCS allocates funding to service activities that are covered by its
performance measures. We also interviewed CNCS officials to obtain
information about CNCS’s grant-making policies and the service activities
that it funds. We also made adjustments to the programs’ budgets to
remove certain administrative expenses identified by CNCS such as
health care stipends and child care assistance for the AmeriCorps
Volunteers in Service to America (VISTA) program. Many of these costs
are beyond the control of programs and should not be directly linked to
the program’s performance. We discussed these adjustments with
agency budget officials and they concurred with our approach. To answer
the second question, we conducted site visits and phone interviews with a
total of 20 grantees, including 3 state service commissions. 4 (For a list of
the grantees we interviewed, see app. II.) We selected these grantees
because they are geographically diverse and cover both urban and rural
locations, provide representation among CNCS’s main programs 5 and
program focus areas, and reflect varying levels of success in measuring
their performance. We conducted site visits with grantees in three
states—California, Maryland, and Texas—as well as in the District of
Columbia. The remaining interviews were conducted by phone. During
our interviews, we used interview protocols with a standard set of
questions about grantees’ experiences measuring performance and any



3
 GAO, Performance Measurement: Better Alignment to Strategic Goals and Data
Verification Needed at the Corporation for National and Community Service, GAO-10-886
(Washington, D.C.: July 28, 2010).
4
 State service commissions are responsible for allocating CNCS’s funding to AmeriCorps
state programs and monitoring program activities.
5
 These main programs include AmeriCorps State and National, VISTA, National Civilian
Community Corps (NCCC), and Senior Corps.




Page 2                                              GAO-12-310 CNCS Strategic Planning
challenges they encountered in the process. The results of our site visits
and phone interviews with these 20 grantees cannot be generalized.
Furthermore, we also met with CNCS officials to identify performance
measurement challenges and reviewed relevant federal laws, regulations,
and agency documents.

We conducted this performance audit from April 2011 to February 2012 in
accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe that
the evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objectives.

We briefed your staff on the results of our analysis on January 11, 2012,
and this report formally conveys the information provided during that
briefing. See appendix I for the briefing slides that were presented on that
date. In summary, we reported the following findings:

•   CNCS officials expect that most grantees from its main programs will
    adopt at least 1 of the agency’s 16 agency-wide performance
    measures. Specifically, officials told us that AmeriCorps State and
    National and VISTA grantees will adopt at least 1 of CNCS’s
    performance measures by fiscal year 2012, and Senior Corps
    grantees will adopt at least 1 measure by fiscal year 2013. However,
    officials also said that they plan to fund some activities that do not fall
    under the performance measures, particularly activities that meet local
    needs and/or are innovative. For example, CNCS has funded
    grantees in the state of Washington to carry out gang violence
    prevention activities to address this community challenge. This could
    present challenges for CNCS as it balances accountability, using its
    performance measures to assess the impact of grantees’ service
    activities, with flexibility, allowing grantees to take on projects that
    meet local needs. CNCS may find it difficult to demonstrate the
    effectiveness of its service activities that fall outside its performance
    measures. Additionally, CNCS faces challenges using its performance
    measures to promote accountability among its Senior Corps grantees,
    as legal restrictions make it difficult for CNCS to remove funding from
    grantees that consistently fail to meet their performance targets, and
    these restrictions include some requirements for Senior Corps
    programs that do not apply to CNCS’s other programs. These
    restrictions can make it more difficult to hold these grantees
    accountable for performance. (For additional information on these



Page 3                                         GAO-12-310 CNCS Strategic Planning
    legal restrictions, see app. III.) Specifically, officials told us that failure
    to meet performance targets is not a sufficient reason to re-compete a
    Senior Corps grant, and that it is very difficult to terminate a grant for
    poor performance. At the time of our review, CNCS officials told us
    that they have not yet sought statutory changes to address these legal
    constraints.

•   Grantees we interviewed reported performance measurement
    challenges in four general areas that may hamper CNCS’s ability to
    measure its impact: understanding and applying CNCS’s performance
    requirements, developing data collection approaches, collecting data,
    and reporting data. For example, one grantee told us that CNCS’s
    guidance was too broad, and as a result, it had a difficult time
    understanding how to match its program work to the agency’s
    performance measurement requirements. With regard to developing
    data collection approaches, one grantee working with elementary and
    high school students reported difficulties designing an approach that
    would allow the program to collect and measure changes in student
    behavior and knowledge to demonstrate their impact. Grantees with
    multiple worksites and limited staff capacity pointed to challenges
    collecting performance data across worksites. Finally, grantees
    commented that it was challenging to report their performance data
    through eGrants, CNCS’s computer system. CNCS has taken some
    steps to address these challenges; however, many of these steps are
    in the early implementation stages.

As CNCS looks for better ways to demonstrate its impact to Congress
and the American public, it will be critical for the agency to hold grantees
accountable for meeting performance targets. To provide other options for
CNCS to hold Senior Corps grantees accountable for performance, we
recommended that the Chief Executive Officer of CNCS take action to
seek statutory changes, as appropriate, that would allow it to re-compete
Senior Corps grants if the agency has determined current grant recipients
do not meet agency performance measures, or financial management or
other requirements.

We provided a draft of this report to officials at CNCS for review and
comment. In response, CNCS provided written comments, which are
reproduced in appendix IV. Overall, CNCS agreed with our findings, and
consistent with our recommendation, the agency said it planned to seek
statutory changes to address the legal constraints on Senior Corps’ ability
to introduce competition into those programs. In CNCS’s Congressional
Budget Justification for fiscal year 2013, released February 13, 2012,



Page 4                                           GAO-12-310 CNCS Strategic Planning
CNCS set forth the statutory changes it is seeking to address these legal
constraints on the Senior Corps programs. If enacted, these changes
would remove most of the constraints that currently prevent CNCS from
holding Senior Corps grantees accountable for meeting performance
targets. Improved accountability could enable CNCS to more fully
demonstrate its impact.

In addition, CNCS provided us with technical comments, which we
incorporated as appropriate. Notably, in its technical comments, CNCS
said it continues to make progress aligning appropriate performance
measures for its major programs. Specifically, it said that NCCC will be
reporting on some of CNCS’s 16 agency-wide performance measures
that will capture its service work, including its activities related to disaster
response. This should enable CNCS to better align NCCC’s activities to
the agency’s goals.

With reference to our findings on CNCS’s performance measures, CNCS
said in its technical comments that it plans to measure performance of
service activities not aligned to its key strategic goals and will require
grantees to submit their programs’ specific performance data in progress
reports to the agency. While it is important to measure performance and
report on performance for all its service activities, the diversity of these
activities may make gathering, combining, and presenting robust and
consistent performance information difficult and lead to challenges
demonstrating the overall impact of CNCS’s efforts. Thus, as emphasized
in this review and our previous work, it will remain critical for CNCS to
focus its performance measures on the vital few that align with its
strategic goals. 6




6
See GAO-10-243.




Page 5                                          GAO-12-310 CNCS Strategic Planning
We are sending this report to CNCS, appropriate congressional
committees, and other interested parties. In addition, the report will be
available at no charge on GAO’s Web site at http://www.gao.gov. If you or
your staff have any questions about this report, please contact me at
(202) 512-7215 or scottg@gao.gov. Contact points for our Offices of
Congressional Relations and Public Affairs may be found on the last page
of this report. GAO staff who made key contributions to this report are
listed in appendix V.




George A. Scott
Director, Education, Workforce and
  Income Security Issues




Page 6                                     GAO-12-310 CNCS Strategic Planning
Appendix I: Briefing Slides
               Appendix I: Briefing Slides




   Measuring Performance: The Corporation
  for National and Community Service Faces
     Challenges Demonstrating Outcomes




      Briefing to Congressional Committees
                 January 11, 2012




                                                                         1




               Page 7                        GAO-12-310 CNCS Strategic Planning
                    Appendix I: Briefing Slides




Overview

• Introduction
• Research Objectives
• Scope and Methodology
• Summary of Findings
• Background
• Findings
• Conclusions


                                                                             2




                    Page 8                        GAO-12-310 CNCS Strategic Planning
                                                  Appendix I: Briefing Slides




Introduction

• In constrained budget environments, it is increasingly important for federal
  agencies – and their grantees – to demonstrate the impact of their work and
  ensure the efficient use of their resources.
• In fiscal year 2011, Congress appropriated approximately $1 billion for the
  Corporation for National and Community Service (CNCS) to administer a
  variety of volunteer and national service programs.
• The Serve America Act mandated GAO to perform a series of reviews
  relating to program effectiveness with respect to CNCS’s performance
  measures. This work is our second review.1
• Our previous work found that CNCS’s performance measures aligned
  poorly with its strategic goals and did not demonstrate the results of its
  work.2
• At the time of our review, CNCS had developed a new strategic plan for
  fiscal years 2011-2015, which included new areas of responsibility identified
   by the Serve America Act (Pub. L. No. 111-13) and new performance measures.

1Pub. L. No. 111-13, § 1712, 123 Stat. 1460, 1551 (2009).
2GAO,  Performance Measurement: Better Alignment to Strategic Goals and Data Verification Needed at the Corporation for National and Community
                                                                                                                                                  3
Service, GAO-10-886 (Washington, D.C.: July 28, 2010).




                                                  Page 9                                                              GAO-12-310 CNCS Strategic Planning
                                     Appendix I: Briefing Slides




GAO-12-310 CNCS Strategic Planning




Research Objectives

• Objective 1: To what extent does CNCS fund service
  activities that are covered by its performance measures?


• Objective 2: What key challenges, if any, affect the ability of
  CNCS and its grantees to measure performance?




                                                                                               4




                                     Page 10                       GAO-12-310 CNCS Strategic Planning
                    Appendix I: Briefing Slides




Scope and Methodology

• To examine the extent to which CNCS funds service
  activities that are covered by its performance measures, we:
   • Examined agency strategic planning and grant-making
     documents and budget data.
   • Interviewed CNCS officials to obtain information about
     CNCS’s grant-making policies and the service activities
     that it funds.




                                                                              5




                    Page 11                       GAO-12-310 CNCS Strategic Planning
                                                    Appendix I: Briefing Slides




Scope and Methodology

• To examine the challenges that may affect CNCS’s ability to
  measure performance, we:

        • Conducted site visits and phone interviews with 20 grantees,3
          including state commissions, that represented CNCS’s main
          programs and focus areas, various geographical locations,
          and varying level of success with measuring performance.
          The information obtained cannot be generalized.

        • Reviewed relevant federal laws, regulations, and agency
          documents, and interviewed CNCS officials to identify
          performance-related challenges.

                                                                                                                                                      6
3 For the purposes of this study, we are using the term “grantee” in general to describe both CNCS grantees and sponsors—those organizations that

receive AmeriCorps Volunteers in Service to America (VISTA) volunteers. We did not include National Civilian Community Corps (NCCC) worksites in
our interviews.




                                                    Page 12                                                               GAO-12-310 CNCS Strategic Planning
                   Appendix I: Briefing Slides




Scope and Methodology

• We conducted this performance audit from April 2011 to
  February 2012 in accordance with generally accepted
  government auditing standards. Those standards require
  that we plan and perform the audit to obtain sufficient,
  appropriate evidence to provide a reasonable basis for our
  findings and conclusions based on our audit objectives. We
  believe that the evidence obtained provides a reasonable
  basis for our findings and conclusions based on our audit
  objectives.



                                                                             7




                   Page 13                       GAO-12-310 CNCS Strategic Planning
                                  Appendix I: Briefing Slides




Summary of Findings

•   CNCS officials expect that most grantees from its main programs will adopt at least one of the
    agency’s 16 performance measures.

      •   Officials told us that AmeriCorps State and National and AmeriCorps VISTA grantees will
          adopt at least one of CNCS’s performance measures by fiscal year 2012, and Senior
          Corps grantees will adopt at least one measure by fiscal year 2013.

      •   However, CNCS faces difficulties balancing the need to measure the outcomes of its
          activities with providing flexibility for its programs to meet local needs.

      •   Further, CNCS is limited in its ability to promote accountability among its Senior Corps
          grantees, as statutory restrictions make it difficult to remove funding from grantees that
          consistently fail to meet performance targets. However, officials told us that they have not
          yet sought statutory changes to address these restrictions.

•   CNCS may also face challenges reporting its impact due to several grantee-related performance
    measurement challenges, such as collecting performance data from partnering worksites.

      •   CNCS has taken steps to address grantees’ performance measurement concerns;
          however, many of these efforts are in the early implementation stages.

                                                                                                          8




                                  Page 14                                      GAO-12-310 CNCS Strategic Planning
                                                Appendix I: Briefing Slides




Background



Four Programs Accounted for 90 Percent of CNCS’s Budget
for Volunteer Service Activities in Fiscal Year 2011

CNCS’s Fiscal Year 2011 Budget for Volunteer Service Activities, by Program, in Millions




Source: GAO analysis of CNCS budget data.                                                                              9
Note: Excludes administrative costs and education awards to volunteers.




                                                Page 15                                    GAO-12-310 CNCS Strategic Planning
                                               Appendix I: Briefing Slides




Background



The 16 Agency-Wide Performance Measures in CNCS’s Fiscal
Year 2011- 2015 Strategic Plan Will Gauge Progress Toward Its
Four Goals
                                                                •     Performance measures help agencies
                                                                      gauge their progress toward achieving
                                                                      their goals.
                                                                •     CNCS officials told us that these 16
                                                                      agency-wide performance measures
                                                                      reflect CNCS’s priorities.4
                                                                •     Officials plan to use them to
                                                                      demonstrate how CNCS uses its
                                                                      resources to fund service activities that
                                                                      contribute to its four goals.




Source: GAO analysis of CNCS’s FY 2011-2015 strategic plan.
4 CNCS’s strategic plan includes other performance measures, but they are not agency-wide.
                                                                                                                        10




                                               Page 16                                       GAO-12-310 CNCS Strategic Planning
                                             Appendix I: Briefing Slides




Background



Nine of the 16 Agency-Wide Performance Measures Are in
Goal 1, Divided into 6 Focus Areas




Source: GAO analysis of CNCS’s FY 2011-2015 strategic plan.                                          11




                                             Page 17                       GAO-12-310 CNCS Strategic Planning
                                     Appendix I: Briefing Slides




Objective 1: Adopting Performance Measures



CNCS Expects That Most Grantees Will Adopt at
Least One of its Performance Measures
•    In fiscal year 2012, CNCS officials said they expect most AmeriCorps State
     and National and VISTA grantees to adopt at least one of CNCS’s 16
     performance measures voluntarily.
      •    Officials said that “adopting” CNCS’s performance measures means
           that grantees choose one or more of the 16 measures that relate to
           their work, even if they do not cover all their service activities.
      •    Officials told us that grantees will generally be expected to measure
           both the outputs and the final outcomes of their service activities. For
           example, the number of students who complete the program is an
           output, while improved academic performance is an outcome.
•    Beginning in fiscal year 2013, CNCS officials also said that they expect to
     require Senior Corps grantees to adopt at least one of the16 performance
     measures. They did not estimate a time frame for requiring other grantees
     to adopt them.

                                                                                             12




                                     Page 18                       GAO-12-310 CNCS Strategic Planning
                                        Appendix I: Briefing Slides




Objective 1: Adopting Performance Measures



CNCS Will Fund Some Service Activities That Do
Not Align With its Performance Measures
• CNCS officials told us that they expect to fund some AmeriCorps State
  and National grantees that do not adopt CNCS’s performance
  measures, but whose work is innovative or meets local needs.

       • Some service activities that may meet local needs, such as
         preventing gang violence or teen pregnancies, do not clearly fit
         within CNCS’s 16 performance measures.

• AmeriCorps state formula grantees are overseen by state service
  commissions,5 which have the authority to decide whether to require
  grantees to use CNCS’s performance measures.




  5State service commissions provide CNCS funding to AmeriCorps state formula grantees in their states through        13
  annual grant competitions. In addition, the Commissions manage, monitor and evaluate these grantees.




                                        Page 19                                             GAO-12-310 CNCS Strategic Planning
                                     Appendix I: Briefing Slides




Objective 1: Adopting Performance Measures



CNCS Will Fund Some Service Activities That Do
Not Align With its Performance Measures
•    CNCS officials anticipate that some grantees that adopt CNCS’s
     performance measures will not use them for all the work they do, because
     some of their service activities do not align with the measures.
      •    For example, officials expect that some Senior Corps grantees will use
           CNCS’s performance measures for about 25 to 50 percent of their
           service activities.
•    Additionally, Senior Corps and NCCC officials told us they are legally
     required to fund some service activities that fall outside the 16 agency-wide
     performance measures, such as providing respite to caregivers of
     homebound seniors.
•    NCCC officials also plan to continue to use more general performance
     measures, rather than specific targets, for some of their work in order to
     retain flexibility for NCCC members to respond to disasters.
                                                                                             14




                                     Page 20                       GAO-12-310 CNCS Strategic Planning
                                      Appendix I: Briefing Slides




Objective 1: Adopting Performance Measures



CNCS Faces Challenges Balancing Performance
Accountability with Flexibility
• According to CNCS officials, it is difficult to hold grantees accountable
  for measuring their activities’ effectiveness without compromising
  grantees’ flexibility to tailor their program activities to local needs.
      • Officials said they seek to promote accountability by using the 16
        performance measures to assess the impact of grantees’ work.
      • However, officials also told us that they wanted to preserve
        grantees’ flexibility to meet local needs, even if those needs fall
        outside the performance measures.

• This is consistent with our previous findings that other federal grant
  making programs face similar challenges, because grant features that
  encourage performance accountability can limit grantee flexibility.6

  6 GAO,
       Grants Management: Enhancing Performance Accountability Provisions Could Lead to Better Results,           15
  GAO-06-1046 (Washington, D.C.: Sept. 29, 2006).




                                      Page 21                                           GAO-12-310 CNCS Strategic Planning
                                     Appendix I: Briefing Slides




Objective 1: Adopting Performance Measures



CNCS Faces Challenges Balancing Performance
Accountability with Flexibility
•    CNCS officials told us they are currently exploring ways to measure the
     performance of service activities that are not covered under the 16 agency-wide
     performance measures.
      •    As previously noted, CNCS plans to focus on its 16 agency-wide
           performance measures. However, the strategic plan contains other
           measures that CNCS may use to assess the activities of individual
           initiatives or programs. For example, among other measures, CNCS plans
           to collect:
             • the percent of economically disadvantaged individuals receiving
               employment-related assistance.
             • the number of uninsured and economically disadvantaged individuals
               connected to or utilizing health care services.
      •    However, the diversity of these activities may make gathering, combining,
           and presenting robust and consistent performance information difficult,
           leading to challenges demonstrating the overall outcomes of these efforts.
                                                                                             16




                                     Page 22                       GAO-12-310 CNCS Strategic Planning
                                     Appendix I: Briefing Slides




Objective 1: Adopting Performance Measures



    CNCS Faces Challenges Holding Its Senior Corps
    Grantees Accountable For Performance Targets
•    Senior Corps programs—Senior Companion, Retired and Senior Volunteer Program (RSVP), and
     Foster Grandparent—are longstanding programs that exist to connect older Americans to service
     opportunities.
•    Officials told us that some of the volunteer organizations they fund have been grantees for several
     decades.
•    CNCS faces legal obstacles to reducing funding for Senior Corps grantees that persistently do not
     meet their performance targets.
      •    Even though CNCS officials told us that they will require Senior Corps grantees to adopt at least
           one of the 16 agency-wide performance measures by 2013, they commented that without a
           change in statutory authority, it can be very difficult to remove a Senior Corps grantee for poor
           performance alone.
      •    Specifically, officials told us that performance alone is not sufficient reason to re-compete a
           Senior Corps grant, if the grantee is otherwise complying with the terms of the grant contract.
      •    CNCS officials said that they have not yet sought statutory changes to address these difficulties.
•    See appendix III for additional information on the legal obstacles that affect competition in the Senior
     Corps programs.



                                                                                                                17




                                     Page 23                                          GAO-12-310 CNCS Strategic Planning
                                         Appendix I: Briefing Slides




Objective 1: Adopting Performance Measures



CNCS Faces Challenges Holding Its Senior Corps
Grantees Accountable For Performance Targets
• CNCS can utilize non-financial means to help grantees
  meet their performance targets, such as providing public
  recognition or increased oversight.7
       • For example, CNCS increases its on-site compliance
         reviews of Senior Corps grantees that do not meet their
         performance targets.
• However, if non-financial methods fail to improve grantee
  performance, CNCS’s ability to use its performance
  measures to promote accountability is potentially limited
  due to difficulties offering these grants for re-competition.
  7For more information about financial and non-financial options for holding grantees accountable for performance,     18
  see GAO-06-1046.




                                         Page 24                                              GAO-12-310 CNCS Strategic Planning
                                     Appendix I: Briefing Slides




Objective 2: Performance Measurement Challenges



Several Grantee-Related Performance Measurement
Challenges Could Hamper CNCS’s Ability to Measure Its
Impact

• Grantees we interviewed
  reported performance
  measurement challenges
  in four areas (see figure).



• CNCS has taken steps to
  address grantees’
  challenges; however,
  many of its efforts are in
  the early implementation
  stages.


                                                                   Source: GAO analysis.                       19




                                     Page 25                                         GAO-12-310 CNCS Strategic Planning
                                     Appendix I: Briefing Slides




Objective 2: Understanding and Applying CNCS
             Performance Requirements

Grantees Faced Challenges Understanding and Applying
Performance Information Communicated by CNCS

•   Grantees we interviewed reported:

      •   The information provided by CNCS
          was too broad and lacked specific
          examples related to their work.

      •   Overcoming the learning curve to
          understanding CNCS’s performance
          requirements was challenging.

•   Since we met with these grantees, the
    AmeriCorps and VISTA programs have
    updated their guidance, and CNCS
    officials expect to do the same for their
    other key programs.



                                                                   Source: GAO analysis.               20




                                     Page 26                                 GAO-12-310 CNCS Strategic Planning
                                       Appendix I: Briefing Slides




Objective 2: Developing Data Collection Approaches


Grantees Faced Challenges Developing Approaches to Collect
Performance Data

•   Grantees we interviewed reported
    challenges developing data collection
    approaches that:

       •   Measure the work performed by
           volunteers.

       •   Link program activities with
           outcomes to show impact.

•   CNCS is developing a performance
    measurement training curriculum, which
    may include information on developing
    appropriate data collection approaches.
    (Expected completion Sept. 2012.)



                                                                     Source: GAO analysis.                 21




                                       Page 27                                   GAO-12-310 CNCS Strategic Planning
                                          Appendix I: Briefing Slides




Objective 2: Collecting Data



Grantees Faced Challenges Collecting Performance Data from
Worksites

                                                                        •   Grantees we interviewed reported
                                                                            challenges with the lack of:

                                                                             •   Staff capacity or resources to collect
                                                                                 data from multiple worksites.

                                                                             •   Technology, such as a computer
                                                                                 system, that could help them avoid
                                                                                 manual entry of data.

                                                                        •   CNCS is revamping its resource center
                                                                            and has plans to examine grantees’ data
                                                                            collection needs and identify possible
                                                                            support the agency can provide.




                      Source: GAO analysis.                                                                             22




                                          Page 28                                             GAO-12-310 CNCS Strategic Planning
                                               Appendix I: Briefing Slides




Objective 2: Reporting Data



Grantees Faced Challenges Tracking Participants and
Reporting Meaningful Outcomes to Show Their Impact

                                                                                    •    Grantees we interviewed reported
                                                                                         challenges:

                                                                                            •    Following up with some past
                                                                                                 program participants,
                                                                                                 particularly those at-risk
                                                                                                 populations.

                                                                                            •    Identifying outcomes within
                                                                                                 CNCS’s one-year reporting
                                                                                                 framework.

                                                                                    •    CNCS has provided individualized
                                                                                         technical assistance through state
                                                                                         service commissions and its state
                                                                                         offices.8


Source: GAO analysis.
8CNCS state offices are responsible for overseeing and providing assistance to Senior Corps and VISTA grantees.
                                                                                                                                        23




                                               Page 29                                                        GAO-12-310 CNCS Strategic Planning
                                        Appendix I: Briefing Slides




Objective 2: Reporting data



Grantees Faced Challenges Reporting Performance Data
Through CNCS’s eGrants Computer System

                                                                 • Grantees we interviewed also
                                                                   reported challenges:

                                                                      • Accessing the eGrants system,
                                                                        CNCS’s electronic web-based
                                                                        system that grantees use to
                                                                        report performance data.

                                                                      • Navigating through eGrants.

                                                                 • CNCS has updated its eGrants
                                                                   performance measurement module
                                                                   for AmeriCorps, and plans to develop
                                                                   a module that can be used across all
                                                                   programs by summer 2012.

                    Source: GAO analysis.                                                                     24




                                        Page 30                                     GAO-12-310 CNCS Strategic Planning
                                 Appendix I: Briefing Slides




Conclusions

•   CNCS faces difficulties in striking a balance between providing grantees with the flexibility
    to meet local needs and holding them accountable for performance.
      •   Currently, not all of the work CNCS funds can be assessed under its performance
          measures. CNCS will face difficulties demonstrating the results of this portion of its
          work.
      •   Additionally, CNCS’s ability to use performance information to promote accountability
          among Senior Corps grantees may be limited, due to statutory restrictions that make
          it difficult to remove funding from persistently poor performers.
      •   At the time of our review, CNCS had not yet sought statutory changes to address
          these restrictions.

•   CNCS has taken some steps to address grantee-related performance challenges, but
    many of its efforts are in the early stages. Thus, it is unclear to what extent these efforts
    will fully address grantees’ challenges.

•   As CNCS moves forward, it will be critical for the agency to continue to monitor its efforts
    and the experiences of its grantees to ensure it is able to obtain meaningful performance
    information to demonstrate its impact to Congress and the American public.


                                                                                                      25




                                 Page 31                                    GAO-12-310 CNCS Strategic Planning
                     Appendix I: Briefing Slides




GAO on the Web
Web site: http://www.gao.gov/

Contact
Chuck Young, Managing Director, Public Affairs, youngc1@gao.gov
(202) 512-4800, U.S. Government Accountability Office
441 G Street NW, Room 7149, Washington, D.C. 20548

Copyright
This is a work of the U.S. government and is not subject to copyright
protection in the United States. The published product may be reproduced and
distributed in its entirety without further permission from GAO. However,
because this work may contain copyrighted images or other material,
permission from the copyright holder may be necessary if you wish to
reproduce this material separately.

                                                                                  26




                     Page 32                            GAO-12-310 CNCS Strategic Planning
Appendix II: List of CNCS Grantees
                                         Appendix II: List of CNCS Grantees
                                         Interviewed



Interviewed


                                                                                                                  a
CNCS grantee                           State        CNCS program                                     Focus area
Amarillo Independent School District   TX           AmeriCorps State                                 Education
AmeriCorps Red Cross Florida Gulf      FL           AmeriCorps State                                 Healthy Futures
Coast                                                                                                Veterans and Military Families
                                                                                                     Disaster Services
Baltimore City Health Department       MD            Senior Corps/ RSVP                              Disaster Services
                                                                                                     Education
                                                                                                     Healthy Futures
California Volunteers                  CA           State Service Commission                         Disaster Services
                                                                                                     Economic Opportunity
                                                                                                     Education
                                                                                                     Environmental Stewardship
                                                                                                     Healthy Futures
                                                                                                     Veterans and Military Families
City of Oakland Office Department of   CA           Senior Corps/ Foster Grandparent and             Education
Human Services                                      Senior Companion                                 Healthy Futures
Conservation Corps North Bay           CA           AmeriCorps State                                 Environmental Stewardship
Council of Churches of the Ozarks      MO           Senior Corps/ RSVP                               Education
                                                                                                     Healthy futures
Montana Prevention Resource Center     MT           AmeriCorps VISTA                                 Healthy Futures
                                                                                                     Veterans and Military Families
Department of Veterans Affairs         CA           AmeriCorps State                                 Veterans and Military Families
Serving America’s Farmworkers          DC           AmeriCorps National                              Healthy Futures
Everywhere (SAFE) AmeriCorps
Maryland Conservation Corps            MD           AmeriCorps State                                 Environmental Stewardship
Maryland Governor’s Office on          MD           State Service Commission                         Economic Opportunity
Service and Volunteerism                                                                             Education
                                                                                                     Environmental Stewardship
                                                                                                     Healthy Futures
OneStar Foundation                     TX           State Service Commission                         Education
                                                                                                     Disaster Services
Panhandle Community Services           TX           Senior Corps/ RSVP                               Healthy Futures
                                                                                                     Disaster Services
                                                                                                     Education
Rural Action, Inc.                     OH           AmeriCorps State                                 Environmental Stewardship
Texas Department of Aging and          TX           Senior Corps/ Foster Grandparent Program         Education
Disability Services
Texas Homeless Network                 TX           AmeriCorps VISTA                                 Economic Opportunity
United Planning Organization           DC           Senior Corps/ Foster Grandparent Program         Education
United Way of the Bay Area             CA           AmeriCorps VISTA                                 Economic Opportunity
Coach Across America– Up2Us            NY           AmeriCorps National                              Healthy Futures
                                         Source: GAO analysis of data from 20 CNCS grantees.
                                         a
                                          These focus areas were used by grantees at the time of our interviews, which were conducted from
                                         July 2011 to September 2011.




                                         Page 33                                                   GAO-12-310 CNCS Strategic Planning
Appendix III: Reported Legal Constraints on
              Appendix III: Reported Legal Constraints on
              Holding Senior Corps Grantees Accountable



Holding Senior Corps Grantees Accountable


               Statutory              Programs
               section                affected                        Description of legal constraint
               42 U.S.C.              Retired and Senior              Beginning in fiscal year 2013, the competitive
               § 5001(e)              Volunteer Program               process to determine Retired and Senior Volunteer
                                      (a Senior Corps                 Program (RSVP) grantees must ensure that the
                                      program)                        number of volunteers in specific geographic areas
                                                                      will remain at least at the previous level, and a
                                                                      similar distribution of programs must be
                                                                      maintained. Additionally, beginning in 2013 the
                                                                      competitive process must ensure that “every effort”
                                                                      is made to minimize the disruption to volunteers.
               42 U.S.C.              Retired and Senior              CNCS is required to continue funding RSVP
               § 5001(i)              Volunteer Program               programs that fail to meet performance measures
                                      (a Senior Corps                 for up to 12 months if a competition does not result
                                      program)                        in a new grantee.
               42 U.S.C.              All Senior Corps                Funding reductions resulting from insufficient
               § 5027(a)(2)           programs                        appropriations must be applied pro rata to all
                                                                      Senior Corps grants. (This pro rata requirement
                                                                      applies regardless of the grantees’ performance.)
               42 U.S.C.              Programs under                  If CNCS wants to deny an application for refunding,
               § 5052(a)              the Domestic                    the grantee must be given (1) at least 75 days
                                      Volunteer Service               notice of the possibility of such denial and the
                                      Act of 1973, which              grounds for the denial, and (2) an opportunity to
                                      includes Senior                 show cause why such action should not be taken.
                                      Corps programs                  If the grantee’s application is denied for
                                                                      noncompliance with the grant or contract terms, the
                                                                      grantee must be given an opportunity for an
                                                                      informal hearing before an impartial hearing officer
                                                                      who has been agreed to by the grantee and CNCS.
                                                                      If CNCS wants to terminate funding for
                                                                      noncompliance with grant or contract terms, the
                                                                      grantee must be given reasonable notice and
                                                                      opportunity for a full and fair hearing.
               42 U.S.C.              All CNCS                        If a grantee is failing to meet performance levels,
               § 12639(l)             programs,                       CNCS must develop a corrective action plan and
               and (m)                including Senior                establish a period of correction for the grantee. For
                                      Corps programs                  these low-performing grantees, CNCS must require
                                                                      the grantee to submit quarterly progress reports. In
                                                                      addition, if such a grantee has been receiving
                                                                      funding for less than 3 years, CNCS is required to
                                                                      provide technical assistance to address the
                                                                      performance problems. Termination or funding
                                                                      reduction of a grantee occurs only if the grantee
                                                                      continues to fail to meet performance measures
                                                                      after such a period of correction.
              Source: CNCS officials, as confirmed by GAO review of relevant federal laws.




              Page 34                                                                        GAO-12-310 CNCS Strategic Planning
Appendix IV: Comments from the
              Appendix IV: Comments from the Corporation
              for National and Community Service



Corporation for National and Community
Service




              Page 35                                      GAO-12-310 CNCS Strategic Planning
Appendix V: GAO Contacts and Staff
                  Appendix V: GAO Contacts and Staff
                  Acknowledgments



Acknowledgments

                  George A. Scott, Director, (202) 512-7215, scottg@gao.gov
GAO Contact
                  In addition to the contact listed above, Bill Keller (Assistant Director),
Acknowledgments   James Bennett, Susannah Compton, Elizabeth Curda, Alex Galuten,
                  Kirsten Lauber, Sheila McCoy, Erin McLaughlin, Brittni Milam, and
                  Ashanta Williams made key contributions to this report.




                  Page 36                                        GAO-12-310 CNCS Strategic Planning
Related GAO Products
             Related GAO Products




             Performance Measurement: Better Alignment to Strategic Goals and Data
             Verification Needed at the Corporation for National and Community
             Service. GAO-10-886. Washington, D.C.: July 28, 2010.

             Grants Management: Enhancing Performance Accountability Provisions
             Could Lead to Better Results. GAO-06-1046. Washington, D.C.:
             September 29, 2006.

             Corporation for National and Community Service: Better Internal
             Control and Revised Practices Would Improve the Management of
             AmeriCorps and the National Service Trust. GAO-04-225. Washington,
             DC.: January 16, 2004.

             Corporation for National and Community Service: Preliminary
             Observations on the National Service Trust and AmeriCorps.
             GAO-03-642T. Washington, D.C.: April 10, 2003.

             National Service Programs: Two AmeriCorps Programs’ Funding and
             Benefits. GAO/HEHS-00-33. Washington, D.C.: February 15, 2000.

             Grant Programs: Design Features Shape Flexibility, Accountability, and
             Performance Information. GAO/GGD-98-137. Washington, D.C.:
             June 22, 1998.

             The Results Act: An Evaluator’s Guide to Assessing Agency Annual
             Performance Plans. GAO/GGD-10.1.20. Washington, D.C.: April 1, 1998.

             National Service Programs: Role of State Commissions in Implementing
             the AmeriCorps Program. GAO/HEHS-97-49. Washington, D.C.:
             February 20, 1997.




(131073)
             Page 37                                    GAO-12-310 CNCS Strategic Planning
GAO’s Mission         The Government Accountability Office, the audit, evaluation, and
                      investigative arm of Congress, exists to support Congress in meeting its
                      constitutional responsibilities and to help improve the performance and
                      accountability of the federal government for the American people. GAO
                      examines the use of public funds; evaluates federal programs and
                      policies; and provides analyses, recommendations, and other assistance
                      to help Congress make informed oversight, policy, and funding decisions.
                      GAO’s commitment to good government is reflected in its core values of
                      accountability, integrity, and reliability.

                      The fastest and easiest way to obtain copies of GAO documents at no
Obtaining Copies of   cost is through GAO’s website (www.gao.gov). Each weekday afternoon,
GAO Reports and       GAO posts on its website newly released reports, testimony, and
                      correspondence. To have GAO e-mail you a list of newly posted products,
Testimony             go to www.gao.gov and select “E-mail Updates.”

Order by Phone        The price of each GAO publication reflects GAO’s actual cost of
                      production and distribution and depends on the number of pages in the
                      publication and whether the publication is printed in color or black and
                      white. Pricing and ordering information is posted on GAO’s website,
                      http://www.gao.gov/ordering.htm.
                      Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
                      TDD (202) 512-2537.
                      Orders may be paid for using American Express, Discover Card,
                      MasterCard, Visa, check, or money order. Call for additional information.
                      Connect with GAO on Facebook, Flickr, Twitter, and YouTube.
Connect with GAO      Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.
                      Visit GAO on the web at www.gao.gov.
                      Contact:
To Report Fraud,
Waste, and Abuse in   Website: www.gao.gov/fraudnet/fraudnet.htm
                      E-mail: fraudnet@gao.gov
Federal Programs      Automated answering system: (800) 424-5454 or (202) 512-7470

                      Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-
Congressional         4400, U.S. Government Accountability Office, 441 G Street NW, Room
Relations             7125, Washington, DC 20548

                      Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800
Public Affairs        U.S. Government Accountability Office, 441 G Street NW, Room 7149
                      Washington, DC 20548




                        Please Print on Recycled Paper.