oversight

OIG's Independent Report on the Department's Performance Summary Report for Fiscal Year 2010

Published by the Department of Education, Office of Inspector General on 2011-01-31.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                          UNITED STATES DEPARTMENT OF EDUCATION

                                      OFFICE OF SAFE AND DRUG-FREE SCHOOLS




                                                    January 3 1 , 2 0 1 1


Honorable R. Gil Kerlikowske
Director
Office of National Drug Control Policy
Executive Office of the President
Washington, D.C. 20500


Dear Director Kerlikowske:


In accordance with the Office of National Drug Control Policy (ONDep) Circular Drug Control
Accounting, enclosed please find detailed information about performance-related measures for
key drug control programs administered by the U.S. Department of Education contained in the
Us. Department of Education's Performance Summary Report for Fiscal Year 2010, along with
the Department of Education Assistant Inspector General's authentication of the management
assertions included in that report.


Please do not hesitate to contact me if you have any questions about this information.


                                      Sincerely,




                                ,L" rJ )y->-��
                                      Kevin Jennings
                                      Assistant Deputy Secretary


Enclosure # I: Department of Education Performance Summary Report for Fiscal Year 20 IO.
January 19,20 1 1


Enclosure #2: Authentication letter from Keith West, Assistant Inspector General for Audit
Services, January 3 1 , 20 1 1


cc:   Keith West




                               400 MARYLAND AVE., S.W.. WASHINGTON, DC 20202
                                                       www.ed.gov

      Our mission is to ensure equal access to education and to promote edu.cationol excellence throughou.t the nation.
       Department of Education 





         Performance Summary Report 



                  Fiscal Year 2010 





                    In Support of the 



             National Drug Control Strategy 



As required by ONDCP Circular: Drug Control Accounting 





                    January 19, 2011 

                                                                            Department of Education 


                                Performance Summary Report for Fiscal Year 2010 


                                                                                TABLE OF CONTENTS 


Transmittal Letter                 . .. . . . . . . . .   .   ......... . . . . . . . . . . . . . . . . . . ..................                       .. ...
                                                                                                                                                         .      ...... . . . . ...... . . . . . . . . . . . . . . . . . . ............                   1



Performance Summary Information .................................................................................2 


        Safe Schools/Heallhy Siudenls                                                              . . . . . ...........................                            . . . . . . . .............             . . . . . . . . ...............              2



        Student Drug Testing                                         . . . . . . . . . . . ... . . . . . . . . . . . . . . . . . . . . . .   .   . . . . . . . . . . . . . . . . ......   .   ..... . . . . . . . . . . . . . . . ............           9


        State Grant Program                                        . . . . . . . . . . .. . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . .   18 


        Grants to Reduce Alcohol Abuse                                                                   . . .   . . . . . ................... . . . . ...........                               . . . . .. . . . . . . . ........ . . . . ...          21 



Assertions    . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .   . . ........          . . .. . ........... . . . . . . . . . . . . .           31 



Criteria for Assertions                            .............. . . . . . . ...... .........................................................................                                                                                          32 

                               UNITED STATES DEPARTMENT OF EDUCATION

                                             OFFICE OF SAFE AND DRUG·FREE SCHOOLS




Kathleen S. Tighe
Inspector General
U.S. Department of Education
400 Maryland Avenue, S.w.
Washington, DC 20202-1510

Dear Ms.Tighe:

As required by Office of National Drug Control Policy (ONDCP) Circular Drug Control
Accounting, enclosed please find detailed information about performance-related
measures for key drug control programs administered by the U.S. Department of
Education, in accordance with the guidelines in the circular dated May 1, 2007. This
information covers the Safe and Drug-Free Schools and Communities program, which is
the Drug Control Budget Decision Unit under which the 2010 budgetary resources for
the Department of Education are displayed in the Fiscal Year 2011 National Drug
Control Budget Summary.


Consistent with the instructions in the ONDCP Circular, please provide your
authentication to me in writing and I will transmit it to ONDCP along with the enclosed
Performance Summary Report. As you know, ONDCP requests these documents by
February 1, 2011. Please do not hesitate to contact me if you have any questions about
the enclosed information.

                                               Sincerely,



                                               Kevin Jennings




                              400 MARYLAND AVE" S.W., WASHINGTON, D.C. 20202
                                              www.ed.gov

 Our mission   i$ 10   ensure equal access 10 education and /0 promote educational excellence Ihroughout ,he nalion.
Performance Summary Information

                         Safe Schools/Healthy Students

Measure 1: The percentage of grantees demonstrating a decrease in substance
abuse over the three-year grant period. (Safe Schools/Healthy Students - FY
2005 and 2006 cohorts)

Table 1

Cohort    FY2006     FY 2007    FY2008 FY2009 FY2010            FY 2010    FY2011
           Actual     Actual    Actual Actual   Target           Actual     Target
2005      n/a        43.8       34.2   Pending n/a              n/a        n/a
2006      n/a        66.7       66.7   66.7    80.0             pending    n/a

The measure. This performance measure is for the Safe Schools/Healthy
Students initiative, a joint project of the Departments of Education, Health and
Human Services, and Justice. The initiative provides grants to local educational
agencies (LEAs) to support the development and implementation of a
comprehensive plan designed to prevent student drug use and violence and
support healthy youth development.

This measure, one of four for this initiative for the 2004, 2005, and 2006 cohorts,
focuses on one of the primary purposes of the initiative - reduced student drug
use. The initiative and this measure, are directly related to the National Drug
Control Strategy's goal of preventing drug use before it begins. Grantees select
and report on one or more measures of prevalence of drug use for students. For
the FY 2004 - 2006 cohorts, the items selected by grantees to respond to this
measure are not common across grant sites but, rather, reflect priority drug use
problems identified by sites.

FY 2010 Performance Results. Sites were not required to provide or collect
baseline data at the time of application or before program interventions were
implemented, so grantees provide baseline data for their selected measures
related to drug use after year one of program implementation (for example in FY
2006 for the FY 2005 cohort). Grantees from the FY 2005 cohort generally
completed no-cost extensions and provided GPRA data in their final grantee
reports; aggregation of this data is currently not completed in order to report FY
2009 actual performance.

The SS/HS National Evaluation contractor (supported by HHS/SAMHSA)
performs the initial review and cleaning of GPRA performance data received by
SS/HS grantees. The contract was not specific about reviewing and
cleaning data received in final reports. FY 2009 data for the 2005 SS/HS cohort
is included in their final performance report. HHS/SAMHSA recently made a

                                         2
decision to review and clean data during the project period, excluding the no-cost
extension year and final report. We are currently discussing options for
compiling these performance results and getting this task completed given
available resources.

The FY 2006 cohort of grantees received no-cost extensions during FY 2009.
Final GPRA data for this cohort was generally submitted at the end of December
2010 and final reports are currently being reviewed, with data available on
grantee performance in FY 2010 available in spring 201 1 .

FY 201 1 Performance Targets. Targets for the two earliest cohorts were initially
established before any performance data for this measure were received, and
represented our best judgment at the time, given the significant size of SS/HS
grants and the emphasis on research-based programs that is central to the
initiative. We elected in 2008 to revise the target for the FY 2005 cohort for this
measure based on the actual performance to date (implementation year two) of
the FY 2004 cohort. Based on our professional judgment, it seemed that the
revised target of 86.25 percent was appropriately aggressive and that attaining
that target would be a meaningful outcome for the program, while acknowledging
that our original target of 90 percent for the initial (FY 2004) cohort may have
been unrealistic. In 2008 we also developed revised targets for the FY 2006
cohort, again, based on the limited data available for this measure. We have
made no additional revisions to targets for these cohorts at this time, except for
substituting a numeric target (of 73.4 percent) in place of "maintain a baseline" as
the 2008 target for the 2006 cohort.

Our ability to establish appropriate targets for these cohorts of the program has
also been impacted by challenges associated with the quality of data supplied by
grant sites. Initially, a significant number of sites failed to provide valid data for
this and some other SS/HS measures. Through technical assistance activities
we have achieved some improvements in data quality for some sites, including
Significantly improved response rates for the 2005 and 2006 grant cohorts, but
have not completed a full grant cycle with cohorts that have received early and
more intensive technical assistance related to gathering performance data.

Subsequently, we have adopted revised GPRA measures for this initiative
beginning with the FY 2007 cohort in order to address implementation challenges
with the measure described above. Those revised GPRA measures for the
program that are relevant to the National Drug Control Strategy are included as
Measures 2 and 3 in this summary report.

Both the 2005 and 2006 grant cohorts are in their close-out phases and the
grants will not be operating in FY 201 1 . Thus, no targets were set for FY 201 1 .

Methodology. Data for these grant cohorts are collected by grantees, generally
using student surveys. Data are furnished in the second of two semi-annual


                                          3
performance reports provided by grantees each project year. If grantees
identified more than one measure of drug abuse or provided data for individual
sChool-building types (for example, separate data for middle and high schools),
grantees were considered to have experienced a decrease in substance abuse if
data for a majority of measures provided reflected a decrease. If a grant site
provided data for an even number of measures and half of those measures
reflected a decrease and half reflected no change or an increase, that grant site
was judged not to have demonstrated a decrease in substance abuse. While
most sites were able to provide some data related to this measure, we
considered as valid data only data from sites that used the same elementslitems
in each of two years. We considered a grant site to have experienced a
decrease if data supplied reflected a decrease over baseline data provided.

The contractor for the SS/HS national evaluation is also using data for this
measure as part of the program evaluation. The evaluation contractor reviews
data submitted, and works with grantees to seek clarifying information and
provide technical assistance if grantees are having difficulty in collecting or
reporting data for this measure. The contractor supplies data for the measure
after it has completed data cleaning processes. If data for this measure are not
available at the time that performance reports are submitted, staff follow-up with
sites to attempt to obtain data for the measure.

Grantees that fail to provide data are not included in the tabulation of data for the
measures. Also, grantees that did not provide data for two consecutive project
years (so that we could determine if a decrease in substance abuse had
occurred) are not included in data reported for the measure. Authorized
representatives for the grant site sign the annual performance report and, in
doing so, certify that to the best of the signer's knowledge and belief, all data in
the perfonmance report are true and correct and that the report fully discloses all
known weaknesses concerning the accuracy, reliability, and completeness of the
data included. Generally, the Department relies on the certification concerning
data supplied by grantees and does not conduct further reviews.

Measure 2: The percentage of SS/HS grantees that report a decrease in
students who report current (30-day) marijuana use (SS/HS FY 2007, -



2008, and 2009 cohorts)

Table 2

        Cohort    FY2008     FY2009     FY2010   FY2010 FY2011
                  Actual     Actual     Target   Actual  Target
        2007      53.8       42.9       56.5     PendinQ 59.9
        2008      nla        50.0       5 1 .0   Pending 52.5
        2009      nla        nla        Set a    Pending Baseline
                                        baseline         +2%



                                          4

The measure. This performance measure is for the Safe Schools/Healthy
Students initiative, a joint project of the Departments of Education, Health and
Human Services, and Justice. The initiative provides grants to local educational
agencies (LEAs) to support the development and implementation of a
comprehensive plan designed to prevent student drug use and violence and
support healthy youth development. Beginning with the FY 2007 cohort, the
project period for SS/HS grants is 48 months.

This measure, one of six for this initiative for cohorts from FY 2007 and forward,
focuses on one of the primary purposes of the initiative - reduced student drug
use. The initiative and this measure are directly related to the National Drug
Control Strategy's goal of preventing drug use before it begins.

FY 2010 Performance Results.
FY 2010 targets for this measure were not established in last year's report as we
were awaiting aggregation and analysis of FY 2008 and FY 2009 data from the
FY 2007 cohort to help inform the setting of targets.

Beginning with the FY 2007 cohort, grantees are required to provide baseline
data prior to implementing interventions. Generally, after the first project year
grantees reported baseline data and year 1 actual performance data. Across all
cohorts (2007, 2008, and 2009) some sites experienced significant delays in
beginning implementation of interventions. Reasons for delays include the need
to finalize partnership agreements, complete a project logic model, develop an
evaluation plan, and for some, to collect baseline data. Delays in implementing
interventions significantly impacted the number of grantees with comparable data
to contribute to performance results.

The FY 2007 cohort provided baseline and year 1 actual [performance] data in
FY 2008. This data was not available in the last report but has since been
received along with actual data for FY 2009. Actual data for FY 201 0 are
currently being reviewed and aggregated, thus are not included in this report.

The FY 2008 cohort provided baseline and year 1 actual [performance] in FY
2009 and is included in this report. Actual data for FY 2010 are currently being
reviewed and aggregated, thus are not included in this report.

The FY 2009 cohort recently submitted baseline and year 1 actual [performance]
data. These data are currently being reviewed and aggregated, thus are not
included in this report.

FY 201 1 Performance Targets.
The setting of FY 201 1 performance targets was based on an analysis of prior
year performance of multiple cohorts. The FY 2007 cohort data (from FY 2008
and 2009) shows that the cohort's initial project year (FY 2008) performance
results were better than second project year (FY 2009) performance results.


                                         5
Staff analysis of grantee data resulted in the identification of numerous factors
thought to contribute to the decline in performance results in the second year of
the project that informed the setting of subsequent targets.

Based on this analysis, and considering the changes made to the GPRA
measures, targets were set using an incremental increase annually of baseline
plus 2, 3, 6, and 9 percent for the 2007, 2008, and 2009 cohorts. Additionally, a
cap of 85 percent was set as the maximum target for the measure given ceiling
effects.

For the 2009 cohort we are awaiting the final aggregation of first year
performance data in order to set a baseline against which to apply the 201 1
target of plus 2 percent on which the target for FY 201 1 will be based.

Methodology. Data are collected by grantees, generally using student surveys.
Data are furnished in the second of two semi-annual performance reports
provided by grantees each project year.

The contractor for the SS/HS national evaluation is also using data for this
measure and from these cohorts as part of the national program evaluation. The
evaluation contractor reviews data submitted by grantees, and works with
grantees to seek clarifying information and provide technical assistance if
grantees are having difficulty in collecting or reporting data for this measure. The
contractor supplies data for the measure after it has completed data cleaning
processes. If data for this measure are not available at the time that
performance reports are submitted, staff follow up with sites to attempt to obtain
data for the measure.

Grantees that fail to provide data or that provide data that does not respond to
the established measure are not included in the tabulation of data for the
measures. Authorized representatives for the grant site sign the annual
performance report and, in doing so, certify that to the best of the signer's
knowledge and belief, all data in the performance report are true and correct and
that the report fully discloses all known weaknesses concerning the accuracy,
reliability, and completeness of the data included. Generally, the Department
relies on the certification concerning data supplied by grantees and does not
conduct further reviews.

Measure 3: The percentage of SS/HS grantees that report a decrease i n
students who report current (30-day) alcohol use (SS/HS FY 2007, 2008,
                                                               -



and 2009 cohorts)




                                         6
Table 3 


        Cohort    FY2008    FY2009     FY2010         FY2010    FY2011
                  Actual    Actual     Tarllet        Actual    Target
       2007       71.4      47.8       75.0       I   pending   79.5
       2008       n/a       56.0       57. 1          pending   58.8
       2009       n/a       n/a        Set a          pending   Baseline
                                       baseline                 +2%


The measure. This performance measure is for the Safe Schools/Heallhy
Students initiative, a jOint project of the Departments of Education, Health and
Human Services, and Justice. The initiative provides grants to local educational
agencies (LEAs) to support the development and implementation of a
comprehensive plan designed to prevent student drug use and violence and
support healthy youth development. Beginning with the FY 2007 cohort, the
project period for SS/HS grants is 48 months.

This measure, one of six for this initiative for cohorts from FY 2007 and forward,
focuses on prevalence of alcohol use. While the National Drug Control Strategy
is focused most intensively on preventing the use of controlled substances, the
strategy does address the role of alcohol as a substance of choice for teenagers.
Data do suggest that early use of alcohol is more likely to result in heavy later
use of alcohol. The initiative and this measure, are directly related to the
National Drug Control Strategy's goal of preventing drug use before it begins.

FY 2010 Performance Results.
FY 2010 targets for this measure were not established in last year's report as we
were awaiting aggregation and analysis of FY 2008 and FY 2009 data from the
FY 2007 cohort to help inform the setting of targets.

Beginning with the FY 2007 cohort, grantees are required to provide baseline
data prior to implementing interventions. Generally, after the first project year
grantees reported baseline data and year 1 actual performance data. Across all
cohorts (2007, 2008, and 2009) some sites experienced significant delays in
beginning implementation of interventions. Reasons for delays include the need
to finalize partnership agreements, complete a project logic model, develop an
evaluation plan, and for some, to collect baseline data. Delays in implementing
interventions significantly impacted the number of grantees with comparable data
to contribute to performance results.

The FY 2007 cohort provided baseline and year 1 actual [performance] data in
FY 2008. This data was not available in the last report but has since been
received along with actual data for FY 2009. Actual dala for FY 2010 are


                                        7
currently being reviewed and aggregated and, thus, are not included in this
report.

The FY 2008 cohort provided baseline and year 1 actual [performance] in FY
2009 and is included in this report. Actual data for FY 2010 are currently being
reviewed and aggregated, thus are not included in this report.

The FY 2009 cohort recently submitted baseline and year 1 actual [performance]
data. These data are currently being reviewed and aggregated, thus are not
included in this report.

FY 201 1 Performance Targets.
The setting of FY 201 1 performance targets was based on an analysis of prior
year performance of multiple cohorts. The FY 2007 cohort data (from FY 2008
and 2009) shows that the cohort's initial project year (FY 2008) performance
results were better than second project year (FY 2009) performance results . .
Staff analysis of grantee data resulted i n the identification of numerous factors
thought to contribute to the decline in performance results in the second year of
the project that informed the setting of subsequent targets.

Based on this analysis, and considering the changes made to the GPRA
measures, targets were set using an incremental increase annually of baseline
plus 2, 3 , 6, and 9 percent for the 2007, 2008, and 2009 cohorts. Additionally, a
cap of 85 percent was set as the maximum target for the measure given ceiling
effects.

For the 2009 cohort we are awaiting the final aggregation of first year
performance data in order to set a baseline against which to apply the 201 1
target of plus 2 percent on which the target for FY 201 1 will be based.


Methodology. Data are collected by grantees, generally using student surveys.
Data are furnished in the second of two semi-annual performance reports
provided by grantees each project year.

The contractor for the SS/HS national evaluation is also using data for this
measure and for these cohorts as part of the national program evaluation. The
evaluation contractor reviews data submitted by grantees, and works with
grantees to seek clarifying information and provide technical assistance if
grantees are having difficulty in collecting or reporting data for this measure. The
contractor supplies data for the measure after it has completed data cleaning
processes. If data for this measure are not available at the time that
performance reports are submitted, staff follow up with sites to attempt to obtain
data for the measure.




                                         8
Grantees that fail to provide data or that provide data that does not respond to
the established measure are not included in the tabulation of data for the
measures. Authorized representatives for the grant site sign the annual
performance report and, in doing so, certify that to the best of the signer's
knowledge and belief, all data in the performance report are true and correct and
that the report fully discloses all known weaknesses concerning the accuracy,
reliability, and completeness of the data included. Generally, the Department
relies on the certification concerning data supplied by grantees and does not
conduct further reviews.


                               Student Drug Testing

Measure 4: The percentage of student drug testing grantees that experience a 5
percent reduction in current (30-day) illegal drug use by students in the target
population. (Student Drug Testing FY 2005, 2006, 2007, and 2008 cohorts)
                                    -




Table 4

Cohort FY2006       FY2007    FY200S         FY2009 FY2010       FY2010    FY2011
       Actual       Actual    Actual          Actual   Target    Actual    Target
2005   nla         no valid no valid         no valid  nla       nla       nla
                   data      data            data
                   available available       available
2006      nla      nla       66.7            12.5      70.0      57.0      nla
2007      nla      nla       33.0            4 1 .7    60.0      50.0      nla
2008      nla      nla       nla             49.0      50.0      65.0      70.0

The measure. This measure is one of two measures for the Student Drug­
Testing Program grant competition. The competition provided discretionary
grants to LEAs, community-based organizations, or other public and private
entities to support implementation of drug testing of students, consistent with the
parameters established by the U.S. Supreme Court or for students and their
families that voluntarily agree to partiCipate in the student drug testing program.

This measure is directly related to the National Drug Control Strategy's goal
related to preventing drug use before it starts. Student drug testing was
prominently featured between 2003 to 2009 in different versions of the strategy
as a recommended drug prevention intervention. FY 2008 was the last cohort of
new grants made under the program.

FY 201 0 Performance Results. FY 20 1 0 performance data for the FY 2006,
2007, and 2008 cohorts are included in table 4 above.

During FY 2008 we completed a review of data submitted to date by the FY 2005
cohort for this measure and identified significant concerns about the quality and

                                         9
comparability of the data. Grant sites had reported on prevalence rates for a
variety of illegal drugs and did not always provide data from the same
items/elements across project years one and two. Also, some sites surveyed
their entire student population and others surveyed only students in the testing
pool.

Based on these concerns, we obtained assistance from the U.S. Department of
Education's Data Quality Initiative contractor. With the contractor's help, we
created and disseminated detailed data collection and reporting guidance for the
program, as well as data standards that we will use to determine what constitutes
valid data for this measure. We disseminated this guidance to FY 2007 grantees
during project implementation, but were able to provide the guidance to the new
cohort of 2008 grantees at the start of their projects. Based on that guidance, as
well as data quality and aggregation checks, in FY 201 0 we aggregated available
data from the FY 2007 and 2008 cohorts and recorded those data in the
Department's software that houses GPRA measures and data.

Data for the FY 2006 cohort came from the evaluation conducted for the
Department of Education by a contractor. Data for this cohort were collected by
the contractor annually; the data reported in the chart above refiect the progress
of cohort grant sites based on aggregate information at the grantee level about
changes in prevalence of drug use among each grantee's students surveyed by
the contractor.

Because of the concerns about data quality discussed previously including
                                                                   I



receiving valid data from only a small percentage of grantees in the cohort, no
aggregate data for the FY 2005 can be provided.

It is difficult to assess why performance results for the 2006 and 2007 cohorts fell
short of the established targets for this measure in FY 2010, whereas the 2008
cohort exceeded the target.

We have carefully considered program performance reports submitted by
grantees, as well as our experience in monitoring and providing technical
assistance to grantees, and have identified some challenges that may have
impeded grant performance. Some common problems include turnover in
leadership (at the school board, authorized representative or project director
level) and challenges with collecting and reporting valid data about the measure.
Another variable that might affect performance in sites is related to project
design. For example, we'fe not certain how to assess the likely impact of a
random student drug testing intervention on students who volunteer to be
included in the testing pool, versus students who are forced to be tested as a
condition of participation in extracurricular activities.

Finally, cohort size and composition varies from cohort to cohort. In some years
funding for a large amount of new awards was available and in others only a


                                         10
handful of sites received grants. To the extent that our peer review results that
are used to rank order applications and select grantees accurately predict project
quality, the range of scores funded (and perhaps the range of project quality)
varies from fiscal year to fiscal year.

We believe that an equally important dimension in assessing performance
against established targets for this measure is our ability to correctly predict
targets for measures. Because there is limited available research and no
"industry" standard available to guide expectations for performance, generally we
have used the performance of prior cohorts on the measure to establish targets
for subsequent cohorts. Because of the need to establish targets for future
cohorts before a single cohort has completed project implementation and
submitted final data, we have based targets on information provided over a fairly
limited amount of time, often reflecting the results of less than a single cohort of
complete performance information.

This situation introduced two possible problems into the target setting process ­
that targets are being based in some part on a subset of grantees that are not
representative of the cohort as a whole, and that those sites able to provide valid
data more quickly may also have superior capacity to effectively implement
program interventions. While we have placed a significant emphasis on
improving data quality and have worked to improve the response rates for
measures over the life of the cohort, this causes performance results from a
relatively small set of grantees in year one to be compared to results for a
Significantly larger percentage of the cohort by the end of project.

Increasingly, it is clear that a series of variables serve to make each cohort
unique, and that the issue of how we have established targets for this measure
has been problematic. Revised FY 2010 processes for establishing targets for
this measure are discussed in the FY 201 1 Performance Targets section below.

FY 201 1 Performance Targets. We first established targets for the percentage of
grantees experiencing a 5 percent reduction in current illegal drug use after
reviewing the first two years of data for the FY 2003 cohort of grant sites.
Consistent with research that suggests that changes in student behavior related
to student drug testing may not be reatized immediately, we assumed that we
could look for an increased number of grantees to experience positive change
and, using our professional judgment, set that target at 50 percent of grantees.

We have since received data for three project years from this single cohort of
sites (the FY 2003 cohort), and the information provided by the grantees did not
provide an adequate basis for revisiting targets for future cohorts. This cohort
was very small (eight grantees), and also experienced extensive delays in
implementation and data collection activities. Because only a handful of
grantees were able to eventually provide data specific to the measure, we do not



                                         II
believe that it would be appropriate to base expectations about the performance
of other cohorts on this limited information.

Similar problems with data quality for the FY 2005 cohort of grant sites meant
that data from that cohort was not helpful in determining whether targets for the
program needed to be readjusted. Challenges with data quality resulted in only a
very limited proportion of grant sites that provided approximately comparable
data. Conversely, for the FY 2006 cohort, because the data from the evaluation
were collected by a contractor using comparable survey items and collection
procedures (in contrast to the varying procedures used by individual grant sites in
the other cohorts), data for the 2006 cohort similarly do not provide an
appropriate basis for making adjustments in existing targets under the program.

Given these challenges, and improvements we have made in data quality
(including generally requiring grantees to collect baseline data for their projects
before interventions are implemented), we modified our process for establishing
targets. While prior cohort performance may provide some insights about
general patterns of performance that we can incorporate into our targeting setting
processes, for any future cohorts for this program we decided to establish
numerical performance targets after baseline data is received for the new cohort.
We entered these targets into the Department's Visual Performance System
(VPS) as "administrative" targets (for example, baseline plus 5 percent), and then
convert the targets to numerical targets after baseline data is collected and
aggregated. We believe that this process revision helped us better match targets
to cohort performance, and reflect the unique characteristics of each cohort.

The 2008 cohort is the only cohort for which a target is established for FY 201 1 ,
and it is based on the revised target setting approach described above. Only a
portion of the FY 2006 cohort, and small number of grants, will be operational in
FY 201 1 in a no-cost extension phase of their grants. Each have different project
end dates making comparisons with prior year actual performance difficult. The
FY 2007 cohort generally ended in FY 2010, with limited no-cost extension
activity and therefore no FY 201 1 targets are set for this cohort.

Methodology With the exception of the FY 2006 cohort, data are collected by
grantees using student surveys. Data are provided as part of the grantees'
annual performance reports. Generally, grantees prior to FY 2008 cohort did not
use the same survey items to collect data for this measure but, rather, self-select
survey items (often from surveys already administered) in order to provide these
data. Survey items may relate to different substances, but must collect
information concerning current use in order to be included in the data reported for
this measure. Grantees did not provide baseline data in their applications, so we
had to wait until grantees provided data both from project year one and two in
order to determine if they have experienced a decrease in substance abuse.




                                        12
Beginning with the FY 2008 cohort, we asked grantees to provide data for current
(prior 30-day) use of marijuana, as a proxy for illegal drug use. Beginning with
the FY 2008 cohort, we also instructed grantees to collect baseline data for this
measure before beginning with implementation of their student drug testing
program.

The FY 2005 cohort of grant sites provided data for FY 2006 and 2007, but only
a small percentage of grant sites provided valid data for the measure. Many
sites requested and received no-cost extensions for their projects, and data for
this measure is included in final reports that were due at the end of December
2009. Significant data quality issues (including inconsistent use of survey items
and changes in respondent populations) affected the majority of grant sites in the
cohort, resulting in no valid data for this cohort.

Authorized representatives for the grant site sign the annual performance report
and, in doing so, certify that to the best of the signer's knowledge and belief, all
data in the performance report are true and correct and that the report fully
discloses all known weaknesses concerning the accuracy, reliability, and
completeness of the data included. Generally, the Department relies on the
certification concerning data supplied by grantees and does not conduct further
reviews.

Data for the FY 2006 cohort were collected as part of an evaluation of student
drug testing. Data through FY 201 0 for the measures were collected by the
evaluation contractor, using common survey items and collection procedures.
Survey responses are analyzed by the contractor and data are provided to the
Department.

The anticipated levels of decrease in substance abuse are consistent with the
national goals for the reduction of underage drug use included in the National
Drug Control Strategy in effect when SOT grants were awarded- five percent per
year. Targets were initially established following the report of baseline data for
grant sites from the FY 2003 cohort but, as discussed above, we adjusted our
method of setting targets.

Measure 5: The percentage of student drug testing grantees that experience a 5
percent reduction in past-year illegal drug use by students in the target
population. (Student Drug Testing FY 2005, 2006, 2007 and 2008 cohorts)
                                    -




Table 5

Cohort FY2006       FY2007       FY2008          FY2009      FY2010   FY2010   FY2011
       Actual        Actual      Actual           Actual     Target   Actual   Taraet
2005   nla          no valid    no valid         no valid    nla      nla      nla
                    data        data             data
                    available   available        available

                                            13
2006     nla        nla        55.5           12.5     60.0      57.0       nla
2007     nla        nla        33.0           33.3     60.0      54.0       nla
2008     nla        nla        nla            58.0     60.0      58.0       65.0

The measure. This measure is one of two measures for the Student Drug­
Testing Programs grant competition. The competition provides discretionary
grants to LEAs, community-based organizations, or other public and private
entities to support implementation of drug testing of students, consistent with the
parameters established by the U.S. Supreme Court or for students and their
families that voluntarily agree to participate in the student drug testing program.

This measure is directly related to the National Drug Control Strategy's goal
related to preventing drug use before it starts. Student drug testing was
prominently featured in annual editions of the National Drug Control Strategy
between 2003 and 2009 as a recommended drug prevention intervention. FY
2008 was the last cohort of new grants made under the program.

FY 201 0 Performance Results.

FY 201 0 performance data for the FY 2006, 2007, and 2008 cohorts are included
in the chart for this measure above.

During FY 2008 we completed a review of data submitted to date by the FY 2005
cohort for this measure and identified significant concerns about the quality and
comparability of the data. Grant sites had reported on prevalence rates for a
variety of illegal drugs and did not always provide data from the same
items/elements across project years one and two. Also, some sites surveyed
their entire student population and others surveyed only students in the testing
pool.

Based on these concerns, we obtained assistance from the U.S. Department of
Education's Data Quality Initiative contractor. With the contractor's help, we
created and disseminated detailed data collection and reporting guidance for the
program, as well as data standards that we will use to determine what constitutes
valid data for this measure. We disseminated this guidance to FY 2007 grantees
during project implementation, but were able to provide the guidance to the new
cohort of 2008 grantees at the start of their projects. Based on that guidance, as
well as data quality and aggregation checks, in FY 201 0 we aggregated available
data from the FY 2007 and 2008 cohorts and recorded those data in the
Department's software that houses GPRA measures and data.

Data for the FY 2006 cohort came from the evaluation conducted for the
Department of Education by a contractor. Data for this cohort were collected by
the contractor annually; the data reported in the chart above refiect the progress
of cohort grant sites based on aggregate information at the grantee level about



                                         14
changes in prevalence of drug use among each grantee's students surveyed by
the contractor.

Because of the concems about data quality discussed previously, including
receiving valid data from only a small percentage of grantees in the cohort, no
aggregate data for the FY 2005 can be provided.

It is difficult to assess why performance results for the 2006, 2007, 2008 cohorts
fell slightly short of the established targets for this measure in FY 2010.

We have carefully considered program performance reports submitted by
grantees, as well as our experience in monitoring and providing technical
assistance to grantees, and have identified some challenges that may have
impeded grant performance. Some common problems include turnover in
leadership (at the school board, authorized representative or project director
level) and challenges with collecting and reporting valid data about the measure.
Another variable that might affect performance in sites is related to project
design. For example, we're not certain how to assess the likely impact of a
random student drug testing intervention on students who volunteer to be
included in the testing pool, versus students who are forced to be tested as a
condition of participation in extracurricular activities.

Finally, cohort size and composition varies from cohort to cohort. In some years
funding for a large amount of new awards was available and in others only a
handful of sites received grants. To the extent that our peer review results that
are used to rank order applications and select grantees accurately predict project
quality, the range of scores funded (and perhaps the range of project quality)
varies from fiscal year to fiscal year.

We believe that an equally important dimension in assessing performance
against established targets for this measure is our ability to correctly predict
targets for measures. Because there is limited available research and no
"industry" standard available to guide expectations for performance, generally we
have used the performance of prior cohorts on the measure to establish targets
for subsequent cohorts. Because of the need to establish targets for future
cohorts before a single cohort has completed project implementation and
submitted final data, we have based targets on information provided over a fairly
limited amount of time, often reflecting the results of less than a single cohort of
complete performance information.

This situation introduced two possible problems into the target setting process­
that targets are being based in some part on a subset of grantees that are not
representative of the cohort as a whole, and that those sites able to provide valid
data more quickly may also have superior capacity to effectively implement
program interventions. While we have placed a significant emphasis on
improving data quality and have worked to improve the response rates for


                                         15
measures over the life of the cohort, this causes performance results from a
relatively small set of grantees in year one to be compared to results for a
significantly larger percentage of the cohort by the end of project.

Increasingly, it is clear that a series of variables serve to make each cohort
unique, and that the issue of how we have established targets for this measure
has been problematic. Revised FY 20 1 0 processes for establishing targets for
this measure are discussed in the FY 201 1 Performance Targets section below.

FY 201 1 Performance Targets. We first established targets for the percentage of
grantees experiencing a 5 percent reduction in past-year illegal drug use after
reviewing the first two years of data for the FY 2003 cohort of grant sites.
Consistent with research that suggests that changes in student behavior related
to student drug testing may not be realized immediately, we assumed that we
could look for an increased number of grantees to experience positive change
and, using our professional judgment, set that target at 50 percent of grantees.

We have since received data for three project years from this single cohort of
sites (the FY 2003 cohort), and the information provided by the grantees did not
provide an adequate basis for revisiting targets for future cohorts. This cohort
was very small (eight grantees), and also experienced extensive delays in
implementation and data collection activities. Because only a handful of
grantees were able to eventually provide data specific to the measure, we do not
believe that it would be appropriate to base expectations about the performance
of other cohorts on this limited information.

Similar problems with data quality for the FY 2005 cohort of grant sites meant
that data from that cohort was not helpful in determining whether targets for the
program needed to be readjusted. Challenges with data quality resulted in only a
very limited proportion of grant sites that provided approximately comparable
data. Conversely, for the FY 2006 cohort, because the data from the evaluation
were collected by a contractor using comparable survey items and collection
procedures (in contrast to the varying procedures used by individual grant sites in
the other cohorts), data for the 2006 cohort similarly do not provide an
appropriate basis for making adjustments in existing targets under the program.

Given these challenges, and improvements we have made in data quality
(including generally requiring grantees to collect baseline data for their projects
before interventions are implemented), we modified our process for establishing
targets. While prior cohort performance may provide some insights about
general patterns of performance that we can incorporate into our targeting setting
processes, for any future cohorts for this program we decided to establish
numerical performance targets after baseline data is received for the new cohort.
We entered these targets into the Department's Visual Performance System
(VPS) as "administrative" targets (for example, baseline plus 5 percent), and then
convert the targets to numerical targets after baseline data is collected and


                                        16
aggregated. We believe that this process revision helped us better match targets
to cohort performance, and reflect the unique characteristics of each cohort.

The 2008 cohort is the only cohort for which a target is established for FY 201 1 ,
and it is based on the revised target setting approach described above. Only a
portion of the FY 2006 cohort, and small number of grants, will be operational in
FY 201 1 in a no-cost extension phase of their grants. Each have different project
end dates making comparisons with prior years actual perfonmance difficult. The
FY 2007 cohort generally ended in FY 201 0, with limited no-cost extension
activity and therefore no FY 201 1 targets are set for this cohort.

Methodology With the exception of the FY 2006 cohort, data are collected by
grantees using student surveys. Data are provided as part of the grantees'
annual performance reports. Generally, grantees prior to the FY 2008 cohort did
not use the same survey items to collect data for this measure but, rather, self­
select survey items (often from surveys already administered) in order to provide
these data. Survey items may relate to different substances, but must collect
information concerning current use in order to be included in the data reported for
this measure. Grantees did not provide baseline data in their applications, so we
had to wait until grantees provided data both from project year one and two in
order to determine if they have experienced a decrease in substance abuse.

Beginning with the FY 2008 cohort, we asked grantees to provide data for past­
year use of marijuana, as a proxy for illegal drug use. Beginning with the FY
2008 cohort, we also instructed grantees to collect baseline data for this measure
before beginning with implementation of their student drug testing program.

The FY 2005 cohort of grant sites provided data for FY 2006 and 2007, but only
a small percentage of grant sites provided valid data for the measure. Many
sites requested and received no-cost extensions for their projects, and data for
this measure is included in final reports that were due at the end of December
2009. Significant data quality issues (including inconsistent use of survey items
and changes in respondent populations) affected the majority of grant sites in the
cohort, resulting in no valid data for this cohort.

Authorized representatives for the grant site sign the annual performance report
and, in doing so, certify that to the best of the signer's knowledge and belief, all
data in the perfonmance report are true and correct and that the report fully
discloses all known weaknesses concerning the accuracy, reliability, and
completeness of the data included. Generally, the Department relies on the
certification concerning data supplied by grantees and does not conduct further
reviews.

Data for the FY 2006 cohort were being collected as part of an evaluation of
student drug testing. Data through FY 201 0 for the measures were collected by
the evaluation contractor, using common survey items and collection procedures.


                                         17
Survey responses are analyzed by the contractor and data are provided to the
Department.

The anticipated levels of decrease in substance abuse are consistent with the
national goals for the reduction of underage drug use included in the National
Drug Control Strategy in effect when SDT grants were awarded- five percent per
year. Targets were initially established following the report of baseline data for
grant sites from the FY 2003 cohort but, as discussed above, we adjusted our
method of setting targets.

              Safe and Drug-Free Schools and Communities State Grants'

Measure 6: The percentage of students in grades 9-12 who were offered, sold,
or given an illegal drug on school property during the past 1 2 months. (Safe and
Drug-Free Schools and Communities State Grants)

Table 6

 FY 2005        FY 2006       FY 2007       FY 2008        FY 2009        FY 2009       FY 2010
  Actual         Actual        Actual        Actual         Target         Actual        Target
25.4            None         22.3           None          26             22.7           None

The measure. This measure is one of three measures directly related to
reducing student drug or alcohol use for the Safe and Drug-Free Schools and
Communities (SDFSC) State Grants. This formula grant program provided funds
to the States, based on school-aged population and the State's relative share of
Elementary and Secondary Education Act Title I concentration grant funds, to
support drug and violence prevention programs. The measure directly relates to
the National Drug Control Strategy Goal of preventing youth drug use by focusing
on the extent to which illegal drugs are available on school property.

FY 2009 and 2010 Performance Results. Data for this measure were collected in
2009, but were not released by the Centers for Disease Control and Prevention
until summer 201 O. No target was established for this measure for FY 2010
because data is collected only every other year, in odd-numbered years and the
SDFSC State Grants program was terminated in FY 2010. In the past year
performance report we promised to provide FY 2009 data in the FY 201 0
performance summary report, but indicated we would end reporting for the
measure at that time. The percentage of students in grades 9-12 who were


I   The FY 2009 Performance Report also included two additional measures for the SDFSC State
Grant program, numbered measures 9 and 1 0 in that report, relating to the percentage of drug
and violence prevention programs/practices supported with SDFSC State Grant funds that are
research based, and the percentage of drug and violence prevention curriculum programs that
are implemented with fidelity. As indicated in that report, FY 2010 targets were not set for these
measures, and no additional performance data would be available for these measures due to the
enacted FY 2010 appropriations statute that did not provide funding for the State Grant program.

                                                 18
offered, sold, or given an illegal drug on school property during the past 1 2
months increased from 1 993 to 1995, and decreased from 1 995 to 2009.

FY 201 1 Performance Targets. The SDFSC State Grants program was
terminated in FY 201 0 and no targets are sel for FY 201 1 .

 Methodology. Data for this measure are collected from a nationally
representative sample of students in grades 9-12 as part of the Youth Risk
Behavior Surveillance System (YRBSS), sponsored by the Centers for Disease
Control and Prevention (CDC). Data are collected in odd years and reported in
the following even years. No data are collected for even years and, as a result,
no targets have been established for even years.

Detailed information about the methodology used to sample and report data for
the YRBSS is available at the CDC website. We rely on the assertions provided
about methodology presented by CDC in using this data to report on
performance of SDFSC State Grants.

Measure 7: The percentage of students in grades 9-12 who used marijuana one
or more times during the past 30 days. (SDFSC State Grants)

Table 7

FY 2005     FY 2006      FY 2007     FY 2008      FY 2009      FY 2009    FY 2010
 Actual      Actual      Actual       Actual       Target       Actual     Target
20.2        None        19.7         None        18.0         20.8        None

The measure. This measure is one of three measures directly related to
reducing student drug and alcohol use for SDFSC State Grants. This formula
grant program provided funds to the States, based on school-aged population
and the State's relative share of Elementary and Secondary Education Act Title I
concentration grant funds, to support drug and violence prevention programs.
The measure is directly related to the National Drug Control Strategy Goal of
preventing youth drug use by focusing on the extent of current use by high
school aged-youth of the most prevalent illegal drug.

FY 2009 and 201 0 Performance Results. Data for this measure were collected in
2009, but was not released by the Centers for Disease Control and Prevention
until summer 2010. No target was established for this measure for FY 201 0
because data are collected only every other year, in odd-numbered years and
the SDFSC State Grants program was terminated in FY 2010. In the past year
performance report we promised to provide FY 2009 data in the FY 2010
performance summary report, but indicated we would end reporting for the
measure at that time. The percentage of students in grades 9-12 who used
marijuana one or more times during the past 30 days increased from 1991



                                         19
(14.7 percent) to 1 999 (26.7 percent) and has decreased since then to 20.8
percent in 2009.

FY 201 1 Performance Targets. The SDFSC State Grants program was
terminated in FY 2010 and no targets are set for FY 201 1 .

Methodology. Data for this measure are collected from a nationally
representative sample of students in grades 9-12 as part of the Youth Risk
Behavior Surveillance System (YRBSS), sponsored by the Centers for Disease
Control and Prevention (CDC). Data are collected in odd years and reported in
the following even years. No data are collected for even years and, as a result,
no targets have been established for even years.

Detailed information about the methodology used to sample and report data for
the YRBSS is available at the CDC website. We rely on the assertions provided
about the methodology presented by CDC in using this data to report on
performance of SDFSC State Grants.

Measure 8: The percentage of students in grades 9-12 who had five or more
drinks of alcohol in a row one or more times during the past 30 days. (SDFSC
State Grants)

Table 8

FY 2005     FY 2006      FY 2007    FY 2008      FY 2009     FY 2009     FY 2010
 Actual      Actual       Actual     Actual       Taraet      Actual      Taraet
25.5        None        26.0        None        25.0        24.2         None

The measure. This measure is one of three measures related to reducing
student drug or alcohol use for SDFSC State Grants. This formula grant program
provided funds to the States, based on school-aged population and the State's
relative share of Elementary and Secondary Education Act Title I concentration
grant funds, to support drug and violence prevention programs. The measure is
directly related to the National Drug Control Strategy Goal of preventing youth
drug use by focusing on the prevalence of binge drinking by high school aged­
students. While alcohol is not explicitly an emphasis of the National Drug Control
Strategy, illegal use of alcohol can be associated with use of other illegal drugs.

FY 2009 and 2010 Performance Results. Data for this measure were collected in
2009, but were not released by the Centers for Disease Control and Prevention
until summer 2010. No target was established for this measure for FY 201 0
because data is collected only every other year, in odd-numbered years, and the
SDFSC State Grants program was terminated in FY 2010. In the past year
performance report we promised to provide FY 2009 data in the FY 201 0
performance summary report, but indicated we would end reporting for the
measure at that time. The percentage of students in grades 9-12 who had five or

                                        20
more drinks of alcohol in row in the past 30 days did not change significantly from
1991 (31.3 percent) to 1 997 (33.4 percent) but has decreased over the last
decade.

FY 201 1 Performance Targets. The SDFSC State Grants program was
terminated in FY 201 0 and no targets are set lor FY 201 1 .

MethodologV. Data for this measure are collected from a nationally
representative sample of students in grades 9-12 as part of the Youth Risk
Behavior Surveillance System (YRBSS), sponsored by the Centers for Disease
Control and Prevention (CDC). Data are collected in odd years and reported in
the following even years. No data are collected for even years and, as a result,
no targets have been established for even years.

Detailed information about the methodology used to sample and report data for
the YRBSS is available at the CDC website. We rely on the assertions provided
about the methodology presented by CDC in using this data to report on
performance of SDFSC State Grants.

                        Grants to Reduce Alcohol Abuse

Measure 11: The percentage of grantees whose target students show a
measurable decrease in binge drinking. (Grants to Reduce Alcohol Abuse
Program - FY 2005, 2007, 2008, 2009 cohorts - no new grants were awarded
under this program in FY 2006.)

Table 11

Cohort     FY        FY        FY         FY         FY           FY      FY
           2006      2007      2008       2009       2010         2010    2011
           Actual    Actual    Actual     Actual     Target       Actual  Target
2005       nla       65.0      59.3       nla        nla          nla     nla
2007       nla       nla       61.5       47.0       49.4     I   pending nla
2008       nla       nla       nla        50.7       53.2         64.0    70.0
2009       nla       nla       nla        nla        nla          57.1    65.0

The measure. This measure examines a key outcome for the Grants to Reduce
Alcohol Abuse (GRAA) program - reduction in binge drinking for the target
population. While the National Drug Control Strategy is focused most intensively
on preventing the use of controlled substances, the strategy does address the
role of alcohol as a drug of choice for teenagers. Research suggests that early
use of alcohol is more likely to result in heavy later use of alcohol.

FY 201 0 Performance Results
We are able to report on performance in FY 2010 for the 2008 and 2009 cohorts
of grants. The 2008 cohort exceeded its target. No FY 201 0 target was set for

                                        21
the FY 2009 cohort but we are reporting actual FY 2010 performance. There is
incomplete data for the 2007 cohort and we are unable to report at this time, but
will do so in 201 1 .

As we have received data from across cohorts for this measure, we continue to
find it difficult to discern a pattern of performance that can serve as a basis for
establishing future targets. For example, it is difficult to assess why performance
results for the 2007 and 2008 cohorts fell short of the established targets for this
measure in the past. Intensive technical assistance (via contract) services were
not available to the FY 2008 cohort for much of FY 2009 but was in FY 2010.
This might be one reason the FY 2008 cohort exceeded its target in FY 2010.

We have carefully considered performance reports submitted by grantees, as
well as our experience in monitoring and providing technical assistance to
grantees, and have identified some challenges that may have impeded grant
performance. Some common problems include turnover in leadership (at the
authorized representative or project director level) and challenges with collecting
and reporting valid data about the measure. Another variable that might affect
performance in sites is related to project design. For example, we're not certain
how to assess the likely impact of a site that is implementing a single research­
based program versus sites that have adopted a more comprehensive strategy
that includes a community-based intervention that complements school-based
curricula. Finally, cohort size and composition varies from cohort to cohort. In
some years funding for a large amount of new awards is available and in others
only a handful of sites will receive grants. To the extent that our peer review
results that are used to rank order applications and select grantees accurately
predict project quality, the range of scores funded (and perhaps the range of
project quality) varies from fiscal year to fiscal year.

We believe that an equally important dimension in assessing performance
against established targets for this measure is our ability to correctly predict
targets for measures. Because there is limited available research and no
"industry" standard available to guide expectations for performance, generally we
have used the performance of prior cohorts on the measure to establish targets
for subsequent cohorts. Because of the need to establish targets for future
cohorts before a single cohort has completed project implementation and
submitted final data, we have in the past based targets on information provided
over a fairly limited amount of time, often reflecting the results of less than a
single cohort of complete performance information. Initial challenges with data
quality, in the past, also resulted in situations where only a limited number of
grantees in a cohort were able to provide valid performance data. This situation
introduced two possible problems into the target setting process - that targets
are being based in some part on a subset of grantees that are not representative
of the cohort as a whole, and that those sites able to provide valid data more
quickly may also have superior capacity to effectively implement program
interventions. While we have placed a significant emphasis on improving data


                                         22
quality and have worked to improve the response rates for measures over the life
of the cohort, this causes performance results from a relatively small set of
grantees in year one to be compared to results for a significantly larger
percentage of the cohort by the end of project.

Increasingly, it has become clear that a series of variables serve to make each
cohort unique, and that the issue of how we have established targets for this
measure in the past has been problematic. Revised processes for establishing
targets for this measure are discussed in the FY 201 1 Performance Targets
section below.

FY 201 1 Performance Targets. We established an FY 2009 target for the FY
2007 cohort based on the performance of prior cohorts in 2009. This cohort
achieved performance levels after one year that were close to those met after
two years by a prior cohort, but experienced a significant decline in FY 2009.
Based on lower than anticipated levels of performance for this measure, we
revised the FY 201 0 targets for both the FY 2007 and FY 2008 cohorts for this
measure. The 2010 target for the FY 2007 cohort was reduced from 80 percent
to 49.4 percent (or a target level that represents a 5 percent increase of the FY
2009 actual data for this measure). The 201 0 target for the FY 2008 cohort was
reduced from 76.87 percent to 53.2 percent (or a target level that represents a 5
percent increase of the FY 2009 actual data for this measure).

Given these challenges, and improvements we have made in data quality
(including generally requiring grantees to collect baseline data for their projects
before intelVentions are implemented), we have modified our process for
establishing targets. While prior cohort performance may provide some insights
about general patterns of performance that we can incorporate into our targeting
setting processes, for any future cohorts for this program we will establish
numerical performance targets after baseline data is received for the new cohort.
We will generally enter these targets into the Department's Visual Performance
System (VPS) as "administrative" targets (for example, baseline plus 5 percent),
and then convert the targets to numerical targets after baseline data is collected
and aggregated. We believe that this process revision will help us better match
targets to cohort performance.

In the case of FY 201 1 targets for the 2008 and 2009 cohorts, these were set
based on FY 2010 actual performance data.

Methodology. Data for this measure are collected by grantees and reported as
part of annual performance reports. If data for this measure are not available at
the time that performance reports are submitted, staff follow-up with sites to
attempt to obtain data for the measure. Grantees that fail to provide data are not
included in the tabulation of data for the measures. Also, grantees that did not
provide data for two consecutive project years (so that we could determine if a
decrease in binge drinking had occurred) are not included in the aggregate data


                                        23
reported for the measure. Authorized representatives for the grant site sign the
annual performance report and, in doing so, certify that to the best of the signer's
knowledge and belief, all data in the performance report are true and correct and
that the report fully discloses all known weaknesses concerning the accuracy,
reliability, and completeness of the data included. Generally, the Department
relies on the certification concerning data supplied by grantees and does not
conduct further reviews.

ED does not mandate data collection protocols or instruments for grantees.
Grantees select a survey item that reflects the concept of binge drinking, and
collect and report data about that survey item as part of their performance
reports. As a result, data are not comparable across grant sites, but individual
grant sites are required to use the same survey items across performance
periods. We consider sites that have experienced a decrease in the rate of binge
drinking of one percent or greater to have achieved a measurable decrease in
binge drinking.

Initially, applicants were not required to furnish baseline data as part of their
applications. Data supplied after year one were considered baseline data for the
projects. Projects required two years of data in order to determine if a decrease
in binge drinking among target students had occurred. However, the FY 2007
and subsequent cohorts were instructed to provide baseline data in their
application, or if that data was not available, to collect it before beginning project
implementation. Thus, we are able to report on grantee and cohort performance
at the end of year one, as is done for the FY 2009 cohort in this report.

We have provided Significantly increased guidance and technical assistance to
grantees beginning with the FY 2007 cohort, and believe that these efforts have
produced data that are of higher quality and more comparable across sites than
those of previous cohorts.

Measure 12: The percentage of grantees that show a measurable increase in
the percentage of target students who believe that alcohol abuse is harmful to
their health. (Grants to Reduce Alcohol Abuse - FY 2005, 2007, 2008, and 2009
cohorts - no new grants were awarded under this program in FY 2006.)

Table 1 2

Cohort      FY        FY         FY         FY         FY         FY         FY
            2006      2007       2008       2009       2010       2010       2011
            Actual    Actual     Actual     Actual     TarQet     Actual     Target
2005        nla       70.0       59.3       nla        nla        nla        nla
2007        nla       nla        69.2       76.5       SO.3       nla        nla 

200S        nla       nla        nla        5S.6       61.5       60.0       65.0
2009        nla       nla        nla        nla        nla        100,0      100.0



                                          24
The measure. This measure examines a key outcome for the Grants to Reduce
Alcohol Abuse (GRAA) program - perception of health risk for alcohol abuse
among target students. While the National Drug Control Strategy is focused
most intensively on preventing the use of controlled substances, the Strategy
does address the role of alcohol use as a drug of choice for teenagers. Data do
suggest that changes in perceptions about risks to health resulting from alcohol
use are positively correlated with reductions in alcohol use.

FY 2010 Performance Results
We are able to report on performance in FY 2010 for the 2008 and 2009 cohorts
of grants. The 2008 cohort fell slightly short of its target. No FY 2010 target was
set for the FY 2009 cohort but we are reporting actual FY 2010 performance.
There is incomplete data for the 2007 cohort and we are unable to report at this
time, but will do so in 201 1 .

As we have received data from across cohorts for this measure, we continue to
find it difficult to discern a pattern of performance that can serve as a basis for
establishing future targets. For example, it is difficult to assess why performance
results for the 2007 and 2008 cohorts fell short of the established targets for this
measure in the past. Intensive technical assistance (via contract) services were
not available to the FY 2008 cohort for much of FY 2009 but was in FY 2010.
This might be one reason the FY 2008 cohort exceeded its FY 2009
performance, although slightly missed meeting the FY 2010 target.

We have carefully considered performance reports submitted by grantees, as
well as our experience in monitoring and providing technical assistance to
grantees, and have identified some challenges that may have impeded grant
performance. Some common problems include turnover in leadership (at the
authorized representative or project director level) and challenges with collecting
and reporting valid data about the measure. Another variable that might affect
performance in sites is related to project design. For example, we're not certain
how to assess the likely impact of a site that is implementing a single research­
based program versus sites that have adopted a more comprehensive strategy
that includes a community-based intervention that complements school-based
curricula. Finally, cohort size and composition varies from cohort to cohort. In
some years funding for a large amount of new awards is available and in others
only a handful of sites will receive grants. To the ex1ent that our peer review
results that are used to rank order applications and select grantees accurately
predict project quality, the range of scores funded (and perhaps the range of
project quality) varies from fiscal year to fiscal year.

We believe that an equally important dimension in assessing performance
against established targets for this measure is our ability to correctly predict
targets for measures. Because there is limited available research and no
"industry" standard available to guide expectations for performance, generally we


                                         25
have used the performance of prior cohorts on the measure to establish targets
for subsequent cohorts. Because of the need to establish targets for future
cohorts before a single cohort has completed project implementation and
submitted final data, we have in the past based targets on information provided
over a fairly limited amount of time, often reflecting the results of less than a
single cohort of complete performance information.

Initial challenges with data quality, in the past, also resulted in situations where
only a limited number of grantees in a cohort were able to provide valid
performance data. This situation introduced two possible problems into the
target setting process - that targets are being based in some part on a subset of
grantees that are not representative of the cohort as a whole, and that those sites
able to provide valid data more quickly may also have superior capacity to
effectively implement program interventions. While we have placed a significant
emphasis on improving data quality and have worked to improve the response
rates for measures over the life of the cohort, this causes performance results
from a relatively small set of grantees in year one to be compared to results for a
significantly larger percentage of the cohort by the end of project.

Increasingly, it has become clear that a series of variables serve to make each
cohort unique, and that the issue of how we have established targets for this
measure in the past has been problematic. Revised processes for establishing
targets for this measure are discussed in the FY 201 1 Performance Targets
section below.

FY 201 1 Performance Targets.
We had established a FY 2009 target for the FY 2007 cohort based on the
performance of prior cohorts in 2008. However, this cohort experienced a
significant decline in performance in FY 2009. Based on lower than anticipated
levels of performance for this measure, we revised the FY 201 0 targets for both
the FY 2007 and FY 2008 cohorts for this measure. The 2010 target for the FY
2007 cohort was reduced from 86.5 percent to 80.3 percent (or a target level that
represents a 5 percent increase of the FY 2009 actual data for this measure).
The 201 0 target for the FY 2008 cohort was reduced from 86.5 percent to 61.5
percent (or a target level that represents a 5 percent increase of the FY 2009
actual data for this measure).

Given these challenges, and improvements we have made in data quality
(including generally requiring grantees to collect baseline data for their projects
before interventions are implemented), we have modified our process for
establishing targets. While prior cohort performance may provide some insights
about general patterns of performance that we can incorporate into our targeting
setting processes, for any future cohorts for this program we will establish
numerical performance targets after baseline data is received for the new cohort.
We will generally enter these targets into the Department's Visual Performance
System (VPS) as "administrative" targets (for example, baseline plus 5 percent),


                                         26
and then convert the targets to numerical targets after baseline data is collected
and aggregated. We believe that this process revision will help us better match
targets to cohort performance.

In the case of FY 201 1 targets for the 2008 and 2009 cohorts, these were set
based on FY 201 0 actual performance data.

Methodology. Data for this measure are collected by grantees and reported as
part of annual performance reports. If data for this measure are not available at
the time that performance reports are submitted, staff follow-up with sites to
attempt to obtain data for the measure. Grantees that fail to provide data are not
included in the tabulation of data for the measures. Also, grantees that did not
provide data for two consecutive project years (so that we could determine if an
increase in the percentage of students who believe alcohol abuse is harmful to
their health had occurred) are not included in the aggregate data reported for the
measure. Authorized representatives for the grant site sign the annual
performance report and, in doing so, certify that to the best of the signer's
knowledge and belief, all data in the performance report are true and correct and
that the report fully discloses all known weaknesses concerning the accuracy,
reliability, and completeness of the data included. Generally, the Department
relies on the certification concerning data supplied by grantees and does not
conduct further reviews.

ED does not mandate data collection protocols or instruments for grantees.
Grantees select a survey item that reflects the concept of perceived harm to
health, and collect and report data about that survey item as part of their
performance reports. As a result, data are not comparable across grant sites,
but individual grant sites are required to use the same survey items across
performance periods. We consider sites that have experienced an increase of
one percent or greater in the percentage of target students who perceive alcohol
abuse is harmful to their health as having achieved a measurable increase for
this measure.

Initially, applicants were not required to furnish baseline data as part of their
applications. Data supplied after year one were considered baseline data for the
projects. Projects required two years of data in order to determine if an increase
in perceptions of alcohol abuse harm had occurred. However, the FY 2007 and
subsequent cohorts were instructed to provide baseline data in their application,
or if that data was not available, to collect it before beginning project
implementation. Thus, we are able to report on grantee and cohort performance
at the end of year one, as is done for the FY 2009 cohort in this report.

We have provided significantly increased guidance and technical assistance to
grantees beginning with the FY 2007 cohort, and believe that these efforts have
produced data that are of higher quality and more comparable across sites than
those of previous cohorts.


                                        27
Measure 13: The percentage of grantees that show a measurable increase in
the percentage of target students who disapprove of alcohol abuse. (Grants to
Reduce Alcohol Abuse - FY 2005, 2007, 2008, and 2009 cohorts - no new
grants were awarded under this program in FY 2006.)

Table 13

Cohort     FY        FY         FY         FY         FY         FY         FY
           2006      2007       2008       2009       2010       2010       2011
           Actual    Actual     Actual     Actual     Target     Actual     Target
2005       nfa       71.0       74.1       nfa        nfa        nfa        nfa
2007       nfa       nfa        69.2       47.0       49.4       nfa        nfa
2008       nfa       nfa        nfa        49.3       51.8       58.3       65.0
2009       nfa       nfa        nfa        nfa        nfa        1 00.0     100.0

The measure. This measure examines a key outcome for the Grants to Reduce
Alcohol Abuse (GRAA) program - disapproval of alcohol abuse among target
students. While the National Drug Control Strategy is focused most intensively
on the preventing the use of controlled substances, the Strategy does address
the role of alcohol use as a drug of choice for teenagers. Data do suggest that
increases in the percentage of target students who believe that alcohol abuse is
not socially acceptable are associated with declines in consumption of alcohol.

FY 201 0 Performance Results
We are able to report on performance in FY 2010 for the 2008 and 2009 cohorts
of grants. The 2008 cohort exceeded its target. No FY 2 0 1 0 target was set for
the FY 2009 cohort but we are reporting actual FY 2 0 1 0 performance. There is
incomplete data for the 2007 cohort and we are unable to report at this time, but
will do so in 201 1 .

As we have received data from across cohorts for this measure, we continue to
find it difficult to discern a pattern of performance that can serve as a basis for
establishing future targets. For example, it is difficult to assess why performance
results for the 2007 and 2008 cohorts fell short of the established targets for this
measure in the past. Intensive technical assistance (via contract) services were
not available to the FY 2008 cohort for much of FY 2009 but was in FY 2010.
This might be one reason the FY 2008 cohort exceeded its target in FY 2010.

We have carefully considered performance reports submitted by grantees, as
well as our experience in monitoring and providing technical assistance to
grantees, and have identified some challenges that may have impeded grant
performance. Some common problems include turnover in leadership (at the
authorized representative or project director level) and challenges with collecting
and reporting valid data about the measure. Another variable that might affect
performance in sites is related to project design. For example, we're not certain

                                         28
how to assess the likely impact of a site that is implementing a single research­
based program versus sites that have adopted a more comprehensive strategy
that includes a community-based intervention that complements school-based
curricula. Finally, cohort size and composition varies from cohort to cohort. In
some years funding for a large amount of new awards is available and in others
only a handful of sites will receive grants. To the extent that our peer review
results that are used to rank order applications and select grantees accurately
predict project quality, the range of scores funded (and perhaps the range of
project quality) varies from fiscal year to fiscal year.

We believe that an equally important dimension in assessing performance
against established targets for this measure is our ability to correctly predict
targets for measures. Because there is limited available research and no
"industry" standard available to guide expectations for performance, generally we
have used the performance of prior cohorts on the measure to establish targets
for subsequent cohorts. Because of the need to establish targets for future
cohorts before a single cohort has completed project implementation and
submitted final data, we have in the past based targets on information provided
over a fairly limited amount of time, often refiecting the results of less than a
single cohort of complete performance information.

Initial challenges with data quality, in the past, also resulted in situations where
only a limited number of grantees in a cohort were able to provide valid
performance data. This situation introduced two possible problems into the
target setting process - that targets are being based in some part on a subset of
grantees that are not representative of the cohort as a whole, and that those sites
able to provide valid data more quickly may also have superior capacity to
effectively implement program interventions. While we have placed a significant
emphasis on improving data quality and have worked to improve the response
rates for measures over the life of the cohort, this causes pertormance results
from a relatively small set of grantees in year one to be compared to results for a
significantly larger percentage of the cohort by the end of project.

Increasingly, it has become clear that a series of variables serve to make each
cohort unique, and that the issue of how we have established targets for this
measure in the past has been problematic. Revised processes for establishing
targets for this measure are discussed in the FY 201 1 Performance Targets
section below.

FY 201 1 Performance Targets.
We had established an FY 2009 target for the FY 2007 cohort based on the
performance of prior cohorts in 2008. However, this cohort experienced a
significant decline in performance in FY 2009. Based on lower than anticipated
levels of performance for this measure, we revised the FY 201 0 targets for both
the FY 2007 and FY 2008 cohorts for this measure. The 201 0 target for the FY
2007 cohort was reduced from 86.5 percent to 49.4 percent (or a target level that


                                        29
represents a 5 percent increase of the FY 2009 actual data for this measure),
The 2010 target for the FY 2008 cohort was reduced from 86.5 percent to 51.8
percent (or a target level that represents a 5 percent increase of the FY 2009
actual data for this measure).

Given these challenges, and improvements we have made in data quality
(including generally requiring grantees to collect baseline data for their projects
before interventions are implemented), we have modified our process for
establishing targets. While prior cohort performance may provide some insights
about general patterns of performance that we can incorporate into our targeting
setting processes, for any future cohorts for this program we will establish
numerical performance targets after baseline data is received for the new cohort.
We will generally enter these targets into the Department's Visual Performance
System (VPS) as "administrative" targets (for example, baseline plus 5 percent),
and then convert the targets to numerical targets after baseline data is collected
and aggregated. We believe that this process revision will help us better match
targets to cohort performance.

I n the case of FY 201 1 targets for the 2008 and 2009 cohorts, these were set
based on FY 201 0 actual performance data.

Methodology. Data for this measure are collected by grantees and reported as 

part of annual performance reports. If data for this measure are not available at
the time that performance reports are submitted, staff follow-up with sites to
attempt to obtain data for the measure. Grantees that fail to provide data are not
included in the tabulation of data for the measures. Also, grantees that did not
provide data for two consecutive project years (so that we could determine if an
increase in the percentage of students who disapprove of alcohol abuse had
occurred) are not included in the aggregate data reported for the measure.
Authorized representatives for the grant site sign the annual performance report
and, in doing so, certify that to the best of the signer's knowledge and belief, all
data in the performance report are true and correct and that the report fully
discloses all known weaknesses concerning the accuracy, reliability, and
completeness of the data included. Generally, the Department relies on the
certification concerning data supplied by grantees and does not conduct further
reviews.

ED does not mandate data collection protocols or instruments for grantees.
Grantees select a survey item that reflects the concept of disapproval of alcohol
abuse, and collect and report data about that survey item as part of their
performance reports. As a result, data are not comparable across grant sites,
but individual grant sites are required to use the same survey items across
performance periods. We consider sites that have experienced an increase in
the percentage of students reporting disapproval of alcohol abuse of one percent
or greater to have achieved a measurable increase for this measure.



                                         30
Initially, applicants were not required to furnish baseline data as part of their
applications. Data supplied after year one were considered baseline data for the
projects. Projects required two years of data in order to determine if an increase
in disapproval of alcohol abuse among target students has occurred. However,
the FY 2007 and subsequent cohorts were instructed to provide baseline data in
their application, or if that data was not available, to collect it before beginning
project implementation. Thus, we are able to report on grantee and cohort
performance and the end of year one, as is done for the FY 2009 cohort in this
report.

We have provided significantly increased guidance and technical assistance to
grantees beginning with the FY 2007 cohort, and believe that these efforts have
produced data that are of higher quality and more comparable across sites than
those of previous cohorts.


Assertions
                          Performance Reporting System

The Department of Education has a system in place to capture performance
information accurately and that system was properly applied to generate the
performance data in this report. In instances in which data are supplied by
grantees as part of required periodic performance reports, the data that are
supplied are accurately reflected in this report.

Data related to the drug control programs included in this Performance Summary
Report for Fiscal Year 2010 are recorded in the Department of Education's
software for recording performance data and are an integral part of our budget
and management processes.


               Explanations for Not Meeting Performance Targets

The explanations provided in the Performance Summary report for Fiscal Year
2010 for not meeting performance targets and for recommendations for plans to
revise performance targets are reasonable given past experience, available
information, and available resources.

                Methodologyfor Establishing Performance Targets

The methodology described in the Performance Summary Report for Fiscal Year
2010 to establish performance targets for the current year is reasonable given
past performance and available resources.




                                         31
          Performance Measures for Significant Drug Control Activities

The Department of Education has established at least one acceptable
performance measure for each Drug Control Decision Unit identified in its
Detailed Accounting of Fiscal Year 2010 Drug Control Funds.

Criteria for Assertions




No workload or participant data support the assertions provided in this report.
Sources of quantitative data used in the report are well documented. These data
are the most recently available and are identified by the year in which the data
was collected.

                           Other Estimation Methods

No estimation methods other than professional judgment were used to make the
required assertions. When professional judgment was used, the objectivity and
strength of those judgments were explained and documented. Professional
judgment was used to establish targets for programs until data from at least one
grant cohort were available to provide additional information needed to set more
accurate targets. We routinely re-evaluate targets set using professional
judgment as additional information about actual performance on measures
becomes available.

                               Reporting Systems

Reporting systems that support the above assertions are current, reliable, and an
integral part of the Department of Education's budget and management
processes. Data collected and reported for the measures discussed in this report
are stored in the Department of Education's Visual Performance System (VPS).
Data from the VPS are used in developing annual budget requests and
justifications, and in preparing reports required under the Government
Performance and Results Act of 1993.




                                       32
                                    UNITED STATES DEPARTMENT OF EDUCATION
                                                           OFFICE OF INSPECTOR GENERAL


                                                                                                                         AUDIT SERVICES




                                                                        JAN 3 1 zon

MEMORANDUM



To:                   Kevin Jennings
                      Assistant Deputy Secretary
                      Office of Safe and Drug-Free Schools


From:                 Keith West           /!                    tvhX
                      Assistant Inspector General for Audit


Subject:              Office of lnspector General's Independent Report on the U.S. Department of
                      Education's Performance Summary Reportfor Fiscal Year 2010, dated
                     January 19, 20 I I


Attached is our authentication of management's assertions contained in the U.S. Department of
Education's Performance Summary Reportjor Fiscal Year 2010, dated January 19, 20 1 1, as
required by section 705(d) of the Office of National Drug Control Policy Reauthorization Act of
1998 (21 U.S.c,          §   1 704(d)).


OUf authentication was conducted in accordance with the guidelines stated in the Office of
National Drug Control Policy Circular: Drug Control Accounting, dated May I, 2007.


Ifyou have any questions or wish to discuss the contents ofthis authentication, please contact
Michele Weaver-Dugan, Director, Operations Internal Audit Team, at (202) 245-694 1 .




Attachment




 The Dcpanmcnt of EduC3tiOfl's missiOfl is to promote student achievement IIIld preparation for global competitiveness by fostering edU<:ldiooal
                                                    excellence and Cflsuring equal access.
                                    UNITED STATES DEPARTMENT OF EDUCATION
                                                          OFFICE OF INSPECTOR GENERAL


                                                                                                                        AUDIT SERVICES

                                                           JAN 3 1 1011


OiTice of Inspector General's Independent Report on the U.S. Department of Education's
PerfOrmance SummaryReporttor Fiscal Year 2010. dated January 1 9,201 1


We have reviewed management's assertions contained in the accompanying Performance
Summary Report Fiscal Year 20l0, dated January 19, 2011 (Perfonnance Summary Report).
The U.S. Department of Education's management is responsible for the Perfonnance Summary
Report and the assertions contained therein.


OUf review was conducted in accordance with attestation standards established by the American
Institute of Certified Public Accountants. A review is substantially less in scope than an
examination, the objective of which is the expression of an opinion on management's assertions.
Accordingly, we do not express such an opinion.


We performed review procedures on the "Performance Summary Information," "'Assertions,"
and "Criteria for Assertions" contained in the accompanying Performance Sununary Report. In
general, our review procedures were limited to inquiries and analytical procedures appropriate
for our review engagement. We did not perform procedures related to controls over the reporting
system noted in the attached report.


Based on our review, nothing came to our attention that caused us to believe that management's
assertions, contained in the accompanying Performance Summary Report, are not fairly stated in
all material respects, based upon the Office of National Drug Control Policy Circular:
Drug Control Accounting, dated May 1 , 2007.




                                                       Keith West
                                                       Assistant Inspector General for Audit




 The Department ofEducatiOll's missioo is 10 pl'Oll1Ole student achievement and preparation for global competitiveness by fostering educational
                                                    excellence and ensuring equal acces.s