oversight

OIG's Independent Report on the Department's Performance Summary Report for Fiscal Year 2008

Published by the Department of Education, Office of Inspector General on 2009-01-30.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                          UNITED STATES DEPARTMENT OF EDUCATION

                                        OFFICE OF SAFE AND DRUG-FREE SCHOOLS




Edward H. Jurith
Acting Director
Office of National Drug Control Policy
Executive Office of the President
Washington, D.C. 20500

Dear Mr. Jurith:

In accordance with the Office of National Drug Control Policy (ONDCP) Circular Drug Control
Accounting, enclosed please find detailed information about performance-related measures for
key drug control programs administered by the U.S. Department of Education contained in the
U.S. Department of Education’s Performance Summary Report for Fiscal Year 2008, along with
the Department of Education Assistant Inspector General’s authentication of the management
assertions included in that report.

Please do not hesitate to contact me if you have any questions about this information.

                                           Sincerely,




                                           William Modzeleski/s/
                                           Assistant Deputy Secretary (Acting)


Enclosure #1: Department of Education Performance Summary Report for Fiscal Year 2008,
dated January 28, 2009

Enclosure #2: Authentication letter from Keith West, Assistant Inspector General for Audit
Services, dated January 30, 2009

cc: Keith West




                         400 MARYLAND AVE., S.W., WASHINGTON, D.C. 20202
                                         www.ed.gov

 Our mission is to ensure equal access to education and to promote educational excellence throughout the nation.
       Department of Education




        Performance Summary Report

                 Fiscal Year 2008




                   In Support of the

            National Drug Control Strategy

As required by ONDCP Circular: Drug Control Accounting



                   January 28, 2009
                                             Department of Education

                        Performance Summary Report for Fiscal Year 2008

                                               TABLE OF CONTENTS

Transmittal Letter ............................................................................................................ 1

Performance Summary Information................................................................................. 2

          Safe Schools/Healthy Students ............................................................................ 2

          Student Drug Testing............................................................................................ 4

          Safe and Drug-Free Schools and Communities State Grants Program................ 9

          Grants to Reduce Alcohol Abuse........................................................................ 15

Assertions ..................................................................................................................... 20

          Performance Reporting System.......................................................................... 20

          Methodology for Establishing Performance Targets ………………………………20

          Performance Measures for Significant Drug Control Activities ........................... 20

Criteria for Assertions ................................................................................................... 20
                          UNITED STATES DEPARTMENT OF EDUCATION

                                        OFFICE OF SAFE AND DRUG-FREE SCHOOLS




Ms. Mary Mitchelson
Inspector General (Acting)
U.S. Department of Education
400 Maryland Avenue, S.W.
Washington, DC 20202-1510

Dear Ms. Mitchelson:

As required by Office of National Drug Control Policy (ONDCP) Circular Drug Control
Accounting, enclosed please find detailed information about performance-related
measures for key drug control programs administered by the U.S. Department of
Education, in accordance with the guidelines in the circular dated May 1, 2007. This
information covers the Safe and Drug-Free Schools and Communities program, which is
the Department’s only Drug Control Budget Decision Unit displayed in the National Drug
Control Budget Summary.

Consistent with the instructions in the ONDCP Circular, please provide your
authentication to me in writing and I will transmit it to ONDCP along with the enclosed
Performance Summary Report. As you know, ONDCP requests these documents by
February 1, 2009 if possible. Please do not hesitate to contact me if you have any
questions about the enclosed information.

                                           Sincerely,




                                           William Modzeleski /s/
                                           Acting Assistant Deputy Secretary for
                                           Safe and Drug-Free Schools




                         400 MARYLAND AVE., S.W., WASHINGTON, D.C. 20202
                                         www.ed.gov

 Our mission is to ensure equal access to education and to promote educational excellence throughout the nation.
                                                       1
Performance Summary Information

                            Safe Schools/Healthy Students

Measure 1: The percentage of grantees demonstrating a decrease in substance abuse
over the three-year grant period. (Safe Schools/Healthy Students – FY 2004, 2005, and
2006 cohorts)

Table 1

Cohort FY 2004      FY 2005    FY 2006    FY 2007   FY 2008    FY 2008    FY 2009
        Actual       Actual     Actual     Actual    Target     Actual     Target
2004   n/a          n/a        75         66.7      90         pending    n/a
2005   n/a          n/a        n/a        43.75     86.25      pending    n/a
2006   n/a          n/a        n/a        n/a       maintain   66.67      76.67%
                                                    a
                                                    baseline

The measure. This performance measure is for the Safe Schools/Healthy Students
initiative, a joint project of the Departments of Education, Health and Human Services,
and Justice. The initiative provides grants to local educational agencies (LEAs) to
support the development and implementation of a comprehensive plan designed to
prevent student drug use and violence and support healthy youth development.

This measure, one of four for this initiative, focuses on one of the primary purposes of
the initiative – reduced student drug use. The initiative, and this measure, are directly
related to the National Drug Control Strategy’s goal of preventing drug use before it
begins. Grantees select and report on one or more measures of prevalence of drug use
for students. For the FY 2004 – 2006 cohorts, those grantee measures are not
common across grant sites but, rather, reflect priority drug use problems identified by
sites.

FY 2008 Performance Results. Because the measure is established to look at progress
over the three-year grant period, it has just a single target for the third year of
implementation of each cohort. Sites were not required to provide or collect baseline
data at the time of application or before program interventions were implemented, so
grantees provide baseline data for their selected measures related to drug use after
year one (for example in FY 2005 for the FY 2004 cohort). Grantees from the FY 2004
cohort generally completed no-cost extension years and will be providing GPRA data in
final grantee reports that were due at the end of December 2008. Those data will be
aggregated later in FY 2009 to determine if the FY 2007 target for the cohort has been
met. The FY 2005 cohort of grantees is also operating under no-cost extensions. Final
GPRA data for this cohort will be submitted at the end of December 2009. First
performance results for the FY 2006 cohort are included in the table above.




                                           2
FY 2009 Performance Targets. Targets for the two earliest cohorts were initially
established before any performance data for this measure were received, and
represented our judgment at the time, given the significant size of SS/HS grants and the
emphasis on research-based programs that is central to the initiative. We elected in
2008 to revise the target for the FY 2005 cohort for this measure based on the actual
performance to date (implementation year two) of the FY 2004 cohort. Based on our
professional judgment, it seemed that the revised target of 86.25 percent was
appropriately aggressive and that attaining that target would be a meaningful outcome
for the program, while acknowledging that our target for the initial (FY 2004) cohort may
have been unrealistic. In 2008 we also developed revised targets for the FY 2006
cohort, again, based on the limited data currently available for this measure.

Our ability to establish appropriate targets for this program has also been impacted by
challenges associated with the quality of data supplied by grant sites. A significant
number of sites have failed to provide valid data for this and some other SS/HS
measures. Through technical assistance activities we have achieved some
improvements in data quality for some sites, but have not completed a full grant cycle
with cohorts that have received early and more intensive technical assistance.

Subsequently, we have adopted revised GPRA measures for this initiative beginning
with the FY 2007 cohort and will establish 2009 targets for sites in that cohort shortly.

Methodology. Data are collected by grantees, generally using student surveys. Data
are furnished in the second of two semi-annual performance reports provided by
grantees each project year. If grantees identified more than one measure of drug abuse
or provided data for individual school-building types (for example, separate data for
middle and high schools), grantees were considered to have experienced a decrease in
substance abuse if data for a majority of measures provided reflected a decrease. If a
grant site provided data for an even number of measures and half of those measures
reflected a decrease and half reflected no change or an increase, that grant site was
judged not to have demonstrated a decrease in substance abuse. The response rate
for the FY 2004 cohort for this measure was 35 percent. While most sites were able to
provide some data related to this measure, we considered as valid data only data from
sites that used the same elements/items in each of years one and two. Nearly 80
percent of grantees from the FY 2006 cohort were able to provide valid data for this
measure.

If data for this measure are not available at the time that performance reports are
submitted, staff follow up with sites to attempt to obtain data for the measure. Grantees
that fail to provide data are not included in the tabulation of data for the measures.
Also, grantees that did not provide data for two consecutive project years (so that we
could determine if a decrease in substance abuse had occurred) are not included in
data reported for the measure. Authorized representatives for the grant site sign the
annual performance report and, in doing so, certify that to the best of the signers’
knowledge and belief, all data in the performance report are true and correct and that
the report fully discloses all known weaknesses concerning the accuracy, reliability, and

                                             3
completeness of the data included. Generally, the Department relies on the certification
concerning data supplied by grantees and does not conduct further reviews.

Targets were established for this measure after the baseline data for the FY 2004
cohort were provided. Based on the available results for this first cohort, targets for
future cohorts have been adjusted. For example, the targets for the FY 2005 and 2006
cohorts were adjusted in 2008.

                                  Student Drug Testing

Measure 2: The percentage of student drug testing grantees that experience a 5
percent reduction in current (30-day) illegal drug use by students in the target
population. (Student Drug Testing – FY 2003, 2005, 2006, and 2007 cohorts—no new
grants were awarded under this program in FY 2004)

Table 2

Cohort FY 2004      FY 2005    FY 2006    FY 2007    FY 2008    FY 2008    FY 2009
        Actual       Actual     Actual     Actual     Target     Actual     Target
2003   n/a          n/a        33         25         n/a        n/a        n/a
2005   n/a          n/a        n/a        pending    50         pending    n/a
2006   n/a          n/a        n/a        n/a        50         66.7       60
2007   n/a          n/a        n/a        n/a        33         pending    50

The measure. This measure is one of two measures for the Student Drug-Testing
Programs grant competition. The competition provides discretionary grants to LEAs,
community-based organizations, or other public and private entities to support
implementation of drug testing of students, consistent with the parameters established
by the U.S. Supreme Court or for students and their families that voluntarily agree to
participate in the student drug testing program.

This measure is directly related to the National Drug Control Strategy’s goal related to
preventing drug use before it starts. Student drug testing has been prominently
featured in recent annual versions of the strategy as a recommended drug prevention
intervention.

FY 2008 Performance Results. FY 2007 data for the FY 2003 cohort were submitted as
part of final reports for these grants, and the aggregate of those data is included in the
chart above.

In FY 2008 we completed a preliminary review of data submitted by the FY 2005 cohort
for this measure and identified significant concerns about the quality and comparability
of the data. Grant sites have reported on prevalence rates for a variety of illegal drugs
and have not always provided data from the same items/elements across project years
one and two. Also, some sites surveyed their entire student population and others
surveyed only students in the testing pool. Based on these concerns, we obtained

                                            4
assistance from the U.S. Department of Education’s Data Quality Initiative contractor.
With their help, we created and disseminated detailed data collection and reporting
guidance for the program, as well as data standards that we will use to determine what
constitutes valid data for this measure, and disseminated this guidance to the new
cohort of 2008 grantees. Based on that guidance, as well as data quality and
aggregation checks, in FY 2009 we will begin aggregating available data from the FY
2007 and 2008 cohorts and record those data in the Department’s software that houses
GPRA measures and data.

Data for the FY 2006 cohort come from the evaluation being conducted for the
Department of Education by a contractor. Data for this cohort were collected by the
contractor in 2007 and again in 2008; the data reported in the chart above reflects the
results of student surveys administered by the contractor.

FY 2009 Performance Targets. We established targets for the percentage of grantees
experiencing a 5 percent reduction in current illegal drug use after reviewing the first two
years of data for the FY 2003 cohort of grant sites. Consistent with research that
suggests that changes in student behavior related to student drug testing may not be
realized immediately, we assumed that we could look for an increased number of
grantees to experience positive change and, using our professional judgment, set that
target at 50 percent of grantees. Although we have received data for three project
years from a single cohort of sites (the FY 2003 cohort), the information provided by the
grantees did not provide an adequate basis for revisiting targets for future cohorts. This
cohort was very small (eight grantees), and also experienced extensive delays in
implementation and data collection activities. Because only a handful of grantees were
able to eventually provide data specific to the measure, we do not believe that it would
be appropriate to base expectations about the performance of other cohorts on this
limited information.

Similar problems with data quality for the FY 2005 cohort of grant sites mean that data
from that cohort will not be helpful in determining if targets for the program will need to
be readjusted. Challenges with data quality have resulted in only a very limited
proportion of grant sites that provided approximately comparable data. Conversely,
because the data from the evaluation are being collected by a contractor using
comparable survey items and collection procedures (in contrast to the varying
procedures used by individual grant sites in the other cohorts), data for the 2006 cohort
does not provide an appropriate basis for making adjustments in existing targets under
the program for the FY 2007 cohort. As a result, we have retained the established FY
2009 target for the FY 2007 cohort of grantees at this time.

We need to establish a revised FY 2009 target for this measure for the FY 2006 cohort
of grant sites since FY 2008 performance already exceeds the current FY 2009 target.
We will consider the data collected and reported by the contractor about changes in the
illegal drug use in grant sites, in conjunction with the limited information about
performance of other cohorts in this grant program to establish an appropriate FY 2009
target for this cohort.

                                             5
Methodology With the exception of the FY 2006 cohort, data are collected by grantees
using student surveys. Data are provided as part of the grantees’ annual performance
reports. Grantees do not use the same survey items to collect data for this measure
but, rather, self-select survey items (often from surveys already administered) in order
to provide these data. Survey items may relate to different substances, but must collect
information concerning current use in order to be included in the data reported for this
measure. Grantees did not provide baseline data in their applications, so we have to
wait until grantees provide data both from project year one and two in order to
determine if they have experienced a decrease in substance abuse. For the FY 2003
cohort, project implementation was delayed for one full year while grantees sought
needed institutional review board clearance to drug test students, so performance data
were initially received in 2005 and 2006. Only 3 of 8 grantees provided comparable
data across the first two years of their project. The FY 2005 cohort of grant sites has
also provided data, but similarly of questionable quality; therefore, data from many sites
cannot be included in aggregating data for the cohort, resulting in only a partial picture
of grantee progress.

Authorized representatives for the grant site sign the annual performance report and, in
doing so, certify that to the best of the signer’s knowledge and belief, all data in the
performance report are true and correct and that the report fully discloses all known
weaknesses concerning the accuracy, reliability, and completeness of the data
included. Generally, the Department relies on the certification concerning data supplied
by grantees and does not conduct further reviews.

Data for the FY 2006 cohort are being collected as part of an evaluation of student drug
testing. Data for the measures are being collected by the evaluation contractor, using
common survey items and collection procedures. Survey responses are analyzed by
the contractor and data are provided to the Department.

The anticipated levels of decrease in substance abuse are consistent with those
included in the National Drug Control Strategy – five percent per year. Targets were
initially established following the report of baseline data for grant sites from the FY 2003
cohort. As discussed above, we do not currently have data of sufficient quality to
support adjustment of targets for this program at this time.

Measure 3: The percentage of student drug testing grantees that experience a 5
percent reduction in past-year illegal drug use by students in the target population.
(Student Drug Testing – FY 2003, 2005, 2006, and 2007 cohorts—no new grants were
awarded under this program in FY 2004)




                                             6
Table 3

Cohort FY 2004      FY 2005    FY 2006    FY 2007    FY 2008    FY 2008    FY 2009
        Actual       Actual     Actual     Actual     Target     Actual     Target
2003   n/a          n/a        25         0          n/a        n/a        n/a
2005   n/a          n/a        n/a        pending    50         pending    n/a
2006   n/a          n/a        n/a        n/a        50         55.5       60
2007   n/a          n/a        n/a        n/a        33         pending    50

The measure. This measure is one of two measures for the Student Drug-Testing
Programs grant competition. The competition provides discretionary grants to LEAs,
community-based organizations, or other public and private entities to support
implementation of drug testing of students, consistent with the parameters established
by the U.S. Supreme Court or for students and their families that voluntarily agree to
participate in the student drug testing program.

This measure is directly related to the National Drug Control Strategy’s goal related to
preventing drug use before it starts. Student drug testing has been prominently
featured in recent annual versions of the strategy as a recommended drug prevention
intervention.

FY 2008 Performance Results. FY 2007 data for the FY 2003 cohort were submitted as
part of final reports for these grants and the aggregate of those data is included in the
chart above.

In FY 2008 we completed a preliminary review of data submitted by the FY 2005 cohort
for this measure and identified significant concerns about the quality and comparability
of the data. Grant sites have reported on prevalence rates for a variety of illegal drugs
and have not always provided data from the same items/elements across project years
one and two. Also, some sites surveyed their entire student population and others
surveyed only students in the testing pool. Based on these concerns, we obtained
assistance from the U.S. Department of Education’s Data Quality Initiative contractor.
With their help, we created and disseminated detailed data collection and reporting
guidance for the program, as well as data standards that we will use to determine what
constitutes valid data for this measure, and disseminated this guidance to the new
cohort of 2008 grantees. Based on that guidance, as well as data quality and
aggregation checks, in FY 2009 we will begin aggregating available data from the FY
2007 and FY 2008 cohorts and record those data in the Department’s software that
houses GPRA measures and data.

Data for the FY 2006 cohort come from the evaluation being conducted for the
Department of Education by a contractor. Data for this cohort were collected by the
contractor in 2007 and again in 2008; the data reported in the chart above reflects the
results of student surveys administered by the contractor. An important note is that data
supplied for the measure for the 2006 cohort represents student drug use in the six
months prior to the survey (rather than the one-year period called for in the measure.)

                                            7
FY 2009 Performance Targets. We established targets for the percentage of grantees
experiencing a 5 percent reduction in annual illegal drug use after reviewing the first two
years of data for the FY 2003 cohort of grant sites. Consistent with research that
suggests that changes in student behavior related to student drug testing may not be
realized immediately, we assumed that we could look for an increased number of
grantees to experience positive change and, using our professional judgment, set that
target at 50 percent of grantees. Although we have received data for three project
years from a single cohort of sites (the FY 2003 cohort), the information provided by the
grantees did not provide an adequate basis for revisiting targets for future cohorts. This
cohort was very small (eight grantees), and also experienced extensive delays in
implementation and data collection activities. Because only a handful of grantees were
able to eventually provide data specific to the measure, we do not believe that it would
be appropriate to base expectations about the performance of other cohorts on this
limited information.

Similar problems with data quality for the FY 2005 cohort of grant sites mean that data
from that cohort will not be helpful in determining if targets for the program will need to
be readjusted. Challenges with data quality have resulted in only a very limited
proportion of grant sites that provided approximately comparable data. Conversely,
because the data from the evaluation are being collected by the contractor using
comparable survey items and collection procedures (in contrast to the varying
procedures used by individual grant sites in the other cohorts), data for the 2006 cohort
does not provide an appropriate basis for making adjustments in existing targets under
the program for the FY 2007 cohort. As a result, we have retained the established FY
2009 target for the FY 2007 cohort of grantees.

Methodology With the exception of the FY 2006 cohort, data are collected by grantees
using student surveys. Data are provided as part of the grantees’ annual performance
reports. Grantees do not use the same survey items to collect data for this measure
but, rather, self-select survey items (often from surveys already administered) in order
to provide these data. Survey items may relate to different substances, but must collect
information concerning annual use in order to be included in the data reported for this
measure. Grantees did not provide baseline data in their applications, so we have to
wait until grantees provide data both from project year one and two in order to
determine if they have experienced a decrease in substance abuse. For the FY 2003
cohort, project implementation was delayed for one full year while grantees sought
needed institutional review board clearance to drug test students, so performance data
were initially received in 2005 and 2006. Only three of eight grantees provided
comparable data across the first two years of their project. The FY 2005 cohort of grant
sites has also provided data, but similarly of questionable quality; therefore, data from
many sites cannot be included in aggregating data for the cohort, resulting in only a
partial picture of grantee progress.

Authorized representatives for the grant site sign the annual performance report and, in
doing so, certify that to the best of the signer’s knowledge and belief, all data in the

                                             8
performance report are true and correct and that the report fully discloses all known
weaknesses concerning the accuracy, reliability, and completeness of the data
included. Generally, the Department relies on the certification concerning data supplied
by grantees and does not conduct further reviews.

Data for the FY 2006 cohort are being collected as part of an evaluation of student drug
testing. Data for the measures are being collected by the evaluation contractor, using
common survey items and collection procedures. Survey responses are analyzed by
the contractor and data are provided to the Department.

The anticipated levels of decrease in substance abuse are consistent with those
included in the National Drug Control Strategy – five percent per year. Targets were
initially established following the report of baseline data for grant sites from the FY 2003
cohort. As discussed above, we do not currently have data of sufficient quality to
support adjustment of targets for this program at this time.

              Safe and Drug-Free Schools and Communities State Grants

Measure 4: The percentage of students in grades 9-12 who were offered, sold, or
given an illegal drug on school property during the past 12 months. (Safe and Drug-
Free Schools and Communities State Grants)

Table 4

FY 2004      FY 2005     FY 2006      FY 2007     FY 2008     FY 2008       FY 2009
 Actual       Actual      Actual       Actual      Target      Actual        Target
None        25.4         None        22.3         None        None         26

The measure. This measure is one of three measures directly related to reducing
student drug or alcohol use for the Safe and Drug-Free Schools and Communities
(SDFSC) State Grants. This formula grant program provides funds to the States, based
on school-aged population and the State’s relative share of Elementary and Secondary
Education Act Title I concentration grant funds, to support drug and violence prevention
programs. The measure directly relates to the National Drug Control Strategy Goal of
preventing youth drug use by focusing on the extent to which illegal drugs are available
on school property.

FY 2008 Performance Results. There is no target and no data collected for this
measure in FY 2008 because data are collected only in odd-numbered years.

FY 2009 Performance Targets. The target identified for this measure in FY 2009 is
currently 26 percent. Given the FY 2007 results, ED will need to revise the target to
reflect the progress achieved in FY 2007.

Methodology. Data for this measure are collected from a nationally representative
sample of students in grades 9-12 as part of the Youth Risk Behavior Surveillance

                                             9
System (YRBSS), sponsored by the Centers for Disease Control and Prevention (CDC).
Data are collected in odd years and reported in the following even years. No data are
collected for even years and, as a result, no targets have been established for even
years.

Detailed information about the methodology used to sample and report data for the
YRBSS is available at the CDC website at:
http://www.cdc.gov/mmwr/preview/mmwrhtml/ss5505a1.htm. We rely on the assertions
provided about methodology presented by CDC in using this data to report on
performance of SDFSC State Grants.

Measure 5: The percentage of students in grades 9-12 who used marijuana one or
more times during the past 30 days. (SDFSC State Grants)

Table 5

FY 2004     FY 2005    FY 2006      FY 2007    FY 2008     FY 2008      FY 2009
 Actual      Actual     Actual       Actual     Target      Actual       Target
None       20          None        19.7        None        None        18

The measure. This measure is one of three measures directly related to reducing
student drug and alcohol use for SDFSC State Grants. This formula grant program
provides funds to the States, based on school-aged population and the State’s relative
share of Elementary and Secondary Education Act Title I concentration grant funds, to
support drug and violence prevention programs. The measure is directly related to the
National Drug Control Strategy Goal of preventing youth drug use by focusing on the
extent of current use by high school aged-youth of the most prevalent illegal drug.

FY 2008 Performance Results. This is no target and no data for this measure in FY
2008 because data are collected only in the odd-numbered years. .

FY 2009 Performance Targets. The target for this measure in FY 2009 is 18. Given the
limited progress made toward achieving the established target level in FY 2007, we do
not plan to revise this target.

Methodology. Data for this measure are collected from a nationally representative
sample of students in grades 9-12 as part of the Youth Risk Behavior Surveillance
System (YRBSS), sponsored by the Centers for Disease Control and Prevention (CDC).
Data are collected in odd years and reported in the following even years. No data are
collected for even years and, as a result, no targets have been established for even
years.

Detailed information about the methodology used to sample and report data for the
YRBSS is available at the CDC website at:
http://www.cdc.gov/mmwr/preview/mmwrhtml/ss5505a1.htm. We rely on the assertions


                                          10
provided about the methodology presented by CDC in using this data to report on
performance of SDFSC State Grants.

Measure 6: The percentage of students in grades 9-12 who had five or more drinks of
alcohol in a row one or more times during the past 30 days. (SDFSC State Grants)

Table 6

FY 2004     FY 2005    FY 2006      FY 2007    FY 2008     FY 2008      FY 2009
 Actual      Actual     Actual       Actual     Target      Actual       Target
None       26          None        26          None        None        25

The measure. This measure is one of three measures related to reducing student drug
or alcohol use for SDFSC Grants. This formula grant program provides funds to the
States, based on school-aged population and the State’s relative share of Elementary
and Secondary Education Act Title I concentration grant funds, to support drug and
violence prevention programs. The measure is directly related to the National Drug
Control Strategy Goal of preventing youth drug use by focusing on the prevalence of
binge drinking by high school aged-students. While alcohol is not explicitly an emphasis
of the National Drug Control Strategy, illegal use of alcohol can be associated with use
of other illegal drugs.

FY 2008 Performance Results. There is no target and no data for measure in FY 2008
because data are collected only in the odd-numbered years.

FY 2009 Performance Targets. The target for this measure for FY 2009 is 25. Given
that there was no change in the data for this measure between 2005 and 2007, we do
not plan to revise the target for FY 2009.

Methodology. Data for this measure are collected from a nationally representative
sample of students in grades 9-12 as part of the Youth Risk Behavior Surveillance
System (YRBSS), sponsored by the Centers for Disease Control and Prevention (CDC).
Data are collected in odd years and reported in the following even years. No data are
collected for even years and as a result no targets have been established for even
years.

Detailed information about the methodology used to sample and report data for the
YRBSS is available at the CDC website at:
http://www.cdc.gov/mmwr/preview/mmwrhtml/ss5505a1.htm. We rely on the assertions
provided about the methodology presented by CDC in using this data to report on
performance of SDFSC State Grants.

Measure 7: The percentage of drug and violence prevention programs/practices
supported with SDFSC State Grant funds that are research based. (SDFSC State
Grants)


                                          11
Table 7

FY 2004     FY 2005    FY 2006     FY 2007     FY 2008      FY 2008     FY 2009
 Actual      Actual     Actual      Actual      Target       Actual      Target
None       7.8         None        None        None        n/a         13

The measure. This measure examines the extent to which programs and practices
supported with SDFSC State Grant funds are based on research. The measure
supports attainment of National Drug Control Strategy goals by focusing on the quality
of programs supported with SDFSC State Grants funds and the likelihood that the
programs will reduce or prevent youth drug use. The 2005 data constitute the baseline
for this measure.

FY 2008 Performance Results. No FY 2008 target is in effect for this measure in FY
2008; data will be collected in 2009 for this measure.

FY 2009 Performance Targets. A contract to collect data to implement this measure
could not be issued in time to permit data collection during FY 2008 as originally
scheduled. As a result, we established a 2009 target against the 2005 baseline that is a
linear extrapolation of a previously established FY 2008 target for this measure. The
next data collection is scheduled to collect information about programs implemented
during the 2008-2009 school year.

Methodology. Baseline data for this measure were collected from a nationally
representative sample of schools under a contract supported by ED. As a first step, the
contractor developed a large list of research-based programs and then screened those
programs to identify programs that were relevant to the SDFSC State Grants program;
had at least two empirical studies completed that met stringent methodological
standards; had implementation materials available; used at least two independent
samples in program evaluations; and demonstrated an adequate level of program
effectiveness.

The contractor collected data for the measure using surveys of national probability
samples of public elementary and secondary schools and the school districts with which
they were associated. The surveys – conducted using both mail and web-based
approaches – gathered information on prevention programs operating during the 2004-
2005 school year. Survey information was collected between fall 2005 and spring 2006.

The sample design included 2,500 districts, and nearly 6,000 schools that were
sampled from the 2,500 districts. The contractor used the National Assessment of
Educational Progress (NAEP) national sample frame. The NAEP sample frame is
derived from the 2003-2004 National Center for Education Statistics Common Core of
Data (CCD) Public Elementary and Secondary School Universe and Agency files.
Using the NAEP sample frame allowed the contractor to take advantage of edits already
made to the CCD files (for example eliminating administrative school districts from the
sample frame).

                                          12
Survey instruments used included 89 prevention programs; respondents were also able
to write in any programs omitted from those listed. The contractor received responses
from 91 percent of the districts included in the sample and 86 percent of schools.

The study conducted by the contractor to obtain data for this measure has some
limitations that are the result of both the research synthesis and survey data collections.
Despite significant efforts to be comprehensive, it is possible that the literature searches
used may not have identified some published studies on prevention programs and, as a
result, the number of research-based program may be understated.

Some other study limitations pertain to the quality of data collected via the surveys.
Recall problems and responses from less knowledgeable respondents in some schools
and districts (particularly among schools and districts that provided information late in
the collection period) may have affected the quality of data. Schools may have also
over-reported the prevention programs operating in their schools if respondents
confused the specific named program in the survey with other similarly named but
different programs.

Measure 8: The percentage of drug and violence prevention curriculum programs that
are implemented with fidelity. (SDFSC State Grants)

Table 8

FY 2004      FY 2005    FY 2006      FY 2007     FY 2008       FY 2008     FY 2009
 Actual       Actual     Actual       Actual      Target        Actual      Target
None        44.3        None         None        None         n/a         53.1

The measure. This measure examines the extent to which research-based curriculum
programs supported with SDFSC State Grant funds are implemented with fidelity. The
measure supports attainment of National Drug Control Strategy goals by focusing on
the quality of implementation of the research-based programs and practices supported
with SDFSC State Grants funds, and the corresponding likelihood that the programs will
reduce or prevent youth drug use. The 2005 data constitute the baseline for this
measure.

FY 2008 Performance Results. No FY 2008 target is in effect for this measure; data will
be collected in 2009 for this measure.

FY 2009 Performance Targets. A contract to collect data to implement this measure
could not be issued in time to permit data collection during FY 2008 as originally
scheduled. As a result, we established a 2009 target against the 2005 baseline that is a
linear extrapolation of a previously established FY 2008 target for this measure. The
next data collection is scheduled to collect information about programs implemented
during the 2008-2009 school year.


                                            13
Methodology. Baseline data for this measure were collected from a nationally
representative sample of schools under a contract supported by ED. Data were
collected in the fall of 2006, and reflected information about programs and practices
implemented during the 2004-2005 school year. The contractor developed a list of
research-based programs and compared information about programs and practices
being implemented with SDFSC State Grants funds with the list of research-based
program and practices. (See discussion for Measure 7)

The contractor then followed up with a subset of respondents to examine the extent to
which research-based programs and practices were implemented in a manner
consistent with implementation keys for individual programs (as determined by program
developers). The contractor focused its review on the 10 programs (from the list of 21
research-based programs) that were implemented most frequently by respondents in
the initial phase of the study.

The contractor mailed copies of questionnaires to principals and program implementers
to each school that reported operating at least one research-based program in the
response to the earlier survey. The response rate for the questionnaire supplied to
program implementers was 78 percent; the response rate for questionnaires completed
by principals was 70 percent.

The study developed program-specific measures of quality implementation for each of
the research-based programs identified by the study. The standards were based on
program developer’s specifications for individual programs. Aspects of implementation
considered included issues such as frequency of student participation; number of
lessons delivered; and topics covered. Based on applying these quality standards to
data supplied on the two questionnaires, the contractor identified the percentage of
research-based programs that were implemented according to the standards identified
by the program developer (which the study refers to as being implemented with
“fidelity”).

This aspect of the study has some limitations related to the application of the program-
specific standards used for assessing the quality of program implementation to
responses provided from respondents concerning their program implementation. Valid
measurement of quality of implementation required that a program developer’s program
specifications be applied to implementer reports on that specific program. In some
cases, responses raised questions about whether respondents were reporting on the
correct program. Study staff worked to confirm that implementers were reporting on the
correct program; in cases where the implementer reported on the wrong program, that
report was considered invalid and not included in the final data. If responses suggested
that the program implementer reported on the wrong program and confirmation could
not be made, those cases were also excluded from analyses.

Similar problems occurred for programs that had multiple components or different
versions that are implemented for different ages or grade levels. Study staff reviewed
program materials for different components or versions and worked to identify the

                                           14
program standards most closely related to the various components or versions. If a
meaningful standard for a component or measure could be developed, the case was
included in the analyses; if not, the program was omitted.

Limitations related to data quality from questionnaires also exist. Because a substantial
number of cases were ineligible for inclusion in the study analyses for the reasons
described above, the number of valid cases was reduced, leading in turn to decreased
precision in estimates and larger than expected standard errors and confidence
intervals. Similar recall problems caused by the gap between program implementation
and data collection (as discussed for the previous measure) may have also impacted
data quality. Finally, the quality of reports varied by the extent to which respondents
were in a position to observe actual implementation and intentionally bias reports.
Program implementers may have difficulty in providing objective information about
programs they are responsible for establishing. However, previous research using
similar measures suggests that this ”social desirability” bias is likely to be low.

                            Grants to Reduce Alcohol Abuse

Measure 9: The percentage of grantees whose target students show a measurable
decrease in binge drinking. (Grants to Reduce Alcohol Abuse Program – FY 2004,
2005 and FY 2007 cohorts – no new grants were awarded under this program in FY
2006.)

Table 9

Cohort    FY         FY         FY         FY         FY         FY         FY 2009
          2004       2005       2006       2007       2008       2008       Target
          Actual     Actual     Actual     Actual     Target     Actual
2004      n/a        n/a        50         pending    n/a        n/a        n/a
2005      n/a        n/a        n/a        65         75         pending    n/a
2007      n/a        n/a        n/a        n/a        set a      61.5       baseline +
                                                      baseline              25%(76.87)

The measure. This measure examines a key outcome for the Grants to Reduce Alcohol
Abuse (GRAA) program – reduction in binge drinking for the target population. While
the National Drug Control Strategy is focused most intensively on preventing the use of
controlled substances, the strategy does address the role of alcohol as a drug of choice
for teenagers. Data do suggest that early use of alcohol is more likely to result in heavy
later use of alcohol.

FY 2008 Performance Results. Actual performance data for the FY 2004 cohort will be
contained in final reports for these grant projects, which were due at the end of
December 2008. Data supplied by grantees in these reports will be aggregated by the
Department and be available in early 2009. Grantees from the FY 2005 cohort are
currently operating in no-cost extensions; generally, their final reports are due at the end
of 2009. Data will be aggregated and available in March 2010.

                                            15
We did not establish a FY 2008 target for the FY 2007 cohort because we were not
certain that grantees would be able to provide both baseline data and year one
performance data in their first annual performance report. However grantees were
generally able to provide these data, and we will use them to establish an FY 2009
target for this cohort. Performance for this measure for this cohort after a single year of
implementation was almost equal to that of the prior cohort after two years.

FY 2009 Performance Targets. We established an FY 2009 target for the FY 2007
cohort based on the performance of prior cohorts. Since this cohort achieved
performance levels after one year that were close to those met after two years by a prior
cohort, we plan to revisit the FY 2009 target for the FY 2007 cohort and enter the target
into ED’s software for recording measures, targets and actual results before the
deadline for revisions to FY 2009 targets.

Methodology. Data for this measure are collected by grantees and reported as part of
annual performance reports. If data for this measure are not available at the time that
performance reports are submitted, staff follow up with sites to attempt to obtain data for
the measure. Grantees that fail to provide data are not included in the tabulation of data
for the measures. Also, grantees that did not provide data for two consecutive project
years (so that we could determine if a decrease in binge drinking had occurred) are not
included in the aggregate data reported for the measure. Authorized representatives for
the grant site sign the annual performance report and, in doing so, certify that to the
best of the signer’s knowledge and belief, all data in the performance report are true
and correct and that the report fully discloses all known weaknesses concerning the
accuracy, reliability, and completeness of the data included. Generally, the Department
relies on the certification concerning data supplied by grantees and does not conduct
further reviews.

ED does not mandate data collection protocols or instruments for grantees. Grantees
select a survey item that reflects the concept of binge drinking, and collect and report
data about that survey item as part of their performance reports. As a result, data are
not comparable across grant sites, but individual grant sites are required to use the
same survey items across performance periods.

Applicants are not required to furnish baseline data as part of their applications. Data
supplied after year one are considered baseline data for the projects. Projects require
two years of data in order to determine if a decrease in binge drinking among target
students has occurred.

We have provided significantly increased guidance and technical assistance beginning
for the FY 2007 cohort, and believe that these efforts have produced data that are of
higher quality and more comparable across sites than those of previous cohorts.




                                            16
Measure 10: The percentage of grantees that show a measurable increase in the
percentage of target students who believe that alcohol abuse is harmful to their health.
(Grants to Reduce Alcohol Abuse – FY 2004, 2005, and 2007 cohorts)

Table 10

Cohort     FY        FY         FY         FY         FY         FY         FY
           2004      2005       2006       2007       2008       2008       2009
           Actual    Actual     Actual     Actual     Target     Actual     Target
2004       n/a       n/a        55.6       pending    n/a        n/a        n/a
2005       n/a       n/a        n/a        70         80         pending    n/a
2007       n/a       n/a        n/a        n/a        set a      69.2       baseline
                                                      baseline              + 25%
                                                                            (86.5)

The measure. This measure examines a key outcome for the Grants to Reduce Alcohol
Abuse (GRAA) program – perception of health risk for alcohol abuse among target
students. While the National Drug Control Strategy is focused most intensively on
preventing the use of controlled substances, the Strategy does address the role of
alcohol use as a drug of choice for teenagers. Data do suggest that changes in
perceptions about risks to health resulting from alcohol use are positively correlated with
reductions in alcohol use.

FY 2008 Performance Results. Actual performance data for the FY 2004 cohort will be
contained in final reports for these grant projects, which were due at the end of
December 2008. Data supplied by grantees in these reports will be aggregated by the
Department and be available in early 2009. Grantees from the FY 2005 cohort are
currently operating in no-cost extensions; generally, their final reports are due at the end
of 2009. Data will be aggregated and available in March 2010.

We did not establish a FY 2008 target for the FY 2007 cohort because we were not
certain that grantees would be able to provide both baseline data and year one
performance data in their first annual performance report. However grantees were
generally able to provide these data, and we will use them to establish an FY 2009
target for this cohort. Performance for this measure for this cohort after a single year of
implementation was almost equal to that of the prior cohort after two years.

FY 2009 Performance Targets. We had established an FY 2009 target for the FY 2007
cohort based on the performance of prior cohorts. Since this cohort achieved
performance levels after one year that were close to those met after two years by a prior
cohort, we plan to revisit the FY 2009 target for the FY 2007 cohort and enter any
revised target into ED’s software for recording measures, targets and actual results
before the deadline for revisions to FY 2009 targets.
Methodology. Data for this measure are collected by grantees and reported as part of
annual performance reports. If data for this measure are not available at the time that
performance reports are submitted, staff follow up with sites to attempt to obtain data for

                                            17
the measure. Grantees that fail to provide data are not included in the tabulation of data
for the measures. Also, grantees that did not provide data for two consecutive project
years (so that we could determine if an increase in the percentage of students who
believe that alcohol abuse is harmful to their health had occurred) are not included in
the aggregate data reported for the measure. Authorized representatives for the grant
site sign the annual performance report and, in doing so, certify that to the best of the
signers’ knowledge and belief, all data in the performance report are true and correct
and that the report fully discloses all known weaknesses concerning the accuracy,
reliability, and completeness of the data included. Generally, the Department relies on
the certification concerning data supplied by grantees and does not conduct further
reviews.

ED does not mandate data collection protocols or instruments for grantees. Grantees
select a survey item that reflects the concept of binge drinking, and collect and report
data about that survey item as part of performance reports. As a result, data are not
comparable across grant sites, but individual grant sites are required to use the same
survey items across performance periods.

Applicants are not required to furnish baseline data as part of their applications. Data
supplied after year one are considered baseline data for the projects. Projects require
two years of data in order to determine if a decrease in binge drinking among target
students has occurred.

We have provided significantly increased guidance and technical assistance beginning
for the FY 2007 cohort, and believe that these efforts have produced data that are of
higher quality and more comparable across sites than those of previous cohorts.

Measure 11: The percentage of grantees that show a measurable increase in the
percentage of target students who disapprove of alcohol abuse. (Grants to Reduce
Alcohol Abuse – FY 2004 and FY 2005 Cohorts)

Table 11

Cohort     FY        FY         FY        FY         FY         FY         FY
           2004      2005       2006      2007       2008       2008       2009
           Actual    Actual     Actual    Actual     Target     Actual     Target
2004       n/a       n/a        66.7      pending    n/a        n/a        n/a
2005       n/a       n/a        n/a       71         87         pending    n/a
2007       n/a       n/a        n/a       n/a        set a      69.2       baseline
                                                     baseline              + 25 %
                                                                           (86.5)

The measure. This measure examines a key outcome for the Grants to Reduce Alcohol
Abuse (GRAA) program – perception of health risk for alcohol abuse among target
students. While the National Drug Control Strategy is focused most intensively on the
preventing the use of controlled substances, the Strategy does address the role of

                                            18
alcohol use as a drug of choice for teenagers. Data do suggest that increases in the
percentage of target students who believe that alcohol abuse is not socially acceptable
are associated with declines in consumption of alcohol.

FY 2008 Performance Results. Actual performance data for the FY 2004 cohort will be
contained in final reports for these grant projects, which were due at the end of
December 2008. Data supplied by grantees in these reports will be aggregated by the
Department and be available in early 2009. Grantees from the FY 2005 cohort are
currently operating in no-cost extensions; generally, their final reports are be due at the
end of 2009. Data will be aggregated and available in March 2010.

We did not establish a FY 2008 target for the FY 2007 cohort because we were not
certain that grantees would be able to provide both baseline data and year one
performance data in their first annual performance report. However grantees were
generally able to provide these data, and we will use them to establish an FY 2009
target for this cohort. Performance for this measure for this cohort after a single year of
implementation was almost equal to that of the prior cohort after two years.

FY 2009 Performance Targets. We had established an FY 2009 target for the FY 2007
cohort based on the performance of prior cohorts. Since this cohort achieved
performance levels after one year that were close to those met after two years by a prior
cohort, we plan to revisit the FY 2009 target for the FY 2007 cohort and enter the target
into ED’s software for recording measures, targets and actual results before the
deadline for revisions to FY 2009 targets.
Methodology. Data for this measure are collected by grantees and reported as part of
annual performance reports. If data for this measure are not available at the time that
performance reports are submitted, staff follow up with sites to attempt to obtain data for
the measure. Grantees that fail to provide data are not included in the tabulation of data
for the measures. Also, grantees that did not provide data for two consecutive project
years (so that we could determine if an increase in the percentage of students that
disapprove of alcohol abuse had occurred) are not included in the aggregate data
reported for the measure. Authorized representatives for the grant site sign the annual
performance report, and in doing so, certify that to the best of the signer’s knowledge
and belief, all data in the performance report are true and correct and that the report
fully discloses all known weaknesses concerning the accuracy, reliability, and
completeness of the data included. Generally, the Department relies on the certification
concerning data supplied by grantees and does not conduct further reviews.

ED does not mandate data collection protocols or instruments for grantees. Grantees
select a survey item that reflects the concept of binge drinking, and collect and report
data about that survey item as part of performance reports. As a result, data are not
comparable across grant sites, but individual grant sites are required to use the same
survey items across performance periods.

Applicants are not required to furnish baseline data as part of their applications. Data
supplied after year one are considered baseline data for the projects. Projects require

                                            19
two years of data in order to determine if a decrease in binge drinking among target
students has occurred.

We have provided significantly increased guidance and technical assistance beginning
for the FY 2007 cohort, and believe that these efforts have produced data that are of
higher quality and more comparable across sites than those of previous cohorts.



Assertions
                            Performance Reporting System

The Department of Education has a system in place to capture performance information
accurately and that system was properly applied to generate the performance data in
this report. In instances in which data are supplied by grantees as part of required
periodic performance reports, the data that are supplied are accurately reflected in this
report.

Data related to the drug control programs included in this Performance Summary
Report for Fiscal Year 2008 are recorded in the Department of Education’s software for
recording performance data and are an integral part of our budget and management
processes.
                   Methodology for Establishing Performance Targets

The methodology described in the Performance Summary Report for Fiscal Year 2008
to establish performance targets for the current year is reasonable given past
performance and available resources.

             Performance Measures for Significant Drug Control Activities

The Department of Education has established at least one acceptable performance
measure for each Drug Control Decision Unit identified in its Detailed Accounting of
Fiscal Year 2008 Drug Control Funds.

Criteria for Assertions

                                          Data

No workload or participant data support the assertions provided in this report. Sources
of quantitative data used in the report are well documented. These data are the most
recently available and are identified by the year in which the data was collected.

                               Other Estimation Methods

No estimation methods other than professional judgment were used to make the
required assertions. When professional judgment was used, the objectivity and strength

                                           20
of those judgments were explained and documented. Professional judgment was used
to establish targets for programs until data from at least one grant cohort were available
to provide additional information needed to set more accurate targets. We routinely re-
evaluate targets set using professional judgment as additional information about actual
performance on measures becomes available.

                                   Reporting Systems

Reporting systems that support the above assertions are current, reliable, and an
integral part of the Department of Education’s budget and management processes.
Data collected and reported for the measures discussed in this report are stored in the
Department of Education’s Visual Performance System (VPS). The VPS includes
appropriate disclosures about data quality issues associated with measures. Data from
the VPS are used in developing annual budget requests and justifications, and in
preparing reports required under the Government Performance and Results Act of
1993.




                                            21
                            UNITED STATES DEPARTMENT OF EDUCATION
                                   OFFICE OF INSPECTOR GENERAL


                                                   January 30, 2009

MEMORANDUM


To:                 William Modzeleski
                    Acting Assistant Deputy Secretary
                    Office of Safe and Drug-Free Schools


From:               Keith West /s/
                    Assistant Inspector General for Audit Services

Subject:            Office of Inspector General’s Independent Report on the U.S. Department
                    of Education’s Performance Summary Report for Fiscal Year 2008, dated
                    January 28, 2009

Attached is our authentication of management’s assertions contained in the U.S.
Department of Education’s Performance Summary Report for Fiscal Year 2008, dated
January 28, 2009, as required by section 705(d) of the Office of National Drug Control
Policy (ONDCP) Reauthorization Act of 1998 (21 U.S.C. § 1704(d)).

Our authentication was conducted in accordance with the guidelines stated in the Office
of National Drug Control Policy Circular: Drug Control Accounting, dated May 1, 2007.

If you have any questions or wish to discuss the contents of this authentication, please
contact Michele Weaver-Dugan, Director, Operations Internal Audit Team, at (202) 245-
6941.


Attachment




 The Department of Education's mission is to promote student achievement and preparation for global competitiveness by fostering
                                       educational excellence and ensuring equal access.
                            UNITED STATES DEPARTMENT OF EDUCATION
                                   OFFICE OF INSPECTOR GENERAL


                                                   January 30, 2009



Office of Inspector General’s Independent Report on the U.S. Department of Education’s
Performance Summary Report for Fiscal Year 2008, dated January 28, 2009

We have reviewed management’s assertions contained in the accompanying Performance
Summary Report, titled Department of Education Performance Summary Report for
Fiscal Year 2008, dated January 28, 2009. The U.S. Department of Education’s
management is responsible for the Performance Summary Report and the assertions
contained therein.

Our review was conducted in accordance with attestation standards established by the
American Institute of Certified Public Accountants. A review is substantially less in
scope than an examination, the objective of which is the expression of an opinion on
management’s assertions. Accordingly, we do not express such an opinion.

We performed review procedures on the “Performance Summary Information,”
“Assertions,” and “Criteria for Assertions” contained in the accompanying Performance
Summary Report. In general, our review procedures were limited to inquiries and
analytical procedures appropriate for our review engagement. We did not perform
procedures related to controls over the reporting system noted in the attached report.

Based on our review, nothing came to our attention that caused us to believe that
management’s assertions, contained in the accompanying Performance Summary Report,
are not fairly stated in all material respects, based upon the Office of National Drug
Control Policy Circular: Drug Control Accounting, dated May 1, 2007.




                                                     Keith West /s/
                                                     Assistant Inspector General for Audit Services




 The Department of Education's mission is to promote student achievement and preparation for global competitiveness by fostering
                                       educational excellence and ensuring equal access.