oversight

The Reading First Program's Grant Application Process.

Published by the Department of Education, Office of Inspector General on 2006-09-22.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

   The Reading First Program’s Grant Application Process




                        FINAL INSPECTION REPORT




                                 ED-OIG/I13-F0017
                                  September 2006




Our mission is to promote the                       U.S Department of Education
efficiency, effectiveness, and                      Office of Inspector General
integrity of the Department's                       Washington, D.C.
programs and operations.
     Statements that managerial practices need improvements, as well as other
  conclusions and recommendations in this report represent the opinions of the
 Office of Inspector General. Determinations of corrective action to be taken will
          be made by the appropriate Department of Education officials.


   In accordance with the Freedom of Information Act (5 U.S.C. § 552), reports
issued by the Office of Inspector General are available, if requested, to members of
   the press and general public to the extent information contained therein is not
                          subject to exemptions in the Act.
                                              TABLE OF CONTENTS


EXECUTIVE SUMMARY ...........................................................................................................1

BACKGROUND ............................................................................................................................4

INSPECTION RESULTS .............................................................................................................6

          FINDING 1A –                  The Department Did Not Select the Expert Review
                                        Panel in Compliance With the Requirements of
                                        NCLB................................................................................................6

          FINDING 1B –                  While Not Required to Screen for Conflicts of
                                        Interest, the Screening Process the Department
                                        Created Was Not Effective .............................................................7

          FINDING 2A –                  The Department Did Not Follow Its Own Guidance
                                        For the Peer Review Process ..........................................................8

          FINDING 2B –                  The Department Awarded Grants to States
                                        Without Documentation That the Subpanels
                                        Approved All Criteria...................................................................11

          FINDING 3 –                   The Department Included Requirements in the
                                        Criteria Used by the Expert Review Panels That
                                        Were Not Specifically Addressed in NCLB ................................12

          FINDING 4 –                   In Implementing the Reading First Program,
                                        Department Officials Obscured the Statutory
                                        Requirements of the ESEA; Acted in
                                        Contravention of the GAO Standards for Internal
                                        Control in the Federal Government; and Took
                                        Actions That Call Into Question Whether They
                                        Violated the Prohibitions Included in the DEOA ......................13

RECOMMENDATIONS.............................................................................................................27

DEPARTMENT COMMENTS ..................................................................................................28

OBJECTIVES, SCOPE, AND METHODOLOGY ..................................................................33
Final Report
ED-OIG/I13-F0017                                                                             Page 1



                                EXECUTIVE SUMMARY


The objectives of our inspection were to:

   1. Determine if the Department of Education (Department, ED, or USDE) selected the
      expert review panel in accordance with the No Child Left Behind Act of 2001 (NCLB),
      Section 1203(c) and if the Department adequately screened the panel members for
      possible conflict of interest issues;

   2. Determine if the expert review panel adequately documented its reasons for stating that
      an application was unready for funding; and

   3. Determine if the expert review panel reviewed the applications in accordance with
      established criteria and applied the criteria consistently.

The selection of the expert review panel was not in compliance with the law because the
Department failed to ensure that each State application was reviewed by a properly constituted
panel.

Although not required, the Department developed a process to screen expert review panelists for
conflicts of interest; however, the Department’s process was not effective. We identified six
panelists whose resumes revealed significant professional connections to a teaching methodology
that requires the use of a specific reading program. The Department did not identify any of these
connections in its conflict of interest screening process; therefore, it would not have been in a
position to deal with the potential conflict raised by these connections should a State have
included this program in its application.

The expert review panel adequately documented its reasons for stating that an application was
unready for funding; however, the Department substituted a Department-created report for the
panel’s comments. As a result, the Department did not follow its own guidance for the peer
review process. Therefore, States did not have the benefit of the expert review panel’s
comments and were not always able to quickly or effectively address problems in their
applications. Additionally, we found that five of the State applications we reviewed were funded
without documentation that they met all of the criteria for approval. The Department has not
provided any documentation that would indicate the subpanels approved the final applications
for these States.

The expert review panel appears to have reviewed the applications in accordance with criteria
developed by the Department and applied the criteria consistently; however, the criteria
developed by the Department included language that was not based on the statutory language.
As a result, State applications were forced to meet standards that were not required by the statute.
Final Report
ED-OIG/I13-F0017                                                                           Page 2

In the course of answering our three objectives, we found that Department officials obscured the
statutory requirements of the Elementary and Secondary Education Act of 1965 (ESEA), as
amended by NCLB; acted in contravention of the Government Accountability Office’s (GAO)
Standards for Internal Control in the Federal Government; and took actions that call into
question whether they violated the prohibitions included in the Department of Education
Organization Act (DEOA). The DEOA at §3403(b) prohibits Department officials from
exercising any direction, supervision, or control over the curriculum or program of instruction of
any educational institution, school, or school system.

Specifically, we found that the Department:

   •   Developed an application package that obscured the requirements of the statute;
   •   Took action with respect to the expert review panel process that was contrary to the
       balanced panel composition envisioned by Congress;
   •   Intervened to release an assessment review document without the permission of the entity
       that contracted for its development;
   •   Intervened to influence a State’s selection of reading programs; and
   •   Intervened to influence reading programs being used by local educational agencies
       (LEAs) after the application process was completed.

These actions demonstrate that the program officials failed to maintain a control environment
that exemplifies management integrity and accountability.

We recommend that the Assistant Secretary for the Office of Elementary and Secondary
Education (OESE):

   1) Develop internal management policies and procedures for OESE program offices that
      address when legal advice will be solicited from the Office of the General Counsel
      (OGC) and how discussions between OGC and the program staff will be resolved to
      ensure that programs are managed in compliance with applicable laws and regulations.
   2) In consultation with OGC, evaluate OESE’s processes for assessing potential conflict of
      interest questions, when a panel review process is used, and make those improvements
      necessary to strengthen the processes.
   3) Review all Reading First applications to determine whether all criteria for funding have
      been met.
   4) Review the management and staff structure of the Reading First program office and make
      changes, as appropriate, to ensure that the program is managed and implemented
      consistent with the statutory requirements of NCLB.
   5) Request that OGC develop guidance for OESE on the prohibitions imposed by §3403(b)
      of the Department of Education Organization Act.
   6) When similar new initiatives are approved by Congress, rely upon an internal advisory
      committee, which includes representatives from other OESE programs, OGC, and the
      Department’s Risk Management Team, to provide feedback on program implementation
      issues and ensure coordination in the delivery of similar or complimentary Department
      programs.
Final Report
ED-OIG/I13-F0017                                                                           Page 3


   7) Rely upon the internal advisory committee to:
          a. Determine whether the implementation of Reading First harmed the Federal
             interest and what course of action is required to resolve any issues identified; and
          b. Ensure that future programs, including other programs for which the Department
             is considering using Reading First as a model, have internal controls in place to
             prevent similar problems from occurring.
   8) Convene a discussion with a broad range of state and local education representatives to
      discuss issues with Reading First as part of the reauthorization process.
Final Report
ED-OIG/I13-F0017                                                                          Page 4



                                     BACKGROUND


The ESEA, as amended by NCLB on January 8, 2002, established the Reading First program.
The purpose of NCLB is to “close the achievement gap with accountability, flexibility, and
choice, so that no child is left behind.”

Title 1, Part B, Section 1201 of the ESEA provides five purposes for the Reading First program:

   1) To provide assistance to State educational agencies (SEAs) and LEAs in establishing
      reading programs for students in kindergarten through grade 3 that are based on
      scientifically based reading research (SBRR), to ensure that every student can read at
      grade level or above not later than the end of grade 3.
   2) To provide assistance to SEAs and LEAs in preparing teachers, including special
      education teachers, through professional development and other support, so the teachers
      can identify specific reading barriers facing their students and so the teachers have the
      tools to effectively help their students learn to read.
   3) To provide assistance to SEAs and LEAs in selecting or administering screening,
      diagnostic, and classroom-based instructional reading assessments.
   4) To provide assistance to SEAs and LEAs in selecting or developing effective
      instructional materials (including classroom-based materials to assist teachers in
      implementing the essential components of reading instruction), programs, learning
      systems, and strategies to implement methods that have been proven to prevent or
      remediate reading failure within a State.
   5) To strengthen coordination among schools, early literacy programs, and family literacy
      programs to improve reading achievement for all children.

Title 1, Part B, Section 1208(6) of the ESEA defines SBRR as research that:

  (A) applies rigorous, systematic, and objective procedures to obtain valid knowledge relevant
      to reading development, reading instruction, and reading difficulties; and
  (B) includes research that —
          (i) employs systematic, empirical methods that draw on observation or experiment;
          (ii) involves rigorous data analyses that are adequate to test the stated hypotheses
                and justify the general conclusions drawn;
          (iii) relies on measurements or observational methods that provide valid data across
                evaluators and observers and across multiple measurements and observations;
                and
          (iv) has been accepted by a peer-reviewed journal or approved by a panel of
                independent experts through a comparably rigorous, objective, and scientific
                review.
Final Report
ED-OIG/I13-F0017                                                                            Page 5

The ESEA does not advocate any particular reading program, assessment, or other product. In
fact, Section 9527(b) of the ESEA prohibits the Department from endorsing, approving, or
sanctioning any curriculum.

Title 1, Part B, Section 1002(b)(1) of the ESEA authorized an appropriation for Reading First of
$900,000,000 for fiscal year 2002 and “sums as may be necessary for each of the 5 succeeding
fiscal years.” The appropriations for fiscal years 2003, 2004, 2005, and 2006 were
$993,500,000, $1,023,923,000, $1,041,600,000, and $1,029,234,000, respectively.

Reading First funds are allotted to SEAs by formula according to the proportion of children aged
5 to 17 who reside within the State and are from families with incomes below the poverty line.
SEAs submit applications to the Department to receive Reading First funding. SEA applications
are reviewed by an expert review panel and are required to meet all statutory requirements before
being awarded funds.

The Department’s Office of Elementary and Secondary Education administers the Reading First
program. Two Department officials, the Reading First Director and an Education Program
Specialist (the Reading First Director’s assistant), were responsible for administering the
Reading First application process and, through the date of this report, continued to administer all
facets of the Reading First program.
Final Report
ED-OIG/I13-F0017                                                                                               Page 6



                                        INSPECTION RESULTS


FINDING 1A – The Department Did Not Select the Expert Review Panel in
             Compliance With the Requirements of NCLB

Our objective was to determine if the Department selected the expert review panel in accordance
with Title 1, Part B, Section 1203(c) of the ESEA, which specifically describes the panel
selection process. Section 1203(c)(2)(A) states that the Secretary, in consultation with the
National Institute for Literacy (NIFL), shall convene a panel to evaluate applications and that, at
a minimum, the panel shall include: three individuals selected by the Secretary, three individuals
selected by NIFL, three individuals selected by the National Research Council of the National
Academy of Sciences (NAS), and three individuals selected by the National Institute of Child
Health and Human Development (NICHD). We have determined that each of the four
organizations nominated at least three individuals to serve on the expert review panel; however,
the Department failed to ensure that each State application was reviewed by a properly
constituted panel.

Section 1203(c)(2)(C) requires a panel to recommend grant applications to the Secretary for
funding or for disapproval. After selecting the panelists, the Department created subpanels made
up of five panelists each to review the State applications and recommend either approval or
disapproval to the Secretary. None of the subpanels possessed adequate representation from
each of the organizations identified under Section 1203(c)(2)(A) of the Act.

The Department created a total of 16 subpanels to review the State applications. A majority of
the panelists were nominated by the Department for 15 of the 16 subpanels; and 7 of the 16
subpanels consisted entirely of Department-selected panelists. None of the subpanels included a
representative from each of the nominating organizations and there is no indication that the
subpanels ever met as one large panel to review the State applications and/or recommend
approval or disapproval to the Secretary.

Prior to forming these subpanels, a Department official expressed concern that the use of
subpanels would not be in compliance with the law. As a result, OGC and high-level
Department officials, including the Assistant Secretary for OESE at the time,1 approved a plan
for the Department to create a 12-member “Advisory and Oversight Panel” that would consist of
three individuals selected by the Department, three individuals selected by NIFL, three
individuals selected by NAS, and three individuals selected by NICHD, as required by the Act.
The Advisory and Oversight Panel’s duties would include examining the progress of the
subpanels, reviewing the recommendations of the subpanels, and making the final funding
recommendation to the Secretary, thus ensuring a common, high level of quality and consistency
across the subpanels. Although the Assistant Secretary for OESE and OGC officials agreed

1
  This official left the Department in January of 2003 and for purposes of consistency and clarity will be referred to
in this section simply as “the Assistant Secretary for OESE.”
Final Report
ED-OIG/I13-F0017                                                                            Page 7

upon this approach, the Advisory and Oversight Panel was never created. To date, no one in the
Department has offered an explanation of why this was not done.

Section 1203(c)(1) states: “The Secretary shall approve an application of a State educational
agency under this section only if such application meets the requirements of this section.”
Because the Department did not meet the requirements at Section 1203(c)(2)(A), it raises the
question of whether any of the applications were approved in compliance with the law.

FINDING 1B – While Not Required to Screen for Conflicts of Interest, the
             Screening Process the Department Created Was Not Effective

Our objective was to determine if the Department adequately screened the panel members for
possible conflict of interest issues. NCLB does not require panel members to be screened for
conflict of interest. However, the Department decided to screen potential panelists for conflicts
of interest and developed and implemented a process to do so. That process was not effective.

To assist with developing the screening process, an OGC ethics attorney provided sample
conflict of interest questions to the Reading First Director’s assistant. These questions were
based on questions used to screen for conflicts in the “Agreement for Grant Application
Reviewers Who Receive Compensation” used for discretionary grants.

The Reading First office incorporated most of these sample questions into the conflict of interest
form it provided to potential Reading First panelists. This form asked each individual to provide
his or her personal information, dates of availability, and answers to six questions. Five of the
six questions focused on conflicts related to States’ applications and/or States’ Reading First
programs. The other question focused on conflicts related to the potential panelists’ financial
interest in commercial products.

The Department’s conflict of interest form did not incorporate one of the sample questions
provided by an OGC ethics attorney. This question was: “Are you aware of any other
circumstances that might cause someone to question your impartiality in serving as a reviewer
for this competition?”

An OGC ethics attorney informed us that the screening process was designed to exclude
individuals who had financial connections to products or programs or who had the appearance of
a conflict of interest. This attorney also informed us that OGC’s review of potential conflicts
was informal and that OGC’s role was to provide guidance rather than decisions. We did
determine that when potential panelists identified specific connections to programs, assessments,
or textbooks on the conflict of interest form, the OGC ethics attorney provided appropriate
guidance, and the Department took appropriate action.

The potential panelists also provided the Department with resumes. The Department did not
review the resumes as part of the conflict of interest screening process. We reviewed the
resumes of 25 of the approved panelists and identified six panelists whose resumes revealed
significant professional connections to a teaching methodology that requires the use of a specific
reading program. The Department did not identify any of these connections in its conflict of
Final Report
ED-OIG/I13-F0017                                                                            Page 8

interest screening process; therefore, it would not have been in a position to deal with the
potential conflict raised by these professional connections should a State have included this
program in its application.

FINDING 2A – The Department Did Not Follow Its Own Guidance For the Peer
             Review Process

Our objective was to determine if the expert review panel adequately documented its reasons for
stating that an application was unready for funding. Section 1203(c) of the ESEA, under the
heading “Peer Review,” states that the panel will recommend grant applications to the Secretary
for funding or disapproval. The Department created the Reviewer Guidance for the Reading
First Program (Reviewer Guidance), which describes the process by which panelists will review
applications and provide their comments. The Reviewer Guidance, which the Department
provided to panelists, states that it is the reviewer’s responsibility to provide a rating for each
review criterion and constructive strength and weakness comments on the Technical Review
Form. The guidance states that the panel chair will complete an additional summary sheet,
called the Panel Chair Summary, which will reflect a consensus rating and supporting comments
for each criterion. The guidance also states that the Panel Chair Summary will provide an
overall consensus recommendation for approval or disapproval of the application.

The Guidance for the Reading First Program (Reading First Guidance), provided to States in
April 2002, states that the expert review panel will “recommend clarifications or changes
deemed necessary to improve the likelihood of the plan’s success.” The Reading First Guidance
also states that SEAs “will have an opportunity to address the issues and concerns raised by the
expert panel reviewers.”

The panelists adequately documented their reasons for stating that an application was unready
for funding. The panelists recorded their individual comments on the Technical Review Forms,
and then met to discuss these comments. The panel chair then entered a consensus rating on a
Panel Chair Summary, which was submitted to the Department’s Reading First office. The Panel
Chair Summary appeared to contain constructive comments to support the panel’s ratings.

While the Panel Chair Summaries appeared to provide constructive comments, the impact of the
expert review panelists’ comments on State revisions is uncertain because of actions taken by the
Department’s Reading First office. After the panel chair submitted the Panel Chair Summaries
to the Reading First office, the Reading First Director and his assistant created what they called
an “Expert Review Team Report.” This report was provided to the States. No other documents
reflecting the expert review panel’s comments were provided to the States. The Department did
not explain this practice in the Reviewer Guidance or in the Reading First Guidance.

We have not found any documentation that the Department informed panelists that the Reading
First Director and his assistant would write the report sent to SEAs. In fact, in e-mails to
panelists the Reading First Director wrote that panelists “should look very closely at the exact
panel comments made on the panel chair summary form as these comments drove the SEA’s
resubmission.” Likewise, we have not found any documentation that the Department informed
Final Report
ED-OIG/I13-F0017                                                                                               Page 9

States that they would receive a Department-written report rather than the expert review panel’s
direct comments.

The Reviewer Guidance states: “the conference call between the panelists and the SEA that will
take place after the review of the SEA’s application has been established so that the State may
receive direct feedback from the expert review panel.” In actuality, only the Reading First
Director and his assistant conducted these calls. By conducting these conferences and writing
the document that was sent to the SEA, the Reading First Director and his assistant cut off any
direct contact between States and the expert review panelists and effectively controlled the
feedback States received on their applications.

Prior to the first round of conference calls with States, the Reading First Director e-mailed the
then Assistant Secretary for OESE2 seeking guidance on the inclusion of panel chairs on the
conference calls. The Reading First Director stated that “it’s generally been mentioned to the
States that they would hear directly from the Panel” but that the Department would “lose a bit of
control” and the Panel Chair might say things that would “*complicate* matter [sic]” if included
on the conference calls. In an earlier e-mail to the Assistant Secretary for OESE, the Reading
First Director stated that “in remarks to groups…or face-to-face meetings about what the Review
Panel will/won’t accept the opportunities for BOLDNESS and, perhaps, extralegal requirements
are many.” [Emphasis in original.] We have not found any documentation that the Department
informed panel chairs that they would not participate in the conference calls as outlined in the
Reviewer Guidance. The Reading First Director’s assistant stated that the expert review
panelists were not included in the conference calls because it was too difficult to schedule.

According to the Reading First Director, he and his assistant created the Expert Review Team
Reports to give States a distilled, organized version of the panel’s comments that would show
them which areas they needed to address. However, we found the Department’s Expert Review
Team Reports were not always accurate representations of the expert review panelists’
comments. The Reading First Director and his assistant changed panelists’ comments, left off
others, and added comments of their own. In a number of cases, the Department generalized or
omitted specific questions or suggestions. In other situations the Department’s Expert Review
Team Report exaggerated or misstated the panelists’ concerns.

The following are specific examples of these practices from the twelve State application files that
we reviewed.

Nevada
Nevada submitted its application five times prior to receiving approval. The Panel Chair
Summary for Nevada’s first submission suggested that the State access A Consumer’s Guide to
Evaluating a Core Reading Program (Consumer’s Guide), a guide for evaluating programs
authored by Edward J. Kame’enui and Deborah C. Simmons of the University of Oregon. The
Panel Chair Summary for Nevada’s third submission suggested that the State seek technical
assistance on instructional assessments. The Panel Chair Summary for Nevada’s fourth
submission referred the State to the website at the University of Oregon, which contains a review

2
  This official left the Department in January of 2003 and for purposes of consistency and clarity will be referred to
in this section simply as “the Assistant Secretary for OESE.”
Final Report
ED-OIG/I13-F0017                                                                            Page 10

of assessments. The Department’s Expert Review Team Report corresponding to each
submission omitted these comments.

The author of Nevada’s Reading First application, Joan Taylor, informed us that the
Department’s Expert Review Team Report was not specific about what the State needed to fix.
She stated that the Reading First Director and his assistant worked with her through a series of
conference calls to “try and figure out” what the subpanel wanted. Taylor informed us that she
finally asked if there was anything more in writing and was told by the Reading First Director
and his assistant that there was nothing. Taylor stated that she felt like she had to guess at what
the subpanel wanted and when she did not know what else to do she asked for technical
assistance. Taylor informed us that the Reading First Director and his assistant suggested that
she look at the Oregon website during a phone conference. Taylor stated that she went to the
Oregon website and found the Consumer’s Guide.

Taylor was not aware that the document she was discussing with the Reading First Director and
his assistant was in fact a Department-written summary of the subpanel’s comments and told us
that she would have wanted to see the actual subpanel comments. We provided her with two of
the Panel Chair Summaries and asked her to compare them to the corresponding Department’s
Expert Review Team Reports. Taylor stated that the additional documents “would have been
very helpful” and “would have saved [her] lots of time” during the application process.

New York
New York submitted its application three times prior to receiving funding. The Department’s
Expert Review Team Report for New York’s first submission did not include specific concerns
that the subpanel had about 8 of the 25 criteria. It appears as though the State was not aware of
these concerns and did not address them in its resubmission because the subpanel repeated some
of these concerns during its second review. The panel chair also stated in the Panel Chair
Summary that the assessment, the Woodcock Reading Mastery Test (WRMT), would be
inappropriate. The Reading First Director and his assistant wrote in the Department’s Expert
Review Team Report that only one of WRMT’s subtests would be appropriate.

Georgia
Georgia submitted its application three times prior to receiving approval. The Department’s
Expert Review Team Report for Georgia’s first submission failed to adequately summarize the
subpanel’s comments about instructional assessments and programs. The Department’s Expert
Review Team Report included the comment that “the review team expressed great concern that,
according to the budget…purchasing materials for classroom libraries is a higher priority than
purchasing and implementing core reading program materials.” This comment was not in the
Panel Chair Summary. The Department’s Expert Review Team Report for the next submission
omitted the subpanel’s request for more information on how the Dynamic Indicators of Basic
Early Learning Skills (DIBELS) assessment could be used for progress monitoring. Throughout
the Georgia application review process, the Department’s Expert Review Team Report used
comments from individual panelists’ Technical Review Forms that were not mentioned in the
Panel Chair Summary as representing the consensus of the subpanel.
Final Report
ED-OIG/I13-F0017                                                                         Page 11

Wisconsin
Wisconsin submitted its application four times prior to receiving approval. The Panel Chair
Summaries for Wisconsin’s submissions often included comments that ran several pages for each
criterion. One Panel Chair Summary was 88 pages long while the Department’s Expert Review
Team Report was only 4 pages and failed to capture most of what was prepared by the panel
chair. Further, the Department’s Expert Review Team Report often did not include any of the
panel chair’s specific concerns and simply restated the application criteria requirements instead.

Virginia
Virginia submitted its application three times prior to receiving approval. The Department’s
Expert Review Team Report for Virginia’s second submission left out a comment from the Panel
Chair Summary that the Partnership for Achieving Successful Schools (PASS) model was
inadequately related to SBRR and simply stated that the corresponding criteria did not meet
standard. In the third Panel Chair Summary, the rating for this criterion was again “does not
meet standard” because this issue had not been addressed. The panel chair wrote: “PASS model
requires justification…in its alignment [with] SBRR – as we requested last time.” Virginia’s
final application includes an addendum stating that PASS schools in Virginia receiving funds
through Reading First will be required to replace their existing programs with comprehensive
SBRR programs.

North Dakota
North Dakota submitted its application five times prior to receiving approval. The Department’s
Expert Review Team Reports for North Dakota’s submissions contained generalized comments
that did not relate to those in the corresponding Panel Chair Summaries. The Department’s
Expert Review Team Reports did not include many of the specific concerns provided in the
Panel Chair Summaries.

The Department, by substituting the expert review panel’s comments with a Department-created
report, did not follow its own guidance for the peer review process. States were not always able
to quickly or effectively address problems in their applications because they did not have the
benefit of the expert review panel’s comments. This increased the resources required by States
to apply and by the federal government to review the applications. As a result, not only was the
application review process not effectively and efficiently carried out but not all recommendations
for improvement were relayed to the States.

FINDING 2B – The Department Awarded Grants to States Without Documentation
             That the Subpanels Approved All Criteria

Of the 12 Reading First grant application files that we reviewed, it does not appear that the
subpanels approved the final applications for four States and one territory: Connecticut, Nevada,
New York, Virginia, and Puerto Rico. The Panel Chair Summaries for each final application
provided an overall rating of “Disapproval.” The Department has not provided any
documentation indicating the subpanels approved the final applications for these five applicants.

For example, the Panel Chair Summary for New York’s final application provided a rating of
“Does Not Meet Standard” for three criteria. The three criteria were Instructional Assessments,
Final Report
ED-OIG/I13-F0017                                                                            Page 12

Coherence, and Process for Awarding Subgrants. In the Instructional Assessments and
Coherence sections, the subpanel noted inconsistencies between the application narrative and the
subgrant request for proposal rubric. In the “Process for Awarding Subgrants” section, the
subpanel recommended that the New York State Education Department (NYSED) either change
the minimum cut-off score from 75 to a minimum cut-off score of 80 or include a sentence
addressed to NYSED’s expert reviewers and LEA applicants that unless all of the bulleted points
for each category were addressed, the category could not be judged to have met standard. The
Department has not provided additional documentation to show that these issues were resolved
prior to the Reading First grant being awarded. In fact, NYSED’s application, posted as
approved on its website, did not address any of the recommendations made in the Panel Chair
Summary.

As a result, some applicants were funded without documentation that they met all of the criteria
for approval raising a question of whether these States should have been funded.

FINDING 3 –        The Department Included Requirements in the Criteria Used by the
                   Expert Review Panels That Were Not Specifically Addressed in
                   NCLB

Our objective was to determine if the expert review panel reviewed the applications in
accordance with established criteria and applied the criteria consistently. The Department
instructed the panelists to use the Reading First: Criteria for Review of State Applications
(Reading First Criteria) when rating the 25 criteria in each State application. Each criterion was
broken up into requirements for a rating of ‘Exemplary,’ ‘Meets Standard,’ and ‘Does Not Meet
Standard.’ The Reading First Criteria states “the ‘Meets Standard’ column describes the
conditions that reviewers will expect all State applications to meet,” while the “‘Exemplary’
column describes conditions that, when met in addition to those listed under ‘Meets Standard,’
would be expected to result in the highest quality Reading First programs.”

Additionally, the Reviewer Guidance states: “The purpose of the review criteria is to ensure that
all applications meet the required standards. Panelists are to use only these criteria in evaluating
applications.” The Technical Review forms and Panel Chair Summary forms, which the expert
review panelists used when reviewing State applications, included the same ‘Meets Standard’
conditions as the Reading First Criteria.

Based on our review of 12 State applications, it appears the expert review panelists reviewed the
applications in accordance with the criteria provided to them and applied the criteria consistently.
The criteria, however, included conditions that were not specifically required in the statute. For
example, the ‘Meets Standard’ column of criterion IV(A), Key Reading First Classroom
Characteristics, required State applications to meet three conditions that were not included in the
statute:

   •   Condition 1b required, “Coherent instructional design that includes explicit instructional
       strategies, coordinated instructional sequences, ample practice opportunities, and aligned
       student materials[.]”
Final Report
ED-OIG/I13-F0017                                                                           Page 13

   •   Condition 1d required, “Protected, dedicated block of time for reading instruction[.]”

   •   Condition 1f required, “Small group instruction as appropriate to meet student needs,
       with placement and movement based on ongoing assessment[.]”

In a document titled, “Pre-reading notes for [the Assistant Secretary for OESE]” (Pre-reading
notes document), the Reading First Director commented on a draft of the Reading First Criteria.
The Reading First Director wrote that the Key Reading First Classroom Characteristics section
of the Reading First Criteria was significant because “[i]t’s another example of our aggressive
approach because, obviously, very little of this section can be pegged to legislative language. It
just makes good sense, of course, to help the States see what we know/want RF [Reading First]
classrooms to look like. OGC may not like this entire section, and I wanted to birddog it for
you.” He continued:

       We realize the Meets Standards column is much more fleshed-out than the
       Exemplary column….What we’ve done – again, extra-legally, really – is push all
       the characteristics that we originally had in Exemplary and moved them into
       Meets, because we want all of those (a, b, …g) characteristics to define ALL RF
       classrooms, not just the star RF classrooms.

The ‘Exemplary’ column in the final Key Reading First Classroom Characteristics section only
includes one condition in addition to the requirement that the section ‘Meets Standard.’ In an
interview, the Reading First Director acknowledged that the Department included conditions in
the ‘Meets Standard’ column of the Reading First Criteria that were not in the law.

Because the Department included language in the ‘Meets Standard’ column of its Reading First
Criteria that was not based on the statute, State applications were reviewed based upon standards
that were not required by the statute.

FINDING 4 –        In Implementing the Reading First Program, Department Officials
                   Obscured the Statutory Requirements of the ESEA; Acted in
                   Contravention of the GAO Standards for Internal Control in the
                   Federal Government; and Took Actions That Call Into Question
                   Whether They Violated the Prohibitions Included in the DEOA

In the course of answering our objectives for the inspection, we found that the Department,
acting through the former Assistant Secretary for OESE, the Reading First Director, and others:

   •   Developed an application package that obscured the requirements of the statute;
   •   Took action with respect to the expert review panel process that was contrary to the
       balanced panel composition envisioned by Congress;
   •   Intervened to release an assessment review document without the permission of the entity
       that contracted for its development;
   •   Intervened to influence a State’s selection of reading programs; and
Final Report
ED-OIG/I13-F0017                                                                                          Page 14

    •   Intervened to influence reading programs being used by LEAs after the application
        process was completed.

These actions demonstrate that the program officials failed to maintain a control environment
that exemplifies management integrity and accountability.

Congress was explicit about what it intended this program to achieve, how it was to be
implemented, and what was to be funded. Congress provided specific legislative guidance on the
application approval process, the composition of peer review panels (Title 1, Part B, Section
1203), and the respective roles of the SEAs, LEAs, and the Department (Title 1, Part B, Section
1202). Congress also spelled out an information dissemination role for NIFL. Title 1, Part B,
Section 1207(a)(1) directs NIFL to “disseminate information on scientifically based reading
research. . . .” As discussed below, program officials tried to purposely obscure the content of the
statute and otherwise took actions that seemed to disregard Congress’ direction and intent.

The Standards for Internal Control in the Federal Government were issued by GAO in 1999.3
In the standards, GAO defines internal control as “[a]n integral component of an organization’s
management that provides reasonable assurance that the following objectives are being achieved:
effectiveness and efficiency of operations, reliability of financial reporting, and compliance with
applicable laws and regulations.” As noted in the Foreword to the standards, appropriate internal
controls is a key factor in improving accountability. In the standards, GAO states: “Management
and employees should establish and maintain an environment throughout the organization that
sets a positive and supportive attitude toward internal control and conscientious management.”
The first element identified in the standards as affecting the control environment is “the integrity
and ethical values maintained and demonstrated by management and staff.”

Further, the Department issued an Administrative Communications System (ACS) Directive
entitled “Federal Managers’ Financial Integrity Act Management/Reporting on Internal
Controls” in 2003. The provisions of the directive apply to all of the Department’s program and
administrative activities, and to all managers at all organizations levels. The directive states that
one of the objectives of internal control is so that “Programs and operations are executed and
resources are used consistently with agency mission and in compliance with applicable
provisions of law, regulation, and government-wide policy requirements.” The directive also
states that a major step in the internal control process is establishing management controls,
including establishing and maintaining a control environment throughout the organization that
sets a positive and supportive attitude toward internal control and conscientious management. A
guiding factor of that control environment is that “management provides leadership to ensure
integrity and ethical values are maintained and demonstrated by management and staff.” As
discussed below, the actions of the program officials demonstrated a lack of integrity and ethical
values that created a control environment that allowed non-compliance with laws and
regulations.

3
 The Federal Managers’ Financial Integrity Act of 1982 (FMFIA) requires GAO to issue standards for internal
control in government. The Office of Management and Budget (OMB) Circular A-123, Management
Accountability and Control, revised June 21, 1995, provides the specific requirements for assessing and reporting on
controls. OMB Circular A-123 was revised on December 21, 2004 and became effective beginning with fiscal year
2006.
Final Report
ED-OIG/I13-F0017                                                                                             Page 15


Finally, the DEOA, which established the Department in October 1979, describes the Federal-
State relationship in education as follows:

         §3403(b) No provision of a program administered by the Secretary or by any
         other officer of the Department shall be construed to authorize the Secretary or
         any such officer to exercise any direction, supervision, or control over the
         curriculum, program of instruction, administration, or personnel of any
         educational institution, school, or school system...over the selection or content
         of…textbooks, or other instructional materials by any educational institution or
         school system, except to the extent authorized by law.

Section 9527 of the ESEA includes a specific prohibition on endorsement of curriculum, which
reinforces the language in §3403(b): “Notwithstanding any other prohibition of Federal law, no
funds provided to the Department under this Act may be used by the Department to endorse,
approve, or sanction any curriculum…” As discussed below, the Department Officials’ actions
raise questions about whether they performed their duties to avoid these prohibitions.

The Department Developed an Application Package That Obscured the Requirements of
the Statute
In January 2002, the then Assistant Secretary for OESE4 worked closely with the Reading First
Director in developing the application package to be provided to States. The package consisted
of the application, the Reading First Criteria, which outlined the requirements each State
application needed to meet to receive funding, and the Reading First Guidance.

The Assistant Secretary for OESE planned for the Reading First Guidance to include language
that was not in the statute and exclude language that was in the statute. After reviewing a
revision to the Department’s draft of the Reading First Guidance, the Assistant Secretary for
OESE wrote to the Reading First Director, “under reading first plan. i’d like not to say ‘this
must include early intervention and reading remediation materials’ which i think could be read as
‘reading recovery’ [a reading program]. even if it says this in the law, i’d like it taken out.” The
subject phrase appears in the law twice.5

The Assistant Secretary for OESE later wrote, “i think we’ve lost our voice in this guidance, and
returned to a business as usual, bureaucratic don’t say what we really want kind of voice.”

4
  This official left the Department in January of 2003 and for purposes of consistency and clarity will be referred to
in this section simply as “the Assistant Secretary for OESE.”
5
  Section 1202(d)(3)(A)(ii)(I) of the ESEA provides States with the option to use a percentage of grant funds to
develop and implement a program of professional development for teachers that shall include “information on
instructional materials, programs, strategies, and approaches based on scientifically based reading research,
including early intervention and reading remediation materials, programs, and approaches. . . .” (Emphasis added.)

Section 1203(b)(4)(B) of the ESEA states that State applications are required to include a description of how “the
State educational agency will assist local educational agencies in identifying instructional materials, programs,
strategies, and approaches, based on scientifically based reading research, including early intervention and reading
remediation materials, programs, and approaches.” (Emphasis added.)
Final Report
ED-OIG/I13-F0017                                                                           Page 16


The Reading First Director noted that the Guidance was as bold as it could be, given the law. He
wrote:

       The guidance – throughout – was written in a way that ED people, and then I,
       thought was the strongest we were able to make it, *given the law.*…I do not
       want to seem insufficiently bold, but on many of the areas where you clearly want
       more boldness and different-ness, the summary of my on-going poll of ED
       insiders is “The law won’t let you do that, no matter what [the Assistant Secretary
       for OESE] wants/says.”

The Reading First Director also acknowledged that he expected State officials to review the
Guidance closely to see that it was in accord with the statute. He added:

       [I]t has been suggested to me that the Guidance may be the most problematic
       place to put some of your suggestions for increased boldness. Why? Guidance is
       the official place where the State people with the closest readings of the law will
       go to see where ED has overstepped what the law lets them say. In remarks to
       groups…or face-to-face meetings about what the Review Panel will/won’t accept
       the opportunities for BOLDNESS and, perhaps, extralegal requirements are many.

The approach outlined by the Reading First Director was eventually reflected in the application
package that was available to every State on April 2, 2002. As discussed in Finding 3, the
Department ultimately included the “bold” language that distorted the requirements of the statute
in the Reading First Criteria.

In the Pre-reading notes document, the Reading First Director wrote:

       OGC could likely have concerns with the overall, near-unrelenting
       aggressiveness of this application…the law does not really require what we are
       quite literally requiring in our (aggressive) application. Such examples are
       manifold and OGC may catch some, many or all of them. We have not
       highlighted them to OGC, of course, and we don’t know how many they’ll focus
       on. On some issues, we may be able to dodge a little by moving some ‘Meets
       Standards’ points to the ‘Exemplary,’ but if we do that too much, the result is a
       less bold application and decreased chances of overall success. We’ll need your
       muscle with OGC on these points across the board.

Ultimately, the Department determined that the Reading First Guidance would be used as a
means of appearing to comply with the requirements of the statute. In the Pre-reading notes
document, the Reading First Director wrote:

       [T]here are things in the Guidance that are not in the application – for strategic
       reasons – because we did not want to remind all applicants of their existence so
       prominently…but we felt we had to compromise, i.e. put in the Guidance, in some
       places where the law makes certain things un-ignorable.
Final Report
ED-OIG/I13-F0017                                                                            Page 17


The Reading First Director illustrated this strategy by providing the following examples in the
Pre-reading notes document:

       Page 19, [Section] F-2, 2 b. [of the Guidance] “Providing expanded
       opportunities to students in kindergarten through grade 3 who are served by
       eligible local educational agencies for receiving reading assistance from
       alternative providers.”

       We make absolutely no mention on this opportunity in the application, because
       we don’t like it and don’t want to open the door to this, but it is in the law and
       needs to be addressed somewhere reasonably official – like the Guidance – as a
       “best” compromise. FYI.

       Also, Page 19, [Section] F-3 Are there any required priorities for funds
       reserved for State use? Yes. A State educational agency shall give priority to
       carrying out the activities described in Question F-2 for schools that are among
       the schools served by eligible local educational agencies with the highest
       percentages…”

       Again, my belief is that this is a potential back door though [sic] which some
       money could flow in unwanted directions, and therefore this required priorities
       for funds reserved for State use element of the law is NOT NOT [sic] in the
       application, but we have to reflect that we know it exists somewhere, so that place
       is the Guidance. FYI.

In the final Reading First Guidance, Section F-2, 2b. remained unchanged from the portion
quoted above. In section F-3, the substantive content from the portion quoted above remained
unchanged.

The Department Took Action With Respect to the Expert Review Panel Process That Was
Contrary to the Balanced Panel Composition Envisioned by Congress
Congress, through Title 1, Part B, Section 1203(c)(2)(A) of the ESEA, envisioned an expert
review panel with equal representation from the Department, NIFL, NAS, and NICHD. As we
reported in Finding 1A, the Department nominated a majority of the individuals serving on the
expert review panel. Additionally, 15 of the 16 subpanels had a majority of Department-
nominated panelists and none had the balanced composition envisioned by Congress.

The Reading First Director took direct action to ensure that a particular approach to reading
instruction was represented on the expert review panel. Direct Instruction (DI) is a model for
teaching that requires the use of Reading Mastery, a program published by SRA/McGraw-Hill,
to teach reading. The Reading First Director formerly served as the Executive Director of the
Baltimore Curriculum Project, which has implemented DI in Baltimore City schools since 1996.

The Reading First Director personally nominated three individuals who had significant
professional connections to DI to serve on the expert review panel. The Reading First Director
Final Report
ED-OIG/I13-F0017                                                                           Page 18

selected these three individuals to serve on a total of seven of the 16 subpanels and one of these
individuals to serve as the panel chair on five subpanels. These three individuals were
collectively involved in reviewing a total of 23 States’ applications.

A Baltimore City Public Schools official contacted a Department official to express concern that
two panelists were involved with or employed by DI and questioned whether those two panelists
indicated their connections to DI on the conflict of interest form. In May 2002, the Department
official forwarded this information to the Reading First Director who passed the concerns on to
one of the panelists in question as a “Confidential FYI.” This panelist replied:

       I suspect that [the Baltimore City Public Schools official’s] assumption is that
       USDE must be warned that there may be DI infiltrators and that somehow USDE
       knows how dangerous that can be. You may remember that [the Baltimore City
       Public Schools official] is a whole language (now called Balanced Literacy)
       proponent.

The subsequent e-mail response from the Reading First Director suggests his intention to ensure
a DI presence on the expert review panel: “Funny that [the Baltimore City Public Schools
official] calls *me* to inform that there may be some pro-DI folks on *my* panel!!! Too rich!”
The panelist then asked, “Does he know who you are? Past and present?” The Reading First
Director replied, “That’s the funniest part – yes! You know the line from Casablanca, ‘I am
SHOCKED that there is gambling going on in this establishment!’ Well, ‘I am SHOCKED that
there are pro-DI people on this panel!’”

Shortly before this exchange, a Department employee reported to the Reading First Director that
the Department had received a question from a member of the media about the panel
composition. The response by the Reading First Director suggests that he may indeed have
intended to “stack” the expert review panel. The employee stated: “The question is…are we
going to ‘stack the panel’ so programs like Reading Recovery don’t get a fair shake[?]” The
Reading First Director responded, “‘Stack the panel?’…I have never *heard* of such a
thing….<harumph, harumph>[.]”

A few days before the Department publicly announced the panelists it had chosen to serve, one
of the Department-nominated panelists contacted the Reading First Director and shared his
strong bias against Reading Recovery and his strategy for responding to any State that planned to
include Reading Recovery in its application. The Reading First Director responded: “I really
like the way you’re viewing/approaching this, and not just because it matches my own approach
:-), I swear!” This individual later served as the panel chair for the subpanel that reviewed
Wisconsin’s State application and in response to the State’s plans to use Reading Recovery, he
included an 11-page negative review of Reading Recovery in his official comments on the
application.

Around the same time, Reid Lyon, the former Chief of the Child Development & Behavior
Branch at the NICHD, advised the Reading First Director, the Assistant Secretary for OESE, and
the Senior Advisor to the Secretary at the time that one of the panelists had been “actively
working to undermine the NRP [National Reading Panel] Report and the RF initiatives.” Lyon
Final Report
ED-OIG/I13-F0017                                                                          Page 19

further stated, “Chances are that other reviewers can trump any bias on her part.” In a written
response to all of the people involved, the former Senior Advisor to the Secretary stated, “We
can’t un-invite her. Just make sure she is on a panel with one of our barracuda types.”

The statute envisioned that the expert review panel would be balanced with representatives from
the Department and three other named organizations. In fact, virtually all of the subpanels had a
majority of Department-nominated panelists. Against this backdrop, the actions of the Director
and the former Senior Advisor to the Secretary become particularly problematic.

Additionally, the first element identified in GAO’s Standards for Internal Control in the Federal
Government as affecting the control environment is “the integrity and ethical values maintained
and demonstrated by management and staff.” The apparent intent of the Reading First Director
to include and to give a significant role to panelists who reflected his personal preference in
reading programs; his specific encouragement to a panelist who held views similar to his on
Reading Recovery; and the intention of the former Senior Advisor to the Secretary to control
another panelist raise significant questions about the control environment in which the program
was being managed.

The Department Intervened to Release an Assessment Review Document Without the
Permission of the Entity That Contracted for Its Development
Title 1, Part B, Section 1207(a)(1) of the ESEA directs NIFL to “disseminate information on
scientifically based reading research. . . .” NIFL contracted for the University of Oregon’s
Institute for the Development of Educational Achievement (IDEA) to perform a review of
assessments. The Reading First Director inappropriately intervened to release the report
produced as a result of this review.

In January and February of 2002, the Department held “The Secretary’s Reading Leadership
Academies” (SRLAs). The Reading Leadership Academy Guidebook (the Guidebook) was
published after the SRLAs and collected “the essential materials offered to more than 600 state-
level administrators and policy makers at the academies.”

The Guidebook included a PowerPoint presentation on “the dimensions of assessment important
to reading instruction” by a group referred to as the “Reading First Academy Assessment
Committee.” This committee consisted of eight members, led by Edward Kame’enui, the
Director of IDEA. In the presentation, this committee listed a sample of assessments that it
planned to review.

The committee’s final report, “An Analysis of Reading Assessment Instruments for K-3,” does
not mention any connection to the Department or to NIFL. Additionally, the committee does not
refer to itself as the “Reading First Academy Assessment Committee,” but rather only as the
“Assessment Committee.”

The report includes an analysis of 29 assessments. Of these 29, the Assessment Committee
found 24 to have “‘sufficient evidence’ for use as screening, diagnosis, progress monitoring,
and/or outcome instruments to assess one or more essential reading components at one or more
Final Report
ED-OIG/I13-F0017                                                                        Page 20

grade levels K-3.” The Assessment Committee found the other five assessments “not to have
‘sufficient evidence’. . . .”

The report, which can be found on the IDEA website, states: “there are literally hundreds of
reading assessment instruments available in the marketplace.” The report states that the
Assessment Committee, as a starting point, used the Southwest Educational Development
Laboratory’s (SEDL) “Reading Assessment Database for Grades K-2: Statewide Assessments”
to help select the assessments it would review. According to the report, this database yielded
154 possible early reading assessment tools. In addition, the Assessment Committee members
submitted 27 personal recommendations of reading assessment measures.

Of the 29 assessments the Assessment Committee selected to review, 11 were chosen solely
from the SEDL database and the remaining 18 were on the list of personal recommendations.

Of the 24 assessments the Assessment Committee found to have “sufficient evidence,” seven
were directly tied to Assessment Committee members:

      •    Five were developed or co-developed by Assessment Committee members;
      •    One lists an Assessment Committee member as the individual who can provide
           technical information; and
      •    One is the product of a company that is directed by an Assessment Committee
           member.

Although NIFL funded the Assessment Committee’s review, the Director of NIFL, Sandra
Baxter, informed us that she decided not to issue the Assessment Committee’s final report.
NIFL officials were concerned that releasing the report might appear as though NIFL was
endorsing specific products. Baxter sought guidance on the endorsement issue from the
Department’s OGC on whether NIFL should release the report. It is unclear if this guidance was
ever provided to NIFL.

When we notified Baxter and Lynn Reddy, the Deputy Director of NIFL, that the report was
released on the IDEA website, Baxter informed us that she was surprised to learn this. Reddy
expressed the view that the Assessment Committee’s report should not have been posted on the
IDEA website.

The Reading First Director was in frequent contact with Kame’enui and Doug Carnine, a
research associate at IDEA and professor at the University of Oregon, concerning the release of
the Assessment Committee’s report. In e-mail correspondence with Carnine in May 2002,
Kame’enui wrote, “If I am asked, who do I say initiated and funded this effort?” Carnine
responded, “Say you received funding from RMC [RMC Research Corporation, a Reading First
contractor]. If they ask who funded RMC say the Dept of Ed. You should not need to answer
such questions in writing.”

Carnine forwarded this string of e-mails to the Reading First Director and wrote, “Should
[Kame’enui’s] report go to RMC? Should it be on the web before being sent to you?” The
Reading First Director responded, “As I/we try to weigh all the elements and make the best
Final Report
ED-OIG/I13-F0017                                                                                             Page 21

decision, I’d like to know whether the Assessment work is ready to go fully live…?” Carnine
responded, “They are ready to go live but I see now that [the Assistant Secretary for OESE]
wants to review.” In a later e-mail, Carnine asked, “Is [the Assistant Secretary for OESE]
thinking of killing [the report]?” The Reading First Director responded, “I don’t *think* so, and
I am arguing strenuously against that. Very strenuously.”

The Reading First Director sent Kame’enui an e-mail requesting that he send the Assessment
Committee’s report to the Assistant Secretary for OESE “prior to the ‘release’ to the public.”
Kame’enui asked the Reading First Director whether he should send a set of report materials for
him too. The Reading First Director responded, “Yes, please! Then I can become a better-
informed ambassador and advocate for the Assessment Committee’s work!”

While waiting for the approval to release the Assessment Committee report, Kame’enui
consulted with the Reading First Director over what to tell publishers that asked about it.
Kame’enui asked, “should I say the report is completed but under review or do I simply say it is
not ready for public release?” The Reading First Director advised Kame’enui to “go with…‘not
ready’ for the reason that the ‘under review’ option could lead to the non-trivial question, ‘Under
review by whom?’ which then gets into the next issue of, ‘is this a NIFL doc, or an ED doc?’”

Kame’enui, in correspondence with the Assessment Committee members and the Reading First
Director shortly before the release of the report, wrote: “[T]he report will be presented and
represented publicly as an independent report completed under the auspices of our institute, the
Institute for the Development of Educational Achievement (IDEA) in the College of Education
at the University of Oregon. It will not be identified as a Reading First document.” Kame’enui
also made the decision to present the report itself as a “singular effort” on his part, rather than as
the effort of the Assessment Committee. One reason he provided was: “Because several of [the
Assessment Committee members] are authors or co-authors of the assessment instruments [the
Assessment Committee] reviewed, the perception of a conflict of interest in shaping the final
report was a concern.”

In May 2002, the Reading First Director asked the Assistant Secretary for OESE if he could give
Kame’enui permission to put the report on the IDEA website and stated, “[NIFL] will slow us
down.” The Assistant Secretary for OESE informed the Reading First Director that Kame’enui
had not yet shared the report with NIFL.

A few days later, the Reading First Director asked the Assistant Secretary for OESE if he had
“permission to bug Sandra Baxter about the…assessment work,” writing that “[s]tates are
desperate for that critical work.” In response, the Assistant Secretary for OESE informed the
Reading First Director that the report could be put on the IDEA website and then the report
would go on The Partnership for Reading6 (Partnership) website after Baxter approved it.

Later that day, the Reading First Director wrote to Kame’enui:

6
  The Partnership for Reading is administered by NIFL and, according to its website, is “a collaborative effort by
three federal agencies - the National Institute for Literacy (NIFL), the National Institute of Child Health and Human
Development (NICHD), and the U.S. Department of Education - to bring the findings of evidence-based reading
research to the educational community, families, and others with an interest in helping all people learn to read well.”
Final Report
ED-OIG/I13-F0017                                                                           Page 22


       Please post your splendid work on your IDEA website, announce it in any way(s)
       you choose, and I will inform you when we can post it as well on the Partnership
       website, NIFL, maybe even ED’s…I will worry about the Baxter/NIFL angle –
       you are completely covered, [The Assistant Secretary for OESE] has approved
       this specific step. No worries on your end.

One day later, Baxter wrote in an e-mail to the Reading First Director: “The copy of the final
report just reached us today…I’ll let you know…what our timeline for releasing the results will
be.” A few days later, Baxter wrote to the Reading First Director: “Clearly, [the Assessment
Committee’s final report] is not in shape for publication as submitted. I’ll need at least a few
days to read the entire report and determine just how to streamline it into a document that is
useful to our intended audience.” The Reading First Director forwarded this e-mail to the
Assistant Secretary for OESE, who responded, “at this rate, it may never get up on the nifl
website.”

The Assessment Committee’s final report never appeared on NIFL’s or the Partnership’s
website.

The Department Intervened to Influence a State’s Selection of Reading Programs
As noted above, the Department is prohibited from endorsing curriculum and from mandating,
directing or controlling a State’s curriculum or program of instruction.

In July 2003, the Reading First Director received information from the Maryland State
Department of Education’s (MSDE) Director of Reading First concerning a clarification to the
State’s plans for selecting a list of core reading programs. The Reading First Director then
forwarded this information to Carnine and Jerry Silbert, a consultant to RMC, and stated:

       This CONFIDENTIAL update comes after a direct call I made to MD [Maryland]
       after a tip/suggestion from [Silbert]. This “clarification” represents a marked shift
       from their earlier comments and, although not settled completely yet, bodes well
       for DI in Baltimore. Who knows [Michael] Coyne [an Assistant Professor at the
       University of Connecticut] well? We need to ENSURE that when MD *does* do
       its application of…Consumer Guide that Reading Mastery [a DI reading program
       published by SRA/McGraw-Hill] is NOT relegated to supplemental status, which
       would be HORRIBLE for so many…schools in Balt[imore].

On August 1, 2003, Carnine sent Kame’enui an e-mail and asked him to forward it to Michael
Coyne. In this e-mail, Carnine stated:

       Just mention that [Baltimore] has many schools using RM [Reading Mastery] as a
       core program with EXCELLENT results but that MSDE, for all the wrong
       reasons, left to its own devices could well shunt RM off to the side with the kiss-
       of-death as a “supplemental” or “intervention” program. And, once MSDE did
       something like that, it would be much harder to have them un-do it. . . .The way
Final Report
ED-OIG/I13-F0017                                                                            Page 23

       it’s set up, if, while applying [the Consumer’s Guide] tool, [MSDE] were to
       classify RM as a comprehensive program…MSDE couldn’t un-do it.

After obtaining Coyne’s e-mail address, Carnine stated to Coyne in an e-mail, dated August 2,
2003:

       Please let me know if you have questions. Also feel free to ask [the Reading First
       Director] to confirm that RM can legitimately be treated as a core program…For
       more on DI results (as a core program) in MD you can contact Muriel Berkeley
       [the President and CEO of the Baltimore Curriculum Project who also served as
       an expert review panelist]. . . .

Carnine ensured that the Reading First Director and Berkeley were fully informed of his actions
by forwarding both of the August 2003 e-mails to them.

In November 2003, MSDE issued the “Final Report of the Maryland Committee for Selecting
Core Reading Programs.” The report stated that all LEA representatives on the committee “were
required to participate in a full day of training by Dr. Michael Coyne on September 16, 2003.”

The report then stated:

       Also during August [2003], Dr. Deborah C. Simmons [the former Associate
       Director of IDEA at the University of Oregon] and her associate, Dr. Michael
       Coyne, were in frequent communication with [MSDE’s] Office of Reading First
       to plan how to use the Consumer’s Guide for Evaluating a Core Reading
       Program (Simmons & Kame’enui, 2003) in order to achieve the goal of
       establishing a Fall 2003 list of Maryland Approved Core Reading Programs.

       Dr. Simmons advised that Maryland base the work of its statewide Committee on
       the Oregon Reading First Center: Review of Comprehensive Programs
       (Curriculum Review Panel, July 2003). Oregon’s distinguished panel of
       reviewers…evaluated nine core reading programs. . . .

       …From the list of nine programs reviewed by Oregon, MSDE, responding to the
       consultant guidance of Drs. Simmons and Coyne, selected the following seven
       core programs receiving the highest Oregon ratings for the Maryland Committee’s
       work:

         •    Houghton Mifflin, The Nation’s Choice 2003
         •    SRA, Open Court 2002
         •    SRA/McGraw-Hill, Reading Mastery Plus 2002
         •    Macmillan/McGraw-Hill, Macmillan/McGraw-Hill Reading 2003
         •    Harcourt, Trophies 2003
         •    Success for All Foundation, Success for All
         •    Scott Foresman, Scott Foresman Reading 2004[.]
Final Report
ED-OIG/I13-F0017                                                                              Page 24

The report stated, “Maryland’s approved application for funding under Reading First specifies
that every Reading First School, kindergarten through third grade, must implement a core
reading program from this list that is based on scientifically based reading research (SBRR).”

As a result of the actions taken with regard to MSDE’s process for selecting core reading
programs, the Reading First Director’s initial fear that Reading Mastery would be “relegated to
supplemental status” was not realized since MSDE, with the help of Simmons and Coyne,
selected Reading Mastery as one of its seven core reading programs. The Reading First
Director’s actions raise a question as to whether he was acting with the restraint envisioned by
Congress in the endorsement of curriculum prohibitions of the DEOA and the ESEA.

The Department Intervened to Influence Reading Programs Being Used by LEAs After the
Application Process Was Completed
After certain States completed the application process and received funding, the Reading First
Director became aware that certain LEAs in these States were using the Rigby Literacy (Rigby)
and Wright Group Literacy (Wright Group) programs. The Reading First Director worked
closely with a Department staff member, a former expert review panelist, who undertook a
review of both of these programs.

In e-mail correspondence with the staff member regarding the Wright Group, the Reading First
Director stated:

       Beat the [expletive deleted] out of them in a way that will stand up to any level of
       legal and [whole language] apologist scrutiny. Hit them over and over with
       definitive evidence that they are not SBRR, never have been and never will be.
       They are trying to crash our party and we need to beat the [expletive deleted] out
       of them in front of all the other would-be party crashers who are standing on the
       front lawn waiting to see how we welcome these dirtbags.

The Reading First Director forwarded the above e-mail to Lyon and stated:

       Confidential FYI. Pardon in-house language I use…with fellow team members
       and friends. Do you know—on the QT—if anyone has done any good review of
       the Wright Group stuff, to date? We have beaten Maine on Rigby and this is cut
       from the same cloth. We are proceeding, of course, but if you knew of a good
       piece of work dissecting The Wright Group’s stuff, it could further strengthen our
       hand.

Lyon responded that he would obtain this information and added, “I like your style.” In
response, the Reading First Director stated, “Additional firepower…may help us make this a
one-punch fight.”

After reviewing the programs, the staff member provided the Reading First Director with notes
and talking points critiquing these programs. The Reading First Director used this information to
convince States using Rigby and Wright Group to change programs. In an e-mail to Lyon, the
Reading First Director wrote, “I spoke to Fred Carrigg [the former New Jersey Director of
Final Report
ED-OIG/I13-F0017                                                                             Page 25

Reading First]…with a roomful of others on their end and they are HALTING the funding of
Rigby and, while we were at it, Wright Group. They STOPPED the districts who wanted to use
those programs.”

In a later e-mail to Lyon, the Reading First Director stated:

       As you may remember, RF got Maine to UNDO its already-made decision to have
       Rigby be one of their two approved core programs (Ha, ha – Rigby as a CORE
       program? When pigs fly!) We also as you may recall, got NJ [New Jersey] to
       stop its districts from using Rigby (and the Wright Group, btw) and are doing the
       same in Mississippi. This is for your FYI, as I think this program-bashing is best
       done off or under the major radar screens.

In a formal letter to Carrigg, the Reading First Director did not specifically name Rigby and
Wright Group as not being aligned with SBRR. The Reading First Director wrote, “It appeared
that New Jersey had not fulfilled its responsibility to ensure that all LEAs and schools selected to
participate in Reading First…would implement comprehensive reading programs that are fully
aligned with scientifically based reading research.” The Reading First Director informed us that
he could not definitively say why he did not formally state in the letter that those specific
programs were not in line with SBRR.

Massachusetts, North Dakota, and Kentucky also encountered post-application problems
concerning the implementation of their reading programs. Massachusetts was approved in its
Reading First application to develop its own guide for LEAs to select reading programs. North
Dakota was approved in its Reading First application to develop a State list of approved
programs using the Consumer’s Guide. Kentucky was approved in its Reading First application
to adopt the Massachusetts guide. LEAs in all three of these States selected reading programs
that were later questioned by the Reading First Director or his assistant.

Massachusetts
In Massachusetts, the Reading First Director raised questions over the SBRR qualifications of
programs in four districts. The Coordinator of the Massachusetts Reading First Program, Cheryl
Liebling, who had worked for RMC as a consultant during the application process, informed us
that the Reading First Director contacted her with concern over the programs currently in use in
these districts. This concern was raised even though the State had approved the LEAs’ choices
for reading programs using the expert review panel-approved criteria. The programs used by the
four districts were Wright Group, Rigby, Literacy Collaborative, and Harcourt Collections.
Liebling said that she contacted the districts and suggested a change, telling the districts that the
Reading First Director had concerns about their programs. The district using Wright Group
elected not to change its program, even after Liebling’s recommendation. This district
eventually had its funding taken away. The three other districts agreed to change to other
programs and all three districts continue to be funded.

North Dakota
During North Dakota’s first year of LEA grants, three schools used the State list of approved
programs to select Rigby as their reading program. After the State approved their use of Rigby,
Final Report
ED-OIG/I13-F0017                                                                          Page 26

the three schools implemented the program. The North Dakota Reading First Assistant Director,
Gail Schauer, in an e-mail to the Reading First Director, wrote, “[The Reading First Director’s
assistant] indicated that North Dakota Reading First could not use the Rigby Literacy Program as
a core reading program. The Rigby Literacy Program, however, could be used as a supplemental
program.” Schauer requested that the three schools “be allowed to implement the Rigby Literacy
Program in their Reading First schools until their three-year subgrant period is
completed…based on the fact that the Rigby Literacy Program was an approved program at the
time they wrote their grant, received approval, and began the implementation phase.” Schauer
contacted the Reading First Director’s assistant two months later to ask whether any decisions
were made on the schools’ use of Rigby. The Reading First Director’s assistant responded that
the Department was waiting for feedback on the State’s monitoring visit to make a final
determination.

A month and a half later, Schauer contacted the Reading First Director’s assistant again asking
for information on Rigby. The Reading First Director’s assistant replied that the Department did
not have the full findings from the monitoring report, but the preliminary report “found that the
core program used in the Fargo School District did not appear to be systematic, explicit or
sequential” and the report found “instruction in classrooms to focus on strategies such as cueing
systems, which are not aligned with scientifically based reading research.” Schauer informed us
that given a choice between receiving Reading First funds or continuing their use of Rigby, the
three schools opted to stay with Rigby. All three lost their funding after the first year of
implementation.

Kentucky
The Reading First Director questioned the use of Reading Recovery and Rigby in three Kentucky
districts. The Reading First Director informed us that he asked the State to provide more
justification of both programs’ alignment with SBRR. State officials informed us that they spoke
with the Reading First Director in a conference call and informed him that they believed Reading
Recovery and Rigby were sufficiently SBRR. State officials said that the Reading First Director
told them he had concerns about Reading Recovery and urged them not to use Reading First
funds on the program. The State officials said that they asked for this request in writing, but the
Reading First Director told them that he would not do so and invited them to defend the two
programs instead. The State officials subsequently provided support for the programs in writing,
but they informed us that they did not receive a response from the Department. A State official
informed us that the districts in question are still using Rigby and Reading Recovery. Despite
the Reading First Director’s assertion that these programs were not SBRR, when the State
provided documentation to support the use of the programs, the Department did not respond.

The actions taken by the Reading First Director again call into question whether his intervention
in these circumstances violated provisions of the DEOA and NCLB that prohibit the Department
from exercising control over the curriculum or program of instruction of any school system.
Final Report
ED-OIG/I13-F0017                                                                          Page 27



                                RECOMMENDATIONS


We recommend that the Assistant Secretary for OESE

   1) Develop internal management policies and procedures for OESE program offices that
      address when legal advice will be solicited from OGC and how discussions between
      OGC and the program staff will be resolved to ensure that programs are managed in
      compliance with applicable laws and regulations.

   2) In consultation with OGC, evaluate OESE’s processes for assessing potential conflict of
      interest questions, when a panel review process is used, and make those improvements
      necessary to strengthen the processes.

   3) Review all Reading First applications to determine whether all criteria for funding have
      been met.

   4) Review the management and staff structure of the Reading First program office and make
      changes, as appropriate, to ensure that the program is managed and implemented
      consistent with the statutory requirements of NCLB.

   5) Request that OGC develop guidance for OESE on the prohibitions imposed by §3403(b)
      of the Department of Education Organization Act.

   6) When similar new initiatives are approved by Congress, rely upon an internal advisory
      committee, which includes representatives from other OESE programs, OGC, and the
      Department’s Risk Management Team, to provide feedback on program implementation
      issues and ensure coordination in the delivery of similar or complimentary Department
      programs.

   7) Rely upon the internal advisory committee to:

          a. Determine whether the implementation of Reading First harmed the Federal
             interest and what course of action is required to resolve any issues identified; and

          b. Ensure that future programs, including other programs for which the Department
             is considering using Reading First as a model, have internal controls in place to
             prevent similar problems from occurring.

   8) Convene a discussion with a broad range of state and local education representatives to
      discuss issues with Reading First as part of the reauthorization process.
Final Report
ED-OIG/I13-F0017                                                                             Page 28



                              DEPARTMENT COMMENTS


On August 29, 2006 we received a letter from the Secretary of Education that stated the
Department concurred with all of our recommendations and that her letter would be followed by
a more detailed response. On September 19, 2006 we received a second letter from the Secretary
that reiterated the Department’s concurrence with all the recommendations in the draft and
outlined a series of steps that the Secretary would take in response to our report. The letter also
stated that while the Department found the draft recommendations and much of the discussion in
the report to be helpful, it did not agree with all of the key points made in the draft findings. An
attached letter from the Assistant Secretary for OESE provided a detailed response to our draft.
We have summarized the Department’s comments and provided our responses below. All of the
Department’s response letters, in their entirety, are attached.


FINDING 1A – The Department Did Not Select the Expert Review Panel in Compliance
             With the Requirements of NCLB

Department Comments
The Department agreed with our finding that it did not fully adhere to the statutory requirement
for establishing the expert review panel. The Department acknowledged that it had considered
and OGC had recommended the establishment of a twelve-member “Advisory and Oversight
Panel” made up of individuals nominated by NAS, NICHD, and NIFL. The Department stated
that it is not aware of any information to show that not establishing this panel resulted in any
inappropriate effects or disadvantage to any State, but stated that it will discuss this issue with
State representatives and review applications to determine if there were any inappropriate
effects.

OIG Response
No changes have been made to this finding. The Department acknowledged that it did not
establish an expert review panel in a manner that was consistent with the statutory language.


FINDING 1B –       While Not Required to Screen for Conflicts of Interest, the Screening
                   Process the Department Created Was Not Effective

Department Comments
The Department stated that it had some concerns with this finding. The Department stated that it
was not required to screen for conflicts of interest, but decided to take the extra step and establish
a screening process in cooperation with the Ethics Division of OGC. The Department stated that
the integrity of the panel process was important, and it took appropriate action to ensure that no
conflict occurred.
Final Report
ED-OIG/I13-F0017                                                                           Page 29

The Department stated that it reasonably adapted the competitive grant conflict of interest
procedures to the Reading First program. The Department stated there was no available
information to show that the inclusion of six panelists who had connections to a teaching
methodology resulted in any problematic behavior or that any of these panelists reviewed a state
application that included such a program. The Department stated that it used the standard of
“avoiding financial connections” to programs or products and that OIG suggests that the
appropriate standard should have been “significant professional connections to a teaching
methodology that requires the use of a specific reading program,” which it labeled as not easy to
define and implement.

The Department stated that the OIG finding suggests that the Department could have been better
off by just meeting the minimal requirement of the law, which is that no conflict of interest
screening process is required, and none needed to be implemented.

OIG Response
No changes have been made to this finding.

As we stated in our report, the Department was not required to screen for conflicts of interest.
However, since it wisely chose to do so, we evaluated the effectiveness of the process it selected.
Our objective was not to discourage the Department from taking such actions, but rather to point
out simple additional actions that might have improved the process, such as including relevant
questions regarding impartiality and reviewing resumes.

The six panelists mentioned had significant professional connections to Direct Instruction (DI), a
model for teaching that requires the use of the program Reading Mastery. At the time of the
screening process, the Department anticipated that States would include specific programs in
their applications. Based upon the screening process used, the Department would not have
known of the potential conflict posed by having any of the six individuals review a State
application that included Direct Instruction.


FINDING 2A – The Department Replaced What the Law Intended to be a Peer Review
             Process With its Own Process

Department Comments
The Department stated that it did not replace the process required by the statute, and that the
statute does not establish the role of reviewers’ comments. The Department stated that although
the program office originally intended in its Reviewer Guidance to have reviewers provide
comments directly to the applicants, it found that this did not always ensure that applicants knew
what needed to be addressed.

The Department stated that its staff did a further review of the summaries of the comments and
overall, the staff found that the summaries did not deviate significantly from the reviewers’
comments.
Final Report
ED-OIG/I13-F0017                                                                             Page 30

The Department stated that it modified the process after it was initiated without amending the
original planning documents and that in the future it will take steps to ensure that all plans for
review of applications are followed or amended when necessary.

OIG Response
We have modified the wording of the report to clarify what occurred. The statute did not provide
a process for expert review; however, the Department issued guidance to establish the role of
reviewers’ comments. The process actually undertaken by the Reading First program office,
however, differed from the process outlined in its guidance. There is no evidence to show that
the Reading First program office communicated these changes to either reviewers or applicants.
In addition, there is no evidence that the Reading First program office ever sent the actual Panel
Chair Summaries to applicants. The report outlines those instances where the Department-
prepared summaries deviated significantly from the reviewers’ comments and were not always
accurate representations of the reviewers’ comments.


FINDING 2B – The Department Awarded Grants to States Without Documentation That
             the Subpanels Approved All Criteria

Department Comments
The Department stated that the program statute did not require the panel to approve final
applications and that the panels were advisory, even though the program office tried to follow the
recommendations of the panels. The Department also stated that in a number of cases a review
of a resubmitted application revealed minor remaining issues, and it was not practical to always
reconvene a subpanel.

OIG Response
No changes have been made to this finding.

We recognize that the panel served an advisory role in recommending the approval of final
applications. In the specific examples provided in our report, the remaining issues were not
minor.


FINDING 3 –        The Department Included Requirements in the Criteria Used by the
                   Expert Review Panels That Were Not Specifically Addressed in NCLB

Department Comments
The Department commented that our report suggested the guidance issued by the Department
may not include review criteria not specifically stated in the statute. The Department stated that
the Reading First Criteria sets forth guidance on what criteria the subpanels are likely to use in
reviewing State applications for Reading First grants. The Department further stated that States
were reminded that they must meet all program requirements in order to receive funding and that
this document was merely guidance.
Final Report
ED-OIG/I13-F0017                                                                           Page 31

The Department stated that the “Meets Standard” column would describe the conditions that
reviewers will expect all State applications to meet. The Department acknowledged that in a few
instances, the information in the “Meets Standard” column was not specifically required; it
merely provided information on a submission that would be good practice and was helpful in
ensuring good program management by the State.

The Department also stated that it did not receive complaints from States that the guidance was
onerous.

OIG Response
No changes have been made to this finding.

We recognize the Department’s interest in developing high-quality programs; however, criteria
not specifically stated in the statute should not have been included in the “Meets Standard”
column, but rather in the “Exemplary” column, which was the place identified by the Department
to list additional criteria that it believed would help encourage high-quality projects.

Despite the Department’s assertion that the document was merely guidance, the Reviewer
Guidance states that panelists who reviewed the applications were to use only the criteria
provided in the guidance.


FINDING 4 –        In Implementing the Reading First Program, Department Officials
                   Obscured the Statutory Requirements of the ESEA; Acted in
                   Contravention of the GAO Standards for Internal Control in the Federal
                   Government; and Took Actions That Call Into Question Whether They
                   Violated the Prohibitions Included in the DEOA

Department Comments
The Department stated that this section of the report cites a number of informal internal
communications. In the introduction to its specific comments, the Department expressed
concern that it may be too easy (and inaccurate in some instances) to read interpretations into, or
draw conclusions about, a person’s professional intent or actions solely from raw, unfiltered,
informal snapshots of information.

The Department commented that while it is clearly the State’s responsibility to select and
approve programs for use by participating LEAs, the Department must ensure that only programs
that meet statutory requirements are implemented. The Department further stated that when it
becomes aware of programs that may not meet these provisions, it is incumbent upon the
Department to raise that issue and question that action. The Department stated that it is not
aware of information showing inappropriate actions to require particular programs or
approaches.

OIG Response
No changes have been made to this finding.
Final Report
ED-OIG/I13-F0017                                                                          Page 32

The statements quoted in our report were not informal communications. The Reading First
Director’s communications were written using a Department e-mail account in his role as the
Reading First Director. In preparing its response, the Department reviewed all of the e-mail
communications quoted in this report. The Department’s response does not suggest any
instances where quotations were used out of context or inaccurately.

We agree that the Department must ensure that only programs that meet statutory requirements
are implemented. If the Department was aware of programs that did not meet the statutory
requirements, it should have shared that information formally and transparently. In this instance,
that is not what occurred.

While the Department stated that it is not aware of information showing inappropriate actions to
require particular programs or approaches, the actions reflected in this report raise serious
concerns about the conduct of Department officials with respect to the limitations imposed by
§3403(b) of the Department of Education Organization Act.


Other Department Comments
In the introduction to its specific comments, the Department fully acknowledged its
responsibility to implement programs with fairness and integrity and to follow the law and its
own guidance. We strongly concur. When, as here, that does not occur, it can only serve to
undermine the public’s confidence in the Department and its implementation of major programs,
such as Reading First. As the Department notes, it is incumbent upon all Department employees
“to take painstaking care to maintain the public trust.”
Final Report
ED-OIG/I13-F0017                                                                          Page 33



                 OBJECTIVES, SCOPE, AND METHODOLOGY


The objectives of our inspection were to:

   1. Determine if the Department selected the expert review panel in accordance with the No
      Child Left Behind Act of 2001, Section 1203(c) and if the Department adequately
      screened the panel members for possible conflict of interest issues;

   2. Determine if the expert review panel adequately documented its reasons for stating that
      an application was unready for funding; and

   3. Determine if the expert review panel reviewed the applications in accordance with
      established criteria and applied the criteria consistently.

Through the course of our fieldwork, we identified a fourth item of concern related to actions
that were contrary to statutory requirements of the ESEA, the GAO Standards for Internal
Control in the Federal Government, and the DEOA.

We began our fieldwork on September 28, 2005, and we conducted an exit conference on July
18, 2006.

We interviewed Department staff in the Reading First program office and in OGC. We reviewed
Department guidance and other documentation. In addition, we reviewed Department
correspondence related to the Reading First program.

We also reviewed the applications from the following 11 States and one territory: Connecticut,
Georgia, Kentucky, Massachusetts, Michigan, Nevada, New York, North Carolina, North
Dakota, Virginia, Wisconsin, and Puerto Rico. We also reviewed the individual panelists’
Technical Review forms, the Panel Chair Summary forms, the Department’s Expert Review
Team Reports, and the official grant file for each of the 12 applicants.

Our review focused on nine of the 25 criteria each application was required to meet:

   •   State Outline and Rationale for Using SBRR;
   •   Instructional Assessments;
   •   Instructional Strategies and Programs;
   •   Instructional Materials;
   •   Instructional Leadership;
   •   District and School Based Professional Development;
   •   Evaluation Strategies (Improving Reading Instruction);
   •   State Professional Development Plan; and
   •   Evaluation Strategies (State Reporting and Evaluation).
Final Report
ED-OIG/I13-F0017                                                                         Page 34


We interviewed officials from 10 of the States in our sample.

We also obtained documentation relating to the nomination of panelists. We interviewed
appropriate officials from NIFL, NICHD, and NAS about the panel nominations. We also
obtained completed conflict of interest questionnaires and panelist resumes.

Our inspection was performed in accordance with the 2005 President’s Council on Integrity and
Efficiency Quality Standards for Inspections appropriate to the scope of the inspection described
above.