UNITED STATES DEPARTMENT OF EDUCATION OFFICE OF INSPECTOR GENERAL THE WANAMAKER BUILDING BUILDING 100 PENN SQUARE EAST, SUITE 502 PHILADELPHIA, PA 19107 19107 PHONE (215) 656-6900 · FAX (215) 656-6397 656-6397 July 28, 2006 Control Number ED-OIG/A03F0003 Ms. Faye P. Taylor Chief Executive Officer Following the Leaders 200 5th Avenue East, Suite B Springfield, TN 37172 Dear Ms. Taylor: This Final Audit Report, entitled Education Leaders Council’s Subcontracting Activities, presents the results of our audit. The objectives of the audit were to determine if (1) the Education Leaders Council’s (ELC’s) subcontracting activities complied with the procurement standards set forth in the Education Department General Administrative Regulations (EDGAR), (2) the subcontract costs for Achievement Technologies, Incorporated (ATI) and the Princeton Review (TPR) charged to the federal grants were reasonable and allocable, and (3) any conflict of interest issues exist between ELC and its subcontractors. Our review covered the period July 1, 2002, through December 31, 2004. BACKGROUND ELC is a non-profit organization, founded in 1995, that was formerly located in Washington, DC. According to its mission statement, ELC was committed to leading educational change focused on improved academic achievement for all students. In November 2005, ELC planned or implemented changes that included focusing exclusively on the Following the Leaders (FTL) project, its school-based initiative. As a result, ELC changed its name to Following the Leaders,1 and relocated to Tennessee. According to the organization, its mission is to serve as a catalyst for the application of technology to core instructional processes in order to raise student achievement and fulfill the goals of the No Child Left Behind Act. To accomplish its mission, the organization states that it will assist schools and districts in the selection and implementation of technology-based resources and support services designed to help students achieve academic proficiency in core subjects. 1 In order to differentiate the FTL organization from the project, the report refers to the organization as ELC, its former name. Our mission is to promote the efficiency, effectiveness, and integrity of the Department’s programs and operations. Final Report ED-OIG/A03F0003 Page 2 of 17 The Department of Education’s (the Department’s) Office of Innovation and Improvement (OII) administers the Fund for the Improvement of Education (FIE). The FIE supports nationally significant programs to improve the quality of elementary and secondary education at the state and local levels. The FIE also supports grants to entities that have been earmarked by Congress in appropriations legislation. During the period July 1, 2002, through December 31, 2004, ELC received three FIE grants totaling $23,376,534 for the FTL project. In addition, for fiscal year 2005, Congress directed six grants, totaling $9,594,623 in awards, to ELC for FTL projects in various states. ELC described the FTL project as a package of tools to assist states and school districts in meeting the requirements of the No Child Left Behind Act. As of December 31, 2004, the FTL project provided technology resources and support to over 600 schools in 11 states. According to ELC, the FTL project helps teachers and administrators incorporate standards-based curriculum into their classrooms, and assists with the analysis and reporting of student performance data. The FTL project’s primary service providers were ATI and TPR. During the period July 1, 2002, through December 31, 2004, the grant expenses for these two service providers accounted for 75 percent of total grant awards made to ELC during the period. The grant expenses for ATI totaled $14,489,593 and the grant expenses for TPR totaled $3,048,427. The FTL project’s primary software products were ATI’s SkillsTutor2 and TPR’s Homeroom. Attachment 1 provides supplemental background information on the FIE grant awards, ELC’s contracts with ATI and TPR, the SkillsTutor and Homeroom software products, and the states participating in the FTL project. AUDIT RESULTS The objectives of the audit were to determine if (1) ELC’s subcontracting activities complied with the procurement standards set forth in EDGAR, (2) the subcontract costs for ATI and TPR charged to the federal grants were reasonable and allocable, and (3) any conflict of interest issues exist between ELC and its subcontractors. We found that ELC’s subcontracting activities did not comply with the procurement standards set forth in EDGAR. Specifically, we found that ELC did not have written procurement procedures and did not comply with procurement standards when it awarded contracts to ATI and TPR. We also found that ELC did not perform and document an adequate cost or price analysis. As a result, while the subcontracts’ costs were allocable to the federal grants, we were unable to determine if the costs charged by ATI and TPR were reasonable relative to other vendors’ prices. However, the prices paid by ELC in 2003 and 2004 for ATI’s SkillsTutor software, TPR’s Homeroom software, and training appeared reasonable relative to the prices paid by other clients of ATI and TPR and the list prices for the software and training. Additionally, while ATI and TPR made significant contributions3 to ELC 2 ELC’s January 2003 contract defined SkillsTutor as ATI’s K-12 MegaSuite, which included modules of ATI’s SkillsTutor, Learning Milestones, and K-2 Learning Milestones software. 3 In 2003, ELC and its affiliate, the Education Leaders Action Council, reported $1,326,817 in revenue from non- federal sources. In the same year, ELC and its affiliate received $325,000 from TPR and $147,000 from ATI. Final Report ED-OIG/A03F0003 Page 3 of 17 and its affiliate in 2003, we found no conflicts of interest issues between ELC and the subcontractors. A draft of this report was provided to ELC for review and comment. In its comments on the draft report, ELC concurred with our finding and did not disagree with the recommendations. We have incorporated ELC’s comments, where appropriate, into the report and provide ELC’s full response as an attachment to the report. FINDING – ELC’s Subcontracting Activities Did Not Comply With the Procurement Standards Set Forth in EDGAR. We found that ELC’s subcontracting activities did not comply with the procurement standards set forth in EDGAR. Specifically, we found that ELC did not have written procurement procedures and did not comply with procurement standards when it awarded contracts to ATI and TPR. These instances of non-compliance appear to be the result of inadequate controls. For example, ELC did not develop written procurement policies and procedures until early 2005, even though the Department's first grant award was made to ELC in July 2002. Recipients are required under EDGAR, to establish policies and procedures for procurement. Regulations in 34 C.F.R. §§ 74.40 through 74.48 contain the Procurement Standards to be used by grantees.4 These Procurement Standards include: • The recipient shall maintain written standards of conduct governing the performance of its employees engaged in the award and administration of contracts. 34 C.F.R. § 74.42 • All procurement transactions shall be conducted in a manner to provide, to the maximum extent practical, open and free competition. 34 C.F.R. § 74.43 • All recipients shall establish written procurement procedures. 34 C.F.R. § 74.44(a) • Recipients shall, on request, make available for the Secretary, procurement documents, such as request for proposals or invitations for bids, independent cost estimates, et cetera, when certain conditions apply. Examples of these conditions include (1) a recipient's procurement procedures fail to comply with the procurement standards, and (2) the procurement is expected to exceed $25,000 and is to be awarded without competition. 34 C.F.R. § 74.44(e) • Some form of cost or price analysis must be made and documented in the procurement files in connection with every procurement action. 34 C.F.R. § 74.45 • Procurement records for purchases in excess of $25,000 must include the following at a minimum: (a) basis for contractor selection, (b) justification for lack of competition when competitive bids or offers are not obtained, and (c) basis for award cost or price. 34 C.F.R. § 74.46 4 Unless otherwise noted, all references to 34 C.F.R. Part 74 are to the July 1, 2002, edition. Final Report ED-OIG/A03F0003 Page 4 of 17 In addition, regulations in 34 C.F.R. §74.53(b) require recipients to retain financial records, supporting documents, statistical records, and all other records pertinent to an award for a period of three years from the date of submission of the final expenditure report. The management of ELC was responsible for establishing and maintaining effective internal control over federal programs that provides reasonable assurance that the grantee is managing federal awards in compliance with laws, regulations, and grant provisions. Additionally, the management of ELC was responsible for complying with laws, regulations, and grant provisions related to its federal programs. ELC Did Not Have Written Procurement Procedures During the period of our audit, July 1, 2002, through December 31, 2004, ELC did not have written procurement policies and procedures in accordance with regulations. Regulations in 34 C.F.R. § 74.44(a) require that all grantees shall establish written procurement procedures. ELC was notified of the requirement to have written procurement policies and procedures through a calendar year 2002 single audit finding that recommended ELC implement written procedures to comply with 34 C.F.R. § 74.44. In the schedule of prior audit findings that accompanied ELC's calendar year 2003 financial statements and single audit report, ELC noted that written policies were continuing to be updated and that completion was scheduled by the fourth quarter of 2004. ELC’s procurement policies entitled Procurement Procedures for Following the Leaders, did not address all the requirements set forth in EDGAR. These procurement policies were prepared in January 2005, in advance of our audit. They consisted mainly of a statement that ELC would follow the procedures laid out in 34 C.F.R. §§ 74.41 through 74.48 and a verbatim copy of regulations in 34 C.F.R. § 74.44. In early 2005, ELC adopted a Finance Protocol Manual that provided a summary of ELC’s financial practices. ELC provided the Finance Protocol Manual to us in April 2005. The procurement procedures established in ELC's Finance Protocol Manual did not address all the requirements set forth in EDGAR. Specifically, competitive bids for contracts under $25,000 and professional service contracts were not required; whereas, regulations in 34 C.F.R. § 74.43 specify that all procurement should be conducted in a manner to provide, to the maximum extent practical, open and free competition. The manual also did not specify the use of solicitations for goods and services as required in 34 C.F.R. § 74.44(a)(3), the type of procurement instrument used shall be determined to be appropriate for the particular procurement and for promoting the best interest of the program or project as required in 34 C.F.R. § 74.44(c), and all the required contract provisions contained in 34 C.F.R. § 74.48 and Appendix A. Furthermore, despite receiving its first grant award in July 2002, ELC did not adopt a conflict of interest policy until May 2004. ELC’s conflict of interest policy did not fully comply with the requirements established in 34 C.F.R. § 74.42. Specifically, ELC’s conflict of interest policy (1) allowed, with the approval of ELC’s board of directors, an officer or director to participate in procurement decisions even if a real or apparent conflict of interest existed; (2) did not address all individuals specified in 34 CF.R. § 74.42, such as employees or agents; (3) did not address the solicitation or acceptance of gratuities, favors, or anything of monetary value that is not a substantial amount; and (4) did not specify the disciplinary actions applied to violations, except that, in certain circumstances, the board could require the resignation of the director or officer. Final Report ED-OIG/A03F0003 Page 5 of 17 We informed ELC that its procurement and conflict of interest policies did not comply with all the requirements set forth in EDGAR. In March 2006, ELC adopted revised policies and procedures covering procurement and conflicts of interest. ELC provided us with an opportunity to review and comment on drafts of the policies. ELC Did Not Comply With Procurement Standards ELC did not comply with procurement standards set forth in regulations when it awarded contracts to the primary FTL service providers. We found that ELC did not award the contracts to ATI and TPR in a manner that provided open and free competition; did not perform and document an adequate cost or price analysis; and did not document the basis for the selection of the contractors, the justification for the lack of competition, and the basis for the award cost or price. ELC did not comply with regulations in 34 C.F.R. § 74.43 that require all procurement transactions to be conducted in a manner to provide, to the maximum extent practical, open and free competition. ELC's former Chief Executive Officer and former Chief of Staff, who were at ELC when the contracts were awarded, noted that ELC did not hold a competition for the FTL project’s vendors, because ATI and TPR were included in the FTL grant proposals as partners. According to the ATI's Chief Executive Officer, while ATI provided special services to ELC, ATI viewed ELC as a customer; ATI was not a partner or sub-grantee on the grants. Similarly, according to TPR's Chief Executive Officer, while TPR treated ELC as a peer, TPR considered itself a vendor for ELC. ELC also did not comply with regulations in 34 C.F.R. § 74.46 that require that procurement records for purchases in excess of $25,000 must include the basis for contractor selection, the justification for lack of competition, and the basis for award cost or price. ELC's former Chief of Staff noted that ELC performed an informal comparison of vendors and concluded that ATI had the best product, and that the only documentation of this comparison was a vendor comparison prepared in November 2002. The former Chief of Staff also added that there was no formal written justification for the selection of TPR. Similarly, based upon our conversation with ELC's former Chief Executive Officer, there was no formal written justification for ELC’s selection of ATI and TPR. The former Chief Executive Officer stated that, at the time of the initial grant, ATI was the only vendor providing the needed services and that ELC thought ATI’s costs were reasonable. Additionally, the former Chief Executive Officer stated that ELC did not have a formal process where it documented the different vendors’ prices in the marketplace, but claimed ELC knew that ATI’s and TPR’s prices were competitive relative to other school-based products. ELC provided us with a two-page matrix comparing nine vendors to nine criteria established by ELC.5 This document was prepared in November 2002, which was after the Department awarded the grant to ELC, and ELC awarded the initial contract to ATI (both of which occurred in July 2002). The document was prepared in response to a Department request for ELC to clarify the justification for selecting ATI in a non-competitive manner. Based upon the information contained in the document, at least one other vendor in addition to ATI met all of the criteria and seven vendors met at least seven of the nine criteria. It is possible that, in addition to 5 Attachment 2 presents a summary of ELC’s comparison of the nine vendors. Final Report ED-OIG/A03F0003 Page 6 of 17 ATI, some of the other vendors could have put together or modified their offerings to meet all of ELC’s criteria. As a result, the document prepared by ELC did not adequately justify the lack of competition for the procurement action and ELC’s basis for the selection of ATI for the FTL project. TPR was not among the nine vendors included in the document. ELC also did not comply with regulations in 34 C.F.R. § 74.45 that require that some form of cost or price analysis to be made and documented in connection with every procurement action. The November 2002 matrix presented pricing information for six of the nine vendors. However, the pricing information was generally presented on a per student basis and is not comparable among vendors (i.e., some specified a range of costs, training costs, and implementation costs, whereas others did not). As a result, the information contained in the matrix was insufficient to conduct a pricing analysis. Furthermore, ELC provided the audit team with no documentation of a cost or price analysis for the subsequent contract that included TPR. One of the rationales cited by ELC officials in their statements to us on selecting ATI and TPR as service providers for the FTL project was a June 2004 Forrester Consulting report.6 ELC commissioned the study that compared software company offerings that provided assessment, diagnostic, and/or remedial solutions to help students improve test scores. According to ELC officials, the Forrester Consulting report indicated that the integrated ATI and TPR products provided a unique tool not found elsewhere in the marketplace. We reviewed the report and interviewed the Project Director at Forrester Consulting. Forrester Consulting assessed the products of ten companies, including ATI and TPR.7 Based upon information contained in the report, it appears that in addition to the ATI and TPR combination, five other vendors met many of the criteria specified by ELC. As a result, the study indicated that, in addition to ATI and TPR, other vendors were capable of providing ELC with the software and services for the FTL project. While we do not know the exact number of vendors that were capable of providing the required software and services in 2002, when ELC contracted with ATI and later TPR, the report indicated that ATI and TPR were most likely not the only vendors capable of providing the software or services for FTL. According to the Project Director at Forrester Consulting, ELC’s use of the ATI and TPR combination met all the criteria, and a combination of other vendors could have met all the criteria. As a result, the Forrester Consulting report did not adequately justify the lack of competition and basis for the selection of ATI and TPR. ELC’s subcontracts with ATI and TPR represented the majority of the costs for the FTL project. During the audit period, the grant expenses of $17,538,020 for ATI and TPR accounted for 75 percent of total grant awards made to ELC during the period. Had ELC issued a solicitation that established all the requirements that bidders were to fulfill and conducted the procurement in a manner to provide free and open competition, ELC may have been able to obtain the needed services at a better value. In addition, because ELC did not comply with the procurement standards set forth in regulations, ELC could not readily demonstrate that it was a prudent steward of federal grant funds when contracting with FTL's service providers. 6 Forrester Consulting report entitled Use of Technology to Improve Student Achievement, Review of Research and Software, dated June 28, 2004. 7 Attachment 3 presents a summary of the software comparison contained in the Forrester Consulting report. Final Report ED-OIG/A03F0003 Page 7 of 17 Recommendations: We recommend that the Department's Chief Financial Officer, in collaboration with the Assistant Deputy Secretary for Innovation and Improvement, require ELC (renamed FTL) to: 1.1 Adopt procurement policies and procedures that comply with all the requirements set forth in regulations in 34 C.F.R. §§ 74.40 through 74.48. In addition, such policies and procedures should address the record retention requirements established in 34 C.F.R. § 74.53. 1.2 Ensure that it complies with procurement and documentation standards set forth in Department regulations when renewing or awarding any federally funded contract. ELC’s Comments: ELC concurred that it did not have the required written procurement policies in place at the time of our audit. ELC added that it has developed written procurement policies with input from the OIG that comply with all EDGAR requirements. ELC also acknowledged that documentation of its procurement process for the primary FTL service providers was deemed insufficient. ELC noted that it had responded to the Department’s questions about its use of sole source procurement, and the Department had approved the applications and budgets, which included ATI and TPR. Additionally, ELC noted that ATI’s and TPR’s products were purchased at a “steep discount.” OIG’s Response: When we provided comments on the drafts of ELC’s policies and procedures, we informed ELC that our comments were for its benefit in drafting polices and procedures and should not be considered as an approval of such policies and procedures. The procurement policies adopted by ELC, incorporated many, but not all, of the comments and suggestions that we provided. Additionally, the Department’s approval of ELC’s grant applications and knowledge of ELC’s sole source contracts with ATI and TPR did not relieve ELC of its requirement to comply with the procurement standards set forth in EDGAR. Furthermore, our comparison of the prices for software and training paid by ELC to the prices paid by other clients of ATI and TPR found that ELC was not the only client to which discounts were extended by ATI and TPR. We have included Attachments 2 and 3 to present summary-level information on ELC’s and Forrester Consulting’s comparison of software vendors’ offerings, both of which are discussed in the finding. Final Report ED-OIG/A03F0003 Page 8 of 17 OTHER MATTERS Our January 2006 audit report entitled The Education Leaders Council’s Drawdown and Expenditure of Federal Funds (control number ED-OIG/A03F0010), reviewed whether federal funds drawndown by ELC for the FTL project were used for ELC’s operations, and whether expenditures allocated to the federally funded FTL project were reasonable, allocable, and allowable in accordance with regulations. In general, we found that ELC drew down grant funds that could have been used to cover operating deficits in ELC’s non-FTL activities, charged indirect costs to the federal grants even though it did not have an approved indirect costs plan, and charged questioned and unsupported costs to the federal grants. A copy of the audit report, including ELC’s comments on the report, may be obtained from the Department’s Office of Inspector General website at http://www.ed.gov/about/offices/list/oig/auditreports/a03f0010.pdf. OBJECTIVES, SCOPE, AND METHODOLOGY The objectives of the audit were to determine if (1) the ELC’s subcontracting activities complied with the procurement standards set forth in EDGAR, (2) the subcontract costs for ATI and TPR charged to the federal grants were reasonable and allocable, and (3) any conflict of interest issues exist between ELC and its subcontractors. Our review covered the period July 1, 2002, through December 31, 2004. To achieve the audit objectives, we performed the following. We reviewed applicable criteria contained in EDGAR (34 C.F.R. Part 74); OMB Circular A-122, Cost Principles for Non-Profit Organizations; and OMB Circular A-110, Uniform Administrative Requirements for Grants and Agreements With Institutions of Higher Education, Hospitals, and Other Non-Profit Organizations. We reviewed ELC’s grant applications and the grant award notices for the fiscal year 2002, 2003, and 2004 grant awards. We reviewed ELC’s single audit reports and management letters for fiscal years 2002 and 2003. We also interviewed ELC’s current Chief Executive Officer and Chief Financial Officer, and ELC’s former Chief Executive Officers, Chief of Staff, Chief Operating Officer, Director of Federal Programs, Chief Policy Officer, and Chairman of the Board of Directors. To determine if ELC’s subcontracting activities complied with procurement standards, we performed the following. We reviewed ELC’s policies and procedures covering procurement. We reviewed ELC’s contracts and memorandum of understanding with ATI and TPR. We reviewed documents provided by ELC regarding the selection of ATI and TPR. We also reviewed the Forrester Consulting study entitled Use of Technology to Improve Student Achievement and interviewed the Project Director at Forrester Consulting. Final Report ED-OIG/A03F0003 Page 9 of 17 To determine if the subcontract costs for ATI and TPR charged to the federal grants were reasonable and allocable, we performed the following. We interviewed officials at ATI, including the Chief Executive Officer, Chief Financial Officer, Director of Finance, and Director of Project Management. We interviewed officials at TPR, including the Chief Executive Officer, Executive Vice President for Strategic Development, Executive Vice President for the K-12 Division, Vice President for the K-12 Division, and Division Controller for Test Preparation. We reviewed financial statements and reports for ATI and TPR covering calendar years 2002, 2003, and 2004. We reviewed all invoices submitted to ELC by ATI and TPR during the audit period. We compared the prices paid by ELC for software and training to ATI’s and TPR’s list prices. We compared ELC’s purchase prices for software and training to the prices paid by ATI’s and TPR’s other clients. We judgmentally selected sales to ATI’s and TPR’s other clients for review. The universe of ATI’s K-12 MegaSuite sales consisted of 13 sales in 2003 and 21 sales in 2004, excluding returns and sales to ELC. For each year, 2003 and 2004, the four largest sales to unique clients were selected. The universe of ATI’s training sales was 51 sales in 2003 and 47 sales in 2004. Three training sales from 2003 and four training sales from 2004 were selected based upon the amount of the sale and the sales amount relative to ATI’s list prices. The universe of TPR’s Homeroom software sales consisted of 908 sales during the period August 2000 through December 2004. We selected the 20 largest Homeroom sales to unique clients and 10 smaller Homeroom sales; these selections were also used to review training sales. In selecting sales for review, we excluded multiple sales to the same client (other than the largest sale) and sales to ELC. Because there is no assurance that the judgmental samples were representative of the respective universes, the results should not be projected to the unsampled records. To determine if any conflict of interest issues exist between ELC and its subcontractors, we performed the following. We reviewed ELC’s Form 990, Return of Organization Exempt From Income Tax, for calendar years 2001, 2002, 2003, and 2004. We compared listings of officers, employees, and board members of ELC and its subcontractors. We reviewed ELC’s sources of non-federal revenue for calendar years 2001, 2002, 2003, and 2004. We compared ELC’s monthly bank account balances for calendar years 2002 through 2004 to estimates of ELC’s monthly operating expenses. We reviewed documents pertaining to the significant contributions that ELC and its affiliate received from ATI and TPR in 2003. The audit team obtained electronic files from ELC's accounting systems of record of ELC's 2001 and 2002 non-federal revenue, 2002 federal grant expenses, and 2003 and 2004 general ledgers. We conducted a preliminary assessment of the computer-processed data that was obtained from ELC. The audit team assessed the completeness of the 2001, 2002, and 2003 electronic data by comparing it to information contained in ELC's respective audited financial statements and, in the case of grant expenses, to GAPS drawdown data. All of the data was considered fairly complete. The audit team also assessed the accuracy of the data that was material to the audit's objectives by verifying it to documentation or through discussions. Nothing came to our attention in this limited review to indicate the data was not accurate. The audit team was unable to assess the completeness of the 2004 electronic data because ELC had not issued audited financial statements at the time our audit was performed. However, the data was obtained from ELC's accounting system of record and represents the same data used by ELC's management for decision-making and reporting purposes. Therefore, we considered the data sufficiently reliable for the purposes of this review. Final Report ED-OIG/A03F0003 Page 10 of 17 As part of our audit, we assessed ELC's system of internal control, policies, procedures and practices applicable to ELC's subcontracting activities. For purposes of the audit, we assessed and classified the significant controls into the following category: procurement of goods and services with federal funds. Because of inherent limitations, a study and evaluation made for the limited purposes described above would not necessarily disclose all material weaknesses in internal controls. However, our assessment disclosed significant internal control weaknesses that adversely affected ELC's ability to administer the federal grant funds. These weaknesses resulted in ELC's procurement activities not complying with regulations. These weaknesses and their effects are fully discussed in the Audit Results section of this audit report. We conducted on-site fieldwork at ELC’s offices in Washington, DC during the period January 25, 2005, through March 28, 2005; at ATI’s offices in Newton, MA during the period March 14, 2005, through March 18, 2005; and at TPR’s offices in New York City, NY during the period May 24, 2005, through May 25, 2005. On March 31, 2006, we held an exit conference with ELC’s management. Our audit was performed in accordance with generally accepted government auditing standards appropriate to the scope of the review described above. ADMINISTRATIVE MATTERS This report incorporates comments you provided in response to the draft report. If you have any additional comments or information that you believe may have a bearing on the resolution of this audit, you should send them directly to the following Department of Education officials, who will consider them before taking the final Departmental action on this audit: William McCabe Acting Chief Financial Officer U.S. Department of Education 400 Maryland Avenue, SW Washington, D.C. 20202-4300 Margo Anderson Associate Assistant Deputy Secretary Office of Innovation and Improvement 400 Maryland Avenue, SW Washington, D.C. 20202-5900 It is the policy of the U.S. Department of Education to expedite the resolution of audits by initiating timely action on the findings and recommendations contained herein. Therefore, receipt of your comments within 30 days would be appreciated. Statements that managerial practices need improvements, as well as other conclusions and recommendations in this report, represent the opinions of the Office of Inspector General. Determinations of corrective action to be taken will be made by the appropriate Department of Education officials. Final Report ED-OIGIA03F0003 Page 11 of17 In accordance with the Freedom of Information Act (5 U.S.C. § 552), reports issued by the Office of Inspector General are available to members ofthe press and general public to the extent information contained therein is not subject to exemptions in the Act. . We appreciate the cooperation and assistance extended by your staff during the audit. If you have any questions, please contact me at (215) 656-6279. Sincerely, Regional Inspector General for Audit cc. James Home, Chairman, FTL Attachments Final Report ED-OIG/A03F0003 Page 12 of 17 Attachment 1: Supplemental Background Information FIE Grants Awards: During the period July 1, 2002, through December 31, 2004, ELC received three FIE grants totaling $23,376,534 for the FTL project, as follows: Award Date Award Number Award Amount Award Type July 1, 2002 R215U020001 $3,501,000 Unsolicited grant May 1, 2003 U215K030213 $9,934,999 Congressionally directed June 25, 2004 U215K040252 $9,940,535 Congressionally directed When the fiscal year 2005 grants, of $9,594,623, to ELC are included, ELC was awarded FIE grants totaling $32,971,157. ATI and TPR Contracts: In July 2002, ELC and ATI entered into a contract for the license, support, and implementation of ATI’s Achieve Resource Center, Project Achieve Software Suite, and SkillsTutor software in up to 15 schools in one state. In October 2002, the contract was amended. The amendments changed the contract’s term and authorized software licenses for six states. In January 2003, the contract was amended and restated. In addition to ELC and ATI, TPR became a party to the contract. The term of the amended and restated contract was January 1, 2003, through December 31, 2005. Under the contract, ELC guaranteed purchases from ATI and TPR equal to a percentage (ranging from 75 percent to 77 percent) of the funding ELC received for the FTL project during the term of the contract. ATI and TPR were to provide licenses, training and support for their products. ATI’s SkillsTutor and TPR’s Homeroom: ELC describes these products as follows. SkillsTutor is a web-based tool designed to help students master core subjects and skills. It focuses on four key areas, consisting of supplemental instruction and tutoring; diagnostic and prescriptive lessons; research-based content; and individualized instruction aligned to state and national standards. Homeroom is an online formative assessment and benchmarking tool aligned to state standards and tests and designed to increase student achievement. It allows the tracking of student performance to assess trends and opportunities for remediation, and the use of data to develop individualized learning plans. States Participating in the FTL Project: State No. of Schools State No. of Schools State No. of Schools Alaska 135 Arizona 7 Florida 25 Iowa 80 Illinois 77 Massachusetts 42 Mississippi 83 Ohio 45 Pennsylvania 27 Tennessee 51 West Virginia 61 Total 633 Source: ATI’s 2004 Billing Summary Final Report ED-OIG/A03F0003 Page 13 of 17 Attachment 2: Summary of ELC’s Comparison of Software Vendors Criteria Aligned With Curriculum Component Assessment Training & Web Based Prescribes Individual Standards Generates Pricing Information & Lesson Learning Tutoring Console Console Student Student Creates Service Company Parent Plans Tests State Path Presented Per Achievement Technologies, √ √ √ √ √ √ √ √ √ Student, Teacher & School Incorporated Company 2 √ √ √ √ √ √ No √ √ Grade & Subject Company 3 √ √ √ √ √ √ √ No No Not Available Company 4 √ No √ √ √ √ √ √ No Student Student, Implementation, Company 5 √ No No √ √ √ √ No No Training Company 6 √ No √ √ Limited √ No No √ Student Company 7 √ √ √ √ √ √ √ √ No Not Available Company 8 √ √ √ √ √ √ √ No √ Not Available Company 9 √ √ √ √ √ √ √ √ √ Module & Student Note: √ indicates that the vendor met the criteria. Source: ELC’s vendor comparison matrix, prepared November 2002. Final Report ED-OIG/A03F0003 Page 14 of 17 Attachment 3: Summary of Forrester Consulting’s Comparison of Software Company Offerings Forrester Consulting reviewed software that provided assessment, diagnostic and/or remedial solutions to help schools improve student test scores. Forrester Consulting reviewed the software from 10 companies, which were identified by ELC, and noted which offerings met specific criteria established by ELC. In the report’s Executive Summary, Forrester Consulting states, “In most cases the companies meet many of the criteria. Some do not have alignment to textbooks or remedial content built into their application. Others are developing this capability. Still others believe that teachers have access to many materials, and as long as teachers know the specific area of students weakness, they can provide their own remedial materials.” The Princeton Technologies, Incorporated Achievement Company 10 Company 3 Company 4 Company 5 Company 6 Company 7 Company 8 Company 9 Review Criteria Software Emphasis Analysis √ - √ √ √ √ √ √ √ √ Assessment √ - √ √ √ √ √ √ √ √ Remediation - √ - √ √ √ - - - - Formative Assessments Aligned to State Standards √ √ √ √ √ √ √ √ √ √ Linked to Specific Textbooks √ √ √ √ Pending Custom No No No √ Remediation Strategies Aligned to State Standards √ √ √ √ No √ No No √ √ Linked to Specific Textbooks √ √ √ √ No Custom No No Some √ Test Item Banks Not Sufficient to Assess Grades 3-12 √ √ √ √ √ √ √ √ √ Available Same Idiom and Format of State Mandated Not - Similar - Possibly √ Custom √ Possibly √ Assessments Available Data Can be Disaggregated According to NCLB √ √ √ √ √ √ √ √ √ √ Specifications Reporting Capabilities for Administrators, √ √ √ √ √ √ √ √ √ √ Principals, Teachers & Parents Final Report ED-OIG/A03F0003 Page 15 of 17 The Princeton Technologies, Incorporated Achievement Company 10 Company 3 Company 4 Company 5 Company 6 Company 7 Company 8 Company 9 Review Criteria Project Management Onsite Training – Initial √ √ √ √ √ √ Varies √ Online √ Onsite Training - Ongoing - √ √ √ √ √ √ Support √ Support Follow-up School Visits - √ √ √ √ √ √ √ √ √ Consultant Services to District & School Not √ √ √ √ √ √ √ √ √ Leadership Applicable Customization Downloading Student Information Into √ √ √ √ √ √ √ √ √ √ System Inputting Textbook Correlations √ √ √ √ √ √ No No No √ Inputting Alignment With State Standards √ √ √ √ √ √ √ No √ √ Hosted / Hosted / Hosted / Hosted / Hardware Compatibility Hosted Hosted Hosted Hosted Hosted Hosted LAN LAN PC LAN Aggregation / Usage and Performance Data Student √ √ √ √ √ √ √ √ √ √ School √ √ √ √ √ √ √ √ √ √ District √ √ √ √ √ √ √ √ √ √ State - No √ √ √ √ √ √ √ No National - No √ Possibly Possibly √ √ Possibly Possibly No Note: √ indicates that the vendor met the criteria. Source: Forrester Consulting report entitled Use of Technology to Improve Student Achievement, Review of Research and Software, dated June 28, 2004. Mr. Bernard Tadley Office of the Inspector General U.S. Department of Education The Wannamaker Building 100 Penn Square East, Suite 502 Philadelphia, PA 19107 June 6,2006 On behalf of Following the Leaders, Inc., we submit the following response to ED-OIG audit number A03-F003, Education Leaders Council's Subcontracting Activities, dated May 8, 2006 and prepared by the U.S. Department of Education's Office of the Inspector General (OIG) In December of2005, The Education Leaders Council (ELC) re-framed its organizational mission and changed its name to Following the Leaders (FTL). A new Chief Executive Officer, Dr. Faye P. Taylor, assumed leadership of the organization. Dr. Taylor replaced Mr. Theodot Rebarber who was the CEO from October, 2004 until December 2005. During their tenures, Dr. Taylor and Mr. Rebarber have worked diligently to correct operational difficulties that ELC had encountered primarily during the first six months of 2004. FTL is now re-organized and in full compliance with all federal regulations governing the use offederal funds awarded by the United States Congress for the Following the Leaders program. Thus, while we concur with the findings concerning certain procurement protocols, we are pleased to report that we have corrected these problems. The OIG's report identified one finding: 1. ELC's Subcontracting Activities Did Not Comply With the Procurement Standards Set Forth in EDGAR. ELC Did Not Have Written Procurement Records Auditee response: We concur that ELC did not have appropriate written procurement pDJicies in place during the period audited by the OIG. However, new management has since developed written procurement policies with input from the OIG that comply with all EDGAR requirements. In addition, procurement activities since June of2005 have been reviewed and approved by the Department of Education. Following the Leaders, Inc. 200 5th Avenue Suite D Springfield, TN 37172 ...* . ,.Following the Leaders ELC Did Not Coniply With Procurement Standards Auditee Response: The OIG audit notes that no public bidding or similar open procurement process was used to enter into contracts with Achievement Technologies (AT!) and The Princeton Review (TPR). As the OIG audit also notes, however, ELC included AT! and TPR in its budget application to the Department of Education. ELC answered the Department's questions about sole source procurement to the satisfaction of the Department and the budget was approved. ELC subsequently included AT! and TPRby name in each budget application, each of which was also approved. ELC submitted program reports as required under the terms of the awards to the Department; these reports contained detailed updates on the objectives and projected outcomes of the grant activities. Again, the Department did not indicate any dissatisfaction with ELC's progress or with AT! and TPR's contribution. Finally, ATI and TPR's products were offered to Following the Leaders participants at a steep discount and neither EtC nor Department staffhad reason to question the value ofthe services received (nor have they challenged costs under current contracts). In conclusion, while we do not disagree with the importance of clear and sound operating protocols and acknowledge that procedures were not always written down as required, we do believe that the value of services and the deliverables received met the program objectives. Clearly, following certain steps prior to engaging a contractor's services can help ensure the quality ofthe product received. ELC believed that they had taken such steps, but documentation ofthe process was deemed insufficient. The organization has added additional requirements to its procurement protocols which will ensure that sufficient documentation exists to demonstrate that procurement decisions are consistent with the letter as well as the spirit ofFederal regulations. Sincerely, Meave G. O'Marah, consulting Chief Financial Officer on behalf of Following the Leaders (formerly Education Leaders Council) Following the Leaders, Inc. 2 200 5th Avenue Suite D Springfield, TN 37172
Education Leaders Council's Subcontracting Activities.
Published by the Department of Education, Office of Inspector General on 2006-07-28.
Below is a raw (and likely hideous) rendition of the original report. (PDF)