oversight

Untimely Resolution of Issues Impacting Performance Validation and Payment Calculations Under the EDUCATE Contract

Published by the Department of Education, Office of Inspector General on 2010-03-09.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                                   UNITED STATES DEPARTMENT OF EDUCATION
                                                        OFFICE OF INSPECTOR GENERAL


                                                                                                                     AUDIT SERVICES
                                                            March 9, 2010

FINAL ALERT MEMORANDUM

TO:                  Danny Harris
                     Chief Information Officer
                     Office of the Chief Information Officer

                     Thomas Skelly 

                     Acting Chief Financial Officer 

                     Office of the Chief Financial Officer 



FROM: 	              Keith West /s/
                     Assistant Inspector General for Audit

SUBJECT:	            Untimely Resolution of Issues Impacting Performance Validation and Payment
                     Calculations Under the EDUCATE Contract
                     Control Number ED-OIG/L19K0001

While responding to allegations received by the Office of Inspector General’s (OIG) Hotline regarding
the U.S. Department of Education’s (Department) management of the Education Department Utility for
Communications, Applications, and Technology Environment (EDUCATE) contract, we became aware
that the Department did not act timely to resolve issues impacting performance validation and payment
calculations after the execution of a contract modification in December 2008. As a result, the Department
may have paid the EDUCATE contractor money it was not entitled to receive under the terms of the
contract. The purpose of this alert memorandum is to bring our concerns to your attention in order to
expedite corrective action.

Contract Background

The EDUCATE contract established a Contractor-Owned Contractor-Operated Information Technology
(IT) service model for the Department under which the contractor is required to provide the total IT
platform and infrastructure to support Department employees in meeting the Department’s mission. This
includes items such as desktop and printer services, helpdesk support, data center operations, e-mail,
network, disaster recovery, and other special services. The rendered IT services are monitored and
evaluated through established Service Level Agreements (SLA) that involve the application of incentives
and disincentives if services provided fall above or below the standards.

The Department’s process to calculate and approve incentives/disincentives under the SLA framework
includes multiple Department officials and the support of an Independent Verification and Validation
contractor (IV&V). The Department’s SLA owners are tasked with validating contractor-provided
performance data and calculations. SLA owners meet weekly with the EDUCATE contractor to resolve
discrepancies, with the accumulated data flowing into a monthly SLA report that summarizes
performance and compares actual results to standards established in the SLA framework. The SLA report
flows through multiple members of the contract administration team, including the Contracting Officer’s

 The Department of Education's mission is to promote student achievement and preparation for global competitiveness by fostering educational
                                                   excellence and ensuring equal access.
Final Alert Memorandum
ED-OIG/L19K0001                                                                               Page 2 of 4
Representative (COR), Assistant COR (ACOR), and Project Manager (PM). The PM makes
recommendations for final approval on invoice payment to the COR, who subsequently submits the
information to Contracts and Acquisition Management (CAM) for payment processing. The IV&V’s
responsibilities include independently verifying and validating the performance of the EDUCATE
contractor to determine if it is meeting desired performance goals. Within this role, the IV&V submits
monthly reports to the Department’s contract administration team that summarize the EDUCATE
contractor’s performance during the prior month.

The Department Did Not Act Timely to Resolve Noted Concerns

Contract Modification (MOD) 23, effective December 1, 2008, modified the SLA framework by revising
and creating SLAs. Among other changes, the modification implemented a new algorithm to determine
the amount of downtime for servers, allowing 30 hours of monthly downtime beginning with the
December 2008 performance period. During our review, we became aware that the Department did not
act timely to resolve conditions impacting the determination and validation of amounts due to the
EDUCATE contractor under the modified SLA framework.

In February 2009, the EDUCATE contractor submitted its December 2008 performance invoice to the
Department in the amount of $3,571,685. The Department withheld $135,767 from the payment issued,
claiming that performance was not reported in accordance with new SLA metrics established under MOD
23.

In July 2009, the EDUCATE contractor submitted an invoice to recoup $129,307 from the amount
withheld for the December 2008 performance period. The ACOR conducted a review of performance by
sampling SLAs and unit-based charges reported in the resubmitted December 2008 invoice, noting that
the results of her review conflicted with the results of the SLA report. The ACOR stated she utilized the
services of the IV&V to validate the SLA algorithms and metrics for the resubmitted December 2008
invoice, and on August 3, 2009, recommended the PM and COR withhold $71,642 from the invoice
payment.

The PM indicated she initiated follow-up with the ACOR regarding the results of her review, but the
ACOR was involved in an automobile accident and was out of the office for several days after submitting
her recommendation. According to the PM, the COR took the ACOR’s recommendation to a regularly
scheduled meeting on August 6, 2009, involving the PM, COR, and CAM. The PM further indicated the
COR reviewed each recommendation with those in attendance and the ACOR was later advised via
telephone that the outcome was to withhold only $2,160 from payment. Ultimately, the ACOR submitted
the required information to CAM on August 7, 2009, to ensure the resubmitted invoice was paid timely.

The PM stated she relied on the December 2008 SLA report provided by the Department’s SLA
coordinator as the basis for approving the invoice because the ACOR’s recommendation was a separate
internal analysis and the related calculations had not been vetted or approved by the SLA owners. She
further stated the ACOR’s recommendation was not based on the official SLA calculations and
determinations of the SLA owners.

During our review, we found concerns regarding SLA calculations had been reported by the IV&V since
April 2009. The IV&V stated uncertainty existed regarding the methodology used to calculate server
downtime and that without defined rules, it could not determine whether SLA performance was within
contractual requirements. In its March 2009 Monthly Report,1 the IV&V first stated it could not monitor
                                                            
1
    IV&V’s March 2009 Monthly Report was released in April 2009.
Final Alert Memorandum
ED-OIG/L19K0001                                                                                   Page 3 of 4
the EDUCATE contractor’s performance following MOD 23 because performance data was either
incomplete or not submitted. The report specifically stated the EDUCATE contractor was not:

        1.   Tracking scheduled downtime following the requirements of MOD 23;
        2.   Using correct SLA initiation times to verify compliance;
        3.   Providing correct data to support SLA compliance; and
        4.   Providing full and complete back-up data to support some SLAs.

Concerns were repeated in monthly IV&V reports through November 2009 (the last report issued at the
time of our review). In December 2009, after completion of our work, we were advised a meeting was
held that included Department officials, including the COR and SLA owners, the EDUCATE contractor,
and the IV&V. Issues noted as being discussed included properly tracking scheduled downtime, using
correct initiation times, and providing correct data to support SLA compliance following MOD 23 (items
1, 2 and 3 above). The COR and IV&V reported the noted issues were adequately addressed and
resolved. However, the IV&V stated full and complete backup data to support other SLAs (item 4 above)
was a recurring issue that was still not resolved.

Departmental Directive, Office of the Chief Financial Officer (OCFO): 2-108, dated August 6 2009
(Directive), Subpart VI.B.2, requires the PM to ensure that effective and cooperative relations and
communications are maintained among the PM, the CORs, and the Contracting Officer (CO)/Contract
Specialist for all monitoring and administration matters, particularly in the sharing of monitoring
information and in the review of reports, deliverables, invoices, and other administrative matters.

Subpart VI.E of the Directive states the COR “Monitors individual contracts to ensure a contractor’s
technical performance is in accordance with the contract.”

Subpart VII.H.2.d notes it is the responsibility of the COR to become familiar with the payment
provisions applicable to each contract he or she must monitor.

Subpart VII.P.2 states “A COR who believes that a contractor has failed (or may be expected to fail) to
perform under a contract must notify the CO promptly so that swift, positive, and firm steps can be taken
to protect the Government’s interests. If a COR fails to initiate corrective action in the face of known
performance deficiencies, such inaction could be judged to constitute a waiver of the Government’s right
later to demand remedy by the contractor.”

The individuals involved in the administration of the EDUCATE contract have not effectively fulfilled
their roles in the oversight and monitoring of the contract, to include establishing an effective process to
assign responsibility for timely resolving issues identified in the IV&V reports. The COR stated he did
not take action to resolve the noted concerns as he believed the responsibility resided with the PM
because the issues were technical and not contractual in nature.

As a result, the Department cannot assure services are being offered following the contracted terms and
that the Department’s interests are being protected. The Department may have already paid the
EDUCATE contractor money it was not entitled to receive under the terms of the contract. In addition,
submission of incomplete data and uncertainties with calculation methodologies limits the ability of the
IV&V to be able to effectively perform its intended role in contract performance monitoring.
Final Alert Memorandum
ED-OIG/L19K0001                                                                                  Page 4 of 4
Recommendations

We recommend the Chief Information Officer and Chief Financial Officer:

1.1 	 Instruct the PM and COR to review all payments made to the EDUCATE contractor for
      performance under the modified SLA framework (MOD 23) and request reimbursement for money
      it was not entitled to receive under the terms of the contract.

1.2 	 Proceed as quickly as possible to resolve the issue repeatedly presented in the IV&V reports
      regarding submission of incomplete data to support some SLAs.

1.3 	 Establish a process, to include the assignment of an accountable official, for timely resolving issues
      identified in the IV&V reports.

1.4	 Ensure any future concerns relating to EDUCATE performance validation or payment calculations
     are resolved in a timely manner and communicated to all critical contract administration team
     officials.

1.5 	 Ensure all EDUCATE contract administration team members understand their responsibilities in
      order to better protect the Department’s interests.

Department Comments

A draft of this memorandum was provided to the Office of the Chief Information Officer (OCIO) and
OCFO for comment. In its response to the draft alert memorandum, OCIO/OCFO concurred with our
finding and recommendations and described the corrective actions already taken or planned. The
response is included in its entirety as an Attachment to this memorandum.

Corrective actions proposed (resolution phase) and implemented (closure phase) will be monitored and
tracked through the Department’s Audit Accountability and Resolution Tracking System (AARTS).

Alert memoranda issued by the Office of Inspector General will be made available to members of the
press and general public to the extent information contained in the memoranda is not subject to
exemptions in the Freedom of Information Act (5 U.S.C. § 552).

For further information, please contact Michele Weaver-Dugan, Director, Operations Internal Audit
Team, at (202) 245-6941.


Attachment
                                                                                                                        Attachment


                              UNITED STATES DEPARTMENT OF EDUCATION
                                                 WASmNGTON. D .C. 20202--_ _        _




                                                                                        FEB 2 6 aJlO
Memorandum

To:                Keith West
                   Assistant Inspector General for Audit
                   Office of the Inspector General
                                                  ~   "
From:              Danny Harris
                   Chief InformatIon UU1cer
                   Office of the Chief lnfonnation Officer

                   Thomas Skelly
                   Acting Chie f Financial Officer
                   Office of the Chief Financia l Officer

Subject:           Response to Draft Alert Memo Entitled Untimely Resolution of issues Impacting
                   Performance Validation and Payment Calculations under the EDUCATE Contract
                   (Control Numbec ED-OIG/ L19KOOO I)



Thank you for providing the subject Alert Memo dated February 4, 2010. The Office of the Chief
Information Officer (OCIO) considers this a serious matter and greatly appreciates your review and
concluding recommendations for corrective action.

Please find be low our comments, corrective action plans and completion dates for each of your respective
recommendations.

Recommendation 1.1

           Instruct the PM and COR to review all payments made to the EDUCATE contractor for
           perfonnance under the modified SLA framework (MOD 23) and request reimbursement for
           money it was not entitled to receive under the tenns of the contract.

DCIO Response

           We concur with this recommendation. The PM, COR and Assistant Contracting Officer's
           Representative (ACOR) as officially de legated by the Contracting Officer, wi ll be instructed to
           review all payments made to the EDUCATE contractor under MOD 23 and withhold from
           payment any monies it was not entitled to receive under the tenns of the contract. This review
           and resu lting withholding will be completed by March 31, 2010.




        OW' mission is to   en$W'4! ~1lQ1   access to mucalion and to promote ed.uca.rtonal ucelknce   thro~lI.oul t~   Nation.
Page 2   -   Response to Draft Alert Memo ED-0IGIL19KOOOI




Recommendation 1.2

         Proceed as quickly as possible to resolve the issue repeatedly presented in the IV&V reports
         regarding submission of incomplete data to support some SLAs.


OCIO Response

         We concur with this recommendation and have already begun corrective action. A team was
         established late December, 2009, to assess the various SLA review processes, identify
         discrepancies between SLA Monitors (SLAMs) the EDUCATE Contractor and the IV&V
         Contractor's interpretation of individual Service Level Agreements, and improve SLA assessment
         processes where necessary. Much progress has been made, as reOeeted in comparative weekly
         I V&V status reports. Training has been initiated, SLAM access to core data provided, analyses
         of COiltractor data conducted, and individual SLAM monitoring practices established. Further
         assessment and improvements arc anticipated, but corrective action is aggressively underway. A
         corrective Action Plan will be in place and fonnal action initiated by March 19,2010.


Recommendation 1.3

         Establish a process, to include the assignment of an accountable official,for timely resolving
         issues identified in the IV&V reports.


OCIO Response

         We concur with this recommendation. An official OCIO Standard Operating Procedure will be
         established which designates the Infonnation Technology Services Dircctor as the Program
         Manager of the EDUCATE contract and addresses the Director as being accountable for timely
         resolution of the issues raised by IV&V. This Standard Operating Procedure will be completed
         and published in the OCIO Procedures That Work library by March 12,2010.


Recommendation 1.4

         Ensure any future concerns relating to EDUCATE perfonnance validation or payment
         calculations are resolved in a timely manner and communicated to all critical contract
         administration team officials.


OCIO Response

         We concur with this recommendation and as st'atcd above, will prepare a Standard Operating
         Procedure detailing fonnal steps to be taken to address EDUCATE Contract concerns relating to
         perfonnance validation and payment calculations in a timely manner. All OCIO staff shall be
         provided copies of these procedures by March 31, 20 I O.
Page 3 - Response 10 Draft Alert Memo ED-OIG/LI9KOOOI




Recommendation 1.5


        Ensure all EDUCATE COntracl administration team members understand their responsibilities in
        order to better protectlhe Department's inlercst's.

aClo Response

        We concur with this recommendation and implemented training of all contract administration
        tcam members early October of last year. The biweekly CUN Owner training program was
        established to address roles and responsibilities of all contract administrative team members, to
        introduce Ihe Integrated Mastcr Schedulc of all EDUCATE Contract Deliverables, to assure
        understanding of contract deliverable processes and 10 provide a forum for continued Service
        Level Agreement assessment. This program has proven to be a tremendous success and has
        helped to bring the team to consensus on contract processes, responsibilities and timeframes. The
        training program will continue indefinitely.


Thank you again for bringing this matter to our attention and for providing recommended action plans to
assure expeditious corrective action. OCIO is committed (0 accurate and timely contract management
and will continue to aggressively address any business management issues that might arise. We look
forward to our continued relationship with your office and arc available to discuss this matter further if
you feel necessary.